Data Governance
Also available as:
PDF
loading table of contents...

Creating a Feed Entity

The feed entity defines the datasets that are cleaned and processed in your data pipeline. For more information, see the feed entity XSD here.

To use the Falcon web UI to define a feed entity:

  1. At the top of the Falcon web UI page, click Feed.

    Figure 2.5. New Feed Configuration Dialog


  2. On the New Feed page, specify the following values:

    Table 2.2. General Feed Configuration Values

    Value

    Description

    Name and Description

    Name and description of the feed entity.

    Tags

    Metadata tagging. For example, you can set the key to "externalTarget" and the corresponding value to "Marketing" tagging this feed for marketing.

    Groups

    Specify the feed group. Feeds can belong to multiple groups.

    Notification

    Enables email notifications that are sent when the scheduled feed instance completes. Specify the email address where you want notifications to be sent. For example, jdoe@xyz.com.

    Access Control List

    Specify the HDFS access permissions. Required for HDFS.

    Schema

    Specify the schema location and provider. This is required for HDFS.


  3. Click Next to advance to the Properties configuration where you can configure the timing and other feed properties.

  4. Click Next to advance to the Location configuration where you can specify the global location across clusters. For HDFS paths, choose File System and for Hive tables, choose Catalog Storage. For example, to specify a data path for a File System location, in the Data path text box, enter /weblogs/${YEAR}-${MONTH}-${DAY}-${HOUR} to point to the web logs.

  5. Click Next to advance to the Clusters configuration where you can:

    • Select the target cluster entity that you defined in Creating a Cluster Entity for retention or replication.

    • Specify the Storage type and Location. If you do not specify a location, the location that you specified in the Properties configuration is used.

    • Select the Validity interval.

  6. Click Next to view a summary of your feed entity definition. The XML file is displayed to the right of the summary. Click Edit XML to edit the XML directly.

  7. If you are satisfied with the feed entity definition, click Save.

  8. To verify that you successfully created the feed entity, enter the feed entity name in the Falcon web UI Search well and press Enter. If the feed entity name appears in the search results, it was successfully created. See Search For and Manage Data Pipeline Entities.