Application Information for Timeline Service 2.0
You can publish different types of information about your application; such entities, events, and metrics, to Timeline Service 2.0.
Timeline Entity Objects
events: A set of timeline events, ordered by the timestamp of the events in descending order. Each event contains an ID and a map to store related information and is associated with one timestamp.
configs: A map from a configuration name to a configuration value and representing all the configurations associated with the entity. Users can post the entire configuration or a part of it in this field. The field is supported for application and generic entities.
metrics: A set of metrics related to a particular entity. There are two types of metrics: single value metric and time series metric. Each metric item contains the metric name, value, and the type of aggregation operation to be performed in the metric. The field is supported for flow run, application and generic entities.
info: A map from an info key name to an (info value object that holds related information for the entity. The field is supported for application and generic entities.
relatestoEntities: Each entity contains
isrelatedtoEntitiesfields to represent relationships with other entities. Both the fields are represented by a map from a relationship name string to a timeline entity.
When posting timeline metrics, you can select how each metric is aggregated. The term
aggregate means applying a
for a set of entities. Timeline Service 2.0 enables aggregating metrics from
different timeline entities within one YARN application. The following are the types
of operations supported in
MAX: Retrieves the maximum value among all
SUM: Retrieves the sum of all
The default value of
NOP means that no real-time aggregation
operation is performed.
- Flow Name: A string that identifies the high-level flow; for example,
distributed grepor any identifiable name that can uniquely represent the application. The default value is the name of the application or the ID of the application if the name is not set.
- Flow Run ID: A monotonically increasing sequence of numbers that distinguish different runs of the same flow. The default value is the application time in UNIX time (milliseconds).
- Flow Version: This optional value is a string identifier that denotes a version of the flow. Flow version can be used to identify changes in the flows, such as code changes or script changes. The default version number is 1.
You can provide the flow context through YARN application tags, as shown in the following example:
ApplicationSubmissionContext appContext = app.getApplicationSubmissionContext(); // set the flow context as YARN application tags Set<String> tags = new HashSet<>(); tags.add(TimelineUtils.generateFlowNameTag("distributed grep")); tags.add(Timelineutils.generateFlowVersionTag ("3df8b0d6100530080d2e0decf9e528e57c42a90a")); tags.add(TimelineUtils.generateFlowRunIdTag(System.currentTimeMillis())); appContext.setApplicationTags(tags);
The ResourceManager converts YARN application tags to lower case values before storing them. Therefore, you must convert flow names and versions to lower case before using them in REST API queries.