@InterfaceAudience.Private @InterfaceStability.Unstable public class RollingLevelDBTimelineStore extends org.apache.hadoop.service.AbstractService implements TimelineStore
An implementation of an application timeline store backed by leveldb.
There are three sections of the db, the start time section, the entity section, and the indexed entity section.
The start time section is used to retrieve the unique start time for a given entity. Its values each contain a start time while its keys are of the form:
START_TIME_LOOKUP_PREFIX + entity type + entity id
The entity section is ordered by entity type, then entity start time descending, then entity ID. There are four sub-sections of the entity section: events, primary filters, related entities, and other info. The event entries have event info serialized into their values. The other info entries have values corresponding to the values of the other info name/value map for the entry (note the names are contained in the key). All other entries have empty values. The key structure is as follows:
ENTITY_ENTRY_PREFIX + entity type + revstarttime + entity id ENTITY_ENTRY_PREFIX + entity type + revstarttime + entity id + DOMAIN_ID_COLUMN ENTITY_ENTRY_PREFIX + entity type + revstarttime + entity id + EVENTS_COLUMN + reveventtimestamp + eventtype ENTITY_ENTRY_PREFIX + entity type + revstarttime + entity id + PRIMARY_FILTERS_COLUMN + name + value ENTITY_ENTRY_PREFIX + entity type + revstarttime + entity id + OTHER_INFO_COLUMN + name ENTITY_ENTRY_PREFIX + entity type + revstarttime + entity id + RELATED_ENTITIES_COLUMN + relatedentity type + relatedentity id
The indexed entity section contains a primary filter name and primary filter value as the prefix. Within a given name/value, entire entity entries are stored in the same format as described in the entity section above (below, "key" represents any one of the possible entity entry keys described above).
INDEXED_ENTRY_PREFIX + primaryfilter name + primaryfilter value + key
TimelineStore.SystemFilter
org.apache.hadoop.service.Service.STATE
TimelineReader.Field
DEFAULT_LIMIT
Constructor and Description |
---|
RollingLevelDBTimelineStore() |
Modifier and Type | Method and Description |
---|---|
org.apache.hadoop.yarn.api.records.timeline.TimelineDomain |
getDomain(String domainId)
This method retrieves the domain information for a given ID.
|
org.apache.hadoop.yarn.api.records.timeline.TimelineDomains |
getDomains(String owner)
This method retrieves all the domains that belong to a given owner.
|
org.apache.hadoop.yarn.api.records.timeline.TimelineEntities |
getEntities(String entityType,
Long limit,
Long windowStart,
Long windowEnd,
String fromId,
Long fromTs,
NameValuePair primaryFilter,
Collection<NameValuePair> secondaryFilters,
EnumSet<TimelineReader.Field> fields,
TimelineDataManager.CheckAcl checkAcl)
This method retrieves a list of entity information,
TimelineEntity ,
sorted by the starting timestamp for the entity, descending. |
org.apache.hadoop.yarn.api.records.timeline.TimelineEntity |
getEntity(String entityId,
String entityType,
EnumSet<TimelineReader.Field> fields)
This method retrieves the entity information for a given entity.
|
org.apache.hadoop.yarn.api.records.timeline.TimelineEvents |
getEntityTimelines(String entityType,
SortedSet<String> entityIds,
Long limit,
Long windowStart,
Long windowEnd,
Set<String> eventType)
This method retrieves the events for a list of entities all of the same
entity type.
|
void |
put(org.apache.hadoop.yarn.api.records.timeline.TimelineDomain domain)
Store domain information to the timeline store.
|
org.apache.hadoop.yarn.api.records.timeline.TimelinePutResponse |
put(org.apache.hadoop.yarn.api.records.timeline.TimelineEntities entities)
Stores entity information to the timeline store.
|
protected void |
serviceInit(org.apache.hadoop.conf.Configuration conf) |
protected void |
serviceStart() |
protected void |
serviceStop() |
close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop
protected void serviceInit(org.apache.hadoop.conf.Configuration conf) throws Exception
serviceInit
in class org.apache.hadoop.service.AbstractService
Exception
protected void serviceStart() throws Exception
serviceStart
in class org.apache.hadoop.service.AbstractService
Exception
protected void serviceStop() throws Exception
serviceStop
in class org.apache.hadoop.service.AbstractService
Exception
public org.apache.hadoop.yarn.api.records.timeline.TimelineEntity getEntity(String entityId, String entityType, EnumSet<TimelineReader.Field> fields) throws IOException
TimelineReader
getEntity
in interface TimelineReader
entityId
- The entity whose information will be retrieved.entityType
- The type of the entity.fields
- Specifies which fields of the entity object to retrieve (see
TimelineReader.Field
). If the set of fields contains
TimelineReader.Field.LAST_EVENT_ONLY
and not TimelineReader.Field.EVENTS
, the
most recent event for each entity is retrieved. If null, retrieves
all fields.TimelineEntity
object.IOException
public org.apache.hadoop.yarn.api.records.timeline.TimelineEvents getEntityTimelines(String entityType, SortedSet<String> entityIds, Long limit, Long windowStart, Long windowEnd, Set<String> eventType) throws IOException
TimelineReader
getEntityTimelines
in interface TimelineReader
entityType
- The type of entities to retrieve events for.entityIds
- The entity IDs to retrieve events for.limit
- A limit on the number of events to return for each entity. If
null, defaults to TimelineReader.DEFAULT_LIMIT
events per entity.windowStart
- If not null, retrieves only events later than the given time
(exclusive)windowEnd
- If not null, retrieves only events earlier than the given time
(inclusive)eventType
- Restricts the events returned to the given types. If null, events
of all types will be returned.TimelineEvents
object.IOException
public org.apache.hadoop.yarn.api.records.timeline.TimelineEntities getEntities(String entityType, Long limit, Long windowStart, Long windowEnd, String fromId, Long fromTs, NameValuePair primaryFilter, Collection<NameValuePair> secondaryFilters, EnumSet<TimelineReader.Field> fields, TimelineDataManager.CheckAcl checkAcl) throws IOException
TimelineReader
TimelineEntity
,
sorted by the starting timestamp for the entity, descending. The starting
timestamp of an entity is a timestamp specified by the client. If it is not
explicitly specified, it will be chosen by the store to be the earliest
timestamp of the events received in the first put for the entity.getEntities
in interface TimelineReader
entityType
- The type of entities to return (required).limit
- A limit on the number of entities to return. If null, defaults to
TimelineReader.DEFAULT_LIMIT
.windowStart
- The earliest start timestamp to retrieve (exclusive). If null,
defaults to retrieving all entities until the limit is reached.windowEnd
- The latest start timestamp to retrieve (inclusive). If null,
defaults to Long.MAX_VALUE
fromId
- If fromId is not null, retrieve entities earlier than and
including the specified ID. If no start time is found for the
specified ID, an empty list of entities will be returned. The
windowEnd parameter will take precedence if the start time of this
entity falls later than windowEnd.fromTs
- If fromTs is not null, ignore entities that were inserted into the
store after the given timestamp. The entity's insert timestamp
used for this comparison is the store's system time when the first
put for the entity was received (not the entity's start time).primaryFilter
- Retrieves only entities that have the specified primary filter. If
null, retrieves all entities. This is an indexed retrieval, and no
entities that do not match the filter are scanned.secondaryFilters
- Retrieves only entities that have exact matches for all the
specified filters in their primary filters or other info. This is
not an indexed retrieval, so all entities are scanned but only
those matching the filters are returned.fields
- Specifies which fields of the entity object to retrieve (see
TimelineReader.Field
). If the set of fields contains
TimelineReader.Field.LAST_EVENT_ONLY
and not TimelineReader.Field.EVENTS
, the
most recent event for each entity is retrieved. If null, retrieves
all fields.TimelineEntities
object.IOException
public org.apache.hadoop.yarn.api.records.timeline.TimelinePutResponse put(org.apache.hadoop.yarn.api.records.timeline.TimelineEntities entities)
TimelineWriter
put
in interface TimelineWriter
entities
- a TimelineEntities
object.TimelinePutResponse
object.public void put(org.apache.hadoop.yarn.api.records.timeline.TimelineDomain domain) throws IOException
TimelineWriter
put
in interface TimelineWriter
domain
- a TimelineDomain
objectIOException
public org.apache.hadoop.yarn.api.records.timeline.TimelineDomain getDomain(String domainId) throws IOException
TimelineReader
getDomain
in interface TimelineReader
TimelineDomain
object.IOException
public org.apache.hadoop.yarn.api.records.timeline.TimelineDomains getDomains(String owner) throws IOException
TimelineReader
getDomains
in interface TimelineReader
owner
- the domain ownerTimelineDomains
object.IOException
Copyright © 2008–2024 Apache Software Foundation. All rights reserved.