public abstract class YarnConfigurationStore extends Object implements AutoCloseable
CapacityScheduler
configuration via key-value
using write-ahead logging. When configuration mutation is requested, caller
should first log it with logMutation
, which persists this pending
mutation. This mutation is merged to the persisted configuration only after
confirmMutation
is called.
On startup/recovery, caller should call retrieve
to get all
confirmed mutations, then get pending mutations which were not confirmed via
getPendingMutations
, and replay/confirm them via
confirmMutation
as in the normal case.Modifier and Type | Class and Description |
---|---|
static class |
YarnConfigurationStore.LogMutation
LogMutation encapsulates the fields needed for configuration mutation
audit logging and recovery.
|
Modifier and Type | Field and Description |
---|---|
static org.slf4j.Logger |
LOG |
Constructor and Description |
---|
YarnConfigurationStore() |
Modifier and Type | Method and Description |
---|---|
void |
checkVersion() |
abstract void |
close()
Closes the configuration store, releasing any required resources.
|
abstract void |
confirmMutation(YarnConfigurationStore.LogMutation pendingMutation,
boolean isValid)
Should be called after
logMutation . |
abstract void |
format()
Format the persisted configuration.
|
abstract long |
getConfigVersion()
Get the last updated config version.
|
abstract List<YarnConfigurationStore.LogMutation> |
getConfirmedConfHistory(long fromId)
Get a list of confirmed configuration mutations starting from a given id.
|
protected abstract org.apache.hadoop.yarn.server.records.Version |
getConfStoreVersion()
Get schema version of persisted conf store, for detecting compatibility
issues when changing conf store schema.
|
protected abstract org.apache.hadoop.yarn.server.records.Version |
getCurrentVersion()
Get the hard-coded schema version, for comparison against the schema
version currently persisted.
|
protected abstract LinkedList<YarnConfigurationStore.LogMutation> |
getLogs()
Get a list of configuration mutations.
|
abstract void |
initialize(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.conf.Configuration schedConf,
RMContext rmContext)
Initialize the configuration store, with schedConf as the initial
scheduler configuration.
|
abstract void |
logMutation(YarnConfigurationStore.LogMutation logMutation)
Logs the configuration change to backing store.
|
abstract org.apache.hadoop.conf.Configuration |
retrieve()
Retrieve the persisted configuration.
|
protected abstract void |
storeVersion()
Persist the hard-coded schema version to the conf store.
|
public abstract void initialize(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.conf.Configuration schedConf, RMContext rmContext) throws Exception
conf
- configuration to initialize store withschedConf
- Initial key-value scheduler configuration to persist.rmContext
- RMContext for this configuration storeIOException
- if initialization failsException
public abstract void close() throws IOException
close
in interface AutoCloseable
IOException
- on failure to closepublic abstract void logMutation(YarnConfigurationStore.LogMutation logMutation) throws Exception
logMutation
- configuration change to be persisted in write ahead logIOException
- if logging failsException
public abstract void confirmMutation(YarnConfigurationStore.LogMutation pendingMutation, boolean isValid) throws Exception
logMutation
. Gets the pending mutation
last logged by logMutation
and marks the mutation as persisted (no
longer pending). If isValid is true, merge the mutation with the persisted
configuration.pendingMutation
- the log mutation to applyisValid
- if true, update persisted configuration with pending
mutation.Exception
- if mutation confirmation failspublic abstract org.apache.hadoop.conf.Configuration retrieve() throws IOException
IOException
- an I/O exception has occurred.public abstract void format() throws Exception
IOException
- on failure to formatException
public abstract long getConfigVersion() throws Exception
Exception
- On version fetch failure.public abstract List<YarnConfigurationStore.LogMutation> getConfirmedConfHistory(long fromId)
fromId
- id from which to start getting mutations, inclusiveprotected abstract org.apache.hadoop.yarn.server.records.Version getConfStoreVersion() throws Exception
Exception
- On version fetch failureprotected abstract LinkedList<YarnConfigurationStore.LogMutation> getLogs() throws Exception
Exception
- On mutation fetch failureprotected abstract void storeVersion() throws Exception
Exception
- On storage failureprotected abstract org.apache.hadoop.yarn.server.records.Version getCurrentVersion()
Copyright © 2008–2024 Apache Software Foundation. All rights reserved.