public abstract class AppPlacementAllocator<N extends SchedulerNode> extends Object
This class has the following functionality:
1) Keeps track of pending resource requests when following events happen:
- New ResourceRequests are added to scheduler.
- New containers get allocated.
2) Determines the order that the nodes given in the CandidateNodeSet
will be used for allocating containers.
And different set of resource requests (E.g., resource requests with the same schedulerKey) can have one instance of AppPlacementAllocator, each AppPlacementAllocator can have different ways to order nodes depends on requests.
Modifier and Type | Field and Description |
---|---|
protected AppSchedulingInfo |
appSchedulingInfo |
protected RMContext |
rmContext |
protected org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey |
schedulerRequestKey |
Constructor and Description |
---|
AppPlacementAllocator() |
Modifier and Type | Method and Description |
---|---|
abstract ContainerRequest |
allocate(org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerKey,
NodeType type,
SchedulerNode node)
Notify container allocated.
|
abstract boolean |
canAllocate(NodeType type,
SchedulerNode node)
We can still have pending requirement for a given NodeType and node
|
abstract boolean |
canDelayTo(String resourceName)
Can delay to give locality?
TODO: This should be moved out of AppPlacementAllocator
and should belong to specific delay scheduling policy impl.
|
abstract int |
getOutstandingAsksCount(String resourceName)
Get #pending-allocations for given resourceName.
|
abstract PendingAsk |
getPendingAsk(String resourceName)
Get pending ask for given resourceName.
|
int |
getPlacementAttempt() |
Iterator<N> |
getPreferredNodeIterator(CandidateNodeSet<N> candidateNodeSet)
Get iterator of preferred node depends on requirement and/or availability.
|
abstract String |
getPrimaryRequestedNodePartition()
It is possible that one request can accept multiple node partition,
So this method returns primary node partition for pending resource /
headroom calculation.
|
abstract Map<String,org.apache.hadoop.yarn.api.records.ResourceRequest> |
getResourceRequests()
Get pending ResourceRequests by given schedulerRequestKey
|
abstract org.apache.hadoop.yarn.api.records.SchedulingRequest |
getSchedulingRequest()
Get pending SchedulingRequest.
|
abstract int |
getUniqueLocationAsks() |
void |
incrementPlacementAttempt() |
void |
initialize(AppSchedulingInfo appSchedulingInfo,
org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerRequestKey,
RMContext rmContext)
Initialize this allocator, this will be called by Factory automatically.
|
abstract boolean |
precheckNode(SchedulerNode schedulerNode,
SchedulingMode schedulingMode) |
abstract boolean |
precheckNode(SchedulerNode schedulerNode,
SchedulingMode schedulingMode,
Optional<DiagnosticsCollector> dcOpt)
Does this
AppPlacementAllocator accept resources on given node? |
abstract void |
showRequests()
Print human-readable requests to LOG debug.
|
abstract PendingAskUpdateResult |
updatePendingAsk(Collection<org.apache.hadoop.yarn.api.records.ResourceRequest> requests,
boolean recoverPreemptedRequestForAContainer)
Replace existing pending asks by the new requests
|
abstract PendingAskUpdateResult |
updatePendingAsk(org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerRequestKey,
org.apache.hadoop.yarn.api.records.SchedulingRequest schedulingRequest,
boolean recoverPreemptedRequestForAContainer)
Replace existing pending asks by the new SchedulingRequest
|
protected AppSchedulingInfo appSchedulingInfo
protected org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerRequestKey
protected RMContext rmContext
public Iterator<N> getPreferredNodeIterator(CandidateNodeSet<N> candidateNodeSet)
candidateNodeSet
- input CandidateNodeSetpublic abstract PendingAskUpdateResult updatePendingAsk(Collection<org.apache.hadoop.yarn.api.records.ResourceRequest> requests, boolean recoverPreemptedRequestForAContainer)
requests
- new asksrecoverPreemptedRequestForAContainer
- if we're recovering resource
requests for preempted containerpublic abstract PendingAskUpdateResult updatePendingAsk(org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerRequestKey, org.apache.hadoop.yarn.api.records.SchedulingRequest schedulingRequest, boolean recoverPreemptedRequestForAContainer)
schedulerRequestKey
- scheduler request keyschedulingRequest
- new asksrecoverPreemptedRequestForAContainer
- if we're recovering resource
requests for preempted containerpublic abstract Map<String,org.apache.hadoop.yarn.api.records.ResourceRequest> getResourceRequests()
public abstract PendingAsk getPendingAsk(String resourceName)
PendingAsk.ZERO
resourceName
- resourceNamepublic abstract int getOutstandingAsksCount(String resourceName)
resourceName
- resourceNamepublic abstract ContainerRequest allocate(org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerKey, NodeType type, SchedulerNode node)
schedulerKey
- SchedulerRequestKey for this ResourceRequesttype
- Type of the allocationnode
- Which node this container allocated onContainerRequest
for more details.public abstract boolean canAllocate(NodeType type, SchedulerNode node)
type
- Locality Typenode
- which node we will allocate onpublic abstract boolean canDelayTo(String resourceName)
resourceName
- resourceNamepublic abstract boolean precheckNode(SchedulerNode schedulerNode, SchedulingMode schedulingMode, Optional<DiagnosticsCollector> dcOpt)
AppPlacementAllocator
accept resources on given node?schedulerNode
- schedulerNodeschedulingMode
- schedulingModedcOpt
- optional diagnostics collectorpublic abstract boolean precheckNode(SchedulerNode schedulerNode, SchedulingMode schedulingMode)
public abstract String getPrimaryRequestedNodePartition()
public abstract int getUniqueLocationAsks()
public abstract void showRequests()
public void initialize(AppSchedulingInfo appSchedulingInfo, org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerRequestKey, RMContext rmContext)
appSchedulingInfo
- appSchedulingInfoschedulerRequestKey
- schedulerRequestKeyrmContext
- rmContextpublic abstract org.apache.hadoop.yarn.api.records.SchedulingRequest getSchedulingRequest()
public int getPlacementAttempt()
public void incrementPlacementAttempt()
Copyright © 2008–2024 Apache Software Foundation. All rights reserved.