@InterfaceAudience.Private @InterfaceStability.Unstable public class DominantResourceFairnessPolicy extends SchedulingPolicy
Modifier and Type | Class and Description |
---|---|
static class |
DominantResourceFairnessPolicy.DominantResourceFairnessComparator
This class compares two
Schedulable instances according to the
DRF policy. |
Modifier and Type | Field and Description |
---|---|
static String |
NAME |
DEFAULT_POLICY
Constructor and Description |
---|
DominantResourceFairnessPolicy() |
Modifier and Type | Method and Description |
---|---|
boolean |
checkIfUsageOverFairShare(org.apache.hadoop.yarn.api.records.Resource usage,
org.apache.hadoop.yarn.api.records.Resource fairShare)
Check if the resource usage is over the fair share under this policy.
|
void |
computeShares(Collection<? extends Schedulable> schedulables,
org.apache.hadoop.yarn.api.records.Resource totalResources)
Computes and updates the shares of
Schedulable s as per
the SchedulingPolicy , to be used later for scheduling decisions. |
void |
computeSteadyShares(Collection<? extends FSQueue> queues,
org.apache.hadoop.yarn.api.records.Resource totalResources)
Computes and updates the steady shares of
FSQueue s as per the
SchedulingPolicy . |
Comparator<Schedulable> |
getComparator()
The comparator returned by this method is to be used for sorting the
Schedulable s in that queue. |
org.apache.hadoop.yarn.api.records.Resource |
getHeadroom(org.apache.hadoop.yarn.api.records.Resource queueFairShare,
org.apache.hadoop.yarn.api.records.Resource queueUsage,
org.apache.hadoop.yarn.api.records.Resource maxAvailable)
Get headroom by calculating the min of
clusterAvailable and
(queueFairShare - queueUsage ) resources that are
applicable to this policy. |
String |
getName() |
org.apache.hadoop.yarn.util.resource.ResourceCalculator |
getResourceCalculator()
The
ResourceCalculator returned by this method should be used
for any calculations involving resources. |
void |
initialize(FSContext fsContext)
Initialize the scheduling policy with a
FSContext object, which has
a pointer to the cluster resources among other information. |
getInstance, initialize, isChildPolicyAllowed, parse
public static final String NAME
public String getName()
getName
in class SchedulingPolicy
SchedulingPolicy
public Comparator<Schedulable> getComparator()
SchedulingPolicy
Schedulable
s in that queue.getComparator
in class SchedulingPolicy
public org.apache.hadoop.yarn.util.resource.ResourceCalculator getResourceCalculator()
SchedulingPolicy
ResourceCalculator
returned by this method should be used
for any calculations involving resources.getResourceCalculator
in class SchedulingPolicy
public void computeShares(Collection<? extends Schedulable> schedulables, org.apache.hadoop.yarn.api.records.Resource totalResources)
SchedulingPolicy
Schedulable
s as per
the SchedulingPolicy
, to be used later for scheduling decisions.
The shares computed are instantaneous and only consider queues with
running applications.computeShares
in class SchedulingPolicy
schedulables
- Schedulable
s whose shares are to be updatedtotalResources
- Total Resource
s in the clusterpublic void computeSteadyShares(Collection<? extends FSQueue> queues, org.apache.hadoop.yarn.api.records.Resource totalResources)
SchedulingPolicy
FSQueue
s as per the
SchedulingPolicy
. The steady share does not differentiate
between queues with and without running applications under them. The
steady share is not used for scheduling, it is displayed on the Web UI
for better visibility.computeSteadyShares
in class SchedulingPolicy
queues
- FSQueue
s whose shares are to be updatedtotalResources
- Total Resource
s in the clusterpublic boolean checkIfUsageOverFairShare(org.apache.hadoop.yarn.api.records.Resource usage, org.apache.hadoop.yarn.api.records.Resource fairShare)
SchedulingPolicy
checkIfUsageOverFairShare
in class SchedulingPolicy
usage
- Resource
the resource usagefairShare
- Resource
the fair sharepublic org.apache.hadoop.yarn.api.records.Resource getHeadroom(org.apache.hadoop.yarn.api.records.Resource queueFairShare, org.apache.hadoop.yarn.api.records.Resource queueUsage, org.apache.hadoop.yarn.api.records.Resource maxAvailable)
SchedulingPolicy
clusterAvailable
and
(queueFairShare
- queueUsage
) resources that are
applicable to this policy. For eg if only memory then leave other
resources such as CPU to same as clusterAvailable
.getHeadroom
in class SchedulingPolicy
queueFairShare
- fairshare in the queuequeueUsage
- resources used in the queuemaxAvailable
- available resource in cluster for this queuepublic void initialize(FSContext fsContext)
SchedulingPolicy
FSContext
object, which has
a pointer to the cluster resources among other information.initialize
in class SchedulingPolicy
fsContext
- a FSContext
object which has a pointer to the
cluster resourcesCopyright © 2008–2024 Apache Software Foundation. All rights reserved.