@InterfaceAudience.Public @InterfaceStability.Evolving public abstract class SchedulingPolicy extends Object
Modifier and Type | Field and Description |
---|---|
static SchedulingPolicy |
DEFAULT_POLICY |
static byte |
DEPTH_ANY |
static byte |
DEPTH_INTERMEDIATE |
static byte |
DEPTH_LEAF |
static byte |
DEPTH_PARENT |
static byte |
DEPTH_ROOT |
Constructor and Description |
---|
SchedulingPolicy() |
Modifier and Type | Method and Description |
---|---|
abstract boolean |
checkIfAMResourceUsageOverLimit(org.apache.hadoop.yarn.api.records.Resource usage,
org.apache.hadoop.yarn.api.records.Resource maxAMResource)
Check if a leaf queue's AM resource usage over its limit under this policy
|
abstract boolean |
checkIfUsageOverFairShare(org.apache.hadoop.yarn.api.records.Resource usage,
org.apache.hadoop.yarn.api.records.Resource fairShare)
Check if the resource usage is over the fair share under this policy
|
abstract void |
computeShares(Collection<? extends Schedulable> schedulables,
org.apache.hadoop.yarn.api.records.Resource totalResources)
Computes and updates the shares of
Schedulable s as per
the SchedulingPolicy , to be used later for scheduling decisions. |
abstract void |
computeSteadyShares(Collection<? extends FSQueue> queues,
org.apache.hadoop.yarn.api.records.Resource totalResources)
Computes and updates the steady shares of
FSQueue s as per the
SchedulingPolicy . |
abstract byte |
getApplicableDepth()
Specifies the depths in the hierarchy, this
SchedulingPolicy
applies to |
abstract Comparator<Schedulable> |
getComparator()
The comparator returned by this method is to be used for sorting the
Schedulable s in that queue. |
abstract org.apache.hadoop.yarn.api.records.Resource |
getHeadroom(org.apache.hadoop.yarn.api.records.Resource queueFairShare,
org.apache.hadoop.yarn.api.records.Resource queueUsage,
org.apache.hadoop.yarn.api.records.Resource maxAvailable)
Get headroom by calculating the min of
clusterAvailable and
(queueFairShare - queueUsage ) resources that are
applicable to this policy. |
static SchedulingPolicy |
getInstance(Class<? extends SchedulingPolicy> clazz)
Returns a
SchedulingPolicy instance corresponding to the passed clazz |
abstract String |
getName() |
abstract org.apache.hadoop.yarn.util.resource.ResourceCalculator |
getResourceCalculator()
The
ResourceCalculator returned by this method should be used
for any calculations involving resources. |
void |
initialize(org.apache.hadoop.yarn.api.records.Resource clusterCapacity) |
static boolean |
isApplicableTo(SchedulingPolicy policy,
byte depth)
Checks if the specified
SchedulingPolicy can be used for a queue at
the specified depth in the hierarchy |
static SchedulingPolicy |
parse(String policy)
Returns
SchedulingPolicy instance corresponding to the
SchedulingPolicy passed as a string. |
public static final SchedulingPolicy DEFAULT_POLICY
public static final byte DEPTH_LEAF
public static final byte DEPTH_INTERMEDIATE
public static final byte DEPTH_ROOT
public static final byte DEPTH_PARENT
public static final byte DEPTH_ANY
public static SchedulingPolicy getInstance(Class<? extends SchedulingPolicy> clazz)
SchedulingPolicy
instance corresponding to the passed clazzpublic static SchedulingPolicy parse(String policy) throws AllocationConfigurationException
SchedulingPolicy
instance corresponding to the
SchedulingPolicy
passed as a string. The policy can be "fair" for
FairSharePolicy, "fifo" for FifoPolicy, or "drf" for
DominantResourceFairnessPolicy. For a custom
SchedulingPolicy
s in the RM classpath, the policy should be
canonical class name of the SchedulingPolicy
.policy
- canonical class name or "drf" or "fair" or "fifo"AllocationConfigurationException
public void initialize(org.apache.hadoop.yarn.api.records.Resource clusterCapacity)
public abstract org.apache.hadoop.yarn.util.resource.ResourceCalculator getResourceCalculator()
ResourceCalculator
returned by this method should be used
for any calculations involving resources.public abstract String getName()
SchedulingPolicy
public abstract byte getApplicableDepth()
SchedulingPolicy
applies toSchedulingPolicy
#DEPTH_*public static boolean isApplicableTo(SchedulingPolicy policy, byte depth)
SchedulingPolicy
can be used for a queue at
the specified depth in the hierarchypolicy
- SchedulingPolicy
we are checking the
depth-applicability fordepth
- queue's depth in the hierarchypublic abstract Comparator<Schedulable> getComparator()
Schedulable
s in that queue.public abstract void computeShares(Collection<? extends Schedulable> schedulables, org.apache.hadoop.yarn.api.records.Resource totalResources)
Schedulable
s as per
the SchedulingPolicy
, to be used later for scheduling decisions.
The shares computed are instantaneous and only consider queues with
running applications.schedulables
- Schedulable
s whose shares are to be updatedtotalResources
- Total Resource
s in the clusterpublic abstract void computeSteadyShares(Collection<? extends FSQueue> queues, org.apache.hadoop.yarn.api.records.Resource totalResources)
FSQueue
s as per the
SchedulingPolicy
. The steady share does not differentiate
between queues with and without running applications under them. The
steady share is not used for scheduling, it is displayed on the Web UI
for better visibility.queues
- FSQueue
s whose shares are to be updatedtotalResources
- Total Resource
s in the clusterpublic abstract boolean checkIfUsageOverFairShare(org.apache.hadoop.yarn.api.records.Resource usage, org.apache.hadoop.yarn.api.records.Resource fairShare)
usage
- Resource
the resource usagefairShare
- Resource
the fair sharepublic abstract boolean checkIfAMResourceUsageOverLimit(org.apache.hadoop.yarn.api.records.Resource usage, org.apache.hadoop.yarn.api.records.Resource maxAMResource)
usage
- Resource
the resource used by application mastersmaxAMResource
- Resource
the maximum allowed resource for
application masterspublic abstract org.apache.hadoop.yarn.api.records.Resource getHeadroom(org.apache.hadoop.yarn.api.records.Resource queueFairShare, org.apache.hadoop.yarn.api.records.Resource queueUsage, org.apache.hadoop.yarn.api.records.Resource maxAvailable)
clusterAvailable
and
(queueFairShare
- queueUsage
) resources that are
applicable to this policy. For eg if only memory then leave other
resources such as CPU to same as clusterAvailable.queueFairShare
- fairshare in the queuequeueUsage
- resources used in the queuemaxAvailable
- available resource in cluster for this queueCopyright © 2017 Apache Software Foundation. All Rights Reserved.