org.apache.hadoop.yarn.server.resourcemanager.scheduler

Class SchedulerNode

java.lang.Object

org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode

Direct Known Subclasses:

FiCaSchedulerNode, FSSchedulerNode

@InterfaceAudience.Private
@InterfaceStability.Unstable
public abstract class SchedulerNode
extends Object

Represents a YARN Cluster Node from the viewpoint of the scheduler.

Constructor Summary

Constructors

Constructor and Description
SchedulerNode(RMNode node, boolean usePortForNodeName)
SchedulerNode(RMNode node, boolean usePortForNodeName, Set<String> labels)

Method Summary

Methods

Modifier and Type	Method and Description
void	allocateContainer(RMContainer rmContainer) The Scheduler has allocated containers on this node to the given application.
protected void	allocateContainer(RMContainer rmContainer, boolean launchedOnNode) The Scheduler has allocated containers on this node to the given application.
void	containerStarted(org.apache.hadoop.yarn.api.records.ContainerId containerId) Inform the node that a container has launched.
void	deductUnallocatedResource(org.apache.hadoop.yarn.api.records.Resource resource) Deduct unallocated resources from the node.
org.apache.hadoop.yarn.api.records.ResourceUtilization	getAggregatedContainersUtilization() Get the resource utilization of the containers in the node.
org.apache.hadoop.yarn.api.records.Resource	getAllocatedResource() Get allocated resources on the node.
protected RMContainer	getContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId) Get the container for the specified container ID.
List<RMContainer>	getCopiedListOfRunningContainers() Get the containers running on the node.
String	getHttpAddress() Get HTTP address for the node.
Set<String>	getLabels() Get the labels for the node.
org.apache.hadoop.yarn.api.records.NodeId	getNodeID() Get the ID of the node which contains both its hostname and port.
String	getNodeName() Get the name of the node for scheduling matching decisions.
org.apache.hadoop.yarn.api.records.ResourceUtilization	getNodeUtilization() Get the resource utilization of the node.
int	getNumContainers() Get number of active containers on the node.
String	getPartition() Get partition of which the node belongs to, if node-labels of this node is empty or null, it belongs to NO_LABEL partition.
String	getRackName() Get rackname.
RMContainer	getReservedContainer() Get the reserved container in the node.
RMNode	getRMNode()
List<RMContainer>	getRunningContainersWithAMsAtTheEnd() Get the containers running on the node with AM containers at the end.
org.apache.hadoop.yarn.api.records.Resource	getTotalResource() Get total resources on the node.
org.apache.hadoop.yarn.api.records.Resource	getUnallocatedResource() Get unallocated resources on the node.
boolean	isValidContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId) Check if a container is launched by this node.
void	recoverContainer(RMContainer rmContainer) Recover a container.
void	releaseContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, boolean releasedByNode) Release an allocated container on this node.
abstract void	reserveResource(SchedulerApplicationAttempt attempt, org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerKey, RMContainer container) Reserve container for the attempt on this node.
void	setAggregatedContainersUtilization(org.apache.hadoop.yarn.api.records.ResourceUtilization containersUtilization) Set the resource utilization of the containers in the node.
void	setNodeUtilization(org.apache.hadoop.yarn.api.records.ResourceUtilization nodeUtilization) Set the resource utilization of the node.
void	setReservedContainer(RMContainer reservedContainer) Set the reserved container in the node.
String	toString()
abstract void	unreserveResource(SchedulerApplicationAttempt attempt) Unreserve resources on this node.
void	updateLabels(Set<String> labels) Update the labels for the node.
protected void	updateResourceForReleasedContainer(org.apache.hadoop.yarn.api.records.Container container) Update the resources of the node when releasing a container.
void	updateTotalResource(org.apache.hadoop.yarn.api.records.Resource resource) Set total resources on the node.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

SchedulerNode

public SchedulerNode(RMNode node,
             boolean usePortForNodeName,
             Set<String> labels)

SchedulerNode

public SchedulerNode(RMNode node,
             boolean usePortForNodeName)

Method Detail

getRMNode

public RMNode getRMNode()

updateTotalResource

public void updateTotalResource(org.apache.hadoop.yarn.api.records.Resource resource)

Set total resources on the node.

Parameters:

resource - Total resources on the node.

getNodeID

public org.apache.hadoop.yarn.api.records.NodeId getNodeID()

Get the ID of the node which contains both its hostname and port.

Returns:

The ID of the node.

getHttpAddress

public String getHttpAddress()

Get HTTP address for the node.

Returns:

HTTP address for the node.

getNodeName

public String getNodeName()

Get the name of the node for scheduling matching decisions.

Typically this is the 'hostname' reported by the node, but it could be configured to be 'hostname:port' reported by the node via the YarnConfiguration.RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME constant. The main usecase of this is Yarn minicluster to be able to differentiate node manager instances by their port number.

Returns:

Name of the node for scheduling matching decisions.

getRackName

public String getRackName()

Get rackname.

Returns:

rackname

allocateContainer

public void allocateContainer(RMContainer rmContainer)

The Scheduler has allocated containers on this node to the given application.

Parameters:

rmContainer - Allocated container

allocateContainer

protected void allocateContainer(RMContainer rmContainer,
                     boolean launchedOnNode)

The Scheduler has allocated containers on this node to the given application.

Parameters:

rmContainer - Allocated container

launchedOnNode - True if the container has been launched

getUnallocatedResource

public org.apache.hadoop.yarn.api.records.Resource getUnallocatedResource()

Get unallocated resources on the node.

Returns:

Unallocated resources on the node

getAllocatedResource

public org.apache.hadoop.yarn.api.records.Resource getAllocatedResource()

Get allocated resources on the node.

Returns:

Allocated resources on the node

getTotalResource

public org.apache.hadoop.yarn.api.records.Resource getTotalResource()

Get total resources on the node.

Returns:

Total resources on the node.

isValidContainer

public boolean isValidContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Check if a container is launched by this node.

Returns:

If the container is launched by the node.

updateResourceForReleasedContainer

protected void updateResourceForReleasedContainer(org.apache.hadoop.yarn.api.records.Container container)

Update the resources of the node when releasing a container.

Parameters:

container - Container to release.

releaseContainer

public void releaseContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                    boolean releasedByNode)

Release an allocated container on this node.

Parameters:

containerId - ID of container to be released.

releasedByNode - whether the release originates from a node update.

containerStarted

public void containerStarted(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Inform the node that a container has launched.

Parameters:

containerId - ID of the launched container

deductUnallocatedResource

public void deductUnallocatedResource(org.apache.hadoop.yarn.api.records.Resource resource)

Deduct unallocated resources from the node. This is used when allocating a container.

Parameters:

resource - Resources to deduct.

reserveResource

public abstract void reserveResource(SchedulerApplicationAttempt attempt,
                   org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerKey,
                   RMContainer container)

Reserve container for the attempt on this node.

Parameters:

attempt - Application attempt asking for the reservation.

schedulerKey - Priority of the reservation.

container - Container reserving resources for.

unreserveResource

public abstract void unreserveResource(SchedulerApplicationAttempt attempt)

Unreserve resources on this node.

Parameters:

attempt - Application attempt that had done the reservation.

toString

public String toString()

Overrides:

toString in class Object

getNumContainers

public int getNumContainers()

Get number of active containers on the node.

Returns:

Number of active containers on the node.

getCopiedListOfRunningContainers

public List<RMContainer> getCopiedListOfRunningContainers()

Get the containers running on the node.

Returns:

A copy of containers running on the node.

getRunningContainersWithAMsAtTheEnd

public List<RMContainer> getRunningContainersWithAMsAtTheEnd()

Get the containers running on the node with AM containers at the end.

Returns:

A copy of running containers with AM containers at the end.

getContainer

protected RMContainer getContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Get the container for the specified container ID.

Parameters:

containerId - The container ID

Returns:

The container for the specified container ID

getReservedContainer

public RMContainer getReservedContainer()

Get the reserved container in the node.

Returns:

Reserved container in the node.

setReservedContainer

public void setReservedContainer(RMContainer reservedContainer)

Set the reserved container in the node.

Parameters:

reservedContainer - Reserved container in the node.

recoverContainer

public void recoverContainer(RMContainer rmContainer)

Recover a container.

Parameters:

rmContainer - Container to recover.

getLabels

public Set<String> getLabels()

Get the labels for the node.

Returns:

Set of labels for the node.

updateLabels

public void updateLabels(Set<String> labels)

Update the labels for the node.

Parameters:

labels - Set of labels for the node.

getPartition

public String getPartition()

Get partition of which the node belongs to, if node-labels of this node is empty or null, it belongs to NO_LABEL partition. And since we only support one partition for each node (YARN-2694), first label will be its partition.

Returns:

Partition for the node.

setAggregatedContainersUtilization

public void setAggregatedContainersUtilization(org.apache.hadoop.yarn.api.records.ResourceUtilization containersUtilization)

Set the resource utilization of the containers in the node.

Parameters:

containersUtilization - Resource utilization of the containers.

getAggregatedContainersUtilization

public org.apache.hadoop.yarn.api.records.ResourceUtilization getAggregatedContainersUtilization()

Get the resource utilization of the containers in the node.

Returns:

Resource utilization of the containers.

setNodeUtilization

public void setNodeUtilization(org.apache.hadoop.yarn.api.records.ResourceUtilization nodeUtilization)

Set the resource utilization of the node. This includes the containers.

Parameters:

nodeUtilization - Resource utilization of the node.

getNodeUtilization

public org.apache.hadoop.yarn.api.records.ResourceUtilization getNodeUtilization()

Get the resource utilization of the node.

Returns:

Resource utilization of the node.