org.apache.hadoop.yarn.client.api.impl

Class NMClientImpl

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.yarn.client.api.NMClient

org.apache.hadoop.yarn.client.api.impl.NMClientImpl

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.service.Service

@InterfaceAudience.Private
@InterfaceStability.Unstable
public class NMClientImpl
extends NMClient

This class implements NMClient. All the APIs are blocking.

By default, this client stops all the running containers that are started by it when it stops. It can be disabled via cleanupRunningContainersOnStop(boolean), in which case containers will continue to run even after this client is stopped and till the application runs at which point ResourceManager will forcefully kill them.

Note that the blocking APIs ensure the RPC calls to NodeManager are executed immediately, and the responses are received before these APIs return. However, when startContainer(org.apache.hadoop.yarn.api.records.Container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext) or stopContainer(org.apache.hadoop.yarn.api.records.ContainerId, org.apache.hadoop.yarn.api.records.NodeId) returns, NodeManager may still need some time to either start or stop the container because of its asynchronous implementation. Therefore, getContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId, org.apache.hadoop.yarn.api.records.NodeId) is likely to return a transit container status if it is executed immediately after startContainer(org.apache.hadoop.yarn.api.records.Container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext) or stopContainer(org.apache.hadoop.yarn.api.records.ContainerId, org.apache.hadoop.yarn.api.records.NodeId).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	NMClientImpl.StartedContainer

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields

Modifier and Type	Field and Description
protected ConcurrentMap<org.apache.hadoop.yarn.api.records.ContainerId,NMClientImpl.StartedContainer>	startedContainers

Constructor Summary

Constructors

Constructor and Description
NMClientImpl()
NMClientImpl(String name)

Method Summary

Methods

Modifier and Type	Method and Description
protected void	cleanupRunningContainers()
void	cleanupRunningContainersOnStop(boolean enabled) Set whether the containers that are started by this client, and are still running should be stopped when the client stops.
AtomicBoolean	getCleanupRunningContainers()
org.apache.hadoop.yarn.api.records.ContainerStatus	getContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.NodeId nodeId) Query the status of a container.
void	increaseContainerResource(org.apache.hadoop.yarn.api.records.Container container) Increase the resource of a container.
protected void	serviceInit(org.apache.hadoop.conf.Configuration conf)
protected void	serviceStop()
Map<String,ByteBuffer>	startContainer(org.apache.hadoop.yarn.api.records.Container container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContext) Start an allocated container.
void	stopContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.NodeId nodeId) Stop an started container.

Methods inherited from class org.apache.hadoop.yarn.client.api.NMClient

createNMClient, createNMClient, getNMTokenCache, setNMTokenCache

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceStart, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

startedContainers

protected ConcurrentMap<org.apache.hadoop.yarn.api.records.ContainerId,NMClientImpl.StartedContainer> startedContainers

Constructor Detail

NMClientImpl

public NMClientImpl()

NMClientImpl

public NMClientImpl(String name)

Method Detail

serviceStop

protected void serviceStop()
                    throws Exception

Overrides:

serviceStop in class org.apache.hadoop.service.AbstractService

Throws:

Exception

cleanupRunningContainers

protected void cleanupRunningContainers()

serviceInit

protected void serviceInit(org.apache.hadoop.conf.Configuration conf)
                    throws Exception

Overrides:

serviceInit in class org.apache.hadoop.service.AbstractService

Throws:

Exception

cleanupRunningContainersOnStop

public void cleanupRunningContainersOnStop(boolean enabled)

Description copied from class: NMClient

Set whether the containers that are started by this client, and are still running should be stopped when the client stops. By default, the feature should be enabled.

However, containers will be stopped only when service is stopped. i.e. after AbstractService.stop().

Specified by:

cleanupRunningContainersOnStop in class NMClient

Parameters:

enabled - whether the feature is enabled or not

startContainer

public Map<String,ByteBuffer> startContainer(org.apache.hadoop.yarn.api.records.Container container,
                                    org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContext)
                                      throws org.apache.hadoop.yarn.exceptions.YarnException,
                                             IOException

Description copied from class: NMClient

Start an allocated container.

The ApplicationMaster or other applications that use the client must provide the details of the allocated container, including the Id, the assigned node's Id and the token via Container. In addition, the AM needs to provide the ContainerLaunchContext as well.

Specified by:

startContainer in class NMClient

Parameters:

container - the allocated container

containerLaunchContext - the context information needed by the NodeManager to launch the container

Returns:

a map between the auxiliary service names and their outputs

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

increaseContainerResource

public void increaseContainerResource(org.apache.hadoop.yarn.api.records.Container container)
                               throws org.apache.hadoop.yarn.exceptions.YarnException,
                                      IOException

Description copied from class: NMClient

Increase the resource of a container.

The ApplicationMaster or other applications that use the client must provide the details of the container, including the Id and the target resource encapsulated in the updated container token via Container.

Specified by:

increaseContainerResource in class NMClient

Parameters:

container - the container with updated token

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

stopContainer

public void stopContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                 org.apache.hadoop.yarn.api.records.NodeId nodeId)
                   throws org.apache.hadoop.yarn.exceptions.YarnException,
                          IOException

Description copied from class: NMClient

Stop an started container.

Specified by:

stopContainer in class NMClient

Parameters:

containerId - the Id of the started container

nodeId - the Id of the NodeManager

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

getContainerStatus

public org.apache.hadoop.yarn.api.records.ContainerStatus getContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                                                                    org.apache.hadoop.yarn.api.records.NodeId nodeId)
                                                                      throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                             IOException

Description copied from class: NMClient

Query the status of a container.

Specified by:

getContainerStatus in class NMClient

Parameters:

containerId - the Id of the started container

nodeId - the Id of the NodeManager

Returns:

the status of a container

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

getCleanupRunningContainers

public AtomicBoolean getCleanupRunningContainers()