Package org.apache.hadoop.yarn.client.api

Class NMClient

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.yarn.client.api.NMClient
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.hadoop.service.Service
Direct Known Subclasses:
NMClientImpl
@Public @Stable public abstract class NMClient extends org.apache.hadoop.service.AbstractService

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    protected static enum 
    NMClient.UpgradeOp
     

    Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

    org.apache.hadoop.service.Service.STATE

  • Constructor Summary

    Constructors
    Modifier
    Constructor
    Description
    protected
    NMClient(String name)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    abstract void
    cleanupRunningContainersOnStop(boolean enabled)
    Set whether the containers that are started by this client, and are still running should be stopped when the client stops.
    abstract void
    commitLastReInitialization(org.apache.hadoop.yarn.api.records.ContainerId containerId)
    Commit last reInitialization of the specified container.
    static NMClient
    createNMClient()
    Create a new instance of NMClient.
    static NMClient
    createNMClient(String name)
    Create a new instance of NMClient.
    abstract org.apache.hadoop.yarn.api.records.ContainerStatus
    getContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.NodeId nodeId)
    Query the status of a container.
    List<org.apache.hadoop.yarn.api.records.LocalizationStatus>
    getLocalizationStatuses(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.NodeId nodeId)
    Get the localization statuses of a container.
    NMTokenCache
    getNMTokenCache()
    Get the NM token cache of the NMClient.
    org.apache.hadoop.yarn.api.records.NodeId
    getNodeIdOfStartedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
    Get the NodeId of the node on which container is running.
    abstract void
    increaseContainerResource(org.apache.hadoop.yarn.api.records.Container container)
    Deprecated.
    void
    localize(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.NodeId nodeId, Map<String,org.apache.hadoop.yarn.api.records.LocalResource> localResources)
    Localize resources for a container.
    abstract void
    reInitializeContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContex, boolean autoCommit)
    Re-Initialize the Container.
    abstract void
    restartContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
    Restart the specified container.
    abstract void
    rollbackLastReInitialization(org.apache.hadoop.yarn.api.records.ContainerId containerId)
    Rollback last reInitialization of the specified container.
    void
    setNMTokenCache(NMTokenCache nmTokenCache)
    Set the NM Token cache of the NMClient.
    abstract Map<String,ByteBuffer>
    startContainer(org.apache.hadoop.yarn.api.records.Container container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContext)
    Start an allocated container.
    abstract void
    stopContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.NodeId nodeId)
    Stop an started container.
    abstract void
    updateContainerResource(org.apache.hadoop.yarn.api.records.Container container)
    Update the resources of a container.

    Methods inherited from class org.apache.hadoop.service.AbstractService

    close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, serviceStart, serviceStop, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • NMClient

      @Private protected NMClient(String name)

  • Method Details

    • createNMClient

      @Public public static NMClient createNMClient()
      Create a new instance of NMClient.

    • createNMClient

      @Public public static NMClient createNMClient(String name)
      Create a new instance of NMClient.

    • startContainer

      public abstract Map<String,ByteBuffer> startContainer(org.apache.hadoop.yarn.api.records.Container container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContext) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException

      Start an allocated container.

      The ApplicationMaster or other applications that use the client must provide the details of the allocated container, including the Id, the assigned node's Id and the token via Container. In addition, the AM needs to provide the ContainerLaunchContext as well.

      Parameters:
      container - the allocated container
      containerLaunchContext - the context information needed by the NodeManager to launch the container
      Returns:
      a map between the auxiliary service names and their outputs
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - YarnException.
      IOException - IOException.

    • increaseContainerResource

      @Deprecated public abstract void increaseContainerResource(org.apache.hadoop.yarn.api.records.Container container) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Deprecated.

      Increase the resource of a container.

      The ApplicationMaster or other applications that use the client must provide the details of the container, including the Id and the target resource encapsulated in the updated container token via Container.

      Parameters:
      container - the container with updated token.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - YarnException.
      IOException - IOException.

    • updateContainerResource

      public abstract void updateContainerResource(org.apache.hadoop.yarn.api.records.Container container) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException

      Update the resources of a container.

      The ApplicationMaster or other applications that use the client must provide the details of the container, including the Id and the target resource encapsulated in the updated container token via Container.

      Parameters:
      container - the container with updated token.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - YarnException.
      IOException - IOException.

    • stopContainer

      public abstract void stopContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.NodeId nodeId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException

      Stop an started container.

      Parameters:
      containerId - the Id of the started container
      nodeId - the Id of the NodeManager
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - YarnException.
      IOException - IOException.

    • getContainerStatus

      public abstract org.apache.hadoop.yarn.api.records.ContainerStatus getContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.NodeId nodeId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException

      Query the status of a container.

      Parameters:
      containerId - the Id of the started container
      nodeId - the Id of the NodeManager
      Returns:
      the status of a container.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - YarnException.
      IOException - IOException.

    • reInitializeContainer

      public abstract void reInitializeContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContex, boolean autoCommit) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException

      Re-Initialize the Container.

      Parameters:
      containerId - the Id of the container to Re-Initialize.
      containerLaunchContex - the updated ContainerLaunchContext.
      autoCommit - commit re-initialization automatically ?
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - YarnException.
      IOException - IOException.

    • restartContainer

      public abstract void restartContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException

      Restart the specified container.

      Parameters:
      containerId - the Id of the container to restart.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - YarnException.
      IOException - IOException.

    • rollbackLastReInitialization

      public abstract void rollbackLastReInitialization(org.apache.hadoop.yarn.api.records.ContainerId containerId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException

      Rollback last reInitialization of the specified container.

      Parameters:
      containerId - the Id of the container to restart.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - YarnException.
      IOException - IOException.

    • commitLastReInitialization

      public abstract void commitLastReInitialization(org.apache.hadoop.yarn.api.records.ContainerId containerId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException

      Commit last reInitialization of the specified container.

      Parameters:
      containerId - the Id of the container to commit reInitialize.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - YarnException.
      IOException - IOException.

    • cleanupRunningContainersOnStop

      public abstract void cleanupRunningContainersOnStop(boolean enabled)

      Set whether the containers that are started by this client, and are still running should be stopped when the client stops. By default, the feature should be enabled.

      However, containers will be stopped only when service is stopped. i.e. after AbstractService.stop().
      Parameters:
      enabled - whether the feature is enabled or not

    • setNMTokenCache

      public void setNMTokenCache(NMTokenCache nmTokenCache)
      Set the NM Token cache of the NMClient. This cache must be shared with the AMRMClient that requested the containers managed by this NMClient

      If a NM token cache is not set, the NMTokenCache.getSingleton() singleton instance will be used.

      Parameters:
      nmTokenCache - the NM token cache to use.

    • getNMTokenCache

      public NMTokenCache getNMTokenCache()
      Get the NM token cache of the NMClient. This cache must be shared with the AMRMClient that requested the containers managed by this NMClient

      If a NM token cache is not set, the NMTokenCache.getSingleton() singleton instance will be used.

      Returns:
      the NM token cache

    • getNodeIdOfStartedContainer

      public org.apache.hadoop.yarn.api.records.NodeId getNodeIdOfStartedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
      Get the NodeId of the node on which container is running. It returns null if the container if container is not found or if it is not running.
      Parameters:
      containerId - Container Id of the container.
      Returns:
      NodeId of the container on which it is running.

    • localize

      @Unstable public void localize(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.NodeId nodeId, Map<String,org.apache.hadoop.yarn.api.records.LocalResource> localResources) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Localize resources for a container.
      Parameters:
      containerId - the ID of the container
      nodeId - node Id of the container
      localResources - resources to localize
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getLocalizationStatuses

      @Unstable public List<org.apache.hadoop.yarn.api.records.LocalizationStatus> getLocalizationStatuses(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.NodeId nodeId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Get the localization statuses of a container.
      Parameters:
      containerId - the Id of the container
      nodeId - node Id of the container
      Returns:
      the status of a container.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - YarnException.
      IOException - IOException.