Package org.apache.hadoop.yarn.server.uam

Class UnmanagedApplicationManager

java.lang.Object
org.apache.hadoop.yarn.server.uam.UnmanagedApplicationManager
@Public @Unstable public class UnmanagedApplicationManager extends Object
UnmanagedApplicationManager is used to register unmanaged application and negotiate for resources from resource managers. An unmanagedAM is an AM that is not launched and managed by the RM. Allocate calls are handled asynchronously using AsyncCallback.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String
    APP_NAME
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    UnmanagedApplicationManager(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId, String queueName, String submitter, String appNameSuffix, boolean keepContainersAcrossApplicationAttempts, String rmName, org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext originalApplicationSubmissionContext)
    Constructor.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    allocateAsync(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request, org.apache.hadoop.yarn.util.AsyncCallback<org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse> callback)
    Sends the specified heart beat request to the resource manager and invokes the callback asynchronously with the response.
    protected AMHeartbeatRequestHandler
    createAMHeartbeatRequestHandler(org.apache.hadoop.conf.Configuration config, org.apache.hadoop.yarn.api.records.ApplicationId appId, AMRMClientRelayer relayer)
     
    protected <T> T
    createRMProxy(Class<T> protocol, org.apache.hadoop.conf.Configuration config, org.apache.hadoop.security.UserGroupInformation user, org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> token)
    Returns RM proxy for the specified protocol type.
    protected void
    createUAMProxy(org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> amrmToken)
     
    protected void
    drainHeartbeatThread()
     
    org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterResponse
    finishApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterRequest request)
    Unregisters from the resource manager and stops the request handler thread.
    org.apache.hadoop.yarn.api.protocolrecords.KillApplicationResponse
    forceKillApplication()
    Force kill the UAM.
    AMRMClientRelayer
    getAMRMClientRelayer()
    Returns the rmProxy relayer of this UAM.
    org.apache.hadoop.yarn.api.records.ApplicationId
    getAppId()
    Returns the application id of the UAM.
    org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext
    getApplicationSubmissionContext()
     
    int
    getRequestQueueSize()
     
    protected org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier>
    getUAMToken()
    Gets the amrmToken of the unmanaged AM.
    protected org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier>
    initializeUnmanagedAM(org.apache.hadoop.yarn.api.records.ApplicationId appId)
    Launch and initialize an unmanaged AM.
    protected boolean
    isHeartbeatThreadAlive()
     
    org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier>
    launchUAM()
    Launch a new UAM in the resource manager.
    void
    reAttachUAM(org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> amrmToken)
    Re-attach to an existing UAM in the resource manager.
    org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse
    registerApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest request)
    Registers this UnmanagedApplicationManager with the resource manager.
    void
    shutDownConnections()
    Shutdown this UAM client, without killing the UAM in the YarnRM side.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Constructor Details

    • UnmanagedApplicationManager

      public UnmanagedApplicationManager(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId, String queueName, String submitter, String appNameSuffix, boolean keepContainersAcrossApplicationAttempts, String rmName, org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext originalApplicationSubmissionContext)
      Constructor.
      Parameters:
      conf - configuration
      appId - application Id to use for this UAM
      queueName - the queue of the UAM
      submitter - user name of the app
      appNameSuffix - the app name suffix to use
      rmName - name of the YarnRM
      originalApplicationSubmissionContext - ApplicationSubmissionContext
      keepContainersAcrossApplicationAttempts - keep container flag for UAM recovery. See #setKeepContainersAcrossApplicationAttempts(boolean)

  • Method Details

    • createAMHeartbeatRequestHandler

      @VisibleForTesting protected AMHeartbeatRequestHandler createAMHeartbeatRequestHandler(org.apache.hadoop.conf.Configuration config, org.apache.hadoop.yarn.api.records.ApplicationId appId, AMRMClientRelayer relayer)

    • launchUAM

      public org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> launchUAM() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Launch a new UAM in the resource manager.
      Returns:
      identifier uam identifier
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if fails
      IOException - if fails

    • reAttachUAM

      public void reAttachUAM(org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> amrmToken) throws IOException, org.apache.hadoop.yarn.exceptions.YarnException
      Re-attach to an existing UAM in the resource manager.
      Parameters:
      amrmToken - the UAM token
      Throws:
      IOException - if re-attach fails
      org.apache.hadoop.yarn.exceptions.YarnException - if re-attach fails

    • createUAMProxy

      protected void createUAMProxy(org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> amrmToken) throws IOException
      Throws:
      IOException

    • registerApplicationMaster

      public org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse registerApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Registers this UnmanagedApplicationManager with the resource manager.
      Parameters:
      request - RegisterApplicationMasterRequest
      Returns:
      register response
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if register fails
      IOException - if register fails

    • finishApplicationMaster

      public org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterResponse finishApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Unregisters from the resource manager and stops the request handler thread.
      Parameters:
      request - the finishApplicationMaster request
      Returns:
      the response
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if finishAM call fails
      IOException - if finishAM call fails

    • forceKillApplication

      public org.apache.hadoop.yarn.api.protocolrecords.KillApplicationResponse forceKillApplication() throws IOException, org.apache.hadoop.yarn.exceptions.YarnException
      Force kill the UAM.
      Returns:
      kill response
      Throws:
      IOException - if fails to create rmProxy
      org.apache.hadoop.yarn.exceptions.YarnException - if force kill fails

    • allocateAsync

      public void allocateAsync(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request, org.apache.hadoop.yarn.util.AsyncCallback<org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse> callback) throws org.apache.hadoop.yarn.exceptions.YarnException
      Sends the specified heart beat request to the resource manager and invokes the callback asynchronously with the response.
      Parameters:
      request - the allocate request
      callback - the callback method for the request
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if registerAM is not called yet

    • shutDownConnections

      public void shutDownConnections()
      Shutdown this UAM client, without killing the UAM in the YarnRM side.

    • getAppId

      public org.apache.hadoop.yarn.api.records.ApplicationId getAppId()
      Returns the application id of the UAM.
      Returns:
      application id of the UAM

    • getAMRMClientRelayer

      public AMRMClientRelayer getAMRMClientRelayer()
      Returns the rmProxy relayer of this UAM.
      Returns:
      rmProxy relayer of the UAM

    • createRMProxy

      protected <T> T createRMProxy(Class<T> protocol, org.apache.hadoop.conf.Configuration config, org.apache.hadoop.security.UserGroupInformation user, org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> token) throws IOException
      Returns RM proxy for the specified protocol type. Unit test cases can override this method and return mock proxy instances.
      Type Parameters:
      T - type of the proxy
      Parameters:
      protocol - protocol of the proxy
      config - configuration
      user - ugi for the proxy connection
      token - token for the connection
      Returns:
      the proxy instance
      Throws:
      IOException - if fails to create the proxy

    • initializeUnmanagedAM

      protected org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> initializeUnmanagedAM(org.apache.hadoop.yarn.api.records.ApplicationId appId) throws IOException, org.apache.hadoop.yarn.exceptions.YarnException
      Launch and initialize an unmanaged AM. First, it creates a new application on the RM and negotiates a new attempt id. Then it waits for the RM application attempt state to reach YarnApplicationAttemptState.LAUNCHED after which it returns the AM-RM token.
      Parameters:
      appId - application id
      Returns:
      the UAM token
      Throws:
      IOException - if initialize fails
      org.apache.hadoop.yarn.exceptions.YarnException - if initialize fails

    • getUAMToken

      protected org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> getUAMToken() throws IOException, org.apache.hadoop.yarn.exceptions.YarnException
      Gets the amrmToken of the unmanaged AM.
      Returns:
      the amrmToken of the unmanaged AM.
      Throws:
      IOException - if getApplicationReport fails
      org.apache.hadoop.yarn.exceptions.YarnException - if getApplicationReport fails

    • getRequestQueueSize

      @VisibleForTesting public int getRequestQueueSize()

    • drainHeartbeatThread

      @VisibleForTesting protected void drainHeartbeatThread()

    • isHeartbeatThreadAlive

      @VisibleForTesting protected boolean isHeartbeatThreadAlive()

    • getApplicationSubmissionContext

      @VisibleForTesting public org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext getApplicationSubmissionContext()