Package org.apache.hadoop.yarn.client.api.impl

Class YarnClientImpl

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.yarn.client.api.YarnClient
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.hadoop.service.Service
@Private @Unstable public class YarnClientImpl extends YarnClient

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

    org.apache.hadoop.service.Service.STATE

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected AHSClient
    historyClient
     
    protected org.apache.hadoop.yarn.api.ApplicationClientProtocol
    rmClient
     
    protected long
    submitPollIntervalMillis
     
    protected org.apache.hadoop.yarn.client.api.TimelineClient
    timelineClient
     
    protected boolean
    timelineServiceBestEffort
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    YarnClientImpl()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    YarnClientApplication
    createApplication()
    Obtain a YarnClientApplication for a new application, which in turn contains the ApplicationSubmissionContext and GetNewApplicationResponse objects.
    org.apache.hadoop.yarn.api.protocolrecords.GetNewReservationResponse
    createReservation()
    Obtain a GetNewReservationResponse for a new reservation, which contains the ReservationId object.
    org.apache.hadoop.yarn.api.protocolrecords.ReservationDeleteResponse
    deleteReservation(org.apache.hadoop.yarn.api.protocolrecords.ReservationDeleteRequest request)
    The interface used by clients to remove an existing Reservation.
    void
    failApplicationAttempt(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId)
    Fail an application attempt identified by given ID.
    List<org.apache.hadoop.yarn.api.records.QueueInfo>
    getAllQueues()
    Get information (QueueInfo) about all queues, recursively if there is a hierarchy
    org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier>
    getAMRMToken(org.apache.hadoop.yarn.api.records.ApplicationId appId)
    Get the AMRM token of the application.
    org.apache.hadoop.yarn.api.records.ApplicationAttemptReport
    getApplicationAttemptReport(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)
    Get a report of the given ApplicationAttempt.
    List<org.apache.hadoop.yarn.api.records.ApplicationAttemptReport>
    getApplicationAttempts(org.apache.hadoop.yarn.api.records.ApplicationId appId)
    Get a report of all (ApplicationAttempts) of Application in the cluster.
    org.apache.hadoop.yarn.api.records.ApplicationReport
    getApplicationReport(org.apache.hadoop.yarn.api.records.ApplicationId appId)
    Get a report of the given Application.
    List<org.apache.hadoop.yarn.api.records.ApplicationReport>
    getApplications()
    Get a report (ApplicationReport) of all Applications in the cluster.
    List<org.apache.hadoop.yarn.api.records.ApplicationReport>
    getApplications(EnumSet<org.apache.hadoop.yarn.api.records.YarnApplicationState> applicationStates)
    Get a report (ApplicationReport) of Applications matching the given application states in the cluster.
    List<org.apache.hadoop.yarn.api.records.ApplicationReport>
    getApplications(Set<String> applicationTypes)
    Get a report (ApplicationReport) of Applications matching the given application types in the cluster.
    List<org.apache.hadoop.yarn.api.records.ApplicationReport>
    getApplications(Set<String> applicationTypes, EnumSet<org.apache.hadoop.yarn.api.records.YarnApplicationState> applicationStates)
    Get a report (ApplicationReport) of Applications matching the given application types and application states in the cluster.
    List<org.apache.hadoop.yarn.api.records.ApplicationReport>
    getApplications(Set<String> applicationTypes, EnumSet<org.apache.hadoop.yarn.api.records.YarnApplicationState> applicationStates, Set<String> applicationTags)
    Get a report (ApplicationReport) of Applications matching the given application types, application states and application tags in the cluster.
    List<org.apache.hadoop.yarn.api.records.ApplicationReport>
    getApplications(Set<String> queues, Set<String> users, Set<String> applicationTypes, EnumSet<org.apache.hadoop.yarn.api.records.YarnApplicationState> applicationStates)
    Get a report (ApplicationReport) of Applications matching the given users, queues, application types and application states in the cluster.
    List<org.apache.hadoop.yarn.api.records.ApplicationReport>
    getApplications(org.apache.hadoop.yarn.api.protocolrecords.GetApplicationsRequest request)
    Get a list of ApplicationReports that match the given GetApplicationsRequest.
    Map<org.apache.hadoop.yarn.api.records.NodeAttributeKey,List<org.apache.hadoop.yarn.api.records.NodeToAttributeValue>>
    getAttributesToNodes(Set<org.apache.hadoop.yarn.api.records.NodeAttributeKey> attributes)
    The interface used by client to get mapping of AttributeKey to associated NodeToAttributeValue list for specified node attributeKeys in the cluster.
    List<org.apache.hadoop.yarn.api.records.QueueInfo>
    getChildQueueInfos(String parent)
    Get information (QueueInfo) about all the immediate children queues of the given queue
    Set<org.apache.hadoop.yarn.api.records.NodeAttributeInfo>
    getClusterAttributes()
    The interface used by client to get node attributes in the cluster.
    List<org.apache.hadoop.yarn.api.records.NodeLabel>
    getClusterNodeLabels()
    The interface used by client to get node labels in the cluster
    org.apache.hadoop.yarn.api.records.ContainerReport
    getContainerReport(org.apache.hadoop.yarn.api.records.ContainerId containerId)
    Get a report of the given Container.
    List<org.apache.hadoop.yarn.api.records.ContainerReport>
    getContainers(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId)
    Get a report of all (Containers) of ApplicationAttempt in the cluster.
    Map<String,Set<org.apache.hadoop.yarn.api.records.NodeId>>
    getLabelsToNodes()
    The interface used by client to get labels to nodes mapping in existing cluster
    Map<String,Set<org.apache.hadoop.yarn.api.records.NodeId>>
    getLabelsToNodes(Set<String> labels)
    The interface used by client to get labels to nodes mapping for specified labels in existing cluster
    List<org.apache.hadoop.yarn.api.records.NodeReport>
    getNodeReports(org.apache.hadoop.yarn.api.records.NodeState... states)
    Get a report of nodes (NodeReport) in the cluster.
    Map<String,Set<org.apache.hadoop.yarn.api.records.NodeAttribute>>
    getNodeToAttributes(Set<String> hostNames)
    The interface used by client to get all node to attribute mapping in existing cluster.
    Map<org.apache.hadoop.yarn.api.records.NodeId,Set<String>>
    getNodeToLabels()
    The interface used by client to get node to labels mappings in existing cluster
    List<org.apache.hadoop.yarn.api.records.QueueUserACLInfo>
    getQueueAclsInfo()
    Get information about acls for current user on all the existing queues.
    org.apache.hadoop.yarn.api.records.QueueInfo
    getQueueInfo(String queueName)
    Get information (QueueInfo) about a given queue.
    org.apache.hadoop.yarn.api.records.QueueInfo
    getQueueInfo(String queueName, String subClusterId)
    Get information (QueueInfo) about a given queue.
    org.apache.hadoop.yarn.api.records.Resource
    getResourceProfile(String profile)
    Get the details of a specific resource profile from the RM.
    Map<String,org.apache.hadoop.yarn.api.records.Resource>
    getResourceProfiles()
    Get the resource profiles available in the RM.
    List<org.apache.hadoop.yarn.api.records.ResourceTypeInfo>
    getResourceTypeInfo()
    Get available resource types supported by RM.
    org.apache.hadoop.yarn.api.records.Token
    getRMDelegationToken(org.apache.hadoop.io.Text renewer)
    Get a delegation token so as to be able to talk to YARN using those tokens.
    List<org.apache.hadoop.yarn.api.records.QueueInfo>
    getRootQueueInfos()
    Get information (QueueInfo) about top level queues.
    org.apache.hadoop.yarn.api.records.YarnClusterMetrics
    getYarnClusterMetrics()
    Get metrics (YarnClusterMetrics) about the cluster.
    protected boolean
    isSecurityEnabled()
     
    void
    killApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId)
    Kill an application identified by given ID.
    void
    killApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String diagnostics)
    Kill an application identified by given ID.
    org.apache.hadoop.yarn.api.protocolrecords.ReservationListResponse
    listReservations(org.apache.hadoop.yarn.api.protocolrecords.ReservationListRequest request)
    The interface used by clients to get the list of reservations in a plan.
    void
    moveApplicationAcrossQueues(org.apache.hadoop.yarn.api.records.ApplicationId appId, String queue)
    Attempts to move the given application to the given queue.
    protected void
    serviceInit(org.apache.hadoop.conf.Configuration conf)
     
    protected void
    serviceStart()
     
    protected void
    serviceStop()
     
    void
    setRMClient(org.apache.hadoop.yarn.api.ApplicationClientProtocol rmClient)
     
    void
    shellToContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.ShellContainerCommand command)
    The interface used by client to get a shell to a container.
    void
    signalToContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.SignalContainerCommand command)
    Signal a container identified by given ID.
    org.apache.hadoop.yarn.api.records.ApplicationId
    submitApplication(org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext appContext)
    Submit a new application to YARN. It is a blocking call - it will not return ApplicationId until the submitted application is submitted successfully and accepted by the ResourceManager.
    org.apache.hadoop.yarn.api.protocolrecords.ReservationSubmissionResponse
    submitReservation(org.apache.hadoop.yarn.api.protocolrecords.ReservationSubmissionRequest request)
    The interface used by clients to submit a new reservation to the ResourceManager.
    org.apache.hadoop.yarn.api.records.Priority
    updateApplicationPriority(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, org.apache.hadoop.yarn.api.records.Priority priority)
    The interface used by client to set priority of an application
    org.apache.hadoop.yarn.api.protocolrecords.UpdateApplicationTimeoutsResponse
    updateApplicationTimeouts(org.apache.hadoop.yarn.api.protocolrecords.UpdateApplicationTimeoutsRequest request)
     
    org.apache.hadoop.yarn.api.protocolrecords.ReservationUpdateResponse
    updateReservation(org.apache.hadoop.yarn.api.protocolrecords.ReservationUpdateRequest request)
    The interface used by clients to update an existing Reservation.

    Methods inherited from class org.apache.hadoop.yarn.client.api.YarnClient

    createYarnClient

    Methods inherited from class org.apache.hadoop.service.AbstractService

    close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Field Details

    • rmClient

      protected org.apache.hadoop.yarn.api.ApplicationClientProtocol rmClient

    • submitPollIntervalMillis

      protected long submitPollIntervalMillis

    • historyClient

      protected AHSClient historyClient

    • timelineClient

      protected volatile org.apache.hadoop.yarn.client.api.TimelineClient timelineClient

    • timelineServiceBestEffort

      protected boolean timelineServiceBestEffort

  • Constructor Details

    • YarnClientImpl

      public YarnClientImpl()

  • Method Details

    • serviceInit

      protected void serviceInit(org.apache.hadoop.conf.Configuration conf) throws Exception
      Overrides:
      serviceInit in class org.apache.hadoop.service.AbstractService
      Throws:
      Exception

    • serviceStart

      protected void serviceStart() throws Exception
      Overrides:
      serviceStart in class org.apache.hadoop.service.AbstractService
      Throws:
      Exception

    • serviceStop

      protected void serviceStop() throws Exception
      Overrides:
      serviceStop in class org.apache.hadoop.service.AbstractService
      Throws:
      Exception

    • createApplication

      public YarnClientApplication createApplication() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Obtain a YarnClientApplication for a new application, which in turn contains the ApplicationSubmissionContext and GetNewApplicationResponse objects.

      Specified by:
      createApplication in class YarnClient
      Returns:
      YarnClientApplication built for a new application
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • submitApplication

      public org.apache.hadoop.yarn.api.records.ApplicationId submitApplication(org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext appContext) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Submit a new application to YARN. It is a blocking call - it will not return ApplicationId until the submitted application is submitted successfully and accepted by the ResourceManager.

      Users should provide an ApplicationId as part of the parameter ApplicationSubmissionContext when submitting a new application, otherwise it will throw the ApplicationIdNotProvidedException.

      This internally calls (SubmitApplicationRequest), and after that, it internally invokes (GetApplicationReportRequest) and waits till it can make sure that the application gets properly submitted. If RM fails over or RM restart happens before ResourceManager saves the application's state, #getApplicationReport(GetApplicationReportRequest) will throw the ApplicationNotFoundException. This API automatically resubmits the application with the same ApplicationSubmissionContext when it catches the ApplicationNotFoundException

      Specified by:
      submitApplication in class YarnClient
      Parameters:
      appContext - ApplicationSubmissionContext containing all the details needed to submit a new application
      Returns:
      ApplicationId of the accepted application
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException
      See Also:

    • isSecurityEnabled

      @Private @VisibleForTesting protected boolean isSecurityEnabled()

    • failApplicationAttempt

      public void failApplicationAttempt(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Fail an application attempt identified by given ID.

      Specified by:
      failApplicationAttempt in class YarnClient
      Parameters:
      attemptId - ApplicationAttemptId of the attempt to fail.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - in case of errors or if YARN rejects the request due to access-control restrictions.
      IOException
      See Also:

    • killApplication

      public void killApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Kill an application identified by given ID.

      Specified by:
      killApplication in class YarnClient
      Parameters:
      applicationId - ApplicationId of the application that needs to be killed
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - in case of errors or if YARN rejects the request due to access-control restrictions.
      IOException
      See Also:

    • killApplication

      public void killApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String diagnostics) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Kill an application identified by given ID.

      Specified by:
      killApplication in class YarnClient
      Parameters:
      applicationId - ApplicationId of the application that needs to be killed
      diagnostics - for killing an application.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - in case of errors or if YARN rejects the request due to access-control restrictions.
      IOException

    • getApplicationReport

      public org.apache.hadoop.yarn.api.records.ApplicationReport getApplicationReport(org.apache.hadoop.yarn.api.records.ApplicationId appId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a report of the given Application.

      In secure mode, YARN verifies access to the application, queue etc. before accepting the request.

      If the user does not have VIEW_APP access then the following fields in the report will be set to stubbed values:

      • host - set to "N/A"
      • RPC port - set to -1
      • client token - set to "N/A"
      • diagnostics - set to "N/A"
      • tracking URL - set to "N/A"
      • original tracking URL - set to "N/A"
      • resource usage report - all values are -1
      Specified by:
      getApplicationReport in class YarnClient
      Parameters:
      appId - ApplicationId of the application that needs a report
      Returns:
      application report
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getAMRMToken

      public org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> getAMRMToken(org.apache.hadoop.yarn.api.records.ApplicationId appId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient
      Get the AMRM token of the application.

      The AMRM token is required for AM to RM scheduling operations. For managed Application Masters YARN takes care of injecting it. For unmanaged Applications Masters, the token must be obtained via this method and set in the UserGroupInformation of the current user.

      The AMRM token will be returned only if all the following conditions are met:

      • the requester is the owner of the ApplicationMaster
      • the application master is an unmanaged ApplicationMaster
      • the application master is in ACCEPTED state
      Else this method returns NULL.
      Specified by:
      getAMRMToken in class YarnClient
      Parameters:
      appId - ApplicationId of the application to get the AMRM token
      Returns:
      the AMRM token if available
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getApplications

      public List<org.apache.hadoop.yarn.api.records.ApplicationReport> getApplications() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a report (ApplicationReport) of all Applications in the cluster.

      If the user does not have VIEW_APP access for an application then the corresponding report will be filtered as described in YarnClient.getApplicationReport(ApplicationId).

      Specified by:
      getApplications in class YarnClient
      Returns:
      a list of reports of all running applications
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getApplications

      public List<org.apache.hadoop.yarn.api.records.ApplicationReport> getApplications(Set<String> applicationTypes) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a report (ApplicationReport) of Applications matching the given application types in the cluster.

      If the user does not have VIEW_APP access for an application then the corresponding report will be filtered as described in YarnClient.getApplicationReport(ApplicationId).

      Specified by:
      getApplications in class YarnClient
      Parameters:
      applicationTypes - set of application types you are interested in
      Returns:
      a list of reports of applications
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getApplications

      public List<org.apache.hadoop.yarn.api.records.ApplicationReport> getApplications(EnumSet<org.apache.hadoop.yarn.api.records.YarnApplicationState> applicationStates) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a report (ApplicationReport) of Applications matching the given application states in the cluster.

      If the user does not have VIEW_APP access for an application then the corresponding report will be filtered as described in YarnClient.getApplicationReport(ApplicationId).

      Specified by:
      getApplications in class YarnClient
      Parameters:
      applicationStates - set of application states you are interested in
      Returns:
      a list of reports of applications
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getApplications

      public List<org.apache.hadoop.yarn.api.records.ApplicationReport> getApplications(Set<String> applicationTypes, EnumSet<org.apache.hadoop.yarn.api.records.YarnApplicationState> applicationStates) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a report (ApplicationReport) of Applications matching the given application types and application states in the cluster.

      If the user does not have VIEW_APP access for an application then the corresponding report will be filtered as described in YarnClient.getApplicationReport(ApplicationId).

      Specified by:
      getApplications in class YarnClient
      Parameters:
      applicationTypes - set of application types you are interested in
      applicationStates - set of application states you are interested in
      Returns:
      a list of reports of applications
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getApplications

      public List<org.apache.hadoop.yarn.api.records.ApplicationReport> getApplications(Set<String> applicationTypes, EnumSet<org.apache.hadoop.yarn.api.records.YarnApplicationState> applicationStates, Set<String> applicationTags) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a report (ApplicationReport) of Applications matching the given application types, application states and application tags in the cluster.

      If the user does not have VIEW_APP access for an application then the corresponding report will be filtered as described in YarnClient.getApplicationReport(ApplicationId).

      Specified by:
      getApplications in class YarnClient
      Parameters:
      applicationTypes - set of application types you are interested in
      applicationStates - set of application states you are interested in
      applicationTags - set of application tags you are interested in
      Returns:
      a list of reports of applications
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getApplications

      public List<org.apache.hadoop.yarn.api.records.ApplicationReport> getApplications(Set<String> queues, Set<String> users, Set<String> applicationTypes, EnumSet<org.apache.hadoop.yarn.api.records.YarnApplicationState> applicationStates) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a report (ApplicationReport) of Applications matching the given users, queues, application types and application states in the cluster. If any of the params is set to null, it is not used when filtering.

      If the user does not have VIEW_APP access for an application then the corresponding report will be filtered as described in YarnClient.getApplicationReport(ApplicationId).

      Specified by:
      getApplications in class YarnClient
      Parameters:
      queues - set of queues you are interested in
      users - set of users you are interested in
      applicationTypes - set of application types you are interested in
      applicationStates - set of application states you are interested in
      Returns:
      a list of reports of applications
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getApplications

      public List<org.apache.hadoop.yarn.api.records.ApplicationReport> getApplications(org.apache.hadoop.yarn.api.protocolrecords.GetApplicationsRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a list of ApplicationReports that match the given GetApplicationsRequest.

      If the user does not have VIEW_APP access for an application then the corresponding report will be filtered as described in YarnClient.getApplicationReport(ApplicationId).

      Overrides:
      getApplications in class YarnClient
      Parameters:
      request - the request object to get the list of applications.
      Returns:
      The list of ApplicationReports that match the request
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - Exception specific to YARN.
      IOException - Exception mostly related to connection errors.

    • getYarnClusterMetrics

      public org.apache.hadoop.yarn.api.records.YarnClusterMetrics getYarnClusterMetrics() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get metrics (YarnClusterMetrics) about the cluster.

      Specified by:
      getYarnClusterMetrics in class YarnClient
      Returns:
      cluster metrics
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getNodeReports

      public List<org.apache.hadoop.yarn.api.records.NodeReport> getNodeReports(org.apache.hadoop.yarn.api.records.NodeState... states) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a report of nodes (NodeReport) in the cluster.

      Specified by:
      getNodeReports in class YarnClient
      Parameters:
      states - The NodeStates to filter on. If no filter states are given, nodes in all states will be returned.
      Returns:
      A list of node reports
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getRMDelegationToken

      public org.apache.hadoop.yarn.api.records.Token getRMDelegationToken(org.apache.hadoop.io.Text renewer) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a delegation token so as to be able to talk to YARN using those tokens.

      Specified by:
      getRMDelegationToken in class YarnClient
      Parameters:
      renewer - Address of the renewer who can renew these tokens when needed by securely talking to YARN.
      Returns:
      a delegation token (Token) that can be used to talk to YARN
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getQueueInfo

      public org.apache.hadoop.yarn.api.records.QueueInfo getQueueInfo(String queueName) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get information (QueueInfo) about a given queue.

      Specified by:
      getQueueInfo in class YarnClient
      Parameters:
      queueName - Name of the queue whose information is needed
      Returns:
      queue information
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - in case of errors or if YARN rejects the request due to access-control restrictions.
      IOException - I/O exception has occurred.

    • getQueueInfo

      public org.apache.hadoop.yarn.api.records.QueueInfo getQueueInfo(String queueName, String subClusterId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get information (QueueInfo) about a given queue.

      Specified by:
      getQueueInfo in class YarnClient
      Parameters:
      queueName - Name of the queue whose information is needed.
      subClusterId - sub-cluster Id.
      Returns:
      queue information.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - in case of errors or if YARN rejects the request due to access-control restrictions.
      IOException - I/O exception has occurred.

    • getQueueAclsInfo

      public List<org.apache.hadoop.yarn.api.records.QueueUserACLInfo> getQueueAclsInfo() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get information about acls for current user on all the existing queues.

      Specified by:
      getQueueAclsInfo in class YarnClient
      Returns:
      a list of queue acls (QueueUserACLInfo) for current user
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getAllQueues

      public List<org.apache.hadoop.yarn.api.records.QueueInfo> getAllQueues() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get information (QueueInfo) about all queues, recursively if there is a hierarchy

      Specified by:
      getAllQueues in class YarnClient
      Returns:
      a list of queue-information for all queues
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getRootQueueInfos

      public List<org.apache.hadoop.yarn.api.records.QueueInfo> getRootQueueInfos() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get information (QueueInfo) about top level queues.

      Specified by:
      getRootQueueInfos in class YarnClient
      Returns:
      a list of queue-information for all the top-level queues
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getChildQueueInfos

      public List<org.apache.hadoop.yarn.api.records.QueueInfo> getChildQueueInfos(String parent) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get information (QueueInfo) about all the immediate children queues of the given queue

      Specified by:
      getChildQueueInfos in class YarnClient
      Parameters:
      parent - Name of the queue whose child-queues' information is needed
      Returns:
      a list of queue-information for all queues who are direct children of the given parent queue.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • setRMClient

      @Private @VisibleForTesting public void setRMClient(org.apache.hadoop.yarn.api.ApplicationClientProtocol rmClient)

    • getApplicationAttemptReport

      public org.apache.hadoop.yarn.api.records.ApplicationAttemptReport getApplicationAttemptReport(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a report of the given ApplicationAttempt.

      In secure mode, YARN verifies access to the application, queue etc. before accepting the request.

      Specified by:
      getApplicationAttemptReport in class YarnClient
      Parameters:
      appAttemptId - ApplicationAttemptId of the application attempt that needs a report
      Returns:
      application attempt report
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      org.apache.hadoop.yarn.exceptions.ApplicationAttemptNotFoundException - if application attempt not found
      IOException

    • getApplicationAttempts

      public List<org.apache.hadoop.yarn.api.records.ApplicationAttemptReport> getApplicationAttempts(org.apache.hadoop.yarn.api.records.ApplicationId appId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a report of all (ApplicationAttempts) of Application in the cluster.

      Specified by:
      getApplicationAttempts in class YarnClient
      Parameters:
      appId - application id of the app
      Returns:
      a list of reports for all application attempts for specified application.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getContainerReport

      public org.apache.hadoop.yarn.api.records.ContainerReport getContainerReport(org.apache.hadoop.yarn.api.records.ContainerId containerId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a report of the given Container.

      In secure mode, YARN verifies access to the application, queue etc. before accepting the request.

      Specified by:
      getContainerReport in class YarnClient
      Parameters:
      containerId - ContainerId of the container that needs a report
      Returns:
      container report
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      org.apache.hadoop.yarn.exceptions.ContainerNotFoundException - if container not found.
      IOException

    • getContainers

      public List<org.apache.hadoop.yarn.api.records.ContainerReport> getContainers(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get a report of all (Containers) of ApplicationAttempt in the cluster.

      Specified by:
      getContainers in class YarnClient
      Parameters:
      applicationAttemptId - application attempt id
      Returns:
      a list of reports of all containers for specified application attempts
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • moveApplicationAcrossQueues

      public void moveApplicationAcrossQueues(org.apache.hadoop.yarn.api.records.ApplicationId appId, String queue) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Attempts to move the given application to the given queue.

      Specified by:
      moveApplicationAcrossQueues in class YarnClient
      Parameters:
      appId - Application to move.
      queue - Queue to place it in to.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • createReservation

      public org.apache.hadoop.yarn.api.protocolrecords.GetNewReservationResponse createReservation() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Obtain a GetNewReservationResponse for a new reservation, which contains the ReservationId object.

      Specified by:
      createReservation in class YarnClient
      Returns:
      The GetNewReservationResponse containing a new ReservationId object.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if reservation cannot be created.
      IOException - if reservation cannot be created.

    • submitReservation

      public org.apache.hadoop.yarn.api.protocolrecords.ReservationSubmissionResponse submitReservation(org.apache.hadoop.yarn.api.protocolrecords.ReservationSubmissionRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      The interface used by clients to submit a new reservation to the ResourceManager.

      The client packages all details of its request in a ReservationSubmissionRequest object. This contains information about the amount of capacity, temporal constraints, and gang needs. Furthermore, the reservation might be composed of multiple stages, with ordering dependencies among them.

      In order to respond, a new admission control component in the ResourceManager performs an analysis of the resources that have been committed over the period of time the user is requesting, verify that the user requests can be fulfilled, and that it respect a sharing policy (e.g., CapacityOverTimePolicy). Once it has positively determined that the ReservationRequest is satisfiable the ResourceManager answers with a ReservationSubmissionResponse that includes a ReservationId. Upon failure to find a valid allocation the response is an exception with the message detailing the reason of failure.

      The semantics guarantees that the ReservationId returned, corresponds to a valid reservation existing in the time-range request by the user. The amount of capacity dedicated to such reservation can vary overtime, depending of the allocation that has been determined. But it is guaranteed to satisfy all the constraint expressed by the user in the ReservationDefinition

      Specified by:
      submitReservation in class YarnClient
      Parameters:
      request - request to submit a new Reservation
      Returns:
      response contains the ReservationId on accepting the submission
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if the reservation cannot be created successfully
      IOException

    • updateReservation

      public org.apache.hadoop.yarn.api.protocolrecords.ReservationUpdateResponse updateReservation(org.apache.hadoop.yarn.api.protocolrecords.ReservationUpdateRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      The interface used by clients to update an existing Reservation. This is referred to as a re-negotiation process, in which a user that has previously submitted a Reservation.

      The allocation is attempted by virtually substituting all previous allocations related to this Reservation with new ones, that satisfy the new ReservationDefinition. Upon success the previous allocation is atomically substituted by the new one, and on failure (i.e., if the system cannot find a valid allocation for the updated request), the previous allocation remains valid.

      Specified by:
      updateReservation in class YarnClient
      Parameters:
      request - to update an existing Reservation (the ReservationUpdateRequest should refer to an existing valid ReservationId)
      Returns:
      response empty on successfully updating the existing reservation
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid or reservation cannot be updated successfully
      IOException

    • deleteReservation

      public org.apache.hadoop.yarn.api.protocolrecords.ReservationDeleteResponse deleteReservation(org.apache.hadoop.yarn.api.protocolrecords.ReservationDeleteRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      The interface used by clients to remove an existing Reservation.

      Specified by:
      deleteReservation in class YarnClient
      Parameters:
      request - to remove an existing Reservation (the ReservationDeleteRequest should refer to an existing valid ReservationId)
      Returns:
      response empty on successfully deleting the existing reservation
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid or reservation cannot be deleted successfully
      IOException

    • listReservations

      public org.apache.hadoop.yarn.api.protocolrecords.ReservationListResponse listReservations(org.apache.hadoop.yarn.api.protocolrecords.ReservationListRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      The interface used by clients to get the list of reservations in a plan. The reservationId will be used to search for reservations to list if it is provided. Otherwise, it will select active reservations within the startTime and endTime (inclusive).

      Specified by:
      listReservations in class YarnClient
      Parameters:
      request - to list reservations in a plan. Contains fields to select String queue, ReservationId reservationId, long startTime, long endTime, and a bool includeReservationAllocations. queue: Required. Cannot be null or empty. Refers to the reservable queue in the scheduler that was selected when creating a reservation submission ReservationSubmissionRequest. reservationId: Optional. If provided, other fields will be ignored. startTime: Optional. If provided, only reservations that end after the startTime will be selected. This defaults to 0 if an invalid number is used. endTime: Optional. If provided, only reservations that start on or before endTime will be selected. This defaults to Long.MAX_VALUE if an invalid number is used. includeReservationAllocations: Optional. Flag that determines whether the entire reservation allocations are to be returned. Reservation allocations are subject to change in the event of re-planning as described by ReservationDefinition.
      Returns:
      response that contains information about reservations that are being searched for.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid
      IOException - if the request failed otherwise

    • getNodeToLabels

      public Map<org.apache.hadoop.yarn.api.records.NodeId,Set<String>> getNodeToLabels() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      The interface used by client to get node to labels mappings in existing cluster

      Specified by:
      getNodeToLabels in class YarnClient
      Returns:
      node to labels mappings
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getLabelsToNodes

      public Map<String,Set<org.apache.hadoop.yarn.api.records.NodeId>> getLabelsToNodes() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      The interface used by client to get labels to nodes mapping in existing cluster

      Specified by:
      getLabelsToNodes in class YarnClient
      Returns:
      node to labels mappings
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getLabelsToNodes

      public Map<String,Set<org.apache.hadoop.yarn.api.records.NodeId>> getLabelsToNodes(Set<String> labels) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      The interface used by client to get labels to nodes mapping for specified labels in existing cluster

      Specified by:
      getLabelsToNodes in class YarnClient
      Parameters:
      labels - labels for which labels to nodes mapping has to be retrieved
      Returns:
      labels to nodes mappings for specific labels
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getClusterNodeLabels

      public List<org.apache.hadoop.yarn.api.records.NodeLabel> getClusterNodeLabels() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      The interface used by client to get node labels in the cluster

      Specified by:
      getClusterNodeLabels in class YarnClient
      Returns:
      cluster node labels collection
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - when there is a failure in ApplicationClientProtocol
      IOException - when there is a failure in ApplicationClientProtocol

    • updateApplicationPriority

      public org.apache.hadoop.yarn.api.records.Priority updateApplicationPriority(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, org.apache.hadoop.yarn.api.records.Priority priority) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      The interface used by client to set priority of an application

      Specified by:
      updateApplicationPriority in class YarnClient
      Returns:
      updated priority of an application.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • signalToContainer

      public void signalToContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.SignalContainerCommand command) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Signal a container identified by given ID.

      Specified by:
      signalToContainer in class YarnClient
      Parameters:
      containerId - ContainerId of the container that needs to be signaled
      command - the signal container command
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • updateApplicationTimeouts

      public org.apache.hadoop.yarn.api.protocolrecords.UpdateApplicationTimeoutsResponse updateApplicationTimeouts(org.apache.hadoop.yarn.api.protocolrecords.UpdateApplicationTimeoutsRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Overrides:
      updateApplicationTimeouts in class YarnClient
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getResourceProfiles

      public Map<String,org.apache.hadoop.yarn.api.records.Resource> getResourceProfiles() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get the resource profiles available in the RM.

      Specified by:
      getResourceProfiles in class YarnClient
      Returns:
      a Map of the resource profile names to their capabilities
      Throws:
      org.apache.hadoop.yarn.exceptions.YARNFeatureNotEnabledException - if resource-profile is disabled
      org.apache.hadoop.yarn.exceptions.YarnException - if any error happens inside YARN
      IOException - in case of other errors

    • getResourceProfile

      public org.apache.hadoop.yarn.api.records.Resource getResourceProfile(String profile) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get the details of a specific resource profile from the RM.

      Specified by:
      getResourceProfile in class YarnClient
      Parameters:
      profile - the profile name
      Returns:
      resource profile name with its capabilities
      Throws:
      org.apache.hadoop.yarn.exceptions.YARNFeatureNotEnabledException - if resource-profile is disabled
      org.apache.hadoop.yarn.exceptions.YarnException - if any error happens inside YARN
      IOException - in case of other others

    • getResourceTypeInfo

      public List<org.apache.hadoop.yarn.api.records.ResourceTypeInfo> getResourceTypeInfo() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      Get available resource types supported by RM.

      Specified by:
      getResourceTypeInfo in class YarnClient
      Returns:
      list of supported resource types with detailed information
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if any issue happens inside YARN
      IOException - in case of other others

    • getClusterAttributes

      public Set<org.apache.hadoop.yarn.api.records.NodeAttributeInfo> getClusterAttributes() throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      The interface used by client to get node attributes in the cluster.

      Specified by:
      getClusterAttributes in class YarnClient
      Returns:
      cluster node attributes collection
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - when there is a failure in ApplicationClientProtocol
      IOException - when there is a failure in ApplicationClientProtocol

    • getAttributesToNodes

      public Map<org.apache.hadoop.yarn.api.records.NodeAttributeKey,List<org.apache.hadoop.yarn.api.records.NodeToAttributeValue>> getAttributesToNodes(Set<org.apache.hadoop.yarn.api.records.NodeAttributeKey> attributes) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      The interface used by client to get mapping of AttributeKey to associated NodeToAttributeValue list for specified node attributeKeys in the cluster.

      Specified by:
      getAttributesToNodes in class YarnClient
      Parameters:
      attributes - AttributeKeys for which associated NodeToAttributeValue mapping value has to be retrieved. If empty or null is set then will return mapping for all attributeKeys in the cluster
      Returns:
      mapping of AttributeKey to List of associated NodeToAttributeValue's.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • getNodeToAttributes

      public Map<String,Set<org.apache.hadoop.yarn.api.records.NodeAttribute>> getNodeToAttributes(Set<String> hostNames) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
      Description copied from class: YarnClient

      The interface used by client to get all node to attribute mapping in existing cluster.

      Specified by:
      getNodeToAttributes in class YarnClient
      Parameters:
      hostNames - HostNames for which host to attributes mapping has to be retrived.If empty or null is set then will return all nodes to attributes mapping in cluster.
      Returns:
      Node to attribute mappings
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException
      IOException

    • shellToContainer

      public void shellToContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.ShellContainerCommand command) throws IOException
      Description copied from class: YarnClient

      The interface used by client to get a shell to a container.

      Specified by:
      shellToContainer in class YarnClient
      Parameters:
      containerId - Container ID
      command - Shell type
      Throws:
      IOException - if connection fails.