Package org.apache.hadoop.yarn.server.resourcemanager

Class RMServerUtils

java.lang.Object
org.apache.hadoop.yarn.server.resourcemanager.RMServerUtils
public class RMServerUtils extends Object
Utility methods to aid serving RM data through the REST and RPC APIs

  • Field Details

    • UPDATE_OUTSTANDING_ERROR

      public static final String UPDATE_OUTSTANDING_ERROR
      See Also:

    • RESOURCE_OUTSIDE_ALLOWED_RANGE

      public static final String RESOURCE_OUTSIDE_ALLOWED_RANGE
      See Also:

    • RECORD_FACTORY

      protected static final org.apache.hadoop.yarn.factories.RecordFactory RECORD_FACTORY

    • DUMMY_APPLICATION_RESOURCE_USAGE_REPORT

      public static final org.apache.hadoop.yarn.api.records.ApplicationResourceUsageReport DUMMY_APPLICATION_RESOURCE_USAGE_REPORT
      Statically defined dummy ApplicationResourceUsageREport. Used as a return value when a valid report cannot be found.

  • Constructor Details

    • RMServerUtils

      public RMServerUtils()

  • Method Details

    • queryRMNodes

      public static List<RMNode> queryRMNodes(RMContext context, EnumSet<org.apache.hadoop.yarn.api.records.NodeState> acceptedStates)

    • validateAndSplitUpdateResourceRequests

      public static ContainerUpdates validateAndSplitUpdateResourceRequests(RMContext rmContext, org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request, org.apache.hadoop.yarn.api.records.Resource maximumAllocation, List<org.apache.hadoop.yarn.api.records.UpdateContainerError> updateErrors)
      Check if we have: - Request for same containerId and different target resource. - If targetResources violates maximum/minimumAllocation.
      Parameters:
      rmContext - RM context.
      request - Allocate Request.
      maximumAllocation - Maximum Allocation.
      updateErrors - Container update errors.
      Returns:
      ContainerUpdateRequests.

    • normalizeAndValidateRequests

      public static void normalizeAndValidateRequests(List<org.apache.hadoop.yarn.api.records.ResourceRequest> ask, org.apache.hadoop.yarn.api.records.Resource maximumAllocation, String queueName, YarnScheduler scheduler, RMContext rmContext, boolean nodeLabelsEnabled) throws org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException
      Utility method to validate a list resource requests, by ensuring that the requested memory/vcore is non-negative and not greater than max.
      Parameters:
      ask - resource request.
      maximumAllocation - Maximum Allocation.
      queueName - queue name.
      scheduler - YarnScheduler.
      rmContext - RMContext.
      nodeLabelsEnabled - the node labels feature enabled.
      Throws:
      org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException - when there is invalid request.

    • checkSchedContainerChangeRequest

      public static void checkSchedContainerChangeRequest(SchedContainerChangeRequest request, boolean increase) throws org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException
      Validate increase/decrease request. 
       - Throw exception when any other error happens
      Parameters:
      request - SchedContainerChangeRequest.
      increase - true, add container; false, decrease container.
      Throws:
      org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException - when there is invalid request.

    • validateBlacklistRequest

      public static void validateBlacklistRequest(org.apache.hadoop.yarn.api.records.ResourceBlacklistRequest blacklistRequest) throws org.apache.hadoop.yarn.exceptions.InvalidResourceBlacklistRequestException
      Throws:
      org.apache.hadoop.yarn.exceptions.InvalidResourceBlacklistRequestException

    • validateContainerReleaseRequest

      public static void validateContainerReleaseRequest(List<org.apache.hadoop.yarn.api.records.ContainerId> containerReleaseList, org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId) throws org.apache.hadoop.yarn.exceptions.InvalidContainerReleaseException
      It will validate to make sure all the containers belong to correct application attempt id. If not then it will throw InvalidContainerReleaseException
      Parameters:
      containerReleaseList - containers to be released as requested by application master.
      appAttemptId - Application attempt Id
      Throws:
      org.apache.hadoop.yarn.exceptions.InvalidContainerReleaseException - an Application Master tries to release containers not belonging to it using.

    • verifyAdminAccess

      public static org.apache.hadoop.security.UserGroupInformation verifyAdminAccess(org.apache.hadoop.yarn.security.YarnAuthorizationProvider authorizer, String method, org.slf4j.Logger LOG) throws IOException
      Throws:
      IOException

    • verifyAdminAccess

      public static org.apache.hadoop.security.UserGroupInformation verifyAdminAccess(org.apache.hadoop.yarn.security.YarnAuthorizationProvider authorizer, String method, String module, org.slf4j.Logger LOG) throws IOException
      Utility method to verify if the current user has access based on the passed AccessControlList
      Parameters:
      authorizer - the AccessControlList to check against
      method - the method name to be logged
      module - like AdminService or NodeLabelManager
      LOG - the logger to use
      Returns:
      UserGroupInformation of the current user
      Throws:
      IOException - an I/O exception has occurred.

    • createApplicationState

      public static org.apache.hadoop.yarn.api.records.YarnApplicationState createApplicationState(RMAppState rmAppState)

    • convertRmAppAttemptStateToYarnApplicationAttemptState

      public static org.apache.hadoop.yarn.api.records.YarnApplicationAttemptState convertRmAppAttemptStateToYarnApplicationAttemptState(RMAppAttemptState currentState, RMAppAttemptState previousState)

    • createApplicationAttemptState

      public static org.apache.hadoop.yarn.api.records.YarnApplicationAttemptState createApplicationAttemptState(RMAppAttemptState rmAppAttemptState)

    • processRMProxyUsersConf

      public static void processRMProxyUsersConf(org.apache.hadoop.conf.Configuration conf)
      Find all configs whose name starts with YarnConfiguration.RM_PROXY_USER_PREFIX, and add a record for each one by replacing the prefix with ProxyUsers.CONF_HADOOP_PROXYUSER.
      Parameters:
      conf - Configuration.

    • validateApplicationTimeouts

      public static void validateApplicationTimeouts(Map<org.apache.hadoop.yarn.api.records.ApplicationTimeoutType,Long> timeouts) throws org.apache.hadoop.yarn.exceptions.YarnException
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException

    • validateISO8601AndConvertToLocalTimeEpoch

      public static Map<org.apache.hadoop.yarn.api.records.ApplicationTimeoutType,Long> validateISO8601AndConvertToLocalTimeEpoch(Map<org.apache.hadoop.yarn.api.records.ApplicationTimeoutType,String> timeoutsInISO8601) throws org.apache.hadoop.yarn.exceptions.YarnException
      Validate ISO8601 format with epoch time.
      Parameters:
      timeoutsInISO8601 - format
      Returns:
      expire time in local epoch
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if given application timeout value is lesser than current time.

    • getApplicableNodeCountForAM

      public static int getApplicableNodeCountForAM(RMContext rmContext, org.apache.hadoop.conf.Configuration conf, List<org.apache.hadoop.yarn.api.records.ResourceRequest> amReqs)
      Get applicable Node count for AM.
      Parameters:
      rmContext - context
      conf - configuration
      amReqs - am resource requests
      Returns:
      applicable node count

    • getOrDefault

      public static Long getOrDefault(Map<String,Long> map, String key, Long defaultValue)