org.apache.hadoop.yarn.server.resourcemanager

Class RMServerUtils

java.lang.Object

org.apache.hadoop.yarn.server.resourcemanager.RMServerUtils

public class RMServerUtils
extends Object

Utility methods to aid serving RM data through the REST and RPC APIs

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.hadoop.yarn.api.records.ApplicationResourceUsageReport	DUMMY_APPLICATION_RESOURCE_USAGE_REPORT Statically defined dummy ApplicationResourceUsageREport.
protected static org.apache.hadoop.yarn.factories.RecordFactory	RECORD_FACTORY

Constructor Summary

Constructors

Constructor and Description
RMServerUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static void	checkSchedContainerChangeRequest(SchedContainerChangeRequest request, boolean increase) Validate increase/decrease request.
static org.apache.hadoop.yarn.api.records.YarnApplicationAttemptState	createApplicationAttemptState(RMAppAttemptState rmAppAttemptState)
static org.apache.hadoop.yarn.api.records.YarnApplicationState	createApplicationState(RMAppState rmAppState)
static void	normalizeAndValidateRequests(List<org.apache.hadoop.yarn.api.records.ResourceRequest> ask, org.apache.hadoop.yarn.api.records.Resource maximumResource, String queueName, YarnScheduler scheduler, RMContext rmContext) Utility method to validate a list resource requests, by insuring that the requested memory/vcore is non-negative and not greater than max
static void	processRMProxyUsersConf(org.apache.hadoop.conf.Configuration conf) Find all configs whose name starts with YarnConfiguration.RM_PROXY_USER_PREFIX, and add a record for each one by replacing the prefix with ProxyUsers.CONF_HADOOP_PROXYUSER
static List<RMNode>	queryRMNodes(RMContext context, EnumSet<org.apache.hadoop.yarn.api.records.NodeState> acceptedStates)
static List<org.apache.hadoop.yarn.api.records.UpdateContainerError>	validateAndSplitUpdateResourceRequests(RMContext rmContext, org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request, org.apache.hadoop.yarn.api.records.Resource maximumAllocation, List<org.apache.hadoop.yarn.api.records.UpdateContainerRequest> increaseResourceReqs, List<org.apache.hadoop.yarn.api.records.UpdateContainerRequest> decreaseResourceReqs) Check if we have: - Request for same containerId and different target resource - If targetResources violates maximum/minimumAllocation
static void	validateBlacklistRequest(org.apache.hadoop.yarn.api.records.ResourceBlacklistRequest blacklistRequest)
static void	validateContainerReleaseRequest(List<org.apache.hadoop.yarn.api.records.ContainerId> containerReleaseList, org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId) It will validate to make sure all the containers belong to correct application attempt id.
static org.apache.hadoop.security.UserGroupInformation	verifyAdminAccess(org.apache.hadoop.yarn.security.YarnAuthorizationProvider authorizer, String method, org.apache.commons.logging.Log LOG)
static org.apache.hadoop.security.UserGroupInformation	verifyAdminAccess(org.apache.hadoop.yarn.security.YarnAuthorizationProvider authorizer, String method, String module, org.apache.commons.logging.Log LOG) Utility method to verify if the current user has access based on the passed AccessControlList

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

RECORD_FACTORY

protected static final org.apache.hadoop.yarn.factories.RecordFactory RECORD_FACTORY

DUMMY_APPLICATION_RESOURCE_USAGE_REPORT

public static final org.apache.hadoop.yarn.api.records.ApplicationResourceUsageReport DUMMY_APPLICATION_RESOURCE_USAGE_REPORT

Statically defined dummy ApplicationResourceUsageREport. Used as a return value when a valid report cannot be found.

Constructor Detail

RMServerUtils

public RMServerUtils()

Method Detail

queryRMNodes

public static List<RMNode> queryRMNodes(RMContext context,
                        EnumSet<org.apache.hadoop.yarn.api.records.NodeState> acceptedStates)

validateAndSplitUpdateResourceRequests

public static List<org.apache.hadoop.yarn.api.records.UpdateContainerError> validateAndSplitUpdateResourceRequests(RMContext rmContext,
                                                                                                   org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request,
                                                                                                   org.apache.hadoop.yarn.api.records.Resource maximumAllocation,
                                                                                                   List<org.apache.hadoop.yarn.api.records.UpdateContainerRequest> increaseResourceReqs,
                                                                                                   List<org.apache.hadoop.yarn.api.records.UpdateContainerRequest> decreaseResourceReqs)

Check if we have: - Request for same containerId and different target resource - If targetResources violates maximum/minimumAllocation

Parameters:

rmContext - RM context

request - Allocate Request

maximumAllocation - Maximum Allocation

increaseResourceReqs - Increase Resource Request

decreaseResourceReqs - Decrease Resource Request

Returns:

List of container Errors

normalizeAndValidateRequests

public static void normalizeAndValidateRequests(List<org.apache.hadoop.yarn.api.records.ResourceRequest> ask,
                                org.apache.hadoop.yarn.api.records.Resource maximumResource,
                                String queueName,
                                YarnScheduler scheduler,
                                RMContext rmContext)
                                         throws org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException

Utility method to validate a list resource requests, by insuring that the requested memory/vcore is non-negative and not greater than max

Throws:

org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException

checkSchedContainerChangeRequest

public static void checkSchedContainerChangeRequest(SchedContainerChangeRequest request,
                                    boolean increase)
                                             throws org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException

Validate increase/decrease request. This function must be called under the queue lock to make sure that the access to container resource is atomic. Refer to LeafQueue.decreaseContainer() and CapacityScheduelr.updateIncreaseRequests()

 - Throw exception when any other error happens
 

Throws:

org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException

validateBlacklistRequest

public static void validateBlacklistRequest(org.apache.hadoop.yarn.api.records.ResourceBlacklistRequest blacklistRequest)
                                     throws org.apache.hadoop.yarn.exceptions.InvalidResourceBlacklistRequestException

Throws:

org.apache.hadoop.yarn.exceptions.InvalidResourceBlacklistRequestException

validateContainerReleaseRequest

public static void validateContainerReleaseRequest(List<org.apache.hadoop.yarn.api.records.ContainerId> containerReleaseList,
                                   org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)
                                            throws org.apache.hadoop.yarn.exceptions.InvalidContainerReleaseException

It will validate to make sure all the containers belong to correct application attempt id. If not then it will throw InvalidContainerReleaseException

Parameters:

containerReleaseList - containers to be released as requested by application master.

appAttemptId - Application attempt Id

Throws:

org.apache.hadoop.yarn.exceptions.InvalidContainerReleaseException

verifyAdminAccess

public static org.apache.hadoop.security.UserGroupInformation verifyAdminAccess(org.apache.hadoop.yarn.security.YarnAuthorizationProvider authorizer,
                                                                String method,
                                                                org.apache.commons.logging.Log LOG)
                                                                         throws IOException

Throws:

IOException

verifyAdminAccess

public static org.apache.hadoop.security.UserGroupInformation verifyAdminAccess(org.apache.hadoop.yarn.security.YarnAuthorizationProvider authorizer,
                                                                String method,
                                                                String module,
                                                                org.apache.commons.logging.Log LOG)
                                                                         throws IOException

Utility method to verify if the current user has access based on the passed AccessControlList

Parameters:

authorizer - the AccessControlList to check against

method - the method name to be logged

module - like AdminService or NodeLabelManager

LOG - the logger to use

Returns:

UserGroupInformation of the current user

Throws:

IOException

createApplicationState

public static org.apache.hadoop.yarn.api.records.YarnApplicationState createApplicationState(RMAppState rmAppState)

createApplicationAttemptState

public static org.apache.hadoop.yarn.api.records.YarnApplicationAttemptState createApplicationAttemptState(RMAppAttemptState rmAppAttemptState)

processRMProxyUsersConf

public static void processRMProxyUsersConf(org.apache.hadoop.conf.Configuration conf)

Find all configs whose name starts with YarnConfiguration.RM_PROXY_USER_PREFIX, and add a record for each one by replacing the prefix with ProxyUsers.CONF_HADOOP_PROXYUSER