Package org.apache.hadoop.yarn.util.resource

Class ResourceUtils

java.lang.Object
org.apache.hadoop.yarn.util.resource.ResourceUtils
public class ResourceUtils extends Object
Helper class to read the resource-types to be supported by the system.

  • Field Details

  • Method Details

    • fetchMaximumAllocationFromConfig

      public static Resource fetchMaximumAllocationFromConfig(org.apache.hadoop.conf.Configuration conf)
      Get maximum allocation from config, *THIS WILL NOT UPDATE INTERNAL DATA.
      Parameters:
      conf - config
      Returns:
      maximum allocation

    • initializeResourcesFromResourceInformationMap

      @VisibleForTesting public static void initializeResourcesFromResourceInformationMap(Map<String,ResourceInformation> resourceInformationMap)
      This method is visible for testing, unit test can construct a resourceInformationMap and pass it to this method to initialize multiple resources.
      Parameters:
      resourceInformationMap - constructed resource information map.

    • getResourceTypeIndex

      public static Map<String,Integer> getResourceTypeIndex()
      Get associate index of resource types such memory, cpu etc. This could help to access each resource types in a resource faster.
      Returns:
      Index map for all Resource Types.

    • getResourceTypes

      public static Map<String,ResourceInformation> getResourceTypes()
      Get the resource types to be supported by the system.
      Returns:
      A map of the resource name to a ResourceInformation object which contains details such as the unit.

    • getResourceTypesArray

      public static ResourceInformation[] getResourceTypesArray()

    • getNumberOfKnownResourceTypes

      public static int getNumberOfKnownResourceTypes()

    • getNumberOfCountableResourceTypes

      public static int getNumberOfCountableResourceTypes()

    • resetResourceTypes

      @VisibleForTesting public static void resetResourceTypes()

    • resetResourceTypes

      @VisibleForTesting public static Map<String,ResourceInformation> resetResourceTypes(org.apache.hadoop.conf.Configuration conf)

    • getUnits

      public static String getUnits(String resourceValue)

    • parseResourceValue

      public static String[] parseResourceValue(String resourceValue)
      Extract unit and actual value from resource value.
      Parameters:
      resourceValue - Value of the resource
      Returns:
      Array containing unit and value. [0]=unit, [1]=value
      Throws:
      IllegalArgumentException - if units contain non alpha characters

    • getValue

      public static long getValue(String resourceValue)

    • getNodeResourceInformation

      public static Map<String,ResourceInformation> getNodeResourceInformation(org.apache.hadoop.conf.Configuration conf)
      Function to get the resources for a node. This function will look at the file YarnConfiguration.NODE_RESOURCES_CONFIGURATION_FILE to determine the node resources.
      Parameters:
      conf - configuration file
      Returns:
      a map to resource name to the ResourceInformation object. The map is guaranteed to have entries for memory and vcores

    • resetNodeResources

      @VisibleForTesting public static void resetNodeResources()

    • getResourceTypesMinimumAllocation

      public static Resource getResourceTypesMinimumAllocation()

    • getResourceTypesMaximumAllocation

      public static Resource getResourceTypesMaximumAllocation()
      Get a Resource object with for the maximum allocation possible.
      Returns:
      a Resource object with the maximum allocation for the scheduler

    • getDefaultUnit

      public static String getDefaultUnit(String resourceType)
      Get default unit by given resource type.
      Parameters:
      resourceType - resourceType
      Returns:
      default unit

    • getResourcesTypeInfo

      public static List<ResourceTypeInfo> getResourcesTypeInfo()
      Get all resource types information from known resource types.
      Returns:
      List of ResourceTypeInfo

    • reinitializeResources

      public static void reinitializeResources(List<ResourceTypeInfo> resourceTypeInfo)
      Reinitialize all resource types from external source (in case of client, server will send the updated list and local resourceutils cache will be updated as per server's list of resources).
      Parameters:
      resourceTypeInfo - List of resource types

    • getRequestedResourcesFromConfig

      public static List<ResourceInformation> getRequestedResourcesFromConfig(org.apache.hadoop.conf.Configuration configuration, String prefix)
      From a given configuration get all entries representing requested resources: entries that match the {prefix}{resourceName}={value}[{units}] pattern.
      Parameters:
      configuration - The configuration
      prefix - Keys with this prefix are considered from the configuration
      Returns:
      The list of requested resources as described by the configuration

    • areMandatoryResourcesAvailable

      public static void areMandatoryResourcesAvailable(Resource res)
      Are mandatory resources like memory-mb, vcores available? If not, throw exceptions. On availability, ensure those values are within boundary.
      Parameters:
      res - resource
      Throws:
      IllegalArgumentException - if mandatory resource is not available or value is not within boundary

    • createResourceTypesArray

      public static ResourceInformation[] createResourceTypesArray(Map<String,Long> res)
      Create an array of ResourceInformation objects corresponding to the passed in map of names to values. The array will be ordered according to the order returned by getResourceTypesArray(). The value of each resource type in the returned array will either be the value given for that resource in the res parameter or, if none is given, 0.
      Parameters:
      res - the map of resource type values
      Returns:
      an array of ResourceInformation instances

    • createResourceWithSameValue

      @Private @Unstable public static Resource createResourceWithSameValue(long value)
      Return a new Resource instance with all resource values initialized to value.
      Parameters:
      value - the value to use for all resources
      Returns:
      a new Resource instance

    • multiplyFloor

      public static Resource multiplyFloor(Resource resource, double multiplier)

    • multiplyRound

      public static Resource multiplyRound(Resource resource, double multiplier)

    • createResourceFromString

      @Private @Unstable public static Resource createResourceFromString(String resourceStr, List<ResourceTypeInfo> resourceTypeInfos)

    • getCustomResourcesStrings

      public static StringBuilder getCustomResourcesStrings(Resource resource)