org.apache.hadoop.yarn.api.records

Class Resource

java.lang.Object

org.apache.hadoop.yarn.api.records.Resource

All Implemented Interfaces:

Comparable<Resource>

@InterfaceAudience.Public
 @InterfaceStability.Stable
public abstract class Resource
extends Object
implements Comparable<Resource>

Resource models a set of computer resources in the cluster.

Currently it models both memory and CPU.

The unit for memory is megabytes. CPU is modeled with virtual cores (vcores), a unit for expressing parallelism. A node's capacity should be configured with virtual cores equal to its number of physical cores. A container should be requested with the number of cores it can saturate, i.e. the average number of threads it expects to have runnable at a time.

Virtual cores take integer values and thus currently CPU-scheduling is very coarse. A complementary axis for CPU requests that represents processing power will likely be added in the future to enable finer-grained resource configuration.

Typically, applications request Resource of suitable capability to run their component tasks.

See Also:

ResourceRequest, ApplicationMasterProtocol.allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest)

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.yarn.api.records.ResourceInformation[]	resources

Constructor Summary

Constructors

Constructor and Description
Resource()

Method Summary

Modifier and Type	Method and Description
protected static int	castToIntSafely(long value) Convert long to int for a resource value safely.
int	compareTo(Resource other)
boolean	equals(Object obj)
String	getFormattedString() This method is to get memory in terms of KB|MB|GB.
String	getFormattedString(long memory)
abstract int	getMemory() Deprecated.
long	getMemorySize() Get memory of the resource.
org.apache.hadoop.yarn.api.records.ResourceInformation	getResourceInformation(String resource) Get ResourceInformation for a specified resource.
long	getResourceValue(String resource) Get the value for a specified resource.
abstract int	getVirtualCores() Get number of virtual cpu cores of the resource.
int	hashCode()
protected static org.apache.hadoop.yarn.api.records.ResourceInformation	newDefaultInformation(String name, String unit, long value) Create ResourceInformation with basic fields.
static Resource	newInstance(int memory, int vCores)
static Resource	newInstance(long memory, int vCores)
static Resource	newInstance(long memory, int vCores, Map<String,Long> others) Create a new Resource instance with the given CPU and memory values and additional resource values as set in the others parameter.
abstract void	setMemory(int memory) Deprecated.
void	setMemorySize(long memory) Set memory of the resource.
void	setResourceInformation(String resource, org.apache.hadoop.yarn.api.records.ResourceInformation resourceInformation) Set the ResourceInformation object for a particular resource.
protected void	setResources(org.apache.hadoop.yarn.api.records.ResourceInformation[] resources)
void	setResourceValue(String resource, long value) Set the value of a resource in the ResourceInformation object.
abstract void	setVirtualCores(int vCores) Set number of virtual cpu cores of the resource.
protected void	throwExceptionWhenArrayOutOfBound(int index)
String	toFormattedString()
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

resources

protected org.apache.hadoop.yarn.api.records.ResourceInformation[] resources

Constructor Detail

Resource

public Resource()

Method Detail

newInstance

@InterfaceAudience.Public
 @InterfaceStability.Stable
public static Resource newInstance(int memory,
                                                                                         int vCores)

newInstance

@InterfaceAudience.Public
 @InterfaceStability.Stable
public static Resource newInstance(long memory,
                                                                                         int vCores)

newInstance

@InterfaceAudience.Public
 @InterfaceStability.Stable
public static Resource newInstance(long memory,
                                                                                         int vCores,
                                                                                         Map<String,Long> others)

Create a new Resource instance with the given CPU and memory values and additional resource values as set in the others parameter. Note that the CPU and memory settings in the others parameter will be ignored.

Parameters:

memory - the memory value

vCores - the CPU value

others - a map of other resource values indexed by resource name

Returns:

a Resource instance with the given resource values

getMemory

@InterfaceAudience.Public
 @Deprecated
public abstract int getMemory()

Deprecated.

This method is DEPRECATED: Use getMemorySize() instead Get memory of the resource. Note - while memory has never had a unit specified, all YARN configurations have specified memory in MB. The assumption has been that the daemons and applications are always using the same units. With the introduction of the ResourceInformation class we have support for units - so this function will continue to return memory but in the units of MB

Returns:

memory(in MB) of the resource

getMemorySize

@InterfaceAudience.Public
 @InterfaceStability.Stable
public long getMemorySize()

Get memory of the resource. Note - while memory has never had a unit specified, all YARN configurations have specified memory in MB. The assumption has been that the daemons and applications are always using the same units. With the introduction of the ResourceInformation class we have support for units - so this function will continue to return memory but in the units of MB

Returns:

memory of the resource

setMemory

@InterfaceAudience.Public
 @Deprecated
public abstract void setMemory(int memory)

Deprecated.

Set memory of the resource. Note - while memory has never had a unit specified, all YARN configurations have specified memory in MB. The assumption has been that the daemons and applications are always using the same units. With the introduction of the ResourceInformation class we have support for units - so this function will continue to set memory but the assumption is that the value passed is in units of MB.

Parameters:

memory - memory(in MB) of the resource

setMemorySize

@InterfaceAudience.Public
 @InterfaceStability.Stable
public void setMemorySize(long memory)

Set memory of the resource.

Parameters:

memory - memory of the resource

getVirtualCores

@InterfaceAudience.Public
 @InterfaceStability.Evolving
public abstract int getVirtualCores()

Get number of virtual cpu cores of the resource. Virtual cores are a unit for expressing CPU parallelism. A node's capacity should be configured with virtual cores equal to its number of physical cores. A container should be requested with the number of cores it can saturate, i.e. the average number of threads it expects to have runnable at a time.

Returns:

num of virtual cpu cores of the resource

setVirtualCores

@InterfaceAudience.Public
 @InterfaceStability.Evolving
public abstract void setVirtualCores(int vCores)

Set number of virtual cpu cores of the resource. Virtual cores are a unit for expressing CPU parallelism. A node's capacity should be configured with virtual cores equal to its number of physical cores. A container should be requested with the number of cores it can saturate, i.e. the average number of threads it expects to have runnable at a time.

Parameters:

vCores - number of virtual cpu cores of the resource

getResourceInformation

@InterfaceAudience.Public
 @InterfaceStability.Unstable
public org.apache.hadoop.yarn.api.records.ResourceInformation getResourceInformation(String resource)

Get ResourceInformation for a specified resource.

Parameters:

resource - name of the resource

Returns:

the ResourceInformation object for the resource

getResourceValue

@InterfaceAudience.Public
 @InterfaceStability.Unstable
public long getResourceValue(String resource)

Get the value for a specified resource. No information about the units is returned.

Parameters:

resource - name of the resource

Returns:

the value for the resource

setResourceInformation

@InterfaceAudience.Public
 @InterfaceStability.Unstable
public void setResourceInformation(String resource,
                                                                                           org.apache.hadoop.yarn.api.records.ResourceInformation resourceInformation)

Set the ResourceInformation object for a particular resource.

Parameters:

resource - the resource for which the ResourceInformation is provided

resourceInformation - ResourceInformation object

setResourceValue

@InterfaceAudience.Public
 @InterfaceStability.Unstable
public void setResourceValue(String resource,
                                                                                     long value)

Set the value of a resource in the ResourceInformation object. The unit of the value is assumed to be the one in the ResourceInformation object.

Parameters:

resource - the resource for which the value is provided.

value - the value to set

throwExceptionWhenArrayOutOfBound

protected void throwExceptionWhenArrayOutOfBound(int index)

equals

public boolean equals(Object obj)

Overrides:

equals in class Object

compareTo

public int compareTo(Resource other)

Specified by:

compareTo in interface Comparable<Resource>

toString

public String toString()

Overrides:

toString in class Object

toFormattedString

public String toFormattedString()

getFormattedString

public String getFormattedString()

This method is to get memory in terms of KB|MB|GB.

Returns:

string containing all resources

hashCode

public int hashCode()

Overrides:

hashCode in class Object

castToIntSafely

protected static int castToIntSafely(long value)

Convert long to int for a resource value safely. This method assumes resource value is positive.

Parameters:

value - long resource value

Returns:

int resource value

newDefaultInformation

protected static org.apache.hadoop.yarn.api.records.ResourceInformation newDefaultInformation(String name,
                                                                                              String unit,
                                                                                              long value)

Create ResourceInformation with basic fields.

Parameters:

name - Resource Type Name

unit - Default unit of provided resource type

value - Value associated with giveb resource

Returns:

ResourceInformation object

setResources

@VisibleForTesting
protected void setResources(org.apache.hadoop.yarn.api.records.ResourceInformation[] resources)

getFormattedString

public String getFormattedString(long memory)