org.apache.hadoop.yarn.util

Class ResourceCalculatorPlugin

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.yarn.util.ResourceCalculatorPlugin

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

Direct Known Subclasses:

LinuxResourceCalculatorPlugin, WindowsResourceCalculatorPlugin

@InterfaceAudience.LimitedPrivate(value={"YARN","MAPREDUCE"})
 @InterfaceStability.Unstable
public class ResourceCalculatorPlugin
extends org.apache.hadoop.conf.Configured

Plugin to calculate resource information on the system.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	ResourceCalculatorPlugin()
	ResourceCalculatorPlugin(org.apache.hadoop.util.SysInfo sys)

Method Summary

Modifier and Type	Method and Description
long	getAvailablePhysicalMemorySize() Obtain the total size of the available physical memory present in the system.
long	getAvailableVirtualMemorySize() Obtain the total size of the available virtual memory present in the system.
static ResourceCalculatorPlugin	getContainersMonitorPlugin(org.apache.hadoop.conf.Configuration conf) Create the ResourceCalculatorPlugin for the containers monitor in the Node Manager and configure it.
long	getCpuFrequency() Obtain the CPU frequency of on the system.
float	getCpuUsagePercentage() Obtain the CPU usage % of the machine.
long	getCumulativeCpuTime() Obtain the cumulative CPU time since the system is on.
long	getNetworkBytesRead() Obtain the aggregated number of bytes read over the network.
long	getNetworkBytesWritten() Obtain the aggregated number of bytes written to the network.
static ResourceCalculatorPlugin	getNodeResourceMonitorPlugin(org.apache.hadoop.conf.Configuration conf) Create the ResourceCalculatorPlugin for the node resource monitor in the Node Manager and configure it.
int	getNumCores() Obtain total number of physical cores present on the system.
int	getNumProcessors() Obtain the total number of logical processors present on the system.
float	getNumVCoresUsed() Obtain the number of VCores used.
long	getPhysicalMemorySize() Obtain the total size of the physical memory present in the system.
static ResourceCalculatorPlugin	getResourceCalculatorPlugin(Class<? extends ResourceCalculatorPlugin> clazz, org.apache.hadoop.conf.Configuration conf) Create the ResourceCalculatorPlugin from the class name and configure it.
long	getStorageBytesRead() Obtain the aggregated number of bytes read from disks.
long	getStorageBytesWritten() Obtain the aggregated number of bytes written to disks.
long	getVirtualMemorySize() Obtain the total size of the virtual memory present in the system.

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ResourceCalculatorPlugin

protected ResourceCalculatorPlugin()

ResourceCalculatorPlugin

public ResourceCalculatorPlugin(org.apache.hadoop.util.SysInfo sys)

Method Detail

getVirtualMemorySize

public long getVirtualMemorySize()

Obtain the total size of the virtual memory present in the system.

Returns:

virtual memory size in bytes.

getPhysicalMemorySize

public long getPhysicalMemorySize()

Obtain the total size of the physical memory present in the system.

Returns:

physical memory size bytes.

getAvailableVirtualMemorySize

public long getAvailableVirtualMemorySize()

Obtain the total size of the available virtual memory present in the system.

Returns:

available virtual memory size in bytes.

getAvailablePhysicalMemorySize

public long getAvailablePhysicalMemorySize()

Obtain the total size of the available physical memory present in the system.

Returns:

available physical memory size bytes.

getNumProcessors

public int getNumProcessors()

Obtain the total number of logical processors present on the system.

Returns:

number of logical processors

getNumCores

public int getNumCores()

Obtain total number of physical cores present on the system.

Returns:

number of physical cores

getCpuFrequency

public long getCpuFrequency()

Obtain the CPU frequency of on the system.

Returns:

CPU frequency in kHz

getCumulativeCpuTime

public long getCumulativeCpuTime()

Obtain the cumulative CPU time since the system is on.

Returns:

cumulative CPU time in milliseconds

getCpuUsagePercentage

public float getCpuUsagePercentage()

Obtain the CPU usage % of the machine. Return -1 if it is unavailable.

Returns:

CPU usage in %

getNumVCoresUsed

public float getNumVCoresUsed()

Obtain the number of VCores used. Return -1 if it is unavailable.

Returns:

Number of VCores used a percentage (from 0 to #VCores)

getNetworkBytesRead

public long getNetworkBytesRead()

Obtain the aggregated number of bytes read over the network.

Returns:

total number of bytes read.

getNetworkBytesWritten

public long getNetworkBytesWritten()

Obtain the aggregated number of bytes written to the network.

Returns:

total number of bytes written.

getStorageBytesRead

public long getStorageBytesRead()

Obtain the aggregated number of bytes read from disks.

Returns:

total number of bytes read.

getStorageBytesWritten

public long getStorageBytesWritten()

Obtain the aggregated number of bytes written to disks.

Returns:

total number of bytes written.

getResourceCalculatorPlugin

public static ResourceCalculatorPlugin getResourceCalculatorPlugin(Class<? extends ResourceCalculatorPlugin> clazz,
                                                                   org.apache.hadoop.conf.Configuration conf)

Create the ResourceCalculatorPlugin from the class name and configure it. If class name is null, this method will try and return a memory calculator plugin available for this system.

Parameters:

clazz - ResourceCalculator plugin class-name

conf - configure the plugin with this.

Returns:

ResourceCalculatorPlugin or null if ResourceCalculatorPlugin is not available for current system

getContainersMonitorPlugin

public static ResourceCalculatorPlugin getContainersMonitorPlugin(org.apache.hadoop.conf.Configuration conf)

Create the ResourceCalculatorPlugin for the containers monitor in the Node Manager and configure it. If the plugin is not configured, this method will try and return a memory calculator plugin available for this system.

Parameters:

conf - Configure the plugin with this.

Returns:

ResourceCalculatorPlugin or null if ResourceCalculatorPlugin is not available for current system.

getNodeResourceMonitorPlugin

public static ResourceCalculatorPlugin getNodeResourceMonitorPlugin(org.apache.hadoop.conf.Configuration conf)

Create the ResourceCalculatorPlugin for the node resource monitor in the Node Manager and configure it. If the plugin is not configured, this method will try and return a memory calculator plugin available for this system.

Parameters:

conf - Configure the plugin with this.

Returns:

ResourceCalculatorPlugin or null if ResourceCalculatorPlugin is not available for current system.