Package org.apache.hadoop.yarn.util

Class ResourceCalculatorPlugin

java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.hadoop.yarn.util.ResourceCalculatorPlugin
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable
Direct Known Subclasses:
LinuxResourceCalculatorPlugin, WindowsResourceCalculatorPlugin
@LimitedPrivate({"YARN","MAPREDUCE"}) @Unstable public class ResourceCalculatorPlugin extends org.apache.hadoop.conf.Configured
Plugin to calculate resource information on the system.

  • Constructor Details

    • ResourceCalculatorPlugin

      protected ResourceCalculatorPlugin()

    • ResourceCalculatorPlugin

      public ResourceCalculatorPlugin(org.apache.hadoop.util.SysInfo sys)

  • Method Details

    • getVirtualMemorySize

      public long getVirtualMemorySize()
      Obtain the total size of the virtual memory present in the system.
      Returns:
      virtual memory size in bytes.

    • getPhysicalMemorySize

      public long getPhysicalMemorySize()
      Obtain the total size of the physical memory present in the system.
      Returns:
      physical memory size bytes.

    • getAvailableVirtualMemorySize

      public long getAvailableVirtualMemorySize()
      Obtain the total size of the available virtual memory present in the system.
      Returns:
      available virtual memory size in bytes.

    • getAvailablePhysicalMemorySize

      public long getAvailablePhysicalMemorySize()
      Obtain the total size of the available physical memory present in the system.
      Returns:
      available physical memory size bytes.

    • getNumProcessors

      public int getNumProcessors()
      Obtain the total number of logical processors present on the system.
      Returns:
      number of logical processors

    • getNumCores

      public int getNumCores()
      Obtain total number of physical cores present on the system.
      Returns:
      number of physical cores

    • getCpuFrequency

      public long getCpuFrequency()
      Obtain the CPU frequency of on the system.
      Returns:
      CPU frequency in kHz

    • getCumulativeCpuTime

      public long getCumulativeCpuTime()
      Obtain the cumulative CPU time since the system is on.
      Returns:
      cumulative CPU time in milliseconds

    • getCpuUsagePercentage

      public float getCpuUsagePercentage()
      Obtain the CPU usage % of the machine. Return -1 if it is unavailable.
      Returns:
      CPU usage in %

    • getNumVCoresUsed

      public float getNumVCoresUsed()
      Obtain the number of VCores used. Return -1 if it is unavailable.
      Returns:
      Number of VCores used a percentage (from 0 to #VCores)

    • getNetworkBytesRead

      public long getNetworkBytesRead()
      Obtain the aggregated number of bytes read over the network.
      Returns:
      total number of bytes read.

    • getNetworkBytesWritten

      public long getNetworkBytesWritten()
      Obtain the aggregated number of bytes written to the network.
      Returns:
      total number of bytes written.

    • getStorageBytesRead

      public long getStorageBytesRead()
      Obtain the aggregated number of bytes read from disks.
      Returns:
      total number of bytes read.

    • getStorageBytesWritten

      public long getStorageBytesWritten()
      Obtain the aggregated number of bytes written to disks.
      Returns:
      total number of bytes written.

    • getResourceCalculatorPlugin

      public static ResourceCalculatorPlugin getResourceCalculatorPlugin(Class<? extends ResourceCalculatorPlugin> clazz, org.apache.hadoop.conf.Configuration conf)
      Create the ResourceCalculatorPlugin from the class name and configure it. If class name is null, this method will try and return a memory calculator plugin available for this system.
      Parameters:
      clazz - ResourceCalculator plugin class-name
      conf - configure the plugin with this.
      Returns:
      ResourceCalculatorPlugin or null if ResourceCalculatorPlugin is not available for current system

    • getContainersMonitorPlugin

      public static ResourceCalculatorPlugin getContainersMonitorPlugin(org.apache.hadoop.conf.Configuration conf)
      Create the ResourceCalculatorPlugin for the containers monitor in the Node Manager and configure it. If the plugin is not configured, this method will try and return a memory calculator plugin available for this system.
      Parameters:
      conf - Configure the plugin with this.
      Returns:
      ResourceCalculatorPlugin or null if ResourceCalculatorPlugin is not available for current system.

    • getNodeResourceMonitorPlugin

      public static ResourceCalculatorPlugin getNodeResourceMonitorPlugin(org.apache.hadoop.conf.Configuration conf)
      Create the ResourceCalculatorPlugin for the node resource monitor in the Node Manager and configure it. If the plugin is not configured, this method will try and return a memory calculator plugin available for this system.
      Parameters:
      conf - Configure the plugin with this.
      Returns:
      ResourceCalculatorPlugin or null if ResourceCalculatorPlugin is not available for current system.