org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources

Class CGroupsCpuResourceHandlerImpl

java.lang.Object

org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsCpuResourceHandlerImpl

All Implemented Interfaces:

CpuResourceHandler, ResourceHandler

@InterfaceStability.Unstable
@InterfaceAudience.Private
public class CGroupsCpuResourceHandlerImpl
extends Object
implements CpuResourceHandler

An implementation for using CGroups to restrict CPU usage on Linux. The implementation supports 3 different controls - restrict usage of all YARN containers, restrict relative usage of individual YARN containers and restrict usage of individual YARN containers. Admins can set the overall CPU to be used by all YARN containers - this is implemented by setting cpu.cfs_period_us and cpu.cfs_quota_us to the ratio desired. If strict resource usage mode is not enabled, cpu.shares is set for individual containers - this prevents containers from exceeding the overall limit for YARN containers but individual containers can use as much of the CPU as available(under the YARN limit). If strict resource usage is enabled, then container can only use the percentage of CPU allocated to them and this is again implemented using cpu.cfs_period_us and cpu.cfs_quota_us.

Method Summary

Methods

Modifier and Type	Method and Description
List<PrivilegedOperation>	bootstrap(org.apache.hadoop.conf.Configuration conf) Bootstrap resource susbsystem.
static boolean	cpuLimitsExist(String path)
static int[]	getOverallLimits(float yarnProcessors)
List<PrivilegedOperation>	postComplete(org.apache.hadoop.yarn.api.records.ContainerId containerId) Perform any tasks necessary after container completion
List<PrivilegedOperation>	preStart(Container container) Prepare a resource environment for container launch
List<PrivilegedOperation>	reacquireContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId) Require state for container that was already launched
List<PrivilegedOperation>	teardown() Teardown environment for resource subsystem if requested.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

bootstrap

public List<PrivilegedOperation> bootstrap(org.apache.hadoop.conf.Configuration conf)
                                    throws ResourceHandlerException

Description copied from interface: ResourceHandler

Bootstrap resource susbsystem.

Specified by:

bootstrap in interface ResourceHandler

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

cpuLimitsExist

@InterfaceAudience.Private
public static boolean cpuLimitsExist(String path)
                              throws IOException

Throws:

IOException

getOverallLimits

@InterfaceAudience.Private
public static int[] getOverallLimits(float yarnProcessors)

preStart

public List<PrivilegedOperation> preStart(Container container)
                                   throws ResourceHandlerException

Description copied from interface: ResourceHandler

Prepare a resource environment for container launch

Specified by:

preStart in interface ResourceHandler

Parameters:

container - Container being launched

Returns:

(possibly empty) list of operations that require elevated privileges e.g a) create a custom cgroup b) add pid for container to tasks file for a cgroup.

Throws:

ResourceHandlerException

reacquireContainer

public List<PrivilegedOperation> reacquireContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
                                             throws ResourceHandlerException

Description copied from interface: ResourceHandler

Require state for container that was already launched

Specified by:

reacquireContainer in interface ResourceHandler

Parameters:

containerId - if of the container being reacquired.

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

postComplete

public List<PrivilegedOperation> postComplete(org.apache.hadoop.yarn.api.records.ContainerId containerId)
                                       throws ResourceHandlerException

Description copied from interface: ResourceHandler

Perform any tasks necessary after container completion

Specified by:

postComplete in interface ResourceHandler

Parameters:

containerId - of the container that was completed.

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

teardown

public List<PrivilegedOperation> teardown()
                                   throws ResourceHandlerException

Description copied from interface: ResourceHandler

Teardown environment for resource subsystem if requested. This method needs to be used with care since it could impact running containers.

Specified by:

teardown in interface ResourceHandler

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException