@InterfaceAudience.Private @InterfaceStability.Unstable public class DockerLinuxContainerRuntime extends Object implements LinuxContainerRuntime
This class is a ContainerRuntime
implementation that uses the
native container-executor
binary via a
PrivilegedOperationExecutor
instance to launch processes inside
Docker containers.
The following environment variables are used to configure the Docker engine:
YARN_CONTAINER_RUNTIME_TYPE
ultimately determines whether a
Docker container will be used. If the value is docker
, a Docker
container will be used. Otherwise a regular process tree container will
be used. This environment variable is checked by the
isDockerContainerRequested(java.util.Map<java.lang.String, java.lang.String>)
method, which is called by the
DelegatingLinuxContainerRuntime
.
YARN_CONTAINER_RUNTIME_DOCKER_IMAGE
names which image
will be used to launch the Docker container.
YARN_CONTAINER_RUNTIME_DOCKER_IMAGE_FILE
is currently ignored.
YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE
controls
whether the Docker container's default command is overridden. When set
to true
, the Docker container's command will be
bash <path_to_launch_script>
. When unset or set to false
the Docker container's default command is used.
YARN_CONTAINER_RUNTIME_DOCKER_CONTAINER_NETWORK
sets the
network type to be used by the Docker container. It must be a valid
value as determined by the
yarn.nodemanager.runtime.linux.docker.allowed-container-networks
property.
YARN_CONTAINER_RUNTIME_DOCKER_CONTAINER_HOSTNAME
sets the
hostname to be used by the Docker container. If not specified, a
hostname will be derived from the container ID.
YARN_CONTAINER_RUNTIME_DOCKER_RUN_PRIVILEGED_CONTAINER
controls whether the Docker container is a privileged container. In order
to use privileged containers, the
yarn.nodemanager.runtime.linux.docker.privileged-containers.allowed
property must be set to true
, and the application owner must
appear in the value of the
yarn.nodemanager.runtime.linux.docker.privileged-containers.acl
property. If this environment variable is set to true
, a
privileged Docker container will be used if allowed. No other value is
allowed, so the environment variable should be left unset rather than
setting it to false.
YARN_CONTAINER_RUNTIME_DOCKER_LOCAL_RESOURCE_MOUNTS
adds
additional volume mounts to the Docker container. The value of the
environment variable should be a comma-separated list of mounts.
All such mounts must be given as source:dest
, where the
source is an absolute path that is not a symlink and that points to a
localized resource.
Modifier and Type | Field and Description |
---|---|
static String |
DOCKER_IMAGE_PATTERN |
static String |
ENV_DOCKER_CONTAINER_HOSTNAME |
static String |
ENV_DOCKER_CONTAINER_IMAGE |
static String |
ENV_DOCKER_CONTAINER_IMAGE_FILE |
static String |
ENV_DOCKER_CONTAINER_LOCAL_RESOURCE_MOUNTS |
static String |
ENV_DOCKER_CONTAINER_NETWORK |
static String |
ENV_DOCKER_CONTAINER_RUN_ENABLE_USER_REMAPPING |
static String |
ENV_DOCKER_CONTAINER_RUN_OVERRIDE_DISABLE |
static String |
ENV_DOCKER_CONTAINER_RUN_PRIVILEGED_CONTAINER |
static String |
HOSTNAME_PATTERN |
Constructor and Description |
---|
DockerLinuxContainerRuntime(PrivilegedOperationExecutor privilegedOperationExecutor)
Create an instance using the given
PrivilegedOperationExecutor
instance for performing operations. |
DockerLinuxContainerRuntime(PrivilegedOperationExecutor privilegedOperationExecutor,
CGroupsHandler cGroupsHandler)
Create an instance using the given
PrivilegedOperationExecutor
instance for performing operations and the given CGroupsHandler
instance. |
Modifier and Type | Method and Description |
---|---|
protected void |
addCGroupParentIfRequired(String resourcesOptions,
String containerIdStr,
DockerRunCommand runCommand)
If CGROUPS in enabled and not set to none, then set the CGROUP parent for
the command instance.
|
Set<String> |
getCapabilities() |
String[] |
getIpAndHost(Container container)
Return the host and ip of the container
|
void |
initialize(org.apache.hadoop.conf.Configuration conf)
Initialize the runtime.
|
static boolean |
isDockerContainerRequested(Map<String,String> env)
Return whether the given environment variables indicate that the operation
is requesting a Docker container.
|
void |
launchContainer(ContainerRuntimeContext ctx)
Launch a container.
|
void |
prepareContainer(ContainerRuntimeContext ctx)
Prepare a container to be ready for launch.
|
void |
reapContainer(ContainerRuntimeContext ctx)
Perform any container cleanup that may be required.
|
void |
signalContainer(ContainerRuntimeContext ctx)
Signal a container.
|
boolean |
useWhitelistEnv(Map<String,String> env)
Whether to propagate the whitelist of environment variables from the
nodemanager environment into the container environment.
|
static void |
validateHostname(String hostname) |
static void |
validateImageName(String imageName) |
protected String |
validateMount(String mount,
Map<org.apache.hadoop.fs.Path,List<String>> localizedResources) |
public static final String DOCKER_IMAGE_PATTERN
public static final String HOSTNAME_PATTERN
@InterfaceAudience.Private public static final String ENV_DOCKER_CONTAINER_IMAGE
@InterfaceAudience.Private public static final String ENV_DOCKER_CONTAINER_IMAGE_FILE
@InterfaceAudience.Private public static final String ENV_DOCKER_CONTAINER_RUN_OVERRIDE_DISABLE
@InterfaceAudience.Private public static final String ENV_DOCKER_CONTAINER_NETWORK
@InterfaceAudience.Private public static final String ENV_DOCKER_CONTAINER_HOSTNAME
@InterfaceAudience.Private public static final String ENV_DOCKER_CONTAINER_RUN_PRIVILEGED_CONTAINER
@InterfaceAudience.Private public static final String ENV_DOCKER_CONTAINER_RUN_ENABLE_USER_REMAPPING
@InterfaceAudience.Private public static final String ENV_DOCKER_CONTAINER_LOCAL_RESOURCE_MOUNTS
public DockerLinuxContainerRuntime(PrivilegedOperationExecutor privilegedOperationExecutor)
PrivilegedOperationExecutor
instance for performing operations.privilegedOperationExecutor
- the PrivilegedOperationExecutor
instancepublic DockerLinuxContainerRuntime(PrivilegedOperationExecutor privilegedOperationExecutor, CGroupsHandler cGroupsHandler)
PrivilegedOperationExecutor
instance for performing operations and the given CGroupsHandler
instance. This constructor is intended for use in testing.privilegedOperationExecutor
- the PrivilegedOperationExecutor
instancecGroupsHandler
- the CGroupsHandler
instancepublic static boolean isDockerContainerRequested(Map<String,String> env)
YARN_CONTAINER_RUNTIME_TYPE
whose value is docker
,
this method will return true. Otherwise it will return false.env
- the environment variable settings for the operationpublic void initialize(org.apache.hadoop.conf.Configuration conf) throws ContainerExecutionException
LinuxContainerRuntime
initialize
in interface LinuxContainerRuntime
conf
- the Configuration
to useContainerExecutionException
- if an error occurs while initializing
the runtimepublic boolean useWhitelistEnv(Map<String,String> env)
ContainerRuntime
useWhitelistEnv
in interface ContainerRuntime
env
- the container's environment variablesYarnConfiguration.NM_ENV_WHITELIST
public void prepareContainer(ContainerRuntimeContext ctx) throws ContainerExecutionException
ContainerRuntime
prepareContainer
in interface ContainerRuntime
ctx
- the ContainerRuntimeContext
ContainerExecutionException
- if an error occurs while preparing
the containerpublic static void validateHostname(String hostname) throws ContainerExecutionException
ContainerExecutionException
protected void addCGroupParentIfRequired(String resourcesOptions, String containerIdStr, DockerRunCommand runCommand)
resourcesOptions
- the resource options to check for "cgroups=none"containerIdStr
- the container IDrunCommand
- the command to set with the CGROUP parentprotected String validateMount(String mount, Map<org.apache.hadoop.fs.Path,List<String>> localizedResources) throws ContainerExecutionException
ContainerExecutionException
public void launchContainer(ContainerRuntimeContext ctx) throws ContainerExecutionException
ContainerRuntime
launchContainer
in interface ContainerRuntime
ctx
- the ContainerRuntimeContext
ContainerExecutionException
- if an error occurs while launching
the containerpublic void signalContainer(ContainerRuntimeContext ctx) throws ContainerExecutionException
ContainerRuntime
signalContainer
in interface ContainerRuntime
ctx
- the ContainerRuntimeContext
ContainerExecutionException
- if an error occurs while signaling
the containerpublic void reapContainer(ContainerRuntimeContext ctx) throws ContainerExecutionException
ContainerRuntime
reapContainer
in interface ContainerRuntime
ctx
- the ContainerRuntimeContext
ContainerExecutionException
- if an error occurs while reaping
the containerpublic String[] getIpAndHost(Container container)
ContainerRuntime
getIpAndHost
in interface ContainerRuntime
container
- the Container
public static void validateImageName(String imageName) throws ContainerExecutionException
ContainerExecutionException
Copyright © 2018 Apache Software Foundation. All Rights Reserved.