org.apache.hadoop.mapred

Class LocalContainerLauncher

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.mapred.LocalContainerLauncher

All Implemented Interfaces:

Closeable, AutoCloseable, ContainerLauncher, org.apache.hadoop.service.Service, org.apache.hadoop.yarn.event.EventHandler<ContainerLauncherEvent>

public class LocalContainerLauncher
extends org.apache.hadoop.service.AbstractService
implements ContainerLauncher

Runs the container task locally in a thread. Since all (sub)tasks share the same local directory, they must be executed sequentially in order to avoid creating/deleting the same files/dirs.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher

ContainerLauncher.EventType

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Constructor Summary

Constructors

Constructor and Description
LocalContainerLauncher(AppContext context, org.apache.hadoop.mapred.TaskUmbilicalProtocol umbilical)
LocalContainerLauncher(AppContext context, org.apache.hadoop.mapred.TaskUmbilicalProtocol umbilical, ClassLoader jobClassLoader)

Method Summary

Modifier and Type	Method and Description
void	handle(ContainerLauncherEvent event)
protected static org.apache.hadoop.mapred.MapOutputFile	renameMapOutputForReduce(org.apache.hadoop.mapred.JobConf conf, org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId mapId, org.apache.hadoop.mapred.MapOutputFile subMapOutputFile) Within the _local_ filesystem (not HDFS), all activity takes place within a subdir inside one of the LOCAL_DIRS (${local.dir}/usercache/$user/appcache/$appId/$contId/), and all sub-MapTasks create the same filename ("file.out").
void	serviceStart()
void	serviceStop()
void	setEncryptedSpillKey(byte[] encryptedSpillKey)

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

LocalContainerLauncher

public LocalContainerLauncher(AppContext context,
                              org.apache.hadoop.mapred.TaskUmbilicalProtocol umbilical)

LocalContainerLauncher

public LocalContainerLauncher(AppContext context,
                              org.apache.hadoop.mapred.TaskUmbilicalProtocol umbilical,
                              ClassLoader jobClassLoader)

Method Detail

serviceStart

public void serviceStart()
                  throws Exception

Overrides:

serviceStart in class org.apache.hadoop.service.AbstractService

Throws:

Exception

serviceStop

public void serviceStop()
                 throws Exception

Overrides:

serviceStop in class org.apache.hadoop.service.AbstractService

Throws:

Exception

handle

public void handle(ContainerLauncherEvent event)

Specified by:

handle in interface org.apache.hadoop.yarn.event.EventHandler<ContainerLauncherEvent>

setEncryptedSpillKey

public void setEncryptedSpillKey(byte[] encryptedSpillKey)

renameMapOutputForReduce

@VisibleForTesting
protected static org.apache.hadoop.mapred.MapOutputFile renameMapOutputForReduce(org.apache.hadoop.mapred.JobConf conf,
                                                                                                    org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId mapId,
                                                                                                    org.apache.hadoop.mapred.MapOutputFile subMapOutputFile)
                                                                                             throws IOException

Within the _local_ filesystem (not HDFS), all activity takes place within a subdir inside one of the LOCAL_DIRS (${local.dir}/usercache/$user/appcache/$appId/$contId/), and all sub-MapTasks create the same filename ("file.out"). Rename that to something unique (e.g., "map_0.out") to avoid possible collisions. Longer-term, we'll modify [something] to use TaskAttemptID-based filenames instead of "file.out". (All of this is entirely internal, so there are no particular compatibility issues.)

Throws:

IOException