org.apache.hadoop.mapred
Class TaskController

java.lang.Object
  org.apache.hadoop.mapred.TaskController

All Implemented Interfaces:

Configurable

Direct Known Subclasses:

DefaultTaskController

public abstract class TaskController
extends Object
implements Configurable

Controls initialization, finalization and clean up of tasks, and also the launching and killing of task JVMs. This class defines the API for initializing, finalizing and cleaning up of tasks, as also the launching and killing task JVMs. Subclasses of this class will implement the logic required for performing the actual actions.
NOTE: This class is internal only class and not intended for users!!

Field Summary

protected LocalDirAllocator	allocator
protected static String	COMMAND_FILE
protected org.apache.hadoop.mapred.TaskTracker.LocalStorage	localStorage
static org.apache.commons.logging.Log	LOG
static FsPermission	TASK_LAUNCH_SCRIPT_PERMISSION

Constructor Summary

TaskController()

Method Summary

abstract void	createLogDir(TaskAttemptID taskID, boolean isCleanup) Creates task log dir
abstract void	deleteAsUser(String user, String subDir) Delete the user's files under all of the task tracker root directories.
abstract void	deleteLogAsUser(String user, String subDir) Delete the user's files under the userlogs directory.
Configuration	getConf() Return the configuration used by this object.
String[]	getLocalDirs()
String	getRunAsUser(JobConf conf) Returns the local unix user that a given job will run as.
abstract void	initializeJob(String user, String jobid, Path credentials, Path jobConf, TaskUmbilicalProtocol taskTracker, InetSocketAddress ttAddr) Create all of the directories necessary for the job to start and download all of the job and private distributed cache files.
abstract int	launchTask(String user, String jobId, String attemptId, List<String> setup, List<String> jvmArguments, File currentWorkDirectory, String stdout, String stderr) Create all of the directories for the task and launches the child jvm.
protected void	logOutput(String output)
void	setConf(Configuration conf) Set the configuration to be used by this object.
abstract void	setup(LocalDirAllocator allocator, org.apache.hadoop.mapred.TaskTracker.LocalStorage localStorage) Does initialization and setup.
abstract void	signalTask(String user, int taskPid, ProcessTree.Signal signal) Send a signal to a task pid as the user.
abstract void	truncateLogsAsUser(String user, List<Task> allAttempts) Run the passed command as the user
protected static String	writeCommand(String cmdLine, FileSystem fs, Path commandFile)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

COMMAND_FILE

protected static final String COMMAND_FILE

See Also:

Constant Field Values

allocator

protected LocalDirAllocator allocator

localStorage

protected org.apache.hadoop.mapred.TaskTracker.LocalStorage localStorage

TASK_LAUNCH_SCRIPT_PERMISSION

public static final FsPermission TASK_LAUNCH_SCRIPT_PERMISSION

Constructor Detail

TaskController

public TaskController()

Method Detail

getConf

public Configuration getConf()

Description copied from interface: Configurable

Return the configuration used by this object.

Specified by:

getConf in interface Configurable

getLocalDirs

public String[] getLocalDirs()

setConf

public void setConf(Configuration conf)

Description copied from interface: Configurable

Set the configuration to be used by this object.

Specified by:

setConf in interface Configurable

setup

public abstract void setup(LocalDirAllocator allocator,
                           org.apache.hadoop.mapred.TaskTracker.LocalStorage localStorage)
                    throws IOException

Does initialization and setup.

Parameters:

allocator - the local dir allocator to use

localStorage - local storage to obtain dirs from

Throws:

IOException

initializeJob

public abstract void initializeJob(String user,
                                   String jobid,
                                   Path credentials,
                                   Path jobConf,
                                   TaskUmbilicalProtocol taskTracker,
                                   InetSocketAddress ttAddr)
                            throws IOException,
                                   InterruptedException

Create all of the directories necessary for the job to start and download all of the job and private distributed cache files. Creates both the user directories and the job log directory.

Parameters:

user - the user name

jobid - the job

credentials - a filename containing the job secrets

jobConf - the path to the localized configuration file

taskTracker - the connection the task tracker

ttAddr - the tasktracker's RPC address

Throws:

IOException

InterruptedException

launchTask

public abstract int launchTask(String user,
                               String jobId,
                               String attemptId,
                               List<String> setup,
                               List<String> jvmArguments,
                               File currentWorkDirectory,
                               String stdout,
                               String stderr)
                        throws IOException

Create all of the directories for the task and launches the child jvm.

Parameters:

user - the user name

jobId - the jobId in question

attemptId - the attempt id (cleanup attempts have .cleanup suffix)

setup - list of shell commands to execute before the jvm

jvmArguments - list of jvm arguments

currentWorkDirectory - the full path of the cwd for the task

stdout - the file to redirect stdout to

stderr - the file to redirect stderr to

Returns:

the exit code for the task

Throws:

IOException

signalTask

public abstract void signalTask(String user,
                                int taskPid,
                                ProcessTree.Signal signal)
                         throws IOException

Send a signal to a task pid as the user.

Parameters:

user - the user name

taskPid - the pid of the task

signal - the id of the signal to send

Throws:

IOException

deleteAsUser

public abstract void deleteAsUser(String user,
                                  String subDir)
                           throws IOException

Delete the user's files under all of the task tracker root directories.

Parameters:

user - the user name

subDir - the path relative to the user's subdirectory under the task tracker root directories.

Throws:

IOException

createLogDir

public abstract void createLogDir(TaskAttemptID taskID,
                                  boolean isCleanup)
                           throws IOException

Creates task log dir

Parameters:

taskID - ID of the task

isCleanup - If the task is cleanup task or not

Throws:

IOException

deleteLogAsUser

public abstract void deleteLogAsUser(String user,
                                     String subDir)
                              throws IOException

Delete the user's files under the userlogs directory.

Parameters:

user - the user to work as

subDir - the path under the userlogs directory.

Throws:

IOException

truncateLogsAsUser

public abstract void truncateLogsAsUser(String user,
                                        List<Task> allAttempts)
                                 throws IOException

Run the passed command as the user

Parameters:

user -

allAttempts - the list of attempts that the JVM ran

Throws:

IOException

getRunAsUser

public String getRunAsUser(JobConf conf)

Returns the local unix user that a given job will run as.

writeCommand

protected static String writeCommand(String cmdLine,
                                     FileSystem fs,
                                     Path commandFile)
                              throws IOException

Throws:

IOException

logOutput

protected void logOutput(String output)