org.apache.hadoop.mapreduce.lib.output
Class FileOutputCommitter

java.lang.Object
  extended by org.apache.hadoop.mapreduce.OutputCommitter
      extended by org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

public class FileOutputCommitter
extends OutputCommitter

An OutputCommitter that commits files specified in job output directory i.e. ${mapred.output.dir}.


Field Summary
static String SUCCEEDED_FILE_NAME
           
protected static String TEMP_DIR_NAME
          Temporary directory name
 
Constructor Summary
FileOutputCommitter(Path outputPath, TaskAttemptContext context)
          Create a file output committer
 
Method Summary
 void abortJob(JobContext context, JobStatus.State state)
          Delete the temporary directory, including all of the work directories.
 void abortTask(TaskAttemptContext context)
          Delete the work directory
 void cleanupJob(JobContext context)
          Deprecated. 
 void commitJob(JobContext context)
          Delete the temporary directory, including all of the work directories.
 void commitTask(TaskAttemptContext context)
          Move the files from the work directory to the job output directory
 Path getWorkPath()
          Get the directory that the task should write results into
 boolean needsTaskCommit(TaskAttemptContext context)
          Did this task write any files in the work directory?
 void setupJob(JobContext context)
          Create the temporary directory that is the root of all of the task work directories.
 void setupTask(TaskAttemptContext context)
          No task setup required.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

TEMP_DIR_NAME

protected static final String TEMP_DIR_NAME
Temporary directory name

See Also:
Constant Field Values

SUCCEEDED_FILE_NAME

public static final String SUCCEEDED_FILE_NAME
See Also:
Constant Field Values
Constructor Detail

FileOutputCommitter

public FileOutputCommitter(Path outputPath,
                           TaskAttemptContext context)
                    throws IOException
Create a file output committer

Parameters:
outputPath - the job's output path
context - the task's context
Throws:
IOException
Method Detail

setupJob

public void setupJob(JobContext context)
              throws IOException
Create the temporary directory that is the root of all of the task work directories.

Specified by:
setupJob in class OutputCommitter
Parameters:
context - the job's context
Throws:
IOException - if temporary output could not be created

commitJob

public void commitJob(JobContext context)
               throws IOException
Delete the temporary directory, including all of the work directories. This is called for all jobs whose final run state is SUCCEEDED

Overrides:
commitJob in class OutputCommitter
Parameters:
context - the job's context.
Throws:
IOException

cleanupJob

@Deprecated
public void cleanupJob(JobContext context)
                throws IOException
Deprecated. 

Description copied from class: OutputCommitter
For cleaning up the job's output after job completion

Overrides:
cleanupJob in class OutputCommitter
Throws:
IOException

abortJob

public void abortJob(JobContext context,
                     JobStatus.State state)
              throws IOException
Delete the temporary directory, including all of the work directories.

Overrides:
abortJob in class OutputCommitter
Parameters:
context - the job's context
state - final run state of the job, should be FAILED or KILLED
Throws:
IOException

setupTask

public void setupTask(TaskAttemptContext context)
               throws IOException
No task setup required.

Specified by:
setupTask in class OutputCommitter
Parameters:
context - Context of the task whose output is being written.
Throws:
IOException

commitTask

public void commitTask(TaskAttemptContext context)
                throws IOException
Move the files from the work directory to the job output directory

Specified by:
commitTask in class OutputCommitter
Parameters:
context - the task context
Throws:
IOException - if commit is not

abortTask

public void abortTask(TaskAttemptContext context)
Delete the work directory

Specified by:
abortTask in class OutputCommitter

needsTaskCommit

public boolean needsTaskCommit(TaskAttemptContext context)
                        throws IOException
Did this task write any files in the work directory?

Specified by:
needsTaskCommit in class OutputCommitter
Parameters:
context - the task's context
Returns:
true/false
Throws:
IOException

getWorkPath

public Path getWorkPath()
                 throws IOException
Get the directory that the task should write results into

Returns:
the work directory
Throws:
IOException


Copyright © 2009 The Apache Software Foundation