org.apache.hadoop.mapreduce.lib.output
Class PartialFileOutputCommitter

java.lang.Object
  extended by org.apache.hadoop.mapreduce.OutputCommitter
      extended by org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
          extended by org.apache.hadoop.mapreduce.lib.output.PartialFileOutputCommitter
All Implemented Interfaces:
PartialOutputCommitter

@Checkpointable
@InterfaceAudience.Public
@InterfaceStability.Evolving
public class PartialFileOutputCommitter
extends FileOutputCommitter
implements PartialOutputCommitter

An OutputCommitter that commits files specified in job output directory i.e. ${mapreduce.output.fileoutputformat.outputdir}.


Field Summary
 
Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
PENDING_DIR_NAME, SUCCEEDED_FILE_NAME, SUCCESSFUL_JOB_OUTPUT_DIR_MARKER, TEMP_DIR_NAME
 
Constructor Summary
PartialFileOutputCommitter(Path outputPath, JobContext context)
           
PartialFileOutputCommitter(Path outputPath, TaskAttemptContext context)
           
 
Method Summary
 void cleanUpPartialOutputForTask(TaskAttemptContext context)
          Remove all previously committed outputs from prior executions of this task.
 Path getCommittedTaskPath(int appAttemptId, TaskAttemptContext context)
          Compute the path where the output of a committed task is stored until the entire job is committed for a specific application attempt.
 
Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
abortJob, abortTask, cleanupJob, commitJob, commitTask, getCommittedTaskPath, getCommittedTaskPath, getJobAttemptPath, getJobAttemptPath, getJobAttemptPath, getTaskAttemptPath, getTaskAttemptPath, getWorkPath, isRecoverySupported, needsTaskCommit, recoverTask, setupJob, setupTask
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

PartialFileOutputCommitter

public PartialFileOutputCommitter(Path outputPath,
                                  TaskAttemptContext context)
                           throws IOException
Throws:
IOException

PartialFileOutputCommitter

public PartialFileOutputCommitter(Path outputPath,
                                  JobContext context)
                           throws IOException
Throws:
IOException
Method Detail

getCommittedTaskPath

public Path getCommittedTaskPath(int appAttemptId,
                                 TaskAttemptContext context)
Description copied from class: FileOutputCommitter
Compute the path where the output of a committed task is stored until the entire job is committed for a specific application attempt.

Overrides:
getCommittedTaskPath in class FileOutputCommitter
Parameters:
appAttemptId - the id of the application attempt to use
context - the context of any task.
Returns:
the path where the output of a committed task is stored.

cleanUpPartialOutputForTask

public void cleanUpPartialOutputForTask(TaskAttemptContext context)
                                 throws IOException
Description copied from interface: PartialOutputCommitter
Remove all previously committed outputs from prior executions of this task.

Specified by:
cleanUpPartialOutputForTask in interface PartialOutputCommitter
Parameters:
context - Context for cleaning up previously promoted output.
Throws:
IOException - If cleanup fails, then the state of the task my not be well defined.


Copyright © 2014 Apache Software Foundation. All Rights Reserved.