@Checkpointable @InterfaceAudience.Public @InterfaceStability.Evolving public class PartialFileOutputCommitter extends FileOutputCommitter implements PartialOutputCommitter
OutputCommitter
that commits files specified
in job output directory i.e. ${mapreduce.output.fileoutputformat.outputdir}.FILEOUTPUTCOMMITTER_ALGORITHM_VERSION, FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT, FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED, FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED_DEFAULT, FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED, FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED_DEFAULT, FILEOUTPUTCOMMITTER_FAILURE_ATTEMPTS, FILEOUTPUTCOMMITTER_FAILURE_ATTEMPTS_DEFAULT, PENDING_DIR_NAME, SUCCEEDED_FILE_NAME, SUCCESSFUL_JOB_OUTPUT_DIR_MARKER, TEMP_DIR_NAME
Constructor and Description |
---|
PartialFileOutputCommitter(Path outputPath,
JobContext context) |
PartialFileOutputCommitter(Path outputPath,
TaskAttemptContext context) |
Modifier and Type | Method and Description |
---|---|
void |
cleanUpPartialOutputForTask(TaskAttemptContext context)
Remove all previously committed outputs from prior executions of this task.
|
Path |
getCommittedTaskPath(int appAttemptId,
TaskAttemptContext context)
Compute the path where the output of a committed task is stored until the
entire job is committed for a specific application attempt.
|
abortJob, abortTask, cleanupJob, commitJob, commitJobInternal, commitTask, getCommittedTaskPath, getCommittedTaskPath, getJobAttemptPath, getJobAttemptPath, getJobAttemptPath, getTaskAttemptPath, getTaskAttemptPath, getWorkPath, isCommitJobRepeatable, isRecoverySupported, needsTaskCommit, recoverTask, setupJob, setupTask
isRecoverySupported
public PartialFileOutputCommitter(Path outputPath, TaskAttemptContext context) throws IOException
IOException
public PartialFileOutputCommitter(Path outputPath, JobContext context) throws IOException
IOException
public Path getCommittedTaskPath(int appAttemptId, TaskAttemptContext context)
FileOutputCommitter
getCommittedTaskPath
in class FileOutputCommitter
appAttemptId
- the id of the application attempt to usecontext
- the context of any task.public void cleanUpPartialOutputForTask(TaskAttemptContext context) throws IOException
PartialOutputCommitter
cleanUpPartialOutputForTask
in interface PartialOutputCommitter
context
- Context for cleaning up previously promoted output.IOException
- If cleanup fails, then the state of the task my not be
well defined.Copyright © 2017 Apache Software Foundation. All rights reserved.