org.apache.hadoop.mapreduce.lib.output

Class LazyOutputFormat<K,V>

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<K,V>

org.apache.hadoop.mapreduce.lib.output.FilterOutputFormat<K,V>

org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat<K,V>

@InterfaceAudience.Public
 @InterfaceStability.Stable
public class LazyOutputFormat<K,V>
extends FilterOutputFormat<K,V>

A Convenience class that creates output lazily. Use in conjuction with org.apache.hadoop.mapreduce.lib.output.MultipleOutputs to recreate the behaviour of org.apache.hadoop.mapred.lib.MultipleTextOutputFormat (etc) of the old Hadoop API. See MultipleOutputs documentation for more information.

Field Summary

Fields

Modifier and Type	Field and Description
static String	OUTPUT_FORMAT

Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FilterOutputFormat

baseOut

Constructor Summary

Constructors

Constructor and Description
LazyOutputFormat()

Method Summary

Modifier and Type	Method and Description
void	checkOutputSpecs(JobContext context) Check for validity of the output-specification for the job.
OutputCommitter	getOutputCommitter(TaskAttemptContext context) Get the output committer for this output format.
RecordWriter<K,V>	getRecordWriter(TaskAttemptContext context) Get the RecordWriter for the given task.
static void	setOutputFormatClass(Job job, Class<? extends OutputFormat> theClass) Set the underlying output format for LazyOutputFormat.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

OUTPUT_FORMAT

public static String OUTPUT_FORMAT

Constructor Detail

LazyOutputFormat

public LazyOutputFormat()

Method Detail

setOutputFormatClass

public static void setOutputFormatClass(Job job,
                                        Class<? extends OutputFormat> theClass)

Set the underlying output format for LazyOutputFormat.

Parameters:

job - the Job to modify

theClass - the underlying class

getRecordWriter

public RecordWriter<K,V> getRecordWriter(TaskAttemptContext context)
                                  throws IOException,
                                         InterruptedException

Description copied from class: OutputFormat

Get the RecordWriter for the given task.

Overrides:

getRecordWriter in class FilterOutputFormat<K,V>

Parameters:

context - the information about the current task.

Returns:

a RecordWriter to write the output for the job.

Throws:

IOException

InterruptedException

checkOutputSpecs

public void checkOutputSpecs(JobContext context)
                      throws IOException,
                             InterruptedException

Description copied from class: OutputFormat

Check for validity of the output-specification for the job.

This is to validate the output specification for the job when it is a job is submitted. Typically checks that it does not already exist, throwing an exception when it already exists, so that output is not overwritten.

Implementations which write to filesystems which support delegation tokens usually collect the tokens for the destination path(s) and attach them to the job context's JobConf.

Overrides:

checkOutputSpecs in class FilterOutputFormat<K,V>

Parameters:

context - information about the job

Throws:

IOException - when output should not be attempted

InterruptedException

getOutputCommitter

public OutputCommitter getOutputCommitter(TaskAttemptContext context)
                                   throws IOException,
                                          InterruptedException

Description copied from class: OutputFormat

Get the output committer for this output format. This is responsible for ensuring the output is committed correctly.

Overrides:

getOutputCommitter in class FilterOutputFormat<K,V>

Parameters:

context - the task context

Returns:

an output committer

Throws:

IOException

InterruptedException