org.apache.hadoop.mapreduce.lib.output

Class LazyOutputFormat<K,V>

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<K,V>

org.apache.hadoop.mapreduce.lib.output.FilterOutputFormat<K,V>

org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat<K,V>

@InterfaceAudience.Public
@InterfaceStability.Stable
public class LazyOutputFormat<K,V>
extends FilterOutputFormat<K,V>

A Convenience class that creates output lazily. Use in conjuction with org.apache.hadoop.mapreduce.lib.output.MultipleOutputs to recreate the behaviour of org.apache.hadoop.mapred.lib.MultipleTextOutputFormat (etc) of the old Hadoop API. See MultipleOutputs documentation for more information.

Field Summary

Fields

Modifier and Type	Field and Description
static String	OUTPUT_FORMAT

Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FilterOutputFormat

baseOut

Constructor Summary

Constructors

Constructor and Description
LazyOutputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
void	checkOutputSpecs(JobContext context) Check for validity of the output-specification for the job.
OutputCommitter	getOutputCommitter(TaskAttemptContext context) Get the output committer for this output format.
RecordWriter<K,V>	getRecordWriter(TaskAttemptContext context) Get the RecordWriter for the given task.
static void	setOutputFormatClass(Job job, Class<? extends OutputFormat> theClass) Set the underlying output format for LazyOutputFormat.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

OUTPUT_FORMAT

public static String OUTPUT_FORMAT

Constructor Detail

LazyOutputFormat

public LazyOutputFormat()

Method Detail

setOutputFormatClass

public static void setOutputFormatClass(Job job,
                        Class<? extends OutputFormat> theClass)

Set the underlying output format for LazyOutputFormat.

Parameters:

job - the Job to modify

theClass - the underlying class

getRecordWriter

public RecordWriter<K,V> getRecordWriter(TaskAttemptContext context)
                                  throws IOException,
                                         InterruptedException

Description copied from class: OutputFormat

Get the RecordWriter for the given task.

Overrides:

getRecordWriter in class FilterOutputFormat<K,V>

Parameters:

context - the information about the current task.

Returns:

a RecordWriter to write the output for the job.

Throws:

IOException

InterruptedException

checkOutputSpecs

public void checkOutputSpecs(JobContext context)
                      throws IOException,
                             InterruptedException

Description copied from class: OutputFormat

Check for validity of the output-specification for the job.

This is to validate the output specification for the job when it is a job is submitted. Typically checks that it does not already exist, throwing an exception when it already exists, so that output is not overwritten.

Overrides:

checkOutputSpecs in class FilterOutputFormat<K,V>

Parameters:

context - information about the job

Throws:

IOException - when output should not be attempted

InterruptedException

getOutputCommitter

public OutputCommitter getOutputCommitter(TaskAttemptContext context)
                                   throws IOException,
                                          InterruptedException

Description copied from class: OutputFormat

Get the output committer for this output format. This is responsible for ensuring the output is committed correctly.

Overrides:

getOutputCommitter in class FilterOutputFormat<K,V>

Parameters:

context - the task context

Returns:

an output committer

Throws:

IOException

InterruptedException