org.apache.hadoop.mapreduce.lib.output
Class LazyOutputFormat<K,V>

java.lang.Object
  extended by org.apache.hadoop.mapreduce.OutputFormat<K,V>
      extended by org.apache.hadoop.mapreduce.lib.output.FilterOutputFormat<K,V>
          extended by org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat<K,V>

@InterfaceAudience.Public
@InterfaceStability.Stable
public class LazyOutputFormat<K,V>
extends FilterOutputFormat<K,V>

A Convenience class that creates output lazily. Use in conjuction with org.apache.hadoop.mapreduce.lib.output.MultipleOutputs to recreate the behaviour of org.apache.hadoop.mapred.lib.MultipleTextOutputFormat (etc) of the old Hadoop API. See MultipleOutputs documentation for more information.


Field Summary
static String OUTPUT_FORMAT
           
 
Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FilterOutputFormat
baseOut
 
Constructor Summary
LazyOutputFormat()
           
 
Method Summary
 void checkOutputSpecs(JobContext context)
          Check for validity of the output-specification for the job.
 OutputCommitter getOutputCommitter(TaskAttemptContext context)
          Get the output committer for this output format.
 RecordWriter<K,V> getRecordWriter(TaskAttemptContext context)
          Get the RecordWriter for the given task.
static void setOutputFormatClass(Job job, Class<? extends OutputFormat> theClass)
          Set the underlying output format for LazyOutputFormat.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

OUTPUT_FORMAT

public static String OUTPUT_FORMAT
Constructor Detail

LazyOutputFormat

public LazyOutputFormat()
Method Detail

setOutputFormatClass

public static void setOutputFormatClass(Job job,
                                        Class<? extends OutputFormat> theClass)
Set the underlying output format for LazyOutputFormat.

Parameters:
job - the Job to modify
theClass - the underlying class

getRecordWriter

public RecordWriter<K,V> getRecordWriter(TaskAttemptContext context)
                                  throws IOException,
                                         InterruptedException
Description copied from class: OutputFormat
Get the RecordWriter for the given task.

Overrides:
getRecordWriter in class FilterOutputFormat<K,V>
Parameters:
context - the information about the current task.
Returns:
a RecordWriter to write the output for the job.
Throws:
IOException
InterruptedException

checkOutputSpecs

public void checkOutputSpecs(JobContext context)
                      throws IOException,
                             InterruptedException
Description copied from class: OutputFormat
Check for validity of the output-specification for the job.

This is to validate the output specification for the job when it is a job is submitted. Typically checks that it does not already exist, throwing an exception when it already exists, so that output is not overwritten.

Overrides:
checkOutputSpecs in class FilterOutputFormat<K,V>
Parameters:
context - information about the job
Throws:
IOException - when output should not be attempted
InterruptedException

getOutputCommitter

public OutputCommitter getOutputCommitter(TaskAttemptContext context)
                                   throws IOException,
                                          InterruptedException
Description copied from class: OutputFormat
Get the output committer for this output format. This is responsible for ensuring the output is committed correctly.

Overrides:
getOutputCommitter in class FilterOutputFormat<K,V>
Parameters:
context - the task context
Returns:
an output committer
Throws:
IOException
InterruptedException


Copyright © 2014 Apache Software Foundation. All Rights Reserved.