org.apache.hadoop.mapreduce.lib.output
Class SequenceFileAsBinaryOutputFormat

java.lang.Object
  extended by org.apache.hadoop.mapreduce.OutputFormat<K,V>
      extended by org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<K,V>
          extended by org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat<BytesWritable,BytesWritable>
              extended by org.apache.hadoop.mapreduce.lib.output.SequenceFileAsBinaryOutputFormat

@InterfaceAudience.Public
@InterfaceStability.Stable
public class SequenceFileAsBinaryOutputFormat
extends SequenceFileOutputFormat<BytesWritable,BytesWritable>

An OutputFormat that writes keys, values to SequenceFiles in binary(raw) format


Nested Class Summary
static class SequenceFileAsBinaryOutputFormat.WritableValueBytes
          Inner class used for appendRaw
 
Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat
FileOutputFormat.Counter
 
Field Summary
static String KEY_CLASS
           
static String VALUE_CLASS
           
 
Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat
BASE_OUTPUT_NAME, PART
 
Constructor Summary
SequenceFileAsBinaryOutputFormat()
           
 
Method Summary
 void checkOutputSpecs(JobContext job)
          Check for validity of the output-specification for the job.
 RecordWriter<BytesWritable,BytesWritable> getRecordWriter(TaskAttemptContext context)
          Get the RecordWriter for the given task.
static Class<? extends WritableComparable> getSequenceFileOutputKeyClass(JobContext job)
          Get the key class for the SequenceFile
static Class<? extends Writable> getSequenceFileOutputValueClass(JobContext job)
          Get the value class for the SequenceFile
protected  SequenceFile.Writer getSequenceWriter(TaskAttemptContext context, Class<?> keyClass, Class<?> valueClass)
           
static void setSequenceFileOutputKeyClass(Job job, Class<?> theClass)
          Set the key class for the SequenceFile
static void setSequenceFileOutputValueClass(Job job, Class<?> theClass)
          Set the value class for the SequenceFile
 
Methods inherited from class org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat
getOutputCompressionType, setOutputCompressionType
 
Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat
getCompressOutput, getDefaultWorkFile, getOutputCommitter, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

KEY_CLASS

public static String KEY_CLASS

VALUE_CLASS

public static String VALUE_CLASS
Constructor Detail

SequenceFileAsBinaryOutputFormat

public SequenceFileAsBinaryOutputFormat()
Method Detail

setSequenceFileOutputKeyClass

public static void setSequenceFileOutputKeyClass(Job job,
                                                 Class<?> theClass)
Set the key class for the SequenceFile

This allows the user to specify the key class to be different from the actual class (BytesWritable) used for writing

Parameters:
job - the Job to modify
theClass - the SequenceFile output key class.

setSequenceFileOutputValueClass

public static void setSequenceFileOutputValueClass(Job job,
                                                   Class<?> theClass)
Set the value class for the SequenceFile

This allows the user to specify the value class to be different from the actual class (BytesWritable) used for writing

Parameters:
job - the Job to modify
theClass - the SequenceFile output key class.

getSequenceFileOutputKeyClass

public static Class<? extends WritableComparable> getSequenceFileOutputKeyClass(JobContext job)
Get the key class for the SequenceFile

Returns:
the key class of the SequenceFile

getSequenceFileOutputValueClass

public static Class<? extends Writable> getSequenceFileOutputValueClass(JobContext job)
Get the value class for the SequenceFile

Returns:
the value class of the SequenceFile

getRecordWriter

public RecordWriter<BytesWritable,BytesWritable> getRecordWriter(TaskAttemptContext context)
                                                          throws IOException
Description copied from class: OutputFormat
Get the RecordWriter for the given task.

Overrides:
getRecordWriter in class SequenceFileOutputFormat<BytesWritable,BytesWritable>
Parameters:
context - the information about the current task.
Returns:
a RecordWriter to write the output for the job.
Throws:
IOException

getSequenceWriter

protected SequenceFile.Writer getSequenceWriter(TaskAttemptContext context,
                                                Class<?> keyClass,
                                                Class<?> valueClass)
                                         throws IOException
Throws:
IOException

checkOutputSpecs

public void checkOutputSpecs(JobContext job)
                      throws IOException
Description copied from class: OutputFormat
Check for validity of the output-specification for the job.

This is to validate the output specification for the job when it is a job is submitted. Typically checks that it does not already exist, throwing an exception when it already exists, so that output is not overwritten.

Overrides:
checkOutputSpecs in class FileOutputFormat<BytesWritable,BytesWritable>
Parameters:
job - information about the job
Throws:
IOException - when output should not be attempted


Copyright © 2009 The Apache Software Foundation