Package org.apache.hadoop.mapreduce.lib.output

Class SequenceFileAsBinaryOutputFormat

java.lang.Object
org.apache.hadoop.mapreduce.OutputFormat<K,V>
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<K,V>
org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat<BytesWritable,BytesWritable>
org.apache.hadoop.mapreduce.lib.output.SequenceFileAsBinaryOutputFormat
@Public @Stable public class SequenceFileAsBinaryOutputFormat extends SequenceFileOutputFormat<BytesWritable,BytesWritable>
An OutputFormat that writes keys, values to SequenceFiles in binary(raw) format

  • Field Details

    • KEY_CLASS

      public static String KEY_CLASS

    • VALUE_CLASS

      public static String VALUE_CLASS

  • Constructor Details

    • SequenceFileAsBinaryOutputFormat

      public SequenceFileAsBinaryOutputFormat()

  • Method Details

    • setSequenceFileOutputKeyClass

      public static void setSequenceFileOutputKeyClass(Job job, Class<?> theClass)
      Set the key class for the SequenceFile

      This allows the user to specify the key class to be different from the actual class (BytesWritable) used for writing

      Parameters:
      job - the Job to modify
      theClass - the SequenceFile output key class.

    • setSequenceFileOutputValueClass

      public static void setSequenceFileOutputValueClass(Job job, Class<?> theClass)
      Set the value class for the SequenceFile

      This allows the user to specify the value class to be different from the actual class (BytesWritable) used for writing

      Parameters:
      job - the Job to modify
      theClass - the SequenceFile output key class.

    • getSequenceFileOutputKeyClass

      public static Class<? extends WritableComparable> getSequenceFileOutputKeyClass(JobContext job)
      Get the key class for the SequenceFile
      Returns:
      the key class of the SequenceFile

    • getSequenceFileOutputValueClass

      public static Class<? extends Writable> getSequenceFileOutputValueClass(JobContext job)
      Get the value class for the SequenceFile
      Returns:
      the value class of the SequenceFile

    • getRecordWriter

      public RecordWriter<BytesWritable,BytesWritable> getRecordWriter(TaskAttemptContext context) throws IOException
      Description copied from class: OutputFormat
      Get the RecordWriter for the given task.
      Overrides:
      getRecordWriter in class SequenceFileOutputFormat<BytesWritable,BytesWritable>
      Parameters:
      context - the information about the current task.
      Returns:
      a RecordWriter to write the output for the job.
      Throws:
      IOException

    • checkOutputSpecs

      public void checkOutputSpecs(JobContext job) throws IOException
      Description copied from class: OutputFormat
      Check for validity of the output-specification for the job.

      This is to validate the output specification for the job when it is a job is submitted. Typically checks that it does not already exist, throwing an exception when it already exists, so that output is not overwritten.

      Implementations which write to filesystems which support delegation tokens usually collect the tokens for the destination path(s) and attach them to the job context's JobConf.
      Overrides:
      checkOutputSpecs in class FileOutputFormat<BytesWritable,BytesWritable>
      Parameters:
      job - information about the job
      Throws:
      IOException - when output should not be attempted