org.apache.hadoop.mapreduce.lib.input
Class CombineFileRecordReader<K,V>

java.lang.Object
  extended by org.apache.hadoop.mapreduce.RecordReader<K,V>
      extended by org.apache.hadoop.mapreduce.lib.input.CombineFileRecordReader<K,V>
All Implemented Interfaces:
Closeable

@InterfaceAudience.Public
@InterfaceStability.Stable
public class CombineFileRecordReader<K,V>
extends RecordReader<K,V>

A generic RecordReader that can hand out different recordReaders for each chunk in a CombineFileSplit. A CombineFileSplit can combine data chunks from multiple files. This class allows using different RecordReaders for processing these data chunks from different files.

See Also:
CombineFileSplit

Field Summary
protected  TaskAttemptContext context
           
protected  RecordReader<K,V> curReader
           
protected  FileSystem fs
           
protected  int idx
           
protected  long progress
           
protected  Class<? extends RecordReader<K,V>> rrClass
           
protected  Constructor<? extends RecordReader<K,V>> rrConstructor
           
protected  CombineFileSplit split
           
 
Constructor Summary
CombineFileRecordReader(CombineFileSplit split, TaskAttemptContext context, Class<? extends RecordReader<K,V>> rrClass)
          A generic RecordReader that can hand out different recordReaders for each chunk in the CombineFileSplit.
 
Method Summary
 void close()
          Close the record reader.
 K getCurrentKey()
          Get the current key
 V getCurrentValue()
          Get the current value.
 float getProgress()
          return progress based on the amount of data processed so far.
 void initialize(InputSplit split, TaskAttemptContext context)
          Called once at initialization.
protected  boolean initNextRecordReader()
          Get the record reader for the next chunk in this CombineFileSplit.
 boolean nextKeyValue()
          Read the next key, value pair.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

split

protected CombineFileSplit split

rrClass

protected Class<? extends RecordReader<K,V>> rrClass

rrConstructor

protected Constructor<? extends RecordReader<K,V>> rrConstructor

fs

protected FileSystem fs

context

protected TaskAttemptContext context

idx

protected int idx

progress

protected long progress

curReader

protected RecordReader<K,V> curReader
Constructor Detail

CombineFileRecordReader

public CombineFileRecordReader(CombineFileSplit split,
                               TaskAttemptContext context,
                               Class<? extends RecordReader<K,V>> rrClass)
                        throws IOException
A generic RecordReader that can hand out different recordReaders for each chunk in the CombineFileSplit.

Throws:
IOException
Method Detail

initialize

public void initialize(InputSplit split,
                       TaskAttemptContext context)
                throws IOException,
                       InterruptedException
Description copied from class: RecordReader
Called once at initialization.

Specified by:
initialize in class RecordReader<K,V>
Parameters:
split - the split that defines the range of records to read
context - the information about the task
Throws:
IOException
InterruptedException

nextKeyValue

public boolean nextKeyValue()
                     throws IOException,
                            InterruptedException
Description copied from class: RecordReader
Read the next key, value pair.

Specified by:
nextKeyValue in class RecordReader<K,V>
Returns:
true if a key/value pair was read
Throws:
IOException
InterruptedException

getCurrentKey

public K getCurrentKey()
                throws IOException,
                       InterruptedException
Description copied from class: RecordReader
Get the current key

Specified by:
getCurrentKey in class RecordReader<K,V>
Returns:
the current key or null if there is no current key
Throws:
IOException
InterruptedException

getCurrentValue

public V getCurrentValue()
                  throws IOException,
                         InterruptedException
Description copied from class: RecordReader
Get the current value.

Specified by:
getCurrentValue in class RecordReader<K,V>
Returns:
the object that was read
Throws:
IOException
InterruptedException

close

public void close()
           throws IOException
Description copied from class: RecordReader
Close the record reader.

Specified by:
close in interface Closeable
Specified by:
close in class RecordReader<K,V>
Throws:
IOException

getProgress

public float getProgress()
                  throws IOException,
                         InterruptedException
return progress based on the amount of data processed so far.

Specified by:
getProgress in class RecordReader<K,V>
Returns:
a number between 0.0 and 1.0 that is the fraction of the data read
Throws:
IOException
InterruptedException

initNextRecordReader

protected boolean initNextRecordReader()
                                throws IOException
Get the record reader for the next chunk in this CombineFileSplit.

Throws:
IOException


Copyright © 2009 The Apache Software Foundation