@InterfaceAudience.Public @InterfaceStability.Stable public class CombineFileRecordReader<K,V> extends RecordReader<K,V>
CombineFileSplit
.
A CombineFileSplit can combine data chunks from multiple files.
This class allows using different RecordReaders for processing
these data chunks from different files.CombineFileSplit
Modifier and Type | Field and Description |
---|---|
protected TaskAttemptContext |
context |
protected RecordReader<K,V> |
curReader |
protected int |
idx |
protected long |
progress |
protected Constructor<? extends RecordReader<K,V>> |
rrConstructor |
protected CombineFileSplit |
split |
Constructor and Description |
---|
CombineFileRecordReader(CombineFileSplit split,
TaskAttemptContext context,
Class<? extends RecordReader<K,V>> rrClass)
A generic RecordReader that can hand out different recordReaders
for each chunk in the CombineFileSplit.
|
Modifier and Type | Method and Description |
---|---|
void |
close()
Close the record reader.
|
K |
getCurrentKey()
Get the current key
|
V |
getCurrentValue()
Get the current value.
|
float |
getProgress()
return progress based on the amount of data processed so far.
|
void |
initialize(InputSplit split,
TaskAttemptContext context)
Called once at initialization.
|
protected boolean |
initNextRecordReader()
Get the record reader for the next chunk in this CombineFileSplit.
|
boolean |
nextKeyValue()
Read the next key, value pair.
|
protected CombineFileSplit split
protected Constructor<? extends RecordReader<K,V>> rrConstructor
protected TaskAttemptContext context
protected int idx
protected long progress
protected RecordReader<K,V> curReader
public CombineFileRecordReader(CombineFileSplit split, TaskAttemptContext context, Class<? extends RecordReader<K,V>> rrClass) throws IOException
IOException
public void initialize(InputSplit split, TaskAttemptContext context) throws IOException, InterruptedException
RecordReader
initialize
in class RecordReader<K,V>
split
- the split that defines the range of records to readcontext
- the information about the taskIOException
InterruptedException
public boolean nextKeyValue() throws IOException, InterruptedException
RecordReader
nextKeyValue
in class RecordReader<K,V>
IOException
InterruptedException
public K getCurrentKey() throws IOException, InterruptedException
RecordReader
getCurrentKey
in class RecordReader<K,V>
IOException
InterruptedException
public V getCurrentValue() throws IOException, InterruptedException
RecordReader
getCurrentValue
in class RecordReader<K,V>
IOException
InterruptedException
public void close() throws IOException
RecordReader
close
in interface Closeable
close
in interface AutoCloseable
close
in class RecordReader<K,V>
IOException
public float getProgress() throws IOException, InterruptedException
getProgress
in class RecordReader<K,V>
IOException
InterruptedException
protected boolean initNextRecordReader() throws IOException
IOException
Copyright © 2015 Apache Software Foundation. All rights reserved.