@InterfaceAudience.Public @InterfaceStability.Stable public class CombineFileRecordReader<K,V> extends Object implements RecordReader<K,V>
CombineFileSplit
.
A CombineFileSplit can combine data chunks from multiple files.
This class allows using different RecordReaders for processing
these data chunks from different files.CombineFileSplit
Modifier and Type | Field and Description |
---|---|
protected RecordReader<K,V> |
curReader |
protected int |
idx |
protected JobConf |
jc |
protected long |
progress |
protected Reporter |
reporter |
protected Constructor<RecordReader<K,V>> |
rrConstructor |
protected CombineFileSplit |
split |
Constructor and Description |
---|
CombineFileRecordReader(JobConf job,
CombineFileSplit split,
Reporter reporter,
Class<RecordReader<K,V>> rrClass)
A generic RecordReader that can hand out different recordReaders
for each chunk in the CombineFileSplit.
|
Modifier and Type | Method and Description |
---|---|
void |
close()
Close this
InputSplit to future operations. |
K |
createKey()
Create an object of the appropriate type to be used as a key.
|
V |
createValue()
Create an object of the appropriate type to be used as a value.
|
long |
getPos()
return the amount of data processed
|
float |
getProgress()
return progress based on the amount of data processed so far.
|
protected boolean |
initNextRecordReader()
Get the record reader for the next chunk in this CombineFileSplit.
|
boolean |
next(K key,
V value)
Reads the next key/value pair from the input for processing.
|
protected CombineFileSplit split
protected JobConf jc
protected Reporter reporter
protected Constructor<RecordReader<K,V>> rrConstructor
protected int idx
protected long progress
protected RecordReader<K,V> curReader
public CombineFileRecordReader(JobConf job, CombineFileSplit split, Reporter reporter, Class<RecordReader<K,V>> rrClass) throws IOException
IOException
public boolean next(K key, V value) throws IOException
RecordReader
next
in interface RecordReader<K,V>
key
- the key to read data intovalue
- the value to read data intoIOException
public K createKey()
RecordReader
createKey
in interface RecordReader<K,V>
public V createValue()
RecordReader
createValue
in interface RecordReader<K,V>
public long getPos() throws IOException
getPos
in interface RecordReader<K,V>
IOException
public void close() throws IOException
RecordReader
InputSplit
to future operations.close
in interface Closeable
close
in interface AutoCloseable
close
in interface RecordReader<K,V>
IOException
public float getProgress() throws IOException
getProgress
in interface RecordReader<K,V>
0.0
to 1.0
.IOException
protected boolean initNextRecordReader() throws IOException
IOException
Copyright © 2021 Apache Software Foundation. All rights reserved.