@InterfaceAudience.Public @InterfaceStability.Stable public abstract class MultiFileInputFormat<K,V> extends FileInputFormat<K,V>
InputFormat
that returns MultiFileSplit
's
in getSplits(JobConf, int)
method. Splits are constructed from
the files under the input paths. Each split returned contains nearly
equal content length. getRecordReader(InputSplit, JobConf, Reporter)
to construct RecordReader
's for MultiFileSplit
's.MultiFileSplit
INPUT_DIR_RECURSIVE, LOG, NUM_INPUT_FILES
Constructor and Description |
---|
MultiFileInputFormat() |
Modifier and Type | Method and Description |
---|---|
abstract RecordReader<K,V> |
getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
Get the
RecordReader for the given InputSplit . |
InputSplit[] |
getSplits(JobConf job,
int numSplits)
Splits files returned by
FileInputFormat.listStatus(JobConf) when
they're too big. |
addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getInputPathFilter, getInputPaths, getSplitHosts, isSplitable, listStatus, makeSplit, makeSplit, setInputPathFilter, setInputPaths, setInputPaths, setMinSplitSize
public MultiFileInputFormat()
public InputSplit[] getSplits(JobConf job, int numSplits) throws IOException
FileInputFormat
FileInputFormat.listStatus(JobConf)
when
they're too big.getSplits
in interface InputFormat<K,V>
getSplits
in class FileInputFormat<K,V>
job
- job configuration.numSplits
- the desired number of splits, a hint.InputSplit
s for the job.IOException
public abstract RecordReader<K,V> getRecordReader(InputSplit split, JobConf job, Reporter reporter) throws IOException
InputFormat
RecordReader
for the given InputSplit
.
It is the responsibility of the RecordReader
to respect
record boundaries while processing the logical split to present a
record-oriented view to the individual task.
getRecordReader
in interface InputFormat<K,V>
getRecordReader
in class FileInputFormat<K,V>
split
- the InputSplit
job
- the job that this split belongs toRecordReader
IOException
Copyright © 2016 Apache Software Foundation. All rights reserved.