@InterfaceAudience.Public @InterfaceStability.Stable public class FixedLengthInputFormat extends FileInputFormat<LongWritable,BytesWritable> implements JobConfigurable
FixedLengthRecordReader
Modifier and Type | Field and Description |
---|---|
static String |
FIXED_RECORD_LENGTH |
INPUT_DIR_RECURSIVE, LOG, NUM_INPUT_FILES
Constructor and Description |
---|
FixedLengthInputFormat() |
Modifier and Type | Method and Description |
---|---|
void |
configure(JobConf conf)
Initializes a new instance from a
JobConf . |
static int |
getRecordLength(Configuration conf)
Get record length value
|
RecordReader<LongWritable,BytesWritable> |
getRecordReader(InputSplit genericSplit,
JobConf job,
Reporter reporter)
Get the
RecordReader for the given InputSplit . |
protected boolean |
isSplitable(FileSystem fs,
Path file)
Is the given filename splitable? Usually, true, but if the file is
stream compressed, it will not be.
|
static void |
setRecordLength(Configuration conf,
int recordLength)
Set the length of each record
|
addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getInputPathFilter, getInputPaths, getSplitHosts, getSplits, listStatus, makeSplit, makeSplit, setInputPathFilter, setInputPaths, setInputPaths, setMinSplitSize
public static final String FIXED_RECORD_LENGTH
public FixedLengthInputFormat()
public static void setRecordLength(Configuration conf, int recordLength)
conf
- configurationrecordLength
- the length of a recordpublic static int getRecordLength(Configuration conf)
conf
- configurationpublic void configure(JobConf conf)
JobConfigurable
JobConf
.configure
in interface JobConfigurable
conf
- the configurationpublic RecordReader<LongWritable,BytesWritable> getRecordReader(InputSplit genericSplit, JobConf job, Reporter reporter) throws IOException
InputFormat
RecordReader
for the given InputSplit
.
It is the responsibility of the RecordReader
to respect
record boundaries while processing the logical split to present a
record-oriented view to the individual task.
getRecordReader
in interface InputFormat<LongWritable,BytesWritable>
getRecordReader
in class FileInputFormat<LongWritable,BytesWritable>
genericSplit
- the InputSplit
job
- the job that this split belongs toRecordReader
IOException
protected boolean isSplitable(FileSystem fs, Path file)
FileInputFormat
FileInputFormat
implementations can override this and return
false
to ensure that individual input files are never split-up
so that Mapper
s process entire files.isSplitable
in class FileInputFormat<LongWritable,BytesWritable>
fs
- the file system that the file is onfile
- the file name to checkCopyright © 2015 Apache Software Foundation. All Rights Reserved.