|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.apache.hadoop.mapred.FileInputFormat<LongWritable,BytesWritable> org.apache.hadoop.mapred.FixedLengthInputFormat
@InterfaceAudience.Public @InterfaceStability.Stable public class FixedLengthInputFormat
FixedLengthInputFormat is an input format used to read input files
which contain fixed length records. The content of a record need not be
text. It can be arbitrary binary data. Users must configure the record
length property by calling:
FixedLengthInputFormat.setRecordLength(conf, recordLength);
or
conf.setInt(FixedLengthInputFormat.FIXED_RECORD_LENGTH, recordLength);
FixedLengthRecordReader
Field Summary | |
---|---|
static String |
FIXED_RECORD_LENGTH
|
Fields inherited from class org.apache.hadoop.mapred.FileInputFormat |
---|
INPUT_DIR_RECURSIVE, LOG, NUM_INPUT_FILES |
Constructor Summary | |
---|---|
FixedLengthInputFormat()
|
Method Summary | |
---|---|
void |
configure(JobConf conf)
Initializes a new instance from a JobConf . |
static int |
getRecordLength(Configuration conf)
Get record length value |
RecordReader<LongWritable,BytesWritable> |
getRecordReader(InputSplit genericSplit,
JobConf job,
Reporter reporter)
Get the RecordReader for the given InputSplit . |
protected boolean |
isSplitable(FileSystem fs,
Path file)
Is the given filename splitable? Usually, true, but if the file is stream compressed, it will not be. |
static void |
setRecordLength(Configuration conf,
int recordLength)
Set the length of each record |
Methods inherited from class org.apache.hadoop.mapred.FileInputFormat |
---|
addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getInputPathFilter, getInputPaths, getSplitHosts, getSplits, listStatus, makeSplit, setInputPathFilter, setInputPaths, setInputPaths, setMinSplitSize |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final String FIXED_RECORD_LENGTH
Constructor Detail |
---|
public FixedLengthInputFormat()
Method Detail |
---|
public static void setRecordLength(Configuration conf, int recordLength)
conf
- configurationrecordLength
- the length of a recordpublic static int getRecordLength(Configuration conf)
conf
- configuration
public void configure(JobConf conf)
JobConfigurable
JobConf
.
configure
in interface JobConfigurable
conf
- the configurationpublic RecordReader<LongWritable,BytesWritable> getRecordReader(InputSplit genericSplit, JobConf job, Reporter reporter) throws IOException
InputFormat
RecordReader
for the given InputSplit
.
It is the responsibility of the RecordReader
to respect
record boundaries while processing the logical split to present a
record-oriented view to the individual task.
getRecordReader
in interface InputFormat<LongWritable,BytesWritable>
getRecordReader
in class FileInputFormat<LongWritable,BytesWritable>
genericSplit
- the InputSplit
job
- the job that this split belongs to
RecordReader
IOException
protected boolean isSplitable(FileSystem fs, Path file)
FileInputFormat
FileInputFormat
implementations can override this and return
false
to ensure that individual input files are never split-up
so that Mapper
s process entire files.
isSplitable
in class FileInputFormat<LongWritable,BytesWritable>
fs
- the file system that the file is onfile
- the file name to check
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |