org.apache.hadoop.mapred.lib
Class CombineFileSplit

java.lang.Object
  extended by org.apache.hadoop.mapred.lib.CombineFileSplit
All Implemented Interfaces:
Writable, InputSplit
Direct Known Subclasses:
MultiFileSplit

public class CombineFileSplit
extends Object
implements InputSplit

A sub-collection of input files. Unlike FileSplit, CombineFileSplit * class does not represent a split of a file, but a split of input files into smaller sets. A split may contain blocks from different file but all the blocks in the same split are probably local to some rack
CombineFileSplit can be used to implement RecordReader's, with reading one record per file.

See Also:
FileSplit, CombineFileInputFormat

Constructor Summary
CombineFileSplit()
          default constructor
CombineFileSplit(CombineFileSplit old)
          Copy constructor
CombineFileSplit(JobConf job, Path[] files, long[] lengths)
           
CombineFileSplit(JobConf job, Path[] files, long[] start, long[] lengths, String[] locations)
           
 
Method Summary
 JobConf getJob()
           
 long getLength()
          Get the total number of bytes in the data of the InputSplit.
 long getLength(int i)
          Returns the length of the ith Path
 long[] getLengths()
          Returns an array containing the lengths of the files in the split
 String[] getLocations()
          Returns all the Paths where this input-split resides
 int getNumPaths()
          Returns the number of Paths in the split
 long getOffset(int i)
          Returns the start offset of the ith Path
 Path getPath(int i)
          Returns the ith Path
 Path[] getPaths()
          Returns all the Paths in the split
 long[] getStartOffsets()
          Returns an array containing the startoffsets of the files in the split
 void readFields(DataInput in)
          Deserialize the fields of this object from in.
 String toString()
           
 void write(DataOutput out)
          Serialize the fields of this object to out.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Constructor Detail

CombineFileSplit

public CombineFileSplit()
default constructor


CombineFileSplit

public CombineFileSplit(JobConf job,
                        Path[] files,
                        long[] start,
                        long[] lengths,
                        String[] locations)

CombineFileSplit

public CombineFileSplit(JobConf job,
                        Path[] files,
                        long[] lengths)

CombineFileSplit

public CombineFileSplit(CombineFileSplit old)
                 throws IOException
Copy constructor

Throws:
IOException
Method Detail

getJob

public JobConf getJob()

getLength

public long getLength()
Description copied from interface: InputSplit
Get the total number of bytes in the data of the InputSplit.

Specified by:
getLength in interface InputSplit
Returns:
the number of bytes in the input split.

getStartOffsets

public long[] getStartOffsets()
Returns an array containing the startoffsets of the files in the split


getLengths

public long[] getLengths()
Returns an array containing the lengths of the files in the split


getOffset

public long getOffset(int i)
Returns the start offset of the ith Path


getLength

public long getLength(int i)
Returns the length of the ith Path


getNumPaths

public int getNumPaths()
Returns the number of Paths in the split


getPath

public Path getPath(int i)
Returns the ith Path


getPaths

public Path[] getPaths()
Returns all the Paths in the split


getLocations

public String[] getLocations()
                      throws IOException
Returns all the Paths where this input-split resides

Specified by:
getLocations in interface InputSplit
Returns:
list of hostnames where data of the InputSplit is located as an array of Strings.
Throws:
IOException

readFields

public void readFields(DataInput in)
                throws IOException
Description copied from interface: Writable
Deserialize the fields of this object from in.

For efficiency, implementations should attempt to re-use storage in the existing object where possible.

Specified by:
readFields in interface Writable
Parameters:
in - DataInput to deseriablize this object from.
Throws:
IOException

write

public void write(DataOutput out)
           throws IOException
Description copied from interface: Writable
Serialize the fields of this object to out.

Specified by:
write in interface Writable
Parameters:
out - DataOuput to serialize this object into.
Throws:
IOException

toString

public String toString()
Overrides:
toString in class Object


Copyright © 2009 The Apache Software Foundation