Package org.apache.hadoop.mapred

Class FileSplit

java.lang.Object
org.apache.hadoop.mapreduce.InputSplit
org.apache.hadoop.mapred.FileSplit
All Implemented Interfaces:
Writable, InputSplit, InputSplitWithLocationInfo
@Public @Stable public class FileSplit extends InputSplit implements InputSplitWithLocationInfo
A section of an input file. Returned by InputFormat.getSplits(JobConf, int) and passed to InputFormat.getRecordReader(InputSplit,JobConf,Reporter).

  • Constructor Details

    • FileSplit

      protected FileSplit()

    • FileSplit

      @Deprecated public FileSplit(Path file, long start, long length, JobConf conf)
      Deprecated.
      Constructs a split.
      Parameters:
      file - the file name
      start - the position of the first byte in the file to process
      length - the number of bytes in the file to process

    • FileSplit

      public FileSplit(Path file, long start, long length, String[] hosts)
      Constructs a split with host information
      Parameters:
      file - the file name
      start - the position of the first byte in the file to process
      length - the number of bytes in the file to process
      hosts - the list of hosts containing the block, possibly null

    • FileSplit

      public FileSplit(Path file, long start, long length, String[] hosts, String[] inMemoryHosts)
      Constructs a split with host information
      Parameters:
      file - the file name
      start - the position of the first byte in the file to process
      length - the number of bytes in the file to process
      hosts - the list of hosts containing the block, possibly null
      inMemoryHosts - the list of hosts containing the block in memory

    • FileSplit

      public FileSplit(FileSplit fs)

  • Method Details

    • getPath

      public Path getPath()
      The file containing this split's data.

    • getStart

      public long getStart()
      The position of the first byte in the file to process.

    • getLength

      public long getLength()
      The number of bytes in the file to process.
      Specified by:
      getLength in interface InputSplit
      Specified by:
      getLength in class InputSplit
      Returns:
      the number of bytes in the split

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • write

      public void write(DataOutput out) throws IOException
      Description copied from interface: Writable
      Serialize the fields of this object to out.
      Specified by:
      write in interface Writable
      Parameters:
      out - DataOuput to serialize this object into.
      Throws:
      IOException - any other problem for write.

    • readFields

      public void readFields(DataInput in) throws IOException
      Description copied from interface: Writable
      Deserialize the fields of this object from in.

      For efficiency, implementations should attempt to re-use storage in the existing object where possible.

      Specified by:
      readFields in interface Writable
      Parameters:
      in - DataInput to deseriablize this object from.
      Throws:
      IOException - any other problem for readFields.

    • getLocations

      public String[] getLocations() throws IOException
      Description copied from class: InputSplit
      Get the list of nodes by name where the data for the split would be local. The locations do not need to be serialized.
      Specified by:
      getLocations in interface InputSplit
      Specified by:
      getLocations in class InputSplit
      Returns:
      a new array of the node nodes.
      Throws:
      IOException

    • getLocationInfo

      @Evolving public SplitLocationInfo[] getLocationInfo() throws IOException
      Description copied from class: InputSplit
      Gets info about which nodes the input split is stored on and how it is stored at each location.
      Specified by:
      getLocationInfo in interface InputSplitWithLocationInfo
      Overrides:
      getLocationInfo in class InputSplit
      Returns:
      list of SplitLocationInfos describing how the split data is stored at each location. A null value indicates that all the locations have the data stored on disk.
      Throws:
      IOException