org.apache.hadoop.io.file.tfile
Class TFile.Writer

java.lang.Object
  extended by org.apache.hadoop.io.file.tfile.TFile.Writer
All Implemented Interfaces:
Closeable
Enclosing class:
TFile

public static class TFile.Writer
extends Object
implements Closeable

TFile Writer.


Constructor Summary
TFile.Writer(FSDataOutputStream fsdos, int minBlockSize, String compressName, String comparator, Configuration conf)
          Constructor
 
Method Summary
 void append(byte[] key, byte[] value)
          Adding a new key-value pair to the TFile.
 void append(byte[] key, int koff, int klen, byte[] value, int voff, int vlen)
          Adding a new key-value pair to TFile.
 void close()
          Close the Writer.
 DataOutputStream prepareAppendKey(int length)
          Obtain an output stream for writing a key into TFile.
 DataOutputStream prepareAppendValue(int length)
          Obtain an output stream for writing a value into TFile.
 DataOutputStream prepareMetaBlock(String name)
          Obtain an output stream for creating a meta block.
 DataOutputStream prepareMetaBlock(String name, String compressName)
          Obtain an output stream for creating a meta block.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

TFile.Writer

public TFile.Writer(FSDataOutputStream fsdos,
                    int minBlockSize,
                    String compressName,
                    String comparator,
                    Configuration conf)
             throws IOException
Constructor

Parameters:
fsdos - output stream for writing. Must be at position 0.
minBlockSize - Minimum compressed block size in bytes. A compression block will not be closed until it reaches this size except for the last block.
compressName - Name of the compression algorithm. Must be one of the strings returned by TFile.getSupportedCompressionAlgorithms().
comparator - Leave comparator as null or empty string if TFile is not sorted. Otherwise, provide the string name for the comparison algorithm for keys. Two kinds of comparators are supported.
  • Algorithmic comparator: binary comparators that is language independent. Currently, only "memcmp" is supported.
  • Language-specific comparator: binary comparators that can only be constructed in specific language. For Java, the syntax is "jclass:", followed by the class name of the RawComparator. Currently, we only support RawComparators that can be constructed through the default constructor (with no parameters). Parameterized RawComparators such as WritableComparator or JavaSerializationComparator may not be directly used. One should write a wrapper class that inherits from such classes and use its default constructor to perform proper initialization.
conf - The configuration object.
Throws:
IOException
Method Detail

close

public void close()
           throws IOException
Close the Writer. Resources will be released regardless of the exceptions being thrown. Future close calls will have no effect. The underlying FSDataOutputStream is not closed.

Specified by:
close in interface Closeable
Throws:
IOException

append

public void append(byte[] key,
                   byte[] value)
            throws IOException
Adding a new key-value pair to the TFile. This is synonymous to append(key, 0, key.length, value, 0, value.length)

Parameters:
key - Buffer for key.
value - Buffer for value.
Throws:
IOException

append

public void append(byte[] key,
                   int koff,
                   int klen,
                   byte[] value,
                   int voff,
                   int vlen)
            throws IOException
Adding a new key-value pair to TFile.

Parameters:
key - buffer for key.
koff - offset in key buffer.
klen - length of key.
value - buffer for value.
voff - offset in value buffer.
vlen - length of value.
Throws:
IOException - Upon IO errors.

If an exception is thrown, the TFile will be in an inconsistent state. The only legitimate call after that would be close


prepareAppendKey

public DataOutputStream prepareAppendKey(int length)
                                  throws IOException
Obtain an output stream for writing a key into TFile. This may only be called when there is no active Key appending stream or value appending stream.

Parameters:
length - The expected length of the key. If length of the key is not known, set length = -1. Otherwise, the application must write exactly as many bytes as specified here before calling close on the returned output stream.
Returns:
The key appending output stream.
Throws:
IOException

prepareAppendValue

public DataOutputStream prepareAppendValue(int length)
                                    throws IOException
Obtain an output stream for writing a value into TFile. This may only be called right after a key appending operation (the key append stream must be closed).

Parameters:
length - The expected length of the value. If length of the value is not known, set length = -1. Otherwise, the application must write exactly as many bytes as specified here before calling close on the returned output stream. Advertising the value size up-front guarantees that the value is encoded in one chunk, and avoids intermediate chunk buffering.
Throws:
IOException

prepareMetaBlock

public DataOutputStream prepareMetaBlock(String name,
                                         String compressName)
                                  throws IOException,
                                         MetaBlockAlreadyExists
Obtain an output stream for creating a meta block. This function may not be called when there is a key append stream or value append stream active. No more key-value insertion is allowed after a meta data block has been added to TFile.

Parameters:
name - Name of the meta block.
compressName - Name of the compression algorithm to be used. Must be one of the strings returned by TFile.getSupportedCompressionAlgorithms().
Returns:
A DataOutputStream that can be used to write Meta Block data. Closing the stream would signal the ending of the block.
Throws:
IOException
MetaBlockAlreadyExists - the Meta Block with the same name already exists.

prepareMetaBlock

public DataOutputStream prepareMetaBlock(String name)
                                  throws IOException,
                                         MetaBlockAlreadyExists
Obtain an output stream for creating a meta block. This function may not be called when there is a key append stream or value append stream active. No more key-value insertion is allowed after a meta data block has been added to TFile. Data will be compressed using the default compressor as defined in Writer's constructor.

Parameters:
name - Name of the meta block.
Returns:
A DataOutputStream that can be used to write Meta Block data. Closing the stream would signal the ending of the block.
Throws:
IOException
MetaBlockAlreadyExists - the Meta Block with the same name already exists.


Copyright © 2009 The Apache Software Foundation