org.apache.hadoop.fs
Class FSOutputSummer

java.lang.Object
  java.io.OutputStream
      org.apache.hadoop.fs.FSOutputSummer

All Implemented Interfaces:

Closeable, Flushable

public abstract class FSOutputSummer
extends OutputStream

This is a generic output stream for generating checksums for data before it is written to the underlying stream

Constructor Summary

protected

FSOutputSummer(Checksum sum, int maxChunkSize, int checksumSize)

Method Summary

static byte[]	convertToByteStream(Checksum sum, int checksumSize) Converts a checksum integer value to a byte stream
protected void	flushBuffer()
protected void	flushBuffer(boolean keep)
protected void	resetChecksumChunk(int size) Resets existing buffer with a new one of the specified size.
void	write(byte[] b, int off, int len) Writes len bytes from the specified byte array starting at offset off and generate a checksum for each data chunk.
void	write(int b) Write one byte
protected abstract void	writeChunk(byte[] b, int offset, int len, byte[] checksum)

Methods inherited from class java.io.OutputStream

close, flush, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FSOutputSummer

protected FSOutputSummer(Checksum sum,
                         int maxChunkSize,
                         int checksumSize)

Method Detail

writeChunk

protected abstract void writeChunk(byte[] b,
                                   int offset,
                                   int len,
                                   byte[] checksum)
                            throws IOException

Throws:

IOException

write

public void write(int b)
           throws IOException

Write one byte

Specified by:

write in class OutputStream

Throws:

IOException

write

public void write(byte[] b,
                  int off,
                  int len)
           throws IOException

Writes len bytes from the specified byte array starting at offset off and generate a checksum for each data chunk.

This method stores bytes from the given array into this stream's buffer before it gets checksumed. The buffer gets checksumed and flushed to the underlying output stream when all data in a checksum chunk are in the buffer. If the buffer is empty and requested length is at least as large as the size of next checksum chunk size, this method will checksum and write the chunk directly to the underlying output stream. Thus it avoids uneccessary data copy.

Overrides:

write in class OutputStream

Parameters:

b - the data.

off - the start offset in the data.

len - the number of bytes to write.

Throws:

IOException - if an I/O error occurs.

flushBuffer

protected void flushBuffer()
                    throws IOException

Throws:

IOException

flushBuffer

protected void flushBuffer(boolean keep)
                    throws IOException

Throws:

IOException

convertToByteStream

public static byte[] convertToByteStream(Checksum sum,
                                         int checksumSize)

Converts a checksum integer value to a byte stream

resetChecksumChunk

protected void resetChecksumChunk(int size)

Resets existing buffer with a new one of the specified size.