|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object java.io.InputStream org.apache.hadoop.fs.FSInputStream org.apache.hadoop.fs.FSInputChecker
public abstract class FSInputChecker
This is a generic input stream for verifying checksums for data before it is read by a user.
Field Summary | |
---|---|
protected Path |
file
The file name from which data is read from |
static org.apache.commons.logging.Log |
LOG
|
Constructor Summary | |
---|---|
protected |
FSInputChecker(Path file,
int numOfRetries)
Constructor |
protected |
FSInputChecker(Path file,
int numOfRetries,
boolean verifyChecksum,
Checksum sum,
int chunkSize,
int checksumSize)
Constructor |
Method Summary | |
---|---|
int |
available()
|
static long |
checksum2long(byte[] checksum)
Convert a checksum byte array to a long |
protected abstract long |
getChunkPosition(long pos)
Return position of beginning of chunk containing pos. |
long |
getPos()
Return the current offset from the start of the file |
void |
mark(int readlimit)
|
boolean |
markSupported()
|
protected boolean |
needChecksum()
Return true if there is a need for checksum verification |
int |
read()
Read one checksum-verified byte |
int |
read(byte[] b,
int off,
int len)
Read checksum verified bytes from this byte-input stream into the specified byte array, starting at the given offset. |
protected abstract int |
readChunk(long pos,
byte[] buf,
int offset,
int len,
byte[] checksum)
Reads in next checksum chunk data into buf at offset
and checksum into checksum . |
protected static int |
readFully(InputStream stm,
byte[] buf,
int offset,
int len)
A utility function that tries to read up to len bytes from
stm |
void |
reset()
|
void |
seek(long pos)
Seek to the given position in the stream. |
protected void |
set(boolean verifyChecksum,
Checksum sum,
int maxChunkSize,
int checksumSize)
Set the checksum related parameters |
long |
skip(long n)
Skips over and discards n bytes of data from the
input stream. |
Methods inherited from class org.apache.hadoop.fs.FSInputStream |
---|
read, readFully, readFully, seekToNewSource |
Methods inherited from class java.io.InputStream |
---|
close, read |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final org.apache.commons.logging.Log LOG
protected Path file
Constructor Detail |
---|
protected FSInputChecker(Path file, int numOfRetries)
file
- The name of the file to be readnumOfRetries
- Number of read retries when ChecksumError occursprotected FSInputChecker(Path file, int numOfRetries, boolean verifyChecksum, Checksum sum, int chunkSize, int checksumSize)
file
- The name of the file to be readnumOfRetries
- Number of read retries when ChecksumError occurssum
- the type of Checksum enginechunkSize
- maximun chunk sizechecksumSize
- the number byte of each checksumMethod Detail |
---|
protected abstract int readChunk(long pos, byte[] buf, int offset, int len, byte[] checksum) throws IOException
buf
at offset
and checksum into checksum
.
The method is used for implementing read, therefore, it should be optimized
for sequential reading
pos
- chunkPosbuf
- desitination bufferoffset
- offset in buf at which to store datalen
- maximun number of bytes to read
IOException
protected abstract long getChunkPosition(long pos)
pos
- a postion in the file
protected boolean needChecksum()
public int read() throws IOException
read
in class InputStream
-1
if the end of the
stream is reached.
IOException
- if an I/O error occurs.public int read(byte[] b, int off, int len) throws IOException
This method implements the general contract of the corresponding
method of
the read
class. As an additional
convenience, it attempts to read as many bytes as possible by repeatedly
invoking the InputStream
read
method of the underlying stream. This
iterated read
continues until one of the following
conditions becomes true:
read
method of the underlying stream returns
-1
, indicating end-of-file.
read
on the underlying stream returns
-1
to indicate end-of-file then this method returns
-1
. Otherwise this method returns the number of bytes
actually read.
read
in class InputStream
b
- destination buffer.off
- offset at which to start storing bytes.len
- maximum number of bytes to read.
-1
if the end of
the stream has been reached.
IOException
- if an I/O error occurs.
ChecksumException if any checksum error occurspublic static long checksum2long(byte[] checksum)
public long getPos() throws IOException
FSInputStream
getPos
in interface Seekable
getPos
in class FSInputStream
IOException
public int available() throws IOException
available
in class InputStream
IOException
public long skip(long n) throws IOException
n
bytes of data from the
input stream.
This method may skip more bytes than are remaining in the backing file. This produces no exception and the number of bytes skipped may include some number of bytes that were beyond the EOF of the backing file. Attempting to read from the stream after skipping past the end will result in -1 indicating the end of the file.
If n
is negative, no bytes are skipped.
skip
in class InputStream
n
- the number of bytes to be skipped.
IOException
- if an I/O error occurs.
ChecksumException if the chunk to skip to is corruptedpublic void seek(long pos) throws IOException
This method may seek past the end of the file. This produces no exception and an attempt to read from the stream will result in -1 indicating the end of the file.
seek
in interface Seekable
seek
in class FSInputStream
pos
- the postion to seek to.
IOException
- if an I/O error occurs.
ChecksumException if the chunk to seek to is corruptedprotected static int readFully(InputStream stm, byte[] buf, int offset, int len) throws IOException
len
bytes from
stm
stm
- an input streambuf
- destiniation bufferoffset
- offset at which to store datalen
- number of bytes to read
IOException
- if there is any IO errorprotected final void set(boolean verifyChecksum, Checksum sum, int maxChunkSize, int checksumSize)
verifyChecksum
- whether to verify checksumsum
- which type of checksum to usemaxChunkSize
- maximun chunk sizechecksumSize
- checksum sizepublic final boolean markSupported()
markSupported
in class InputStream
public final void mark(int readlimit)
mark
in class InputStream
public final void reset() throws IOException
reset
in class InputStream
IOException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |