org.apache.hadoop.hdfs

Class DFSInotifyEventInputStream

java.lang.Object

org.apache.hadoop.hdfs.DFSInotifyEventInputStream

@InterfaceAudience.Public
 @InterfaceStability.Unstable
public class DFSInotifyEventInputStream
extends Object

Stream for reading inotify events. DFSInotifyEventInputStreams should not be shared among multiple threads.

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Method Summary

Modifier and Type	Method and Description
long	getTxidsBehindEstimate() Return a estimate of how many transaction IDs behind the NameNode's current state this stream is.
EventBatch	poll() Returns the next batch of events in the stream or null if no new batches are currently available.
EventBatch	poll(long time, TimeUnit tu) Returns the next event batch in the stream, waiting up to the specified amount of time for a new batch.
EventBatch	take() Returns the next batch of events in the stream, waiting indefinitely if a new batch is not immediately available.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Method Detail

poll

public EventBatch poll()
                throws IOException,
                       MissingEventsException

Returns the next batch of events in the stream or null if no new batches are currently available.

Throws:

IOException - because of network error or edit log corruption. Also possible if JournalNodes are unresponsive in the QJM setting (even one unresponsive JournalNode is enough in rare cases), so catching this exception and retrying at least a few times is recommended.

MissingEventsException - if we cannot return the next batch in the stream because the data for the events (and possibly some subsequent events) has been deleted (generally because this stream is a very large number of transactions behind the current state of the NameNode). It is safe to continue reading from the stream after this exception is thrown The next available batch of events will be returned.

getTxidsBehindEstimate

public long getTxidsBehindEstimate()

Return a estimate of how many transaction IDs behind the NameNode's current state this stream is. Clients should periodically call this method and check if its result is steadily increasing, which indicates that they are falling behind (i.e. transaction are being generated faster than the client is reading them). If a client falls too far behind events may be deleted before the client can read them.

A return value of -1 indicates that an estimate could not be produced, and should be ignored. The value returned by this method is really only useful when compared to previous or subsequent returned values.

poll

public EventBatch poll(long time,
                       TimeUnit tu)
                throws IOException,
                       InterruptedException,
                       MissingEventsException

Returns the next event batch in the stream, waiting up to the specified amount of time for a new batch. Returns null if one is not available at the end of the specified amount of time. The time before the method returns may exceed the specified amount of time by up to the time required for an RPC to the NameNode.

Parameters:

time - number of units of the given TimeUnit to wait

tu - the desired TimeUnit

Throws:

IOException - see poll()

MissingEventsException - see poll()

InterruptedException - if the calling thread is interrupted

take

public EventBatch take()
                throws IOException,
                       InterruptedException,
                       MissingEventsException

Returns the next batch of events in the stream, waiting indefinitely if a new batch is not immediately available.

Throws:

IOException - see poll()

MissingEventsException - see poll()

InterruptedException - if the calling thread is interrupted