Package org.apache.hadoop.hdfs

Class DFSInotifyEventInputStream

java.lang.Object
org.apache.hadoop.hdfs.DFSInotifyEventInputStream
@Public @Unstable public class DFSInotifyEventInputStream extends Object
Stream for reading inotify events. DFSInotifyEventInputStreams should not be shared among multiple threads.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final org.slf4j.Logger
    LOG
     

  • Method Summary

    Modifier and Type
    Method
    Description
    long
    getTxidsBehindEstimate()
    Return a estimate of how many transaction IDs behind the NameNode's current state this stream is.
    EventBatch
    poll()
    Returns the next batch of events in the stream or null if no new batches are currently available.
    EventBatch
    poll(long time, TimeUnit tu)
    Returns the next event batch in the stream, waiting up to the specified amount of time for a new batch.
    EventBatch
    take()
    Returns the next batch of events in the stream, waiting indefinitely if a new batch is not immediately available.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • LOG

      public static final org.slf4j.Logger LOG

  • Method Details

    • poll

      public EventBatch poll() throws IOException, MissingEventsException
      Returns the next batch of events in the stream or null if no new batches are currently available.
      Throws:
      IOException - because of network error or edit log corruption. Also possible if JournalNodes are unresponsive in the QJM setting (even one unresponsive JournalNode is enough in rare cases), so catching this exception and retrying at least a few times is recommended.
      MissingEventsException - if we cannot return the next batch in the stream because the data for the events (and possibly some subsequent events) has been deleted (generally because this stream is a very large number of transactions behind the current state of the NameNode). It is safe to continue reading from the stream after this exception is thrown The next available batch of events will be returned.

    • getTxidsBehindEstimate

      public long getTxidsBehindEstimate()
      Return a estimate of how many transaction IDs behind the NameNode's current state this stream is. Clients should periodically call this method and check if its result is steadily increasing, which indicates that they are falling behind (i.e. transaction are being generated faster than the client is reading them). If a client falls too far behind events may be deleted before the client can read them.

      A return value of -1 indicates that an estimate could not be produced, and should be ignored. The value returned by this method is really only useful when compared to previous or subsequent returned values.

    • poll

      public EventBatch poll(long time, TimeUnit tu) throws IOException, InterruptedException, MissingEventsException
      Returns the next event batch in the stream, waiting up to the specified amount of time for a new batch. Returns null if one is not available at the end of the specified amount of time. The time before the method returns may exceed the specified amount of time by up to the time required for an RPC to the NameNode.
      Parameters:
      time - number of units of the given TimeUnit to wait
      tu - the desired TimeUnit
      Throws:
      IOException - see poll()
      MissingEventsException - see poll()
      InterruptedException - if the calling thread is interrupted

    • take

      public EventBatch take() throws IOException, InterruptedException, MissingEventsException
      Returns the next batch of events in the stream, waiting indefinitely if a new batch is not immediately available.
      Throws:
      IOException - see poll()
      MissingEventsException - see poll()
      InterruptedException - if the calling thread is interrupted