Package org.apache.hadoop.yarn.util

Class Log4jWarningErrorMetricsAppender

java.lang.Object
org.apache.log4j.AppenderSkeleton
org.apache.hadoop.yarn.util.Log4jWarningErrorMetricsAppender
All Implemented Interfaces:
org.apache.log4j.Appender, org.apache.log4j.spi.OptionHandler
@Private @Unstable public class Log4jWarningErrorMetricsAppender extends org.apache.log4j.AppenderSkeleton

  • Field Details

  • Constructor Details

    • Log4jWarningErrorMetricsAppender

      public Log4jWarningErrorMetricsAppender()
      Create an appender to keep track of the errors and warnings logged by the system. It will keep purge messages older than 2 days. It will store upto the last 500 unique errors and the last 500 unique warnings. The thread to purge message will run every 5 minutes, unless the 500 message limit is hit earlier.

    • Log4jWarningErrorMetricsAppender

      public Log4jWarningErrorMetricsAppender(int cleanupIntervalSeconds, long messageAgeLimitSeconds, int maxUniqueMessages)
      Create an appender to keep track of the errors and warnings logged by the system.
      Parameters:
      cleanupIntervalSeconds - the interval at which old messages are purged to prevent the message stores from growing unbounded
      messageAgeLimitSeconds - the maximum age of a message in seconds before it is purged from the store
      maxUniqueMessages - the maximum number of unique messages of each type we keep before we start purging

  • Method Details

    • append

      protected void append(org.apache.log4j.spi.LoggingEvent event)
      Specified by:
      append in class org.apache.log4j.AppenderSkeleton

    • close

      public void close()

    • requiresLayout

      public boolean requiresLayout()

    • getErrorCounts

      public List<Integer> getErrorCounts(List<Long> cutoffs)
      Get the counts of errors in the time periods provided. Note that the counts provided by this function may differ from the ones provided by getErrorMessagesAndCounts since the message store is purged at regular intervals to prevent it from growing without bounds, while the store for the counts is purged less frequently.
      Parameters:
      cutoffs - list of timestamp cutoffs(in seconds) for which the counts are desired
      Returns:
      list of error counts in the time periods corresponding to cutoffs

    • getWarningCounts

      public List<Integer> getWarningCounts(List<Long> cutoffs)
      Get the counts of warnings in the time periods provided. Note that the counts provided by this function may differ from the ones provided by getWarningMessagesAndCounts since the message store is purged at regular intervals to prevent it from growing without bounds, while the store for the counts is purged less frequently.
      Parameters:
      cutoffs - list of timestamp cutoffs(in seconds) for which the counts are desired
      Returns:
      list of warning counts in the time periods corresponding to cutoffs

    • getErrorMessagesAndCounts

      public List<Map<String,Log4jWarningErrorMetricsAppender.Element>> getErrorMessagesAndCounts(List<Long> cutoffs)
      Get the errors and the number of occurrences for each of the errors for the time cutoffs provided. Note that the counts provided by this function may differ from the ones provided by getErrorCounts since the message store is purged at regular intervals to prevent it from growing without bounds, while the store for the counts is purged less frequently.
      Parameters:
      cutoffs - list of timestamp cutoffs(in seconds) for which the counts are desired
      Returns:
      list of maps corresponding for each cutoff provided; each map contains the error and the number of times the error occurred in the time period

    • getWarningMessagesAndCounts

      public List<Map<String,Log4jWarningErrorMetricsAppender.Element>> getWarningMessagesAndCounts(List<Long> cutoffs)
      Get the warning and the number of occurrences for each of the warnings for the time cutoffs provided. Note that the counts provided by this function may differ from the ones provided by getWarningCounts since the message store is purged at regular intervals to prevent it from growing without bounds, while the store for the counts is purged less frequently.
      Parameters:
      cutoffs - list of timestamp cutoffs(in seconds) for which the counts are desired
      Returns:
      list of maps corresponding for each cutoff provided; each map contains the warning and the number of times the error occurred in the time period

    • getCleanupInterval

      public long getCleanupInterval()

    • setCleanupInterval

      public void setCleanupInterval(long cleanupInterval)

    • getMessageAgeLimitSeconds

      public long getMessageAgeLimitSeconds()

    • setMessageAgeLimitSeconds

      public void setMessageAgeLimitSeconds(long messageAgeLimitSeconds)

    • getMaxUniqueMessages

      public int getMaxUniqueMessages()

    • setMaxUniqueMessages

      public void setMaxUniqueMessages(int maxUniqueMessages)

    • findAppender

      public static Log4jWarningErrorMetricsAppender findAppender()