Package org.apache.hadoop.metrics

This package defines an API for reporting performance metric information.

See: Description

Package org.apache.hadoop.metrics Description

This package defines an API for reporting performance metric information.

The API is abstract so that it can be implemented on top of a variety of metrics client libraries. The choice of client library is a configuration option, and different modules within the same application can use different metrics implementation libraries.

Sub-packages:

org.apache.hadoop.metrics.spi

The abstract Server Provider Interface package. Those wishing to integrate the metrics API with a particular metrics client library should extend this package.

org.apache.hadoop.metrics.file

An implementation package which writes the metric data to a file, or sends it to the standard output stream.

org.apache.hadoop.metrics.ganglia

An implementation package which sends metric data to Ganglia.

Introduction to the Metrics API

Here is a simple example of how to use this package to report a single metric value:

    private ContextFactory contextFactory = ContextFactory.getFactory();
    
    void reportMyMetric(float myMetric) {
        MetricsContext myContext = contextFactory.getContext("myContext");
        MetricsRecord myRecord = myContext.getRecord("myRecord");
        myRecord.setMetric("myMetric", myMetric);
        myRecord.update();
    }

In this example there are three names:

myContext

The context name will typically identify either the application, or else a module within an application or library.

myRecord

The record name generally identifies some entity for which a set of metrics are to be reported. For example, you could have a record named "cacheStats" for reporting a number of statistics relating to the usage of some cache in your application.

myMetric

This identifies a particular metric. For example, you might have metrics named "cache_hits" and "cache_misses".

Tags

In some cases it is useful to have multiple records with the same name. For example, suppose that you want to report statistics about each disk on a computer. In this case, the record name would be something like "diskStats", but you also need to identify the disk which is done by adding a tag to the record. The code could look something like this:

    private MetricsRecord diskStats =
            contextFactory.getContext("myContext").getRecord("diskStats");
            
    void reportDiskMetrics(String diskName, float diskBusy, float diskUsed) {
        diskStats.setTag("diskName", diskName);
        diskStats.setMetric("diskBusy", diskBusy);
        diskStats.setMetric("diskUsed", diskUsed);
        diskStats.update();
    }

Buffering and Callbacks

Data is not sent immediately to the metrics system when MetricsRecord.update() is called. Instead it is stored in an internal table, and the contents of the table are sent periodically. This can be important for two reasons:

1. It means that a programmer is free to put calls to this API in an inner loop, since updates can be very frequent without slowing down the application significantly.
2. Some implementations can gain efficiency by combining many metrics into a single UDP message.

The API provides a timer-based callback via the registerUpdater() method. The benefit of this versus using java.util.Timer is that the callbacks will be done immediately before sending the data, making the data as current as possible.

Configuration

It is possible to programmatically examine and modify configuration data before creating a context, like this:

    ContextFactory factory = ContextFactory.getFactory();
    ... examine and/or modify factory attributes ...
    MetricsContext context = factory.getContext("myContext");

The factory attributes can be examined and modified using the following ContextFactorymethods:

• Object getAttribute(String attributeName)
• String[] getAttributeNames()
• void setAttribute(String name, Object value)
• void removeAttribute(attributeName)

ContextFactory.getFactory() initializes the factory attributes by reading the properties file hadoop-metrics.properties if it exists on the class path.

A factory attribute named:

contextName.class

should have as its value the fully qualified name of the class to be instantiated by a call of the CodeFactory method getContext(contextName). If this factory attribute is not specified, the default is to instantiate org.apache.hadoop.metrics.file.FileContext.

Other factory attributes are specific to a particular implementation of this API and are documented elsewhere. For example, configuration attributes for the file and Ganglia implementations can be found in the javadoc for their respective packages.