org.apache.hadoop.mapred
Class ShuffleExceptionTracker

java.lang.Object
  org.apache.hadoop.mapred.ShuffleExceptionTracker

public class ShuffleExceptionTracker
extends Object

This class is used to track shuffle exceptions. It contains routines to check an exception that occurred while fetching the Map output to see if it matches what was configured. It also provides functions to keep track of the number of exceptions that occurred and if a limit is set, it will abort the TT. The limit is a percent of exceptions of the last X number of requests.

Field Summary

static org.apache.commons.logging.Log

LOG

Method Summary

boolean	checkException(IOException ie) Parse the exception to see if it matches the regular expression you configured.
protected void	doAbort()
void	exception() Mark the request as an exception occurred.
int	getNumRequests() Gets the number of requests we are tracking
float	getPercentExceptions() Gets the percent of the requests that had exceptions occur.
void	success() Mark the request as success.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

Method Detail

getNumRequests

public int getNumRequests()

Gets the number of requests we are tracking

Returns:

number of requests

getPercentExceptions

public float getPercentExceptions()

Gets the percent of the requests that had exceptions occur.

Returns:

percent failures as float

success

public void success()

Mark the request as success.

exception

public void exception()

Mark the request as an exception occurred.

checkException

public boolean checkException(IOException ie)

Parse the exception to see if it matches the regular expression you configured. If both msgRegex and StackRegex are set then make sure both match, otherwise only one has to match. Abort if the limit is hit.

Parameters:

ie - - the shuffle exception that occurred

Returns:

true if the exception matches, false otherwise

doAbort

protected void doAbort()