org.apache.hadoop.mapred
Class JobHistory.JobInfo

java.lang.Object
  org.apache.hadoop.mapred.JobHistory.JobInfo

Enclosing class:

JobHistory

public static class JobHistory.JobInfo
extends Object

Helper class for logging or reading back events related to job start, finish or failure.

Constructor Summary

JobHistory.JobInfo(String jobId)
Create new JobInfo

Method Summary

static String	decodeJobHistoryFileName(String logFileName) Helper function to decode the URL of the filename of the job-history log file.
static String	encodeJobHistoryFileName(String logFileName) Helper function to encode the URL of the filename of the job-history log file.
static String	encodeJobHistoryFilePath(String logFile) Helper function to encode the URL of the path of the job-history log file.
String	get(JobHistory.Keys k) Get 'String' value for given key.
Map<String,JobHistory.Task>	getAllTasks() Returns all map and reduce tasks .
int	getInt(JobHistory.Keys k) Convert value from history to int and return.
Map<JobACL,AccessControlList>	getJobACLs() Get the job acls.
static String	getJobHistoryFileName(JobConf jobConf, JobID id) Recover the job history filename from the history folder.
static Path	getJobHistoryLogLocation(String logFileName) Get the job history file path given the history filename
static Path	getJobHistoryLogLocationForUser(String logFileName, JobConf jobConf) Get the user job history file path
static String	getLocalJobFilePath(JobID jobId) Get the path of the locally stored job file
long	getLong(JobHistory.Keys k) Convert value from history to int and return.
static String	getUserName(JobConf jobConf) Get the user name from the job conf
Map<JobHistory.Keys,String>	getValues() Returns Map containing all key-values.
void	handle(Map<JobHistory.Keys,String> values) Reads values back from the history, input is same Map as passed to Listener by parseHistory().
static void	logFailed(JobID jobid, long timestamp, int finishedMaps, int finishedReduces, String failReason) Logs job failed event.
static void	logFinished(JobID jobId, long finishTime, int finishedMaps, int finishedReduces, int failedMaps, int failedReduces, Counters mapCounters, Counters reduceCounters, Counters counters) Log job finished.
static void	logInited(JobID jobId, long startTime, int totalMaps, int totalReduces) Logs launch time of job.
static void	logJobInfo(JobID jobid, long submitTime, long launchTime)
static void	logJobInfo(JobID jobid, long submitTime, long launchTime, int restartCount) Deprecated. Use logJobInfo(JobID, long, long) instead.
static void	logJobPriority(JobID jobid, JobPriority priority) Log job's priority.
static void	logKilled(JobID jobid, long timestamp, int finishedMaps, int finishedReduces) Logs job killed event.
static void	logStarted(JobID jobId) Logs job as running
static void	logStarted(JobID jobId, long startTime, int totalMaps, int totalReduces) Deprecated. Use logInited(JobID, long, int, int) and logStarted(JobID)
static void	logSubmitted(JobID jobId, JobConf jobConf, String jobConfPath, long submitTime) Deprecated. Use logSubmitted(JobID, JobConf, String, long, boolean) instead.
static void	logSubmitted(JobID jobId, JobConf jobConf, String jobConfPath, long submitTime, boolean restarted)
static Path	recoverJobHistoryFile(JobConf conf, Path logFilePath) Selects one of the two files generated as a part of recovery.
void	set(JobHistory.Keys k, String s) Set value for the key.
void	set(Map<JobHistory.Keys,String> m) Adds all values in the Map argument to its own values.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JobHistory.JobInfo

public JobHistory.JobInfo(String jobId)

Create new JobInfo

Method Detail

getAllTasks

public Map<String,JobHistory.Task> getAllTasks()

Returns all map and reduce tasks .

getJobACLs

public Map<JobACL,AccessControlList> getJobACLs()

Get the job acls.

Returns:

a Map from JobACL to AccessControlList

handle

public void handle(Map<JobHistory.Keys,String> values)

Reads values back from the history, input is same Map as passed to Listener by parseHistory().

getLocalJobFilePath

public static String getLocalJobFilePath(JobID jobId)

Get the path of the locally stored job file

Parameters:

jobId - id of the job

Returns:

the path of the job file on the local file system

encodeJobHistoryFilePath

public static String encodeJobHistoryFilePath(String logFile)
                                       throws IOException

Helper function to encode the URL of the path of the job-history log file.

Parameters:

logFile - path of the job-history file

Returns:

URL encoded path

Throws:

IOException

encodeJobHistoryFileName

public static String encodeJobHistoryFileName(String logFileName)
                                       throws IOException

Helper function to encode the URL of the filename of the job-history log file.

Parameters:

logFileName - file name of the job-history file

Returns:

URL encoded filename

Throws:

IOException

decodeJobHistoryFileName

public static String decodeJobHistoryFileName(String logFileName)
                                       throws IOException

Helper function to decode the URL of the filename of the job-history log file.

Parameters:

logFileName - file name of the job-history file

Returns:

URL decoded filename

Throws:

IOException

getUserName

public static String getUserName(JobConf jobConf)

Get the user name from the job conf

getJobHistoryLogLocation

public static Path getJobHistoryLogLocation(String logFileName)

Get the job history file path given the history filename

getJobHistoryLogLocationForUser

public static Path getJobHistoryLogLocationForUser(String logFileName,
                                                   JobConf jobConf)

Get the user job history file path

getJobHistoryFileName

public static String getJobHistoryFileName(JobConf jobConf,
                                           JobID id)
                                    throws IOException

Recover the job history filename from the history folder. Uses the following pattern $jt-hostname_[0-9]*_$job-id_$user_$job-name*

Parameters:

jobConf - the job conf

id - job id

Throws:

IOException

recoverJobHistoryFile

public static Path recoverJobHistoryFile(JobConf conf,
                                         Path logFilePath)
                                  throws IOException

Selects one of the two files generated as a part of recovery. The thumb rule is that always select the oldest file. This call makes sure that only one file is left in the end.

Parameters:

conf - job conf

logFilePath - Path of the log file

Throws:

IOException

logSubmitted

@Deprecated
public static void logSubmitted(JobID jobId,
                                           JobConf jobConf,
                                           String jobConfPath,
                                           long submitTime)
                         throws IOException

Deprecated. Use logSubmitted(JobID, JobConf, String, long, boolean) instead.

Log job submitted event to history. Creates a new file in history for the job. if history file creation fails, it disables history for all other events.

Parameters:

jobId - job id assigned by job tracker.

jobConf - job conf of the job

jobConfPath - path to job conf xml file in HDFS.

submitTime - time when job tracker received the job

Throws:

IOException

logSubmitted

public static void logSubmitted(JobID jobId,
                                JobConf jobConf,
                                String jobConfPath,
                                long submitTime,
                                boolean restarted)
                         throws IOException

Throws:

IOException

logInited

public static void logInited(JobID jobId,
                             long startTime,
                             int totalMaps,
                             int totalReduces)

Logs launch time of job.

Parameters:

jobId - job id, assigned by jobtracker.

startTime - start time of job.

totalMaps - total maps assigned by jobtracker.

totalReduces - total reduces.

logStarted

@Deprecated
public static void logStarted(JobID jobId,
                                         long startTime,
                                         int totalMaps,
                                         int totalReduces)

Deprecated. Use logInited(JobID, long, int, int) and logStarted(JobID)

Logs the job as RUNNING.

Parameters:

jobId - job id, assigned by jobtracker.

startTime - start time of job.

totalMaps - total maps assigned by jobtracker.

totalReduces - total reduces.

logStarted

public static void logStarted(JobID jobId)

Logs job as running

Parameters:

jobId - job id, assigned by jobtracker.

logFinished

public static void logFinished(JobID jobId,
                               long finishTime,
                               int finishedMaps,
                               int finishedReduces,
                               int failedMaps,
                               int failedReduces,
                               Counters mapCounters,
                               Counters reduceCounters,
                               Counters counters)

Log job finished. closes the job file in history.

Parameters:

jobId - job id, assigned by jobtracker.

finishTime - finish time of job in ms.

finishedMaps - no of maps successfully finished.

finishedReduces - no of reduces finished sucessfully.

failedMaps - no of failed map tasks.

failedReduces - no of failed reduce tasks.

counters - the counters from the job

logFailed

public static void logFailed(JobID jobid,
                             long timestamp,
                             int finishedMaps,
                             int finishedReduces,
                             String failReason)

Logs job failed event. Closes the job history log file.

Parameters:

jobid - job id

timestamp - time when job failure was detected in ms.

finishedMaps - no finished map tasks.

finishedReduces - no of finished reduce tasks.

logKilled

public static void logKilled(JobID jobid,
                             long timestamp,
                             int finishedMaps,
                             int finishedReduces)

Logs job killed event. Closes the job history log file.

Parameters:

jobid - job id

timestamp - time when job killed was issued in ms.

finishedMaps - no finished map tasks.

finishedReduces - no of finished reduce tasks.

logJobPriority

public static void logJobPriority(JobID jobid,
                                  JobPriority priority)

Log job's priority.

Parameters:

jobid - job id

priority - Jobs priority

logJobInfo

@Deprecated
public static void logJobInfo(JobID jobid,
                                         long submitTime,
                                         long launchTime,
                                         int restartCount)

Deprecated. Use logJobInfo(JobID, long, long) instead.

Log job's submit-time/launch-time

Parameters:

jobid - job id

submitTime - job's submit time

launchTime - job's launch time

restartCount - number of times the job got restarted

logJobInfo

public static void logJobInfo(JobID jobid,
                              long submitTime,
                              long launchTime)

get

public String get(JobHistory.Keys k)

Get 'String' value for given key. Most of the places use Strings as values so the default get' method returns 'String'. This method never returns null to ease on GUIs. if no value is found it returns empty string ""

Parameters:

k -

Returns:

if null it returns empty string - ""

getInt

public int getInt(JobHistory.Keys k)

Convert value from history to int and return. if no value is found it returns 0.

Parameters:

k - key

getLong

public long getLong(JobHistory.Keys k)

Convert value from history to int and return. if no value is found it returns 0.

Parameters:

k -

set

public void set(JobHistory.Keys k,
                String s)

Set value for the key.

Parameters:

k -

s -

set

public void set(Map<JobHistory.Keys,String> m)

Adds all values in the Map argument to its own values.

Parameters:

m -

getValues

public Map<JobHistory.Keys,String> getValues()

Returns Map containing all key-values.