org.apache.hadoop.mapred
Class JobHistory.JobInfo

java.lang.Object
  extended by org.apache.hadoop.mapred.JobHistory.JobInfo
Enclosing class:
JobHistory

public static class JobHistory.JobInfo
extends Object

Helper class for logging or reading back events related to job start, finish or failure.


Constructor Summary
JobHistory.JobInfo(String jobId)
          Create new JobInfo
 
Method Summary
static String decodeJobHistoryFileName(String logFileName)
          Helper function to decode the URL of the filename of the job-history log file.
static String encodeJobHistoryFileName(String logFileName)
          Helper function to encode the URL of the filename of the job-history log file.
static String encodeJobHistoryFilePath(String logFile)
          Helper function to encode the URL of the path of the job-history log file.
 String get(JobHistory.Keys k)
          Get 'String' value for given key.
 Map<String,JobHistory.Task> getAllTasks()
          Returns all map and reduce tasks .
 int getInt(JobHistory.Keys k)
          Convert value from history to int and return.
 Map<JobACL,AccessControlList> getJobACLs()
          Get the job acls.
static String getJobHistoryFileName(JobConf jobConf, JobID id)
          Recover the job history filename from the history folder.
static Path getJobHistoryLogLocation(String logFileName)
          Get the job history file path given the history filename
static Path getJobHistoryLogLocationForUser(String logFileName, JobConf jobConf)
          Get the user job history file path
static String getLocalJobFilePath(JobID jobId)
          Get the path of the locally stored job file
 long getLong(JobHistory.Keys k)
          Convert value from history to int and return.
static String getUserName(JobConf jobConf)
          Get the user name from the job conf
 Map<JobHistory.Keys,String> getValues()
          Returns Map containing all key-values.
static String getWorkflowAdjacencies(Configuration conf)
          Get the workflow adjacencies from the job conf The string returned is of the form "key"="value" "key"="value" ...
 void handle(Map<JobHistory.Keys,String> values)
          Reads values back from the history, input is same Map as passed to Listener by parseHistory().
static void logFailed(JobID jobid, long timestamp, int finishedMaps, int finishedReduces, String failReason)
          Logs job failed event.
static void logFinished(JobID jobId, long finishTime, int finishedMaps, int finishedReduces, int failedMaps, int failedReduces, Counters mapCounters, Counters reduceCounters, Counters counters)
          Log job finished.
static void logInited(JobID jobId, long startTime, int totalMaps, int totalReduces)
          Logs launch time of job.
static void logJobInfo(JobID jobid, long submitTime, long launchTime)
           
static void logJobInfo(JobID jobid, long submitTime, long launchTime, int restartCount)
          Deprecated. Use logJobInfo(JobID, long, long) instead.
static void logJobPriority(JobID jobid, JobPriority priority)
          Log job's priority.
static void logKilled(JobID jobid, long timestamp, int finishedMaps, int finishedReduces)
          Logs job killed event.
static void logStarted(JobID jobId)
          Logs job as running
static void logStarted(JobID jobId, long startTime, int totalMaps, int totalReduces)
          Deprecated. Use logInited(JobID, long, int, int) and logStarted(JobID)
static void logSubmitted(JobID jobId, JobConf jobConf, String jobConfPath, long submitTime)
          Deprecated. Use logSubmitted(JobID, JobConf, String, long, boolean) instead.
static void logSubmitted(JobID jobId, JobConf jobConf, String jobConfPath, long submitTime, boolean restarted)
           
static Path recoverJobHistoryFile(JobConf conf, Path logFilePath)
          Selects one of the two files generated as a part of recovery.
 void set(JobHistory.Keys k, String s)
          Set value for the key.
 void set(Map<JobHistory.Keys,String> m)
          Adds all values in the Map argument to its own values.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

JobHistory.JobInfo

public JobHistory.JobInfo(String jobId)
Create new JobInfo

Method Detail

getAllTasks

public Map<String,JobHistory.Task> getAllTasks()
Returns all map and reduce tasks .


getJobACLs

public Map<JobACL,AccessControlList> getJobACLs()
Get the job acls.

Returns:
a Map from JobACL to AccessControlList

handle

public void handle(Map<JobHistory.Keys,String> values)
Reads values back from the history, input is same Map as passed to Listener by parseHistory().


getLocalJobFilePath

public static String getLocalJobFilePath(JobID jobId)
Get the path of the locally stored job file

Parameters:
jobId - id of the job
Returns:
the path of the job file on the local file system

encodeJobHistoryFilePath

public static String encodeJobHistoryFilePath(String logFile)
                                       throws IOException
Helper function to encode the URL of the path of the job-history log file.

Parameters:
logFile - path of the job-history file
Returns:
URL encoded path
Throws:
IOException

encodeJobHistoryFileName

public static String encodeJobHistoryFileName(String logFileName)
                                       throws IOException
Helper function to encode the URL of the filename of the job-history log file.

Parameters:
logFileName - file name of the job-history file
Returns:
URL encoded filename
Throws:
IOException

decodeJobHistoryFileName

public static String decodeJobHistoryFileName(String logFileName)
                                       throws IOException
Helper function to decode the URL of the filename of the job-history log file.

Parameters:
logFileName - file name of the job-history file
Returns:
URL decoded filename
Throws:
IOException

getUserName

public static String getUserName(JobConf jobConf)
Get the user name from the job conf


getWorkflowAdjacencies

public static String getWorkflowAdjacencies(Configuration conf)
Get the workflow adjacencies from the job conf The string returned is of the form "key"="value" "key"="value" ...


getJobHistoryLogLocation

public static Path getJobHistoryLogLocation(String logFileName)
Get the job history file path given the history filename


getJobHistoryLogLocationForUser

public static Path getJobHistoryLogLocationForUser(String logFileName,
                                                   JobConf jobConf)
Get the user job history file path


getJobHistoryFileName

public static String getJobHistoryFileName(JobConf jobConf,
                                           JobID id)
                                    throws IOException
Recover the job history filename from the history folder. Uses the following pattern $jt-hostname_[0-9]*_$job-id_$user_$job-name*

Parameters:
jobConf - the job conf
id - job id
Throws:
IOException

recoverJobHistoryFile

public static Path recoverJobHistoryFile(JobConf conf,
                                         Path logFilePath)
                                  throws IOException
Selects one of the two files generated as a part of recovery. The thumb rule is that always select the oldest file. This call makes sure that only one file is left in the end.

Parameters:
conf - job conf
logFilePath - Path of the log file
Throws:
IOException

logSubmitted

@Deprecated
public static void logSubmitted(JobID jobId,
                                           JobConf jobConf,
                                           String jobConfPath,
                                           long submitTime)
                         throws IOException
Deprecated. Use logSubmitted(JobID, JobConf, String, long, boolean) instead.

Log job submitted event to history. Creates a new file in history for the job. if history file creation fails, it disables history for all other events.

Parameters:
jobId - job id assigned by job tracker.
jobConf - job conf of the job
jobConfPath - path to job conf xml file in HDFS.
submitTime - time when job tracker received the job
Throws:
IOException

logSubmitted

public static void logSubmitted(JobID jobId,
                                JobConf jobConf,
                                String jobConfPath,
                                long submitTime,
                                boolean restarted)
                         throws IOException
Throws:
IOException

logInited

public static void logInited(JobID jobId,
                             long startTime,
                             int totalMaps,
                             int totalReduces)
Logs launch time of job.

Parameters:
jobId - job id, assigned by jobtracker.
startTime - start time of job.
totalMaps - total maps assigned by jobtracker.
totalReduces - total reduces.

logStarted

@Deprecated
public static void logStarted(JobID jobId,
                                         long startTime,
                                         int totalMaps,
                                         int totalReduces)
Deprecated. Use logInited(JobID, long, int, int) and logStarted(JobID)

Logs the job as RUNNING.

Parameters:
jobId - job id, assigned by jobtracker.
startTime - start time of job.
totalMaps - total maps assigned by jobtracker.
totalReduces - total reduces.

logStarted

public static void logStarted(JobID jobId)
Logs job as running

Parameters:
jobId - job id, assigned by jobtracker.

logFinished

public static void logFinished(JobID jobId,
                               long finishTime,
                               int finishedMaps,
                               int finishedReduces,
                               int failedMaps,
                               int failedReduces,
                               Counters mapCounters,
                               Counters reduceCounters,
                               Counters counters)
Log job finished. closes the job file in history.

Parameters:
jobId - job id, assigned by jobtracker.
finishTime - finish time of job in ms.
finishedMaps - no of maps successfully finished.
finishedReduces - no of reduces finished sucessfully.
failedMaps - no of failed map tasks.
failedReduces - no of failed reduce tasks.
counters - the counters from the job

logFailed

public static void logFailed(JobID jobid,
                             long timestamp,
                             int finishedMaps,
                             int finishedReduces,
                             String failReason)
Logs job failed event. Closes the job history log file.

Parameters:
jobid - job id
timestamp - time when job failure was detected in ms.
finishedMaps - no finished map tasks.
finishedReduces - no of finished reduce tasks.

logKilled

public static void logKilled(JobID jobid,
                             long timestamp,
                             int finishedMaps,
                             int finishedReduces)
Logs job killed event. Closes the job history log file.

Parameters:
jobid - job id
timestamp - time when job killed was issued in ms.
finishedMaps - no finished map tasks.
finishedReduces - no of finished reduce tasks.

logJobPriority

public static void logJobPriority(JobID jobid,
                                  JobPriority priority)
Log job's priority.

Parameters:
jobid - job id
priority - Jobs priority

logJobInfo

@Deprecated
public static void logJobInfo(JobID jobid,
                                         long submitTime,
                                         long launchTime,
                                         int restartCount)
Deprecated. Use logJobInfo(JobID, long, long) instead.

Log job's submit-time/launch-time

Parameters:
jobid - job id
submitTime - job's submit time
launchTime - job's launch time
restartCount - number of times the job got restarted

logJobInfo

public static void logJobInfo(JobID jobid,
                              long submitTime,
                              long launchTime)

get

public String get(JobHistory.Keys k)
Get 'String' value for given key. Most of the places use Strings as values so the default get' method returns 'String'. This method never returns null to ease on GUIs. if no value is found it returns empty string ""

Parameters:
k -
Returns:
if null it returns empty string - ""

getInt

public int getInt(JobHistory.Keys k)
Convert value from history to int and return. if no value is found it returns 0.

Parameters:
k - key

getLong

public long getLong(JobHistory.Keys k)
Convert value from history to int and return. if no value is found it returns 0.

Parameters:
k -

set

public void set(JobHistory.Keys k,
                String s)
Set value for the key.

Parameters:
k -
s -

set

public void set(Map<JobHistory.Keys,String> m)
Adds all values in the Map argument to its own values.

Parameters:
m -

getValues

public Map<JobHistory.Keys,String> getValues()
Returns Map containing all key-values.



Copyright © 2009 The Apache Software Foundation