public class SimpleExponentialTaskRuntimeEstimator extends Object
Modifier and Type | Field and Description |
---|---|
protected AppContext |
context |
protected Set<Task> |
doneTasks |
protected Map<Job,DataStatistics> |
mapperStatistics |
protected Map<Job,DataStatistics> |
reducerStatistics |
protected Map<org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId,Long> |
startTimes |
Constructor and Description |
---|
SimpleExponentialTaskRuntimeEstimator() |
Modifier and Type | Method and Description |
---|---|
long |
attemptEnrolledTime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID) |
void |
contextualize(org.apache.hadoop.conf.Configuration conf,
AppContext context) |
protected DataStatistics |
dataStatisticsForTask(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID) |
void |
enrollAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status,
long timestamp) |
long |
estimatedNewAttemptRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId id)
Estimates how long a new attempt on this task will take if we start
one now
|
long |
estimatedRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id)
Estimate a task attempt's total runtime.
|
boolean |
hasStagnatedProgress(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id,
long timeStamp)
Returns true if the estimator has no updates records for a threshold time
window.
|
long |
runtimeEstimateVariance(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id)
Computes the width of the error band of our estimate of the task
runtime as returned by
TaskRuntimeEstimator.estimatedRuntime(TaskAttemptId) |
long |
thresholdRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)
Find a maximum reasonable execution wallclock time.
|
void |
updateAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status,
long timestamp) |
protected AppContext context
protected final Map<org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId,Long> startTimes
protected final Map<Job,DataStatistics> mapperStatistics
protected final Map<Job,DataStatistics> reducerStatistics
public SimpleExponentialTaskRuntimeEstimator()
public void contextualize(org.apache.hadoop.conf.Configuration conf, AppContext context)
contextualize
in interface TaskRuntimeEstimator
public long estimatedRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id)
TaskRuntimeEstimator
id
- the TaskAttemptId
of the attempt we are asking about-1
if
we don't have enough information yet to produce an estimate.public long estimatedNewAttemptRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId id)
TaskRuntimeEstimator
estimatedNewAttemptRuntime
in interface TaskRuntimeEstimator
id
- the TaskId
of the task we are asking about-1
if
we don't have enough information yet to produce an estimate.public boolean hasStagnatedProgress(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id, long timeStamp)
TaskRuntimeEstimator
id
- the TaskAttemptId
of the attempt we are asking abouttimeStamp
- the time of the report we compare withpublic long runtimeEstimateVariance(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id)
TaskRuntimeEstimator
TaskRuntimeEstimator.estimatedRuntime(TaskAttemptId)
id
- the TaskAttemptId
of the attempt we are asking about-1
if
we don't have enough information yet to produce an estimate.public void updateAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status, long timestamp)
updateAttempt
in interface TaskRuntimeEstimator
public void enrollAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status, long timestamp)
enrollAttempt
in interface TaskRuntimeEstimator
public long attemptEnrolledTime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
attemptEnrolledTime
in interface TaskRuntimeEstimator
protected DataStatistics dataStatisticsForTask(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)
public long thresholdRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)
TaskRuntimeEstimator
thresholdRuntime
in interface TaskRuntimeEstimator
taskID
- the TaskId
of the task we are asking aboutCopyright © 2008–2024 Apache Software Foundation. All rights reserved.