public interface PlanFollower extends Runnable
ResourceScheduler
with the Plan
(s) and viceversa.
While different implementations might operate differently, the key idea is to
map the current allocation of resources for each active reservation in the
plan(s), to a corresponding notion in the underlying scheduler (e.g., tuning
capacity of queues, set pool weights, or tweak application priorities). The
goal is to affect the dynamic allocation of resources done by the scheduler
so that the jobs obtain access to resources in a way that is consistent with
the reservations in the plan. A key conceptual step here is to convert the
absolute-valued promises made in the reservations to appropriate relative
priorities/queue sizes etc.
Symmetrically the PlanFollower exposes changes in cluster conditions (as
tracked by the scheduler) to the plan, e.g., the overall amount of physical
resources available. The Plan in turn can react by replanning its allocations
if appropriate.
The implementation can assume that is run frequently enough to be able to
observe and react to normal operational changes in cluster conditions on the
fly (e.g., if cluster resources drop, we can update the relative weights of a
queue so that the absolute promises made to the job at reservation time are
respected).
However, due to RM restarts and the related downtime, it is advisable for
implementations to operate in a stateless way, and be able to synchronize the
state of plans/scheduler regardless of how big is the time gap between
executions.Modifier and Type | Method and Description |
---|---|
void |
init(org.apache.hadoop.yarn.util.Clock clock,
ResourceScheduler sched,
Collection<Plan> plans)
Init function that configures the PlanFollower, by providing:
|
void |
setPlans(Collection<Plan> plans)
Setter for the list of plans.
|
void |
synchronizePlan(Plan plan,
boolean shouldReplan)
The function performing the actual synchronization operation for a given
Plan.
|
void init(org.apache.hadoop.yarn.util.Clock clock, ResourceScheduler sched, Collection<Plan> plans)
clock
- a reference to the system clock.sched
- a reference to the underlying schedulerplans
- references to the plans we should keep synchronized at every
time tick.void synchronizePlan(Plan plan, boolean shouldReplan)
plan
- the Plan to synchronizeshouldReplan
- replan on reduction of plan capacity if true or
proportionally scale down reservations if falsevoid setPlans(Collection<Plan> plans)
plans
- the collection of Plans we operate on at every time tick.Copyright © 2017 Apache Software Foundation. All Rights Reserved.