All Classes and Interfaces
Class
Description
Base abstract class for
FederationAMRMProxyPolicy implementations,
that provides common validation for reinitialization.Base abstract class for a weighted
ConfigurableFederationPolicy.This class provides basic implementation for common methods that multiple
policies will need to implement.
Base abstract class for
FederationRouterPolicy implementations, that
provides common validation for reinitialization.Partial implementation of
SubClusterResolver, containing basic
implementations of the read methods.
The request sent by the
Router to Federation state
store to map the home subcluster of a newly submitted application.Protocol buffer based implementation of
AddApplicationHomeSubClusterRequest.AddApplicationHomeSubClusterResponse contains the answer from the
FederationApplicationHomeSubClusterStore to a request to insert a
newly generated applicationId and its owner.Protocol buffer based implementation of
AddApplicationHomeSubClusterResponse.
The request sent by the
Router to Federation state
store to map the home subcluster of a newly submitted Reservation.Protocol buffer based implementation of
AddReservationHomeSubClusterRequest.AddReservationHomeSubClusterResponse contains the answer from the
FederationReservationHomeSubClusterStore to a request to insert a
newly generated ReservationId and its owner.Protocol buffer based implementation of
AddReservationHomeSubClusterResponse.Extends Thread and provides an implementation that is used for processing the
AM heart beat request asynchronously and sending back the response using the
callback method registered with the system.
Data structure that encapsulates AllocateRequest and AsyncCallback
instance.
A component that sits in between AMRMClient(Impl) and Yarn RM.
Metrics for FederationInterceptor Internals.
Easier classification of request types for logging metrics.
Constants for AM Secret Keys.
Classes implementing this interface are used in the
LogServlet
for providing various application related information.
ApplicationHomeSubCluster is a report of the runtime information of the
application that is running in the federated cluster.
Protocol buffer based implementation of
ApplicationHomeSubCluster.SecretManager for ContainerTokens.
An implementation of the
FederationAMRMProxyPolicy that simply
broadcasts each ResourceRequest to all the available sub-clusters.Builder utilities to construct various objects.
The protocol between an
TimelineCollectorManager and a
NodeManager to report a new application collector get launched.This interface provides a general method to reinitialize a policy.
Records the allocation history from YarnRM and provide aggregated insights.
ContainerLogsInfo includes the log meta-data of containers.Used to hold max wait time / queue length information to be
passed back to the NodeManager.
Implementation of ContainerQueuingLimit interface.
CSI constants.
Default simple sub-cluster and rack resolver class.
The request to
Federation state store to delete the mapping of
home subcluster of a submitted application.Protocol buffer based implementation of
DeleteApplicationHomeSubClusterRequest.DeleteApplicationHomeSubClusterResponse contains the answer from the
FederationApplicationHomeSubClusterStore to a request to delete the mapping
of home subcluster of a submitted application.Protocol buffer based implementation of
DeleteApplicationHomeSubClusterResponse.This class is used for handling queue policy deletion requests.
This class is used to respond to requests to delete PoliciesConfigurations.
The request to
Federation state store to delete the mapping of
home subcluster of a submitted reservation.Protocol buffer based implementation of
DeleteReservationHomeSubClusterRequest.DeleteReservationHomeSubClusterResponse contains the answer from the
FederationReservationHomeSubClusterStore to a request to delete the mapping
of home subcluster of a submitted reservation.Protocol buffer based implementation of
DeleteReservationHomeSubClusterResponse.This class is used to respond to queue deletion requests and contains a list of queues.
Protocol buffer based implementation of
DeleteSubClusterPoliciesConfigurationsRequest.DeleteSubClusterPoliciesConfigurationsResponse contains the answer from the
FederationPolicyStore to a request to delete policy configurations
for given queues.Protocol buffer based implementation of
DeleteSubClusterPoliciesConfigurationsResponse.
The DistributedOpportunisticContainerAllocator allocates containers on a
given list of nodes, after modifying the container sizes to respect the
limits set by the ResourceManager.
Object used by the Application Master when distributed scheduling is enabled,
in order to forward the
AllocateRequest for GUARANTEED containers to
the Resource Manager, and to notify the Resource Manager about the allocation
of OPPORTUNISTIC containers through the Distributed Scheduler.Implementation of
DistributedSchedulingAllocateRequest.This is the response of the Resource Manager to the
DistributedSchedulingAllocateRequest, when distributed scheduling is
enabled.Implementation of
DistributedSchedulingAllocateResponse.
This protocol extends the
ApplicationMasterProtocol.Implementation of
DistributedSchedulingAMProtocol, used when
distributed scheduling is enabled.Implementation of
DistributedSchedulingAMProtocolPB.Implementors of this interface provide logic to split the list of
ResourceRequests received by the AM among various RMs.FederationApplicationHomeSubClusterStore maintains the state of all
Applications that have been submitted to the federated cluster
Utility class to validate the inputs to
FederationApplicationHomeSubClusterStore, allows a fail fast
mechanism for invalid user inputs.CaffeineCache is a high-performance caching library for Java, offering better performance compared to Ehcache and Guava Cache.
FederationDelegationTokenStateStore maintains the state of all
DelegationToken that have been submitted to the federated cluster.
FederationMembershipStateStore maintains the state of all
subcluster(s) as encapsulated by
SubClusterInfo for all the
subcluster(s) that are participating in federation.Utility class to validate the inputs to
FederationMembershipStateStore, allows a fail fast mechanism for
invalid user inputs.Generic policy exception.
Context to (re)initialize a
FederationAMRMProxyPolicy and
FederationRouterPolicy.Helper class used to factor out common validation steps for policies.
This exception is thrown when the initialization of a federation policy is
not successful.
Implementors need to provide the ability to serialize a policy and its
configuration as a
SubClusterPolicyConfiguration, as well as provide
(re)initialization mechanics for the underlying
FederationAMRMProxyPolicy and FederationRouterPolicy.The FederationPolicyStore provides a key-value interface to access the
policies configured for the system.
Utility class to validate the inputs to
FederationPolicyStore, allows
a fail fast mechanism for invalid user inputs.Utility class for Federation policy.
Utility class that creates proxy for specified protocols when federation is
enabled.
QueryRunner is used to execute stored procedure SQL and parse the returned results.
Helper class that handles reads and writes to Yarn Registry to support UAM HA
and second attempt.
FederationReservationHomeSubClusterStore maintains the state of all
Reservations that have been submitted to the federated cluster
Utility class to validate the inputs to
FederationReservationHomeSubClusterStore, allows a fail fast
mechanism for invalid user inputs.A FailoverProxyProvider implementation that uses the
FederationStateStore to determine the ResourceManager to connect to.Implements the logic for determining the routing of an application submission
based on a policy.
SQLOutParameter is used to set the output parameters of the stored procedure.
FederationStore extends the three interfaces used to coordinate the state of
a federated cluster:
FederationApplicationHomeSubClusterStore,
FederationMembershipStateStore, FederationPolicyStore, and
FederationReservationHomeSubClusterStore.Performance metrics for FederationStateStore implementations.
Exception thrown by the
FederationStateStore.The FederationStateStoreFacade is an utility wrapper that provides singleton
access to the Federation state store.
Exception thrown by the
FederationMembershipStateStoreInputValidator,
FederationApplicationHomeSubClusterStoreInputValidator,
FederationPolicyStoreInputValidator if the input is invalid.Exception thrown by the
FederationStateStore, if it is a retriable
exception.Common utility methods used by the store implementations.
Request class to obtain the home sub-cluster for the specified
ApplicationId.Protocol buffer based implementation of
GetApplicationHomeSubClusterRequest.
The response sent by
Federation state
store to a query for the home subcluster of a newly submitted
application.Protocol buffer based implementation of
GetApplicationHomeSubClusterResponse.Request class to obtain the home sub-cluster mapping of all active
applications.
Protocol buffer based implementation of
GetApplicationsHomeSubClusterRequest.
The response sent by
Federation state
store to a query for the home subcluster of all submitted
applications.Protocol buffer based implementation of
GetApplicationsHomeSubClusterResponse.Request class to obtain the home sub-cluster for the specified
ReservationId.Protocol buffer based implementation of
GetReservationHomeSubClusterRequest.
The response sent by
Federation state
store to a query for the home subcluster of a newly submitted
reservation.Protocol buffer based implementation of
GetReservationHomeSubClusterResponse.Request class to obtain the home sub-cluster mapping of all active
reservations.
Protocol buffer based implementation of
GetReservationsHomeSubClusterRequest.
The response sent by
Federation state
store to a query for the home subcluster of all submitted
reservations.Protocol buffer based implementation of
GetReservationsHomeSubClusterResponse.Request class to obtain information about a sub-cluster identified by its
SubClusterId.Protocol buffer based implementation of
GetSubClusterInfoRequest.Response to a query with
SubClusterInfo about a sub-cluster.Protocol buffer based implementation of
GetSubClusterInfoResponse.GetSubClusterPoliciesConfigurationsRequest is a request to the
FederationPolicyStore to obtain all policy configurations.Protocol buffer based implementation of
GetSubClusterPoliciesConfigurationsRequest.GetSubClusterPolicyConfigurationResponse contains the answer from the
FederationPolicyStore to a request to get all the policies configured in the
system via a SubClusterPolicyConfiguration.Protocol buffer based implementation of
GetSubClusterPoliciesConfigurationsResponse.GetSubClusterPolicyConfigurationRequest is a request to the
FederationPolicyStore to get the configuration of a policy for a
given queue.Protocol buffer based implementation of
GetSubClusterPolicyConfigurationRequest.GetSubClusterPolicyConfigurationResponse contains the answer from the
FederationPolicyStore to a request to get the information about how a policy
should be configured via a SubClusterPolicyConfiguration.Protocol buffer based implementation of
GetSubClusterPolicyConfigurationResponse.Request class to obtain information about all sub-clusters that are
participating in federation.
Protocol buffer based implementation of
GetSubClustersInfoRequest.Response to a query with list of
SubClusterInfo about all
sub-clusters that are currently participating in Federation.Protocol buffer based implementation of
GetSubClustersInfoResponse.This
FederationRouterPolicy pick a subcluster based on the hash of
the job's queue name.Policy that routes applications via hashing of their queuename, and broadcast
resource requests.
An implementation of the
FederationAMRMProxyPolicy that simply
sends the ResourceRequest to the home subcluster.Policy manager which uses the
UniformRandomRouterPolicy for the
Router and HomeAMRMProxyPolicy as the AMRMProxy policy to find the
RM.This exception is thrown when a volume is found not valid.
A wrapper for a DBIterator to translate the raw RuntimeExceptions that
can be thrown into DBExceptions.
This implements a simple load-balancing policy.
An implementation of the
FederationAMRMProxyPolicy interface that
carefully multicasts the requests with the following behavior:This policy selects the subcluster depending on the node where the Client
wants to run its application.
Annotation to document locking order.
LogAggregationReport is a report for log aggregation status
in one NodeManager of an application.Extracts aggregated logs and related information.
Support only ATSv2 client only.
Log web service utils class.
In-memory implementation of
FederationStateStore.NMContainerStatus includes the current information of a container.
This exception is thrown when policies cannot locate any active cluster.
The NodeManager is instructed to perform the given action.
NodeHealthStatus is a summary of the health status of the node.Node Manager's heartbeat response.
PBImpl class for NodeHeartbeatResponse.
NodeStatus is a summary of the status of the node.
Base abstract class for Opportunistic container allocations, that provides
common functions required for Opportunistic container allocation.
This class encapsulates container and resourceName for an allocation.
This class encapsulates application specific parameters used to build a
Container.
A Container Id Generator.
This class encapsulates Resource Request and provides requests per
node and rack.
Class that includes two lists of
ResourceRequests: one for
GUARANTEED and one for OPPORTUNISTIC ResourceRequests.This encapsulates application specific information used by the
Opportunistic Container Allocator to allocate containers.
OpportunisticContainersStatus captures information
pertaining to the state of execution of the opportunistic containers within a
node.Protocol Buffer implementation of OpportunisticContainersStatus.
Metrics for Opportunistic Scheduler.
Policy that allows operator to configure "weights" for routing.
This implements a policy that interprets "weights" as a ordered list of
preferences among sub-clusters.
This is the response to registering an Application Master when distributed
scheduling is enabled.
Implementation of
RegisterDistributedSchedulingAMResponse.Node Manager's register response.
PBImpl class for RegisterNodeManagerResponse.
This class represents a simple implementation of a
FederationPolicyManager.An implementation of the
FederationAMRMProxyPolicy that simply
rejects all requests.This
FederationRouterPolicy simply rejects all incoming requests.A remote log path for a log aggregation file controller.
Container of a list of
RemoteLogPathEntry.This class is used to encapsulate the
NodeId as well as the HTTP
address that can be used to communicate with the Node.Implementation of
RemoteNode.
ReservationHomeSubCluster is a report of the runtime information of the
reservation that is running in the federated cluster.
Protocol buffer based implementation of
ReservationHomeSubCluster.A set of resource requests of the same scheduler key
ResourceRequestSetKey.The scheduler key for a group of
ResourceRequest.This is used by the Node Manager to register/nodeHeartbeat/unregister with
the ResourceManager.
Result Set Handler.
RouterMasterKey Handler.
This class provides a facade to the policy subsystem, and handles the
lifecycle of policies (e.g., refresh from remote, default behaviors etc.).
RouterStoreToken Handler.
Protocol buffer based implementation of
RouterStoreToken.RowCount Handler.
Composite key for outstanding scheduler requests for any schedulable entity.
The request from the NodeManager to the
SharedCacheManager that
requests whether it can upload a resource in the shared cache.
The response from the SharedCacheManager to the NodeManager that indicates
whether the NodeManager can upload the resource to the shared cache.
The request from the NodeManager to the
SharedCacheManager that
notifies that a resource has been uploaded to the shared cache.
The response from the SharedCacheManager to the NodeManager that indicates
whether the NodeManager needs to delete the cached resource it was sending
the notification for.
The protocol between a
NodeManager's
SharedCacheUploadService and the
SharedCacheManager.SetSubClusterPolicyConfigurationRequest is a request to the
FederationPolicyStore to set the policy configuration corresponding
to a queue.Protocol buffer based implementation of
SetSubClusterPolicyConfigurationRequest.SetSubClusterPolicyConfigurationResponse contains the answer from the
FederationPolicyStore to a request to set for a policy configuration
for a given queue.Protocol buffer based implementation of
SetSubClusterPolicyConfigurationResponse.A utility class that contains helper methods for dealing with the internal
shared cache structure.
SQL implementation of
FederationStateStore.
The request sent to set the state of a subcluster to either
SC_DECOMMISSIONED, SC_LOST, or SC_DEREGISTERED.
Protocol buffer based implementation of
SubClusterDeregisterRequest.SubClusterDeregisterResponse contains the answer from the
FederationMembershipStateStore to a request to deregister the sub cluster.Protocol buffer based implementation of
SubClusterDeregisterResponse.
SubClusterHeartbeatRequest is a report of the runtime information of the
subcluster that is participating in federation.
Protocol buffer based implementation of
SubClusterHeartbeatRequest.SubClusterHeartbeatResponse contains the response from the
FederationMembershipStateStore to a periodic heartbeat to indicate
liveliness from a ResourceManager participating in federation.Protocol buffer based implementation of
SubClusterHeartbeatResponse.
SubClusterId represents the globally unique identifier for a
subcluster that is participating in federation.
This class represent a sub-cluster identifier in the JSON representation
of the policy configuration.
Protocol buffer based implementation of
SubClusterId.
SubClusterInfo is a report of the runtime information of the subcluster that
is participating in federation.
Protocol buffer based implementation of
SubClusterInfo.SubClusterPolicyConfiguration is a class that represents a
configuration of a policy.Protobuf based implementation of
SubClusterPolicyConfiguration.
SubClusterRegisterRequest is a request by a sub-cluster
ResourceManager to participate in federation.Protocol buffer based implementation of
SubClusterRegisterRequest.SubClusterRegisterResponse contains the response from the
FederationMembershipStateStore to a registration request from a
ResourceManager to participate in federation.Protocol buffer based implementation of
SubClusterRegisterResponse.An utility that helps to determine the sub-cluster that a specified node or
rack belongs to.
State of a
SubCluster.Marker interface for starting services from RM.
Timeline authentication filter provides delegation token support for ATSv1
and ATSv2.
Initializes
TimelineAuthenticationFilter which provides support for
Kerberos HTTP SPNEGO authentication.Abstract implementation of delegation token manager service for different
versions of timeline service.
Set of utility methods to be used across timeline reader and collector.
This class represents a simple implementation of a
FederationPolicyManager.This simple policy picks at uniform random among any of the currently active
subclusters.
This exception is thrown whenever a policy is given a
SubClusterId
that is unknown.A service that manages a pool of UAM managers in
UnmanagedApplicationManager.UnmanagedApplicationManager is used to register unmanaged application and
negotiate for resources from resource managers.
Node Manager's unregister request.
PBImpl class for UnRegisterNodeManagerRequest.
Node Manager's unregister response.
PBImpl class for UnRegisterNodeManagerResponse.
The request sent by the
Router to
Federation state store to update the home subcluster of a newly
submitted application.Protocol buffer based implementation of
UpdateApplicationHomeSubClusterRequest .UpdateApplicationHomeSubClusterResponse contains the answer from the
FederationApplicationHomeSubClusterStore to a request to register the
home subcluster of a submitted application.Protocol buffer based implementation of
UpdateApplicationHomeSubClusterResponse.
The request sent by the
Router to
Federation state store to update the home subcluster of a newly
submitted reservation.Protocol buffer based implementation of
UpdateReservationHomeSubClusterRequest .UpdateReservationHomeSubClusterResponse contains the answer from the
FederationReservationHomeSubClusterStore to a request to register the
home subcluster of a submitted reservation.Protocol buffer based implementation of
UpdateReservationHomeSubClusterResponse.The version information for state get stored in YARN components,
i.e.
Volume capability range that specified in a volume resource request,
this range defines the desired min/max capacity.
The builder used to build a VolumeCapabilityRange instance.
Base class for all volume related exceptions.
Unique ID for a volume.
VolumeMetaData defines all valid info for a CSI compatible volume.
The builder used to build a VolumeMetaData instance.
Exception throws when volume provisioning is failed.
Policy that allows operator to configure "weights" for routing.
Policy that allows operator to configure "weights" for routing.
This is a DAO class for the configuration of parameters for federation
policies.
This policy implements a weighted random sample among currently active
sub-clusters.
WrappedLogMetaRequest is wrapping a log request initiated by the client.
Server Builder utilities to construct various objects.
Utility class that contains commonly used server methods.
Common web service parameters which could be used in
RM/NM/AHS WebService.
ZK Registration Library
currently does not use any authorization
ZooKeeper implementation of
FederationStateStore.