Package org.apache.hadoop.yarn.server.resourcemanager

Class ResourceManager

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.service.CompositeService
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.hadoop.service.Service, Recoverable, ResourceManagerMXBean
public class ResourceManager extends org.apache.hadoop.service.CompositeService implements Recoverable, ResourceManagerMXBean
The ResourceManager is the main class that is a set of components. "I am the ResourceManager. All your resources belong to us..."

  • Field Details

    • SHUTDOWN_HOOK_PRIORITY

      public static final int SHUTDOWN_HOOK_PRIORITY
      Priority of the ResourceManager shutdown hook.
      See Also:

    • EPOCH_BIT_SHIFT

      public static final int EPOCH_BIT_SHIFT
      Used for generation of various ids.
      See Also:

    • UI2_WEBAPP_NAME

      public static final String UI2_WEBAPP_NAME
      See Also:

    • SCHEDULER_UI_WEBAPP_NAME

      public static final String SCHEDULER_UI_WEBAPP_NAME
      See Also:

    • rmContext

      @VisibleForTesting protected RMContextImpl rmContext
      "Always On" services. Services that need to run always irrespective of the HA state of the RM.

    • adminService

      @VisibleForTesting protected AdminService adminService

    • activeServices

      protected ResourceManager.RMActiveServices activeServices
      "Active" services. Services that need to run only on the Active RM. These services are managed (initialized, started, stopped) by the CompositeService RMActiveServices. RM is active when (1) HA is disabled, or (2) HA is enabled and the RM is in Active state.

    • rmSecretManagerService

      protected RMSecretManagerService rmSecretManagerService

    • scheduler

      protected ResourceScheduler scheduler

    • reservationSystem

      protected ReservationSystem reservationSystem

    • masterService

      protected ApplicationMasterService masterService

    • nmLivelinessMonitor

      protected NMLivelinessMonitor nmLivelinessMonitor

    • nodesListManager

      protected NodesListManager nodesListManager

    • rmAppManager

      protected RMAppManager rmAppManager

    • applicationACLsManager

      protected org.apache.hadoop.yarn.server.security.ApplicationACLsManager applicationACLsManager

    • queueACLsManager

      protected QueueACLsManager queueACLsManager

    • resourceTracker

      protected ResourceTrackerService resourceTracker

    • webAppAddress

      @VisibleForTesting protected String webAppAddress

  • Constructor Details

    • ResourceManager

      public ResourceManager()

  • Method Details

    • getRMContext

      public RMContext getRMContext()

    • getClusterTimeStamp

      public static long getClusterTimeStamp()

    • getRMLoginUser

      public String getRMLoginUser()

    • setClusterTimeStamp

      @VisibleForTesting protected static void setClusterTimeStamp(long timestamp)

    • createResourceProfileManager

      @VisibleForTesting protected ResourceProfilesManager createResourceProfileManager()

    • serviceInit

      protected void serviceInit(org.apache.hadoop.conf.Configuration conf) throws Exception
      Overrides:
      serviceInit in class org.apache.hadoop.service.CompositeService
      Throws:
      Exception

    • createEmbeddedElector

      protected EmbeddedElector createEmbeddedElector() throws IOException
      Throws:
      IOException

    • createAndStartZKManager

      public org.apache.hadoop.util.curator.ZKCuratorManager createAndStartZKManager(org.apache.hadoop.conf.Configuration config) throws IOException
      Get ZooKeeper Curator manager, creating and starting if not exists.
      Parameters:
      config - Configuration for the ZooKeeper curator.
      Returns:
      ZooKeeper Curator manager.
      Throws:
      IOException - If it cannot create the manager.

    • getZKManager

      public org.apache.hadoop.util.curator.ZKCuratorManager getZKManager()

    • getCurator

      public org.apache.curator.framework.CuratorFramework getCurator()

    • getZkRootNodePassword

      public String getZkRootNodePassword()

    • createQueueACLsManager

      protected QueueACLsManager createQueueACLsManager(ResourceScheduler scheduler, org.apache.hadoop.conf.Configuration conf)

    • setRMStateStore

      @VisibleForTesting protected void setRMStateStore(RMStateStore rmStore)

    • createSchedulerEventDispatcher

      protected org.apache.hadoop.yarn.event.EventHandler<SchedulerEvent> createSchedulerEventDispatcher()

    • createDispatcher

      protected org.apache.hadoop.yarn.event.Dispatcher createDispatcher()

    • createScheduler

      protected ResourceScheduler createScheduler()

    • createReservationSystem

      protected ReservationSystem createReservationSystem()

    • createServiceManager

      protected org.apache.hadoop.yarn.server.service.SystemServiceManager createServiceManager()

    • createAMLauncher

      protected ApplicationMasterLauncher createAMLauncher()

    • createAMLivelinessMonitor

      protected AMLivelinessMonitor createAMLivelinessMonitor()

    • createNodeLabelManager

      protected RMNodeLabelsManager createNodeLabelManager() throws InstantiationException, IllegalAccessException
      Throws:
      InstantiationException
      IllegalAccessException

    • createNodeAttributesManager

      protected org.apache.hadoop.yarn.nodelabels.NodeAttributesManager createNodeAttributesManager()

    • createAllocationTagsManager

      protected AllocationTagsManager createAllocationTagsManager()

    • createPlacementConstraintManager

      protected PlacementConstraintManagerService createPlacementConstraintManager()

    • createDelegationTokenRenewer

      protected DelegationTokenRenewer createDelegationTokenRenewer()

    • createRMAppManager

      protected RMAppManager createRMAppManager()

    • createRMApplicationHistoryWriter

      protected RMApplicationHistoryWriter createRMApplicationHistoryWriter()

    • createMultiNodeSortingManager

      protected MultiNodeSortingManager<SchedulerNode> createMultiNodeSortingManager()

    • createSystemMetricsPublisher

      protected SystemMetricsPublisher createSystemMetricsPublisher()

    • validateConfigs

      protected static void validateConfigs(org.apache.hadoop.conf.Configuration conf)

    • httpServerTemplateForRM

      public static org.apache.hadoop.http.HttpServer2.Builder httpServerTemplateForRM(org.apache.hadoop.conf.Configuration conf, InetSocketAddress httpAddr, InetSocketAddress httpsAddr, String name) throws IOException
      Return a HttpServer.Builder that the journalnode / namenode / secondary namenode can use to initialize their HTTP / HTTPS server.
      Parameters:
      conf - configuration object
      httpAddr - HTTP address
      httpsAddr - HTTPS address
      name - Name of the server
      Returns:
      builder object
      Throws:
      IOException - from Builder

    • startWepApp

      protected void startWepApp()

    • createAndInitActiveServices

      protected void createAndInitActiveServices(boolean fromActive)
      Helper method to create and init activeServices. This creates an instance of ResourceManager.RMActiveServices and initializes it.
      Parameters:
      fromActive - Indicates if the call is from the active state transition or the RM initialization.

    • areActiveServicesRunning

      @VisibleForTesting protected boolean areActiveServicesRunning()

    • serviceStart

      protected void serviceStart() throws Exception
      Overrides:
      serviceStart in class org.apache.hadoop.service.CompositeService
      Throws:
      Exception

    • doSecureLogin

      protected void doSecureLogin() throws IOException
      Throws:
      IOException

    • serviceStop

      protected void serviceStop() throws Exception
      Overrides:
      serviceStop in class org.apache.hadoop.service.CompositeService
      Throws:
      Exception

    • createResourceTrackerService

      protected ResourceTrackerService createResourceTrackerService()

    • createClientRMService

      protected ClientRMService createClientRMService()

    • createApplicationMasterService

      protected ApplicationMasterService createApplicationMasterService()

    • createAdminService

      protected AdminService createAdminService()

    • createRMSecretManagerService

      protected RMSecretManagerService createRMSecretManagerService()

    • createRMDelegatedNodeLabelsUpdater

      protected RMDelegatedNodeLabelsUpdater createRMDelegatedNodeLabelsUpdater()
      Create RMDelegatedNodeLabelsUpdater based on configuration.
      Returns:
      RMDelegatedNodeLabelsUpdater.

    • getClientRMService

      @Private public ClientRMService getClientRMService()

    • getResourceScheduler

      @Private public ResourceScheduler getResourceScheduler()
      return the scheduler.
      Returns:
      the scheduler for the Resource Manager.

    • getResourceTrackerService

      @Private public ResourceTrackerService getResourceTrackerService()
      return the resource tracking component.
      Returns:
      the resource tracking component.

    • getApplicationMasterService

      @Private public ApplicationMasterService getApplicationMasterService()

    • getApplicationACLsManager

      @Private public org.apache.hadoop.yarn.server.security.ApplicationACLsManager getApplicationACLsManager()

    • getQueueACLsManager

      @Private public QueueACLsManager getQueueACLsManager()

    • getFederationStateStoreService

      @Private @VisibleForTesting public FederationStateStoreService getFederationStateStoreService()

    • recover

      public void recover(RMStateStore.RMState state) throws Exception
      Specified by:
      recover in interface Recoverable
      Throws:
      Exception

    • main

      public static void main(String[] argv)

    • getBindAddress

      public static InetSocketAddress getBindAddress(org.apache.hadoop.conf.Configuration conf)
      Retrieve RM bind address from configuration.
      Parameters:
      conf - Configuration.
      Returns:
      InetSocketAddress

    • createRMAppLifetimeMonitor

      protected RMAppLifetimeMonitor createRMAppLifetimeMonitor()

    • isSecurityEnabled

      public boolean isSecurityEnabled()
      Description copied from interface: ResourceManagerMXBean
      Gets if security is enabled.
      Specified by:
      isSecurityEnabled in interface ResourceManagerMXBean
      Returns:
      true, if security is enabled.