Package org.apache.hadoop.yarn.client.api

Class NMTokenCache

java.lang.Object
org.apache.hadoop.yarn.client.api.NMTokenCache
@Public @Evolving public class NMTokenCache extends Object
NMTokenCache manages NMTokens required for an Application Master communicating with individual NodeManagers.

By default YARN client libraries AMRMClient and NMClient use getSingleton() instance of the cache.

  • Using the singleton instance of the cache is appropriate when running a single ApplicationMaster in the same JVM.
  • When using the singleton, users don't need to do anything special, AMRMClient and NMClient are already set up to use the default singleton NMTokenCache
If running multiple Application Masters in the same JVM, a different cache instance should be used for each Application Master.
  • If using the AMRMClient and the NMClient, setting up and using an instance cache is as follows: 
       NMTokenCache nmTokenCache = new NMTokenCache();
   AMRMClient rmClient = AMRMClient.createAMRMClient();
   NMClient nmClient = NMClient.createNMClient();
   nmClient.setNMTokenCache(nmTokenCache);
   ...
  • If using the AMRMClientAsync and the NMClientAsync, setting up and using an instance cache is as follows: 
       NMTokenCache nmTokenCache = new NMTokenCache();
   AMRMClient rmClient = AMRMClient.createAMRMClient();
   NMClient nmClient = NMClient.createNMClient();
   nmClient.setNMTokenCache(nmTokenCache);
   AMRMClientAsync rmClientAsync = new AMRMClientAsync(rmClient, 1000, [AMRM_CALLBACK]);
   NMClientAsync nmClientAsync = new NMClientAsync("nmClient", nmClient, [NM_CALLBACK]);
   ...
  • If using ApplicationMasterProtocol and ContainerManagementProtocol directly, setting up and using an instance cache is as follows: 
       NMTokenCache nmTokenCache = new NMTokenCache();
   ...
   ApplicationMasterProtocol amPro = ClientRMProxy.createRMProxy(conf, ApplicationMasterProtocol.class);
   ...
   AllocateRequest allocateRequest = ...
   ...
   AllocateResponse allocateResponse = rmClient.allocate(allocateRequest);
   for (NMToken token : allocateResponse.getNMTokens()) {
     nmTokenCache.setToken(token.getNodeId().toString(), token.getToken());
   }
   ...
   ContainerManagementProtocolProxy nmPro = ContainerManagementProtocolProxy(conf, nmTokenCache);
   ...
   nmPro.startContainer(container, containerContext);
   ...
It is also possible to mix the usage of a client (AMRMClient or NMClient, or the async versions of them) with a protocol proxy (ContainerManagementProtocolProxy or ApplicationMasterProtocol).

  • Constructor Details

    • NMTokenCache

      public NMTokenCache()
      Creates a NM token cache instance.

  • Method Details

    • getSingleton

      public static NMTokenCache getSingleton()
      Returns the singleton NM token cache.
      Returns:
      the singleton NM token cache.

    • getNMToken

      @Public public static org.apache.hadoop.yarn.api.records.Token getNMToken(String nodeAddr)
      Returns NMToken, null if absent. Only the singleton obtained from getSingleton() is looked at for the tokens. If you are using your own NMTokenCache that is different from the singleton, use getToken(String)
      Parameters:
      nodeAddr -
      Returns:
      Token NMToken required for communicating with node manager

    • setNMToken

      @Public public static void setNMToken(String nodeAddr, org.apache.hadoop.yarn.api.records.Token token)
      Sets the NMToken for node address only in the singleton obtained from getSingleton(). If you are using your own NMTokenCache that is different from the singleton, use setToken(String, Token)
      Parameters:
      nodeAddr - node address (host:port)
      token - NMToken

    • getToken

      @Public @Evolving public org.apache.hadoop.yarn.api.records.Token getToken(String nodeAddr)
      Returns NMToken, null if absent
      Parameters:
      nodeAddr -
      Returns:
      Token NMToken required for communicating with node manager

    • setToken

      @Public @Evolving public void setToken(String nodeAddr, org.apache.hadoop.yarn.api.records.Token token)
      Sets the NMToken for node address
      Parameters:
      nodeAddr - node address (host:port)
      token - NMToken

    • containsToken

      @Private @VisibleForTesting public boolean containsToken(String nodeAddr)
      Returns true if NMToken is present in cache.

    • numberOfTokensInCache

      @Private @VisibleForTesting public int numberOfTokensInCache()
      Returns the number of NMTokens present in cache.

    • removeToken

      @Private @VisibleForTesting public void removeToken(String nodeAddr)
      Removes NMToken for specified node manager
      Parameters:
      nodeAddr - node address (host:port)

    • clearCache

      @Private @VisibleForTesting public void clearCache()
      It will remove all the nm tokens from its cache