Package org.apache.hadoop.hdfs.client

Class HdfsAdmin

java.lang.Object
org.apache.hadoop.hdfs.client.HdfsAdmin
@Public @Evolving public class HdfsAdmin extends Object
The public API for performing administrative functions on HDFS. Those writing applications against HDFS should prefer this interface to directly accessing functionality in DistributedFileSystem or DFSClient. Note that this is distinct from the similarly-named DFSAdmin, which is a class that provides the functionality for the CLI `hdfs dfsadmin ...' commands.

  • Field Details

    • TRASH_PERMISSION

      public static final FsPermission TRASH_PERMISSION

  • Constructor Details

    • HdfsAdmin

      public HdfsAdmin(URI uri, Configuration conf) throws IOException
      Create a new HdfsAdmin client.
      Parameters:
      uri - the unique URI of the HDFS file system to administer
      conf - configuration
      Throws:
      IOException - in the event the file system could not be created

  • Method Details

    • setQuota

      public void setQuota(Path src, long quota) throws IOException
      Set the namespace quota (count of files, directories, and sym links) for a directory.
      Parameters:
      src - the path to set the quota for
      quota - the value to set for the quota
      Throws:
      IOException - in the event of error

    • clearQuota

      public void clearQuota(Path src) throws IOException
      Clear the namespace quota (count of files, directories and sym links) for a directory.
      Parameters:
      src - the path to clear the quota of
      Throws:
      IOException - in the event of error

    • setSpaceQuota

      public void setSpaceQuota(Path src, long spaceQuota) throws IOException
      Set the storage space quota (size of files) for a directory. Note that directories and sym links do not occupy storage space.
      Parameters:
      src - the path to set the space quota of
      spaceQuota - the value to set for the space quota
      Throws:
      IOException - in the event of error

    • clearSpaceQuota

      public void clearSpaceQuota(Path src) throws IOException
      Clear the storage space quota (size of files) for a directory. Note that directories and sym links do not occupy storage space.
      Parameters:
      src - the path to clear the space quota of
      Throws:
      IOException - in the event of error

    • setQuotaByStorageType

      public void setQuotaByStorageType(Path src, StorageType type, long quota) throws IOException
      Set the quota by storage type for a directory. Note that directories and sym links do not occupy storage type quota.
      Parameters:
      src - the target directory to set the quota by storage type
      type - the storage type to set for quota by storage type
      quota - the value to set for quota by storage type
      Throws:
      IOException - in the event of error

    • clearQuotaByStorageType

      public void clearQuotaByStorageType(Path src, StorageType type) throws IOException
      Clear the space quota by storage type for a directory. Note that directories and sym links do not occupy storage type quota.
      Parameters:
      src - the target directory to clear the quota by storage type
      type - the storage type to clear for quota by storage type
      Throws:
      IOException - in the event of error

    • allowSnapshot

      public void allowSnapshot(Path path) throws IOException
      Allow snapshot on a directory.
      Parameters:
      path - The path of the directory where snapshots will be taken.
      Throws:
      IOException

    • provisionSnapshotTrash

      public Path provisionSnapshotTrash(Path path) throws IOException
      Provision a trash directory for a given snapshottable directory.
      Parameters:
      path - the root of the snapshottable directory
      Returns:
      Path of the provisioned trash root
      Throws:
      IOException - if the trash directory can not be created.

    • disallowSnapshot

      public void disallowSnapshot(Path path) throws IOException
      Disallow snapshot on a directory.
      Parameters:
      path - The path of the snapshottable directory.
      Throws:
      IOException

    • addCacheDirective

      public long addCacheDirective(CacheDirectiveInfo info, EnumSet<CacheFlag> flags) throws IOException
      Add a new CacheDirectiveInfo.
      Parameters:
      info - Information about a directive to add.
      flags - CacheFlags to use for this operation.
      Returns:
      the ID of the directive that was created.
      Throws:
      IOException - if the directive could not be added

    • modifyCacheDirective

      public void modifyCacheDirective(CacheDirectiveInfo info, EnumSet<CacheFlag> flags) throws IOException
      Modify a CacheDirective.
      Parameters:
      info - Information about the directive to modify. You must set the ID to indicate which CacheDirective you want to modify.
      flags - CacheFlags to use for this operation.
      Throws:
      IOException - if the directive could not be modified

    • removeCacheDirective

      public void removeCacheDirective(long id) throws IOException
      Remove a CacheDirective.
      Parameters:
      id - identifier of the CacheDirectiveInfo to remove
      Throws:
      IOException - if the directive could not be removed

    • listCacheDirectives

      public org.apache.hadoop.fs.RemoteIterator<CacheDirectiveEntry> listCacheDirectives(CacheDirectiveInfo filter) throws IOException
      List cache directives. Incrementally fetches results from the server.
      Parameters:
      filter - Filter parameters to use when listing the directives, null to list all directives visible to us.
      Returns:
      A RemoteIterator which returns CacheDirectiveInfo objects.
      Throws:
      IOException

    • addCachePool

      public void addCachePool(CachePoolInfo info) throws IOException
      Add a cache pool.
      Parameters:
      info - The request to add a cache pool.
      Throws:
      IOException - If the request could not be completed.

    • modifyCachePool

      public void modifyCachePool(CachePoolInfo info) throws IOException
      Modify an existing cache pool.
      Parameters:
      info - The request to modify a cache pool.
      Throws:
      IOException - If the request could not be completed.

    • removeCachePool

      public void removeCachePool(String poolName) throws IOException
      Remove a cache pool.
      Parameters:
      poolName - Name of the cache pool to remove.
      Throws:
      IOException - if the cache pool did not exist, or could not be removed.

    • listCachePools

      public org.apache.hadoop.fs.RemoteIterator<CachePoolEntry> listCachePools() throws IOException
      List all cache pools.
      Returns:
      A remote iterator from which you can get CachePoolEntry objects. Requests will be made as needed.
      Throws:
      IOException - If there was an error listing cache pools.

    • getKeyProvider

      public KeyProvider getKeyProvider() throws IOException
      Get KeyProvider if present.
      Returns:
      the key provider if encryption is enabled on HDFS. Otherwise, it returns null.
      Throws:
      IOException - on RPC exception to the NN.

    • createEncryptionZone

      @Deprecated public void createEncryptionZone(Path path, String keyName) throws IOException, AccessControlException, FileNotFoundException
      Deprecated.
      Create an encryption zone rooted at an empty existing directory, using the specified encryption key. An encryption zone has an associated encryption key used when reading and writing files within the zone.
      Parameters:
      path - The path of the root of the encryption zone. Must refer to an empty, existing directory.
      keyName - Name of key available at the KeyProvider.
      Throws:
      IOException - if there was a general IO exception
      AccessControlException - if the caller does not have access to path
      FileNotFoundException - if the path does not exist

    • createEncryptionZone

      public void createEncryptionZone(Path path, String keyName, EnumSet<CreateEncryptionZoneFlag> flags) throws IOException, AccessControlException, FileNotFoundException, HadoopIllegalArgumentException
      Create an encryption zone rooted at an empty existing directory, using the specified encryption key. An encryption zone has an associated encryption key used when reading and writing files within the zone. Additional options, such as provisioning the trash directory, can be specified using CreateEncryptionZoneFlag flags.
      Parameters:
      path - The path of the root of the encryption zone. Must refer to an empty, existing directory.
      keyName - Name of key available at the KeyProvider.
      flags - flags for this operation.
      Throws:
      IOException - if there was a general IO exception
      AccessControlException - if the caller does not have access to path
      FileNotFoundException - if the path does not exist
      HadoopIllegalArgumentException - if the flags are invalid

    • provisionEncryptionZoneTrash

      public void provisionEncryptionZoneTrash(Path path) throws IOException
      Provision a trash directory for a given encryption zone.
      Parameters:
      path - the root of the encryption zone
      Throws:
      IOException - if the trash directory can not be created.

    • getEncryptionZoneForPath

      public EncryptionZone getEncryptionZoneForPath(Path path) throws IOException, AccessControlException
      Get the path of the encryption zone for a given file or directory.
      Parameters:
      path - The path to get the ez for.
      Returns:
      An EncryptionZone, or null if path does not exist or is not in an ez.
      Throws:
      IOException - if there was a general IO exception
      AccessControlException - if the caller does not have access to path

    • listEncryptionZones

      public org.apache.hadoop.fs.RemoteIterator<EncryptionZone> listEncryptionZones() throws IOException
      Returns a RemoteIterator which can be used to list the encryption zones in HDFS. For large numbers of encryption zones, the iterator will fetch the list of zones in a number of small batches.

      Since the list is fetched in batches, it does not represent a consistent snapshot of the entire list of encryption zones.

      This method can only be called by HDFS superusers.

      Throws:
      IOException

    • reencryptEncryptionZone

      public void reencryptEncryptionZone(Path zone, HdfsConstants.ReencryptAction action) throws IOException
      Performs re-encryption action for a given encryption zone.
      Parameters:
      zone - the root of the encryption zone
      action - the re-encrypt action
      Throws:
      IOException - If any error occurs when handling re-encrypt action.

    • listReencryptionStatus

      public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.hdfs.protocol.ZoneReencryptionStatus> listReencryptionStatus() throws IOException
      Returns a RemoteIterator which can be used to list all re-encryption information. For large numbers of re-encryptions, the iterator will fetch the list in a number of small batches.

      Since the list is fetched in batches, it does not represent a consistent snapshot of the entire list of encryption zones.

      This method can only be called by HDFS superusers.

      Throws:
      IOException

    • getFileEncryptionInfo

      public org.apache.hadoop.fs.FileEncryptionInfo getFileEncryptionInfo(Path path) throws IOException
      Returns the FileEncryptionInfo on the HdfsFileStatus for the given path. The return value can be null if the path points to a directory, or a file that is not in an encryption zone.
      Throws:
      FileNotFoundException - if the path does not exist.
      AccessControlException - if no execute permission on parent path.
      IOException

    • getInotifyEventStream

      public DFSInotifyEventInputStream getInotifyEventStream() throws IOException
      Exposes a stream of namesystem events. Only events occurring after the stream is created are available. See DFSInotifyEventInputStream for information on stream usage. See Event for information on the available events.

      Inotify users may want to tune the following HDFS parameters to ensure that enough extra HDFS edits are saved to support inotify clients that fall behind the current state of the namespace while reading events. The default parameter values should generally be reasonable. If edits are deleted before their corresponding events can be read, clients will see a MissingEventsException on DFSInotifyEventInputStream method calls. It should generally be sufficient to tune these parameters: dfs.namenode.num.extra.edits.retained dfs.namenode.max.extra.edits.segments.retained Parameters that affect the number of created segments and the number of edits that are considered necessary, i.e. do not count towards the dfs.namenode.num.extra.edits.retained quota): dfs.namenode.checkpoint.period dfs.namenode.checkpoint.txns dfs.namenode.num.checkpoints.retained dfs.ha.log-roll.period

      It is recommended that local journaling be configured (dfs.namenode.edits.dir) for inotify (in addition to a shared journal) so that edit transfers from the shared journal can be avoided.

      Throws:
      IOException - If there was an error obtaining the stream.

    • getInotifyEventStream

      public DFSInotifyEventInputStream getInotifyEventStream(long lastReadTxid) throws IOException
      A version of getInotifyEventStream() meant for advanced users who are aware of HDFS edits up to lastReadTxid (e.g. because they have access to an FSImage inclusive of lastReadTxid) and only want to read events after this point.
      Throws:
      IOException

    • setStoragePolicy

      public void setStoragePolicy(Path src, String policyName) throws IOException
      Set the source path to the specified storage policy.
      Parameters:
      src - The source path referring to either a directory or a file.
      policyName - The name of the storage policy.
      Throws:
      IOException

    • unsetStoragePolicy

      public void unsetStoragePolicy(Path src) throws IOException
      Unset the storage policy set for a given file or directory.
      Parameters:
      src - file or directory path.
      Throws:
      IOException

    • getStoragePolicy

      public BlockStoragePolicySpi getStoragePolicy(Path src) throws IOException
      Query the effective storage policy ID for the given file or directory.
      Parameters:
      src - file or directory path.
      Returns:
      storage policy for the given file or directory.
      Throws:
      IOException

    • getAllStoragePolicies

      public Collection<? extends BlockStoragePolicySpi> getAllStoragePolicies() throws IOException
      Retrieve all the storage policies supported by HDFS file system.
      Returns:
      all storage policies supported by HDFS file system.
      Throws:
      IOException

    • setErasureCodingPolicy

      public void setErasureCodingPolicy(Path path, String ecPolicyName) throws IOException
      Set the source path to the specified erasure coding policy.
      Parameters:
      path - The source path referring to a directory.
      ecPolicyName - The erasure coding policy name for the directory.
      Throws:
      IOException
      HadoopIllegalArgumentException - if the specified EC policy is not enabled on the cluster

    • getErasureCodingPolicy

      public org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy getErasureCodingPolicy(Path path) throws IOException
      Get the erasure coding policy information for the specified path
      Parameters:
      path -
      Returns:
      Returns the policy information if file or directory on the path is erasure coded. Null otherwise.
      Throws:
      IOException

    • satisfyStoragePolicy

      public void satisfyStoragePolicy(Path path) throws IOException
      Set the source path to the specified storage policy.
      Parameters:
      path - The source path referring to either a directory or a file.
      Throws:
      IOException

    • getErasureCodingPolicies

      public org.apache.hadoop.hdfs.protocol.ErasureCodingPolicyInfo[] getErasureCodingPolicies() throws IOException
      Get the Erasure coding policies supported.
      Throws:
      IOException

    • unsetErasureCodingPolicy

      public void unsetErasureCodingPolicy(Path path) throws IOException
      Unset erasure coding policy from the directory.
      Parameters:
      path - The source path referring to a directory.
      Throws:
      IOException

    • addErasureCodingPolicies

      public org.apache.hadoop.hdfs.protocol.AddErasureCodingPolicyResponse[] addErasureCodingPolicies(org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy[] policies) throws IOException
      Add Erasure coding policies to HDFS. For each policy input, schema and cellSize are musts, name and id are ignored. They will be automatically created and assigned by Namenode once the policy is successfully added, and will be returned in the response; policy states will be set to DISABLED automatically.
      Parameters:
      policies - The user defined ec policy list to add.
      Returns:
      Return the response list of adding operations.
      Throws:
      IOException

    • removeErasureCodingPolicy

      public void removeErasureCodingPolicy(String ecPolicyName) throws IOException
      Remove erasure coding policy.
      Parameters:
      ecPolicyName - The name of the policy to be removed.
      Throws:
      IOException

    • enableErasureCodingPolicy

      public void enableErasureCodingPolicy(String ecPolicyName) throws IOException
      Enable erasure coding policy.
      Parameters:
      ecPolicyName - The name of the policy to be enabled.
      Throws:
      IOException

    • disableErasureCodingPolicy

      public void disableErasureCodingPolicy(String ecPolicyName) throws IOException
      Disable erasure coding policy.
      Parameters:
      ecPolicyName - The name of the policy to be disabled.
      Throws:
      IOException

    • listOpenFiles

      @Deprecated public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.hdfs.protocol.OpenFileEntry> listOpenFiles() throws IOException
      Deprecated.
      Returns a RemoteIterator which can be used to list all open files currently managed by the NameNode. For large numbers of open files, iterator will fetch the list in batches of configured size.

      Since the list is fetched in batches, it does not represent a consistent snapshot of the all open files.

      This method can only be called by HDFS superusers.

      Throws:
      IOException

    • listOpenFiles

      @Deprecated public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.hdfs.protocol.OpenFileEntry> listOpenFiles(EnumSet<OpenFilesIterator.OpenFilesType> openFilesTypes) throws IOException
      Deprecated.
      Throws:
      IOException

    • listOpenFiles

      public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.hdfs.protocol.OpenFileEntry> listOpenFiles(EnumSet<OpenFilesIterator.OpenFilesType> openFilesTypes, String path) throws IOException
      Throws:
      IOException