@InterfaceAudience.Public @InterfaceStability.Evolving public class ViewFs extends AbstractFileSystem
To use viewfs one would typically set the default file system in the config (i.e. fs.defaultFS < = viewfs:///) along with the mount table config variables as described below.
** Config variables to specify the mount table entries **
The file system is initialized from the standard Hadoop config through
config variables.
See FsConstants
for URI and Scheme constants;
See Constants
for config var constants;
see ConfigUtil
for convenient lib.
All the mount table config entries for view fs are prefixed by fs.viewfs.mounttable. For example the above example can be specified with the following config variables:
**** Merge Mounts **** (NOTE: merge mounts are not implemented yet.)
One can also use "MergeMounts" to merge several directories (this is sometimes called union-mounts or junction-mounts in the literature. For example of the home directories are stored on say two file systems (because they do not fit on one) then one could specify a mount entry such as following merges two dirs:
statistics
Constructor and Description |
---|
ViewFs(Configuration conf) |
Modifier and Type | Method and Description |
---|---|
void |
access(Path path,
FsAction mode)
The specification of this method matches that of
FileContext.access(Path, FsAction)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path. |
FSDataOutputStream |
createInternal(Path f,
EnumSet<CreateFlag> flag,
FsPermission absolutePermission,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt,
boolean createParent)
The specification of this method matches that of
AbstractFileSystem.create(Path, EnumSet, Options.CreateOpts...) except that the opts
have been declared explicitly. |
Path |
createSnapshot(Path path,
String snapshotName)
The specification of this method matches that of
FileContext.createSnapshot(Path, String) . |
void |
createSymlink(Path target,
Path link,
boolean createParent)
The specification of this method matches that of
FileContext.createSymlink(Path, Path, boolean) ; |
boolean |
delete(Path f,
boolean recursive)
The specification of this method matches that of
FileContext.delete(Path, boolean) except that Path f must be for
this file system. |
void |
deleteSnapshot(Path path,
String snapshotName)
The specification of this method matches that of
FileContext.deleteSnapshot(Path, String) . |
AclStatus |
getAclStatus(Path path)
Gets the ACLs of files and directories.
|
List<Token<?>> |
getDelegationTokens(String renewer)
Get one or more delegation tokens associated with the filesystem.
|
BlockLocation[] |
getFileBlockLocations(Path f,
long start,
long len)
The specification of this method matches that of
FileContext.getFileBlockLocations(Path, long, long) except that
Path f must be for this file system. |
FileChecksum |
getFileChecksum(Path f)
The specification of this method matches that of
FileContext.getFileChecksum(Path) except that Path f must be for
this file system. |
FileStatus |
getFileLinkStatus(Path f)
The specification of this method matches that of
FileContext.getFileLinkStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path leading up to the final path component. |
FileStatus |
getFileStatus(Path f)
The specification of this method matches that of
FileContext.getFileStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path. |
FsStatus |
getFsStatus()
The specification of this method matches that of
FileContext.getFsStatus(Path) . |
Path |
getHomeDirectory()
Return the current user's home directory in this file system.
|
Path |
getLinkTarget(Path f)
Partially resolves the path.
|
org.apache.hadoop.fs.viewfs.ViewFs.MountPoint[] |
getMountPoints() |
FsServerDefaults |
getServerDefaults()
Deprecated.
|
FsServerDefaults |
getServerDefaults(Path f)
Return a set of server default configuration values based on path.
|
BlockStoragePolicySpi |
getStoragePolicy(Path src)
Retrieve the storage policy for a given file or directory.
|
int |
getUriDefaultPort()
The default port of this file system.
|
byte[] |
getXAttr(Path path,
String name)
Get an xattr for a file or directory.
|
Map<String,byte[]> |
getXAttrs(Path path)
Get all of the xattrs for a file or directory.
|
Map<String,byte[]> |
getXAttrs(Path path,
List<String> names)
Get all of the xattrs for a file or directory.
|
boolean |
isValidName(String src)
Returns true if the specified string is considered valid in the path part
of a URI by this file system.
|
org.apache.hadoop.fs.RemoteIterator<LocatedFileStatus> |
listLocatedStatus(Path f)
The specification of this method matches that of
FileContext.listLocatedStatus(Path) except that Path f
must be for this file system. |
FileStatus[] |
listStatus(Path f)
The specification of this method matches that of
FileContext.Util.listStatus(Path) except that Path f must be
for this file system. |
org.apache.hadoop.fs.RemoteIterator<FileStatus> |
listStatusIterator(Path f)
The specification of this method matches that of
FileContext.listStatus(Path) except that Path f must be for this
file system. |
List<String> |
listXAttrs(Path path)
Get all of the xattr names for a file or directory.
|
void |
mkdir(Path dir,
FsPermission permission,
boolean createParent)
The specification of this method matches that of
FileContext.mkdir(Path, FsPermission, boolean) except that the Path
f must be fully qualified and the permission is absolute (i.e. |
void |
modifyAclEntries(Path path,
List<AclEntry> aclSpec)
Modifies ACL entries of files and directories.
|
FSDataInputStream |
open(Path f,
int bufferSize)
The specification of this method matches that of
FileContext.open(Path, int) except that Path f must be for this
file system. |
void |
removeAcl(Path path)
Removes all but the base ACL entries of files and directories.
|
void |
removeAclEntries(Path path,
List<AclEntry> aclSpec)
Removes ACL entries from files and directories.
|
void |
removeDefaultAcl(Path path)
Removes all default ACL entries from files and directories.
|
void |
removeXAttr(Path path,
String name)
Remove an xattr of a file or directory.
|
void |
renameInternal(Path src,
Path dst)
The specification of this method matches that of
FileContext.rename(Path, Path, Options.Rename...) except that Path
f must be for this file system and NO OVERWRITE is performed. |
void |
renameInternal(Path src,
Path dst,
boolean overwrite)
The specification of this method matches that of
FileContext.rename(Path, Path, Options.Rename...) except that Path
f must be for this file system. |
void |
renameSnapshot(Path path,
String snapshotOldName,
String snapshotNewName)
The specification of this method matches that of
FileContext.renameSnapshot(Path, String, String) . |
Path |
resolvePath(Path f)
Return the fully-qualified path of path f resolving the path
through any internal symlinks or mount point
|
void |
satisfyStoragePolicy(Path path)
Set the source path to satisfy storage policy.
|
void |
setAcl(Path path,
List<AclEntry> aclSpec)
Fully replaces ACL of files and directories, discarding all existing
entries.
|
void |
setOwner(Path f,
String username,
String groupname)
The specification of this method matches that of
FileContext.setOwner(Path, String, String) except that Path f must
be for this file system. |
void |
setPermission(Path f,
FsPermission permission)
The specification of this method matches that of
FileContext.setPermission(Path, FsPermission) except that Path f
must be for this file system. |
boolean |
setReplication(Path f,
short replication)
The specification of this method matches that of
FileContext.setReplication(Path, short) except that Path f must be
for this file system. |
void |
setStoragePolicy(Path path,
String policyName)
Set the storage policy for a given file or directory.
|
void |
setTimes(Path f,
long mtime,
long atime)
The specification of this method matches that of
FileContext.setTimes(Path, long, long) except that Path f must be
for this file system. |
void |
setVerifyChecksum(boolean verifyChecksum)
The specification of this method matches that of
FileContext.setVerifyChecksum(boolean, Path) except that Path f
must be for this file system. |
void |
setXAttr(Path path,
String name,
byte[] value,
EnumSet<XAttrSetFlag> flag)
Set an xattr of a file or directory.
|
boolean |
supportsSymlinks()
Returns true if the file system supports symlinks, false otherwise.
|
boolean |
truncate(Path f,
long newLength)
The specification of this method matches that of
FileContext.truncate(Path, long) except that Path f must be for
this file system. |
void |
unsetStoragePolicy(Path src)
Unset the storage policy set for a given file or directory.
|
checkPath, checkScheme, clearStatistics, create, createFileSystem, equals, get, getAllStatistics, getAllStoragePolicies, getCanonicalServiceName, getFsStatus, getInitialWorkingDirectory, getStatistics, getStatistics, getUri, getUriPath, hashCode, hasPathCapability, listCorruptFileBlocks, makeQualified, open, openFileWithOptions, printStatistics, rename, setXAttr
public ViewFs(Configuration conf) throws IOException, URISyntaxException
IOException
URISyntaxException
@Deprecated public FsServerDefaults getServerDefaults() throws IOException
AbstractFileSystem
getServerDefaults
in class AbstractFileSystem
IOException
- an I/O error occurredpublic FsServerDefaults getServerDefaults(Path f) throws IOException
AbstractFileSystem
getServerDefaults
in class AbstractFileSystem
f
- path to fetch server defaultsIOException
- an I/O error occurredpublic int getUriDefaultPort()
AbstractFileSystem
getUriDefaultPort
in class AbstractFileSystem
public Path getHomeDirectory()
AbstractFileSystem
getHomeDirectory
in class AbstractFileSystem
public Path resolvePath(Path f) throws FileNotFoundException, AccessControlException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
resolvePath
in class AbstractFileSystem
f
- path to be resolvedFileNotFoundException
AccessControlException
org.apache.hadoop.fs.UnresolvedLinkException
- if symbolic link on path cannot be
resolved internallyIOException
public FSDataOutputStream createInternal(Path f, EnumSet<CreateFlag> flag, FsPermission absolutePermission, int bufferSize, short replication, long blockSize, Progressable progress, org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt, boolean createParent) throws AccessControlException, FileAlreadyExistsException, FileNotFoundException, ParentNotDirectoryException, UnsupportedFileSystemException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
AbstractFileSystem.create(Path, EnumSet, Options.CreateOpts...)
except that the opts
have been declared explicitly.createInternal
in class AbstractFileSystem
AccessControlException
FileAlreadyExistsException
FileNotFoundException
ParentNotDirectoryException
UnsupportedFileSystemException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public boolean delete(Path f, boolean recursive) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.delete(Path, boolean)
except that Path f must be for
this file system.delete
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public BlockLocation[] getFileBlockLocations(Path f, long start, long len) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.getFileBlockLocations(Path, long, long)
except that
Path f must be for this file system.getFileBlockLocations
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FileChecksum getFileChecksum(Path f) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.getFileChecksum(Path)
except that Path f must be for
this file system.getFileChecksum
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FileStatus getFileStatus(Path f) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.getFileStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path.getFileStatus
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void access(Path path, FsAction mode) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.access(Path, FsAction)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FileStatus getFileLinkStatus(Path f) throws AccessControlException, FileNotFoundException, UnsupportedFileSystemException, IOException
AbstractFileSystem
FileContext.getFileLinkStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path leading up to the final path component.
If the file system does not support symlinks then the behavior is
equivalent to AbstractFileSystem.getFileStatus(Path)
.getFileLinkStatus
in class AbstractFileSystem
AccessControlException
FileNotFoundException
UnsupportedFileSystemException
IOException
public FsStatus getFsStatus() throws AccessControlException, FileNotFoundException, IOException
AbstractFileSystem
FileContext.getFsStatus(Path)
.getFsStatus
in class AbstractFileSystem
AccessControlException
FileNotFoundException
IOException
public org.apache.hadoop.fs.RemoteIterator<FileStatus> listStatusIterator(Path f) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.listStatus(Path)
except that Path f must be for this
file system.listStatusIterator
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public org.apache.hadoop.fs.RemoteIterator<LocatedFileStatus> listLocatedStatus(Path f) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.listLocatedStatus(Path)
except that Path f
must be for this file system.
In HDFS implementation, the BlockLocation of returned LocatedFileStatus
will have different formats for replicated and erasure coded file. Please
refer to FileSystem.getFileBlockLocations(FileStatus, long, long)
for more details.listLocatedStatus
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FileStatus[] listStatus(Path f) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.Util.listStatus(Path)
except that Path f must be
for this file system.listStatus
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void mkdir(Path dir, FsPermission permission, boolean createParent) throws AccessControlException, FileAlreadyExistsException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.mkdir(Path, FsPermission, boolean)
except that the Path
f must be fully qualified and the permission is absolute (i.e.
umask has been applied).mkdir
in class AbstractFileSystem
AccessControlException
FileAlreadyExistsException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FSDataInputStream open(Path f, int bufferSize) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.open(Path, int)
except that Path f must be for this
file system.open
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public boolean truncate(Path f, long newLength) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.truncate(Path, long)
except that Path f must be for
this file system.truncate
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void renameInternal(Path src, Path dst, boolean overwrite) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException
AbstractFileSystem
FileContext.rename(Path, Path, Options.Rename...)
except that Path
f must be for this file system.renameInternal
in class AbstractFileSystem
IOException
org.apache.hadoop.fs.UnresolvedLinkException
public void renameInternal(Path src, Path dst) throws AccessControlException, FileAlreadyExistsException, FileNotFoundException, ParentNotDirectoryException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.rename(Path, Path, Options.Rename...)
except that Path
f must be for this file system and NO OVERWRITE is performed.
File systems that do not have a built in overwrite need implement only this
method and can take advantage of the default impl of the other
AbstractFileSystem.renameInternal(Path, Path, boolean)
renameInternal
in class AbstractFileSystem
AccessControlException
FileAlreadyExistsException
FileNotFoundException
ParentNotDirectoryException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public boolean supportsSymlinks()
AbstractFileSystem
supportsSymlinks
in class AbstractFileSystem
public void createSymlink(Path target, Path link, boolean createParent) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException
AbstractFileSystem
FileContext.createSymlink(Path, Path, boolean)
;createSymlink
in class AbstractFileSystem
IOException
org.apache.hadoop.fs.UnresolvedLinkException
public Path getLinkTarget(Path f) throws IOException
AbstractFileSystem
FSLinkResolver
, and differs from the similarly named method
FileContext.getLinkTarget(Path)
.getLinkTarget
in class AbstractFileSystem
IOException
- subclass implementations may throw IOExceptionpublic void setOwner(Path f, String username, String groupname) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.setOwner(Path, String, String)
except that Path f must
be for this file system.setOwner
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void setPermission(Path f, FsPermission permission) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.setPermission(Path, FsPermission)
except that Path f
must be for this file system.setPermission
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public boolean setReplication(Path f, short replication) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.setReplication(Path, short)
except that Path f must be
for this file system.setReplication
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void setTimes(Path f, long mtime, long atime) throws AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
AbstractFileSystem
FileContext.setTimes(Path, long, long)
except that Path f must be
for this file system.setTimes
in class AbstractFileSystem
AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void setVerifyChecksum(boolean verifyChecksum) throws AccessControlException, IOException
AbstractFileSystem
FileContext.setVerifyChecksum(boolean, Path)
except that Path f
must be for this file system.setVerifyChecksum
in class AbstractFileSystem
AccessControlException
IOException
public org.apache.hadoop.fs.viewfs.ViewFs.MountPoint[] getMountPoints()
public List<Token<?>> getDelegationTokens(String renewer) throws IOException
AbstractFileSystem
renewer
- the account name that is allowed to renew the token.IOException
public boolean isValidName(String src)
AbstractFileSystem
isValidName
in class AbstractFileSystem
src
- String source filename to check, path part of the URIpublic void modifyAclEntries(Path path, List<AclEntry> aclSpec) throws IOException
AbstractFileSystem
modifyAclEntries
in class AbstractFileSystem
path
- Path to modifyaclSpec
- List<AclEntry> describing modificationsIOException
- if an ACL could not be modifiedpublic void removeAclEntries(Path path, List<AclEntry> aclSpec) throws IOException
AbstractFileSystem
removeAclEntries
in class AbstractFileSystem
path
- Path to modifyaclSpec
- List<AclEntry> describing entries to removeIOException
- if an ACL could not be modifiedpublic void removeDefaultAcl(Path path) throws IOException
AbstractFileSystem
removeDefaultAcl
in class AbstractFileSystem
path
- Path to modifyIOException
- if an ACL could not be modifiedpublic void removeAcl(Path path) throws IOException
AbstractFileSystem
removeAcl
in class AbstractFileSystem
path
- Path to modifyIOException
- if an ACL could not be removedpublic void setAcl(Path path, List<AclEntry> aclSpec) throws IOException
AbstractFileSystem
setAcl
in class AbstractFileSystem
path
- Path to modifyaclSpec
- List<AclEntry> describing modifications, must
include entries for user, group, and others for compatibility with
permission bits.IOException
- if an ACL could not be modifiedpublic AclStatus getAclStatus(Path path) throws IOException
AbstractFileSystem
getAclStatus
in class AbstractFileSystem
path
- Path to getIOException
- if an ACL could not be readpublic void setXAttr(Path path, String name, byte[] value, EnumSet<XAttrSetFlag> flag) throws IOException
AbstractFileSystem
Refer to the HDFS extended attributes user documentation for details.
setXAttr
in class AbstractFileSystem
path
- Path to modifyname
- xattr name.value
- xattr value.flag
- xattr set flagIOException
public byte[] getXAttr(Path path, String name) throws IOException
AbstractFileSystem
Refer to the HDFS extended attributes user documentation for details.
getXAttr
in class AbstractFileSystem
path
- Path to get extended attributename
- xattr name.IOException
public Map<String,byte[]> getXAttrs(Path path) throws IOException
AbstractFileSystem
Refer to the HDFS extended attributes user documentation for details.
getXAttrs
in class AbstractFileSystem
path
- Path to get extended attributesIOException
public Map<String,byte[]> getXAttrs(Path path, List<String> names) throws IOException
AbstractFileSystem
Refer to the HDFS extended attributes user documentation for details.
getXAttrs
in class AbstractFileSystem
path
- Path to get extended attributesnames
- XAttr names.IOException
public List<String> listXAttrs(Path path) throws IOException
AbstractFileSystem
Refer to the HDFS extended attributes user documentation for details.
listXAttrs
in class AbstractFileSystem
path
- Path to get extended attributesIOException
public void removeXAttr(Path path, String name) throws IOException
AbstractFileSystem
Refer to the HDFS extended attributes user documentation for details.
removeXAttr
in class AbstractFileSystem
path
- Path to remove extended attributename
- xattr nameIOException
public Path createSnapshot(Path path, String snapshotName) throws IOException
AbstractFileSystem
FileContext.createSnapshot(Path, String)
.createSnapshot
in class AbstractFileSystem
IOException
public void renameSnapshot(Path path, String snapshotOldName, String snapshotNewName) throws IOException
AbstractFileSystem
FileContext.renameSnapshot(Path, String, String)
.renameSnapshot
in class AbstractFileSystem
IOException
public void deleteSnapshot(Path path, String snapshotName) throws IOException
AbstractFileSystem
FileContext.deleteSnapshot(Path, String)
.deleteSnapshot
in class AbstractFileSystem
IOException
public void satisfyStoragePolicy(Path path) throws IOException
AbstractFileSystem
satisfyStoragePolicy
in class AbstractFileSystem
path
- The source path referring to either a directory or a file.IOException
public void setStoragePolicy(Path path, String policyName) throws IOException
AbstractFileSystem
setStoragePolicy
in class AbstractFileSystem
path
- file or directory path.policyName
- the name of the target storage policy. The list
of supported Storage policies can be retrieved
via AbstractFileSystem.getAllStoragePolicies()
.IOException
public void unsetStoragePolicy(Path src) throws IOException
AbstractFileSystem
unsetStoragePolicy
in class AbstractFileSystem
src
- file or directory path.IOException
public BlockStoragePolicySpi getStoragePolicy(Path src) throws IOException
getStoragePolicy
in class AbstractFileSystem
src
- file or directory path.IOException
Copyright © 2020 Apache Software Foundation. All rights reserved.