|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.apache.hadoop.fs.AbstractFileSystem
@InterfaceAudience.Public @InterfaceStability.Evolving public abstract class AbstractFileSystem
This class provides an interface for implementors of a Hadoop file system
(analogous to the VFS of Unix). Applications do not access this class;
instead they access files across all file systems using FileContext
.
Pathnames passed to AbstractFileSystem can be fully qualified URI that
matches the "this" file system (ie same scheme and authority)
or a Slash-relative name that is assumed to be relative
to the root of the "this" file system .
Field Summary | |
---|---|
protected org.apache.hadoop.fs.FileSystem.Statistics |
statistics
The statistics for this file system. |
Constructor Summary | |
---|---|
AbstractFileSystem(URI uri,
String supportedScheme,
boolean authorityNeeded,
int defaultPort)
Constructor to be called by subclasses. |
Method Summary | |
---|---|
void |
checkPath(Path path)
Check that a Path belongs to this FileSystem. |
void |
checkScheme(URI uri,
String supportedScheme)
Check that the Uri's scheme matches |
static void |
clearStatistics()
|
FSDataOutputStream |
create(Path f,
EnumSet<CreateFlag> createFlag,
org.apache.hadoop.fs.Options.CreateOpts... opts)
The specification of this method matches that of FileContext.create(Path, EnumSet, Options.CreateOpts...) except
that the Path f must be fully qualified and the permission is absolute
(i.e. |
static AbstractFileSystem |
createFileSystem(URI uri,
Configuration conf)
Create a file system instance for the specified uri using the conf. |
abstract FSDataOutputStream |
createInternal(Path f,
EnumSet<CreateFlag> flag,
FsPermission absolutePermission,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt,
boolean createParent)
The specification of this method matches that of create(Path, EnumSet, Options.CreateOpts...) except that the opts
have been declared explicitly. |
void |
createSymlink(Path target,
Path link,
boolean createParent)
The specification of this method matches that of FileContext.createSymlink(Path, Path, boolean) ; |
abstract boolean |
delete(Path f,
boolean recursive)
The specification of this method matches that of FileContext.delete(Path, boolean) except that Path f must be for
this file system. |
boolean |
equals(Object other)
|
static AbstractFileSystem |
get(URI uri,
Configuration conf)
The main factory method for creating a file system. |
AclStatus |
getAclStatus(Path path)
Gets the ACLs of files and directories. |
protected static Map<URI,org.apache.hadoop.fs.FileSystem.Statistics> |
getAllStatistics()
|
String |
getCanonicalServiceName()
Get a canonical name for this file system. |
abstract BlockLocation[] |
getFileBlockLocations(Path f,
long start,
long len)
The specification of this method matches that of FileContext.getFileBlockLocations(Path, long, long) except that
Path f must be for this file system. |
abstract FileChecksum |
getFileChecksum(Path f)
The specification of this method matches that of FileContext.getFileChecksum(Path) except that Path f must be for
this file system. |
FileStatus |
getFileLinkStatus(Path f)
The specification of this method matches that of FileContext.getFileLinkStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path leading up to the final path component. |
abstract FileStatus |
getFileStatus(Path f)
The specification of this method matches that of FileContext.getFileStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path. |
abstract FsStatus |
getFsStatus()
The specification of this method matches that of FileContext.getFsStatus(Path) . |
FsStatus |
getFsStatus(Path f)
The specification of this method matches that of FileContext.getFsStatus(Path) except that Path f must be for this
file system. |
Path |
getHomeDirectory()
Return the current user's home directory in this file system. |
Path |
getInitialWorkingDirectory()
Some file systems like LocalFileSystem have an initial workingDir that is used as the starting workingDir. |
Path |
getLinkTarget(Path f)
Partially resolves the path. |
abstract FsServerDefaults |
getServerDefaults()
Return a set of server default configuration values. |
org.apache.hadoop.fs.FileSystem.Statistics |
getStatistics()
|
protected static org.apache.hadoop.fs.FileSystem.Statistics |
getStatistics(URI uri)
Get the statistics for a particular file system. |
URI |
getUri()
Returns a URI whose scheme and authority identify this FileSystem. |
abstract int |
getUriDefaultPort()
The default port of this file system. |
String |
getUriPath(Path p)
Get the path-part of a pathname. |
int |
hashCode()
|
boolean |
isValidName(String src)
Returns true if the specified string is considered valid in the path part of a URI by this file system. |
org.apache.hadoop.fs.RemoteIterator<Path> |
listCorruptFileBlocks(Path path)
|
org.apache.hadoop.fs.RemoteIterator<LocatedFileStatus> |
listLocatedStatus(Path f)
The specification of this method matches that of FileContext.listLocatedStatus(Path) except that Path f
must be for this file system. |
abstract FileStatus[] |
listStatus(Path f)
The specification of this method matches that of FileContext.Util.listStatus(Path) except that Path f must be
for this file system. |
org.apache.hadoop.fs.RemoteIterator<FileStatus> |
listStatusIterator(Path f)
The specification of this method matches that of FileContext.listStatus(Path) except that Path f must be for this
file system. |
Path |
makeQualified(Path path)
Make the path fully qualified to this file system |
abstract void |
mkdir(Path dir,
FsPermission permission,
boolean createParent)
The specification of this method matches that of FileContext.mkdir(Path, FsPermission, boolean) except that the Path
f must be fully qualified and the permission is absolute (i.e. |
void |
modifyAclEntries(Path path,
List<AclEntry> aclSpec)
Modifies ACL entries of files and directories. |
FSDataInputStream |
open(Path f)
The specification of this method matches that of FileContext.open(Path) except that Path f must be for this
file system. |
abstract FSDataInputStream |
open(Path f,
int bufferSize)
The specification of this method matches that of FileContext.open(Path, int) except that Path f must be for this
file system. |
static void |
printStatistics()
Prints statistics for all file systems. |
void |
removeAcl(Path path)
Removes all but the base ACL entries of files and directories. |
void |
removeAclEntries(Path path,
List<AclEntry> aclSpec)
Removes ACL entries from files and directories. |
void |
removeDefaultAcl(Path path)
Removes all default ACL entries from files and directories. |
void |
rename(Path src,
Path dst,
org.apache.hadoop.fs.Options.Rename... options)
The specification of this method matches that of FileContext.rename(Path, Path, Options.Rename...) except that Path
f must be for this file system. |
abstract void |
renameInternal(Path src,
Path dst)
The specification of this method matches that of FileContext.rename(Path, Path, Options.Rename...) except that Path
f must be for this file system and NO OVERWRITE is performed. |
void |
renameInternal(Path src,
Path dst,
boolean overwrite)
The specification of this method matches that of FileContext.rename(Path, Path, Options.Rename...) except that Path
f must be for this file system. |
Path |
resolvePath(Path p)
Return the fully-qualified path of path f resolving the path through any internal symlinks or mount point |
void |
setAcl(Path path,
List<AclEntry> aclSpec)
Fully replaces ACL of files and directories, discarding all existing entries. |
abstract void |
setOwner(Path f,
String username,
String groupname)
The specification of this method matches that of FileContext.setOwner(Path, String, String) except that Path f must
be for this file system. |
abstract void |
setPermission(Path f,
FsPermission permission)
The specification of this method matches that of FileContext.setPermission(Path, FsPermission) except that Path f
must be for this file system. |
abstract boolean |
setReplication(Path f,
short replication)
The specification of this method matches that of FileContext.setReplication(Path, short) except that Path f must be
for this file system. |
abstract void |
setTimes(Path f,
long mtime,
long atime)
The specification of this method matches that of FileContext.setTimes(Path, long, long) except that Path f must be
for this file system. |
abstract void |
setVerifyChecksum(boolean verifyChecksum)
The specification of this method matches that of FileContext.setVerifyChecksum(boolean, Path) except that Path f
must be for this file system. |
boolean |
supportsSymlinks()
Returns true if the file system supports symlinks, false otherwise. |
Methods inherited from class java.lang.Object |
---|
clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected org.apache.hadoop.fs.FileSystem.Statistics statistics
Constructor Detail |
---|
public AbstractFileSystem(URI uri, String supportedScheme, boolean authorityNeeded, int defaultPort) throws URISyntaxException
uri
- for this file system.supportedScheme
- the scheme supported by the implementorauthorityNeeded
- if true then theURI must have authority, if false
then the URI must have null authority.
URISyntaxException
- uri
has syntax errorMethod Detail |
---|
public org.apache.hadoop.fs.FileSystem.Statistics getStatistics()
public boolean isValidName(String src)
src
- String source filename to check, path part of the URI
public static AbstractFileSystem createFileSystem(URI uri, Configuration conf) throws UnsupportedFileSystemException
uri
- URI of the file systemconf
- Configuration for the file system
UnsupportedFileSystemException
- file system for uri
is
not foundprotected static org.apache.hadoop.fs.FileSystem.Statistics getStatistics(URI uri)
uri
- used as key to lookup STATISTICS_TABLE. Only scheme and authority
part of the uri are used.
public static void clearStatistics()
public static void printStatistics()
protected static Map<URI,org.apache.hadoop.fs.FileSystem.Statistics> getAllStatistics()
public static AbstractFileSystem get(URI uri, Configuration conf) throws UnsupportedFileSystemException
uri
determines a configuration property name,
fs.AbstractFileSystem.scheme.impl whose value names the
AbstractFileSystem class.
The entire URI and conf is passed to the AbstractFileSystem factory method.
uri
- for the file system to be created.conf
- which is passed to the file system impl.
UnsupportedFileSystemException
- if the file system for
uri
is not supported.public void checkScheme(URI uri, String supportedScheme)
uri
- supportedScheme
- public abstract int getUriDefaultPort()
public URI getUri()
public void checkPath(Path path)
InvalidPathException
- if the path is invalidpublic String getUriPath(Path p)
p
- path
public Path makeQualified(Path path)
path
-
public Path getInitialWorkingDirectory()
public Path getHomeDirectory()
public abstract FsServerDefaults getServerDefaults() throws IOException
IOException
- an I/O error occurredpublic Path resolvePath(Path p) throws FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, org.apache.hadoop.security.AccessControlException, IOException
p
- path to be resolved
FileNotFoundException,
- AccessControlException, IOException
UnresolvedLinkException if symbolic link on path cannot be resolved
internally
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
org.apache.hadoop.security.AccessControlException
IOException
public final FSDataOutputStream create(Path f, EnumSet<CreateFlag> createFlag, org.apache.hadoop.fs.Options.CreateOpts... opts) throws org.apache.hadoop.security.AccessControlException, FileAlreadyExistsException, FileNotFoundException, ParentNotDirectoryException, UnsupportedFileSystemException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.create(Path, EnumSet, Options.CreateOpts...)
except
that the Path f must be fully qualified and the permission is absolute
(i.e. umask has been applied).
org.apache.hadoop.security.AccessControlException
FileAlreadyExistsException
FileNotFoundException
ParentNotDirectoryException
UnsupportedFileSystemException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public abstract FSDataOutputStream createInternal(Path f, EnumSet<CreateFlag> flag, FsPermission absolutePermission, int bufferSize, short replication, long blockSize, Progressable progress, org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt, boolean createParent) throws org.apache.hadoop.security.AccessControlException, FileAlreadyExistsException, FileNotFoundException, ParentNotDirectoryException, UnsupportedFileSystemException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
create(Path, EnumSet, Options.CreateOpts...)
except that the opts
have been declared explicitly.
org.apache.hadoop.security.AccessControlException
FileAlreadyExistsException
FileNotFoundException
ParentNotDirectoryException
UnsupportedFileSystemException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public abstract void mkdir(Path dir, FsPermission permission, boolean createParent) throws org.apache.hadoop.security.AccessControlException, FileAlreadyExistsException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.mkdir(Path, FsPermission, boolean)
except that the Path
f must be fully qualified and the permission is absolute (i.e.
umask has been applied).
org.apache.hadoop.security.AccessControlException
FileAlreadyExistsException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public abstract boolean delete(Path f, boolean recursive) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.delete(Path, boolean)
except that Path f must be for
this file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FSDataInputStream open(Path f) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.open(Path)
except that Path f must be for this
file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public abstract FSDataInputStream open(Path f, int bufferSize) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.open(Path, int)
except that Path f must be for this
file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public abstract boolean setReplication(Path f, short replication) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.setReplication(Path, short)
except that Path f must be
for this file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public final void rename(Path src, Path dst, org.apache.hadoop.fs.Options.Rename... options) throws org.apache.hadoop.security.AccessControlException, FileAlreadyExistsException, FileNotFoundException, ParentNotDirectoryException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.rename(Path, Path, Options.Rename...)
except that Path
f must be for this file system.
org.apache.hadoop.security.AccessControlException
FileAlreadyExistsException
FileNotFoundException
ParentNotDirectoryException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public abstract void renameInternal(Path src, Path dst) throws org.apache.hadoop.security.AccessControlException, FileAlreadyExistsException, FileNotFoundException, ParentNotDirectoryException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.rename(Path, Path, Options.Rename...)
except that Path
f must be for this file system and NO OVERWRITE is performed.
File systems that do not have a built in overwrite need implement only this
method and can take advantage of the default impl of the other
renameInternal(Path, Path, boolean)
org.apache.hadoop.security.AccessControlException
FileAlreadyExistsException
FileNotFoundException
ParentNotDirectoryException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void renameInternal(Path src, Path dst, boolean overwrite) throws org.apache.hadoop.security.AccessControlException, FileAlreadyExistsException, FileNotFoundException, ParentNotDirectoryException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.rename(Path, Path, Options.Rename...)
except that Path
f must be for this file system.
org.apache.hadoop.security.AccessControlException
FileAlreadyExistsException
FileNotFoundException
ParentNotDirectoryException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public boolean supportsSymlinks()
public void createSymlink(Path target, Path link, boolean createParent) throws IOException, org.apache.hadoop.fs.UnresolvedLinkException
FileContext.createSymlink(Path, Path, boolean)
;
IOException
org.apache.hadoop.fs.UnresolvedLinkException
public Path getLinkTarget(Path f) throws IOException
FSLinkResolver
, and differs from the similarly named method
FileContext.getLinkTarget(Path)
.
IOException
public abstract void setPermission(Path f, FsPermission permission) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.setPermission(Path, FsPermission)
except that Path f
must be for this file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public abstract void setOwner(Path f, String username, String groupname) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.setOwner(Path, String, String)
except that Path f must
be for this file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public abstract void setTimes(Path f, long mtime, long atime) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.setTimes(Path, long, long)
except that Path f must be
for this file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public abstract FileChecksum getFileChecksum(Path f) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.getFileChecksum(Path)
except that Path f must be for
this file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public abstract FileStatus getFileStatus(Path f) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.getFileStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FileStatus getFileLinkStatus(Path f) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, UnsupportedFileSystemException, IOException
FileContext.getFileLinkStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path leading up to the final path component.
If the file system does not support symlinks then the behavior is
equivalent to getFileStatus(Path)
.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
UnsupportedFileSystemException
IOException
public abstract BlockLocation[] getFileBlockLocations(Path f, long start, long len) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.getFileBlockLocations(Path, long, long)
except that
Path f must be for this file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FsStatus getFsStatus(Path f) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.getFsStatus(Path)
except that Path f must be for this
file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public abstract FsStatus getFsStatus() throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, IOException
FileContext.getFsStatus(Path)
.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
IOException
public org.apache.hadoop.fs.RemoteIterator<FileStatus> listStatusIterator(Path f) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.listStatus(Path)
except that Path f must be for this
file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public org.apache.hadoop.fs.RemoteIterator<LocatedFileStatus> listLocatedStatus(Path f) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.listLocatedStatus(Path)
except that Path f
must be for this file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public abstract FileStatus[] listStatus(Path f) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnresolvedLinkException, IOException
FileContext.Util.listStatus(Path)
except that Path f must be
for this file system.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public org.apache.hadoop.fs.RemoteIterator<Path> listCorruptFileBlocks(Path path) throws IOException
IOException
public abstract void setVerifyChecksum(boolean verifyChecksum) throws org.apache.hadoop.security.AccessControlException, IOException
FileContext.setVerifyChecksum(boolean, Path)
except that Path f
must be for this file system.
org.apache.hadoop.security.AccessControlException
IOException
public String getCanonicalServiceName()
public void modifyAclEntries(Path path, List<AclEntry> aclSpec) throws IOException
path
- Path to modifyaclSpec
- ListIOException
- if an ACL could not be modifiedpublic void removeAclEntries(Path path, List<AclEntry> aclSpec) throws IOException
path
- Path to modifyaclSpec
- ListIOException
- if an ACL could not be modifiedpublic void removeDefaultAcl(Path path) throws IOException
path
- Path to modify
IOException
- if an ACL could not be modifiedpublic void removeAcl(Path path) throws IOException
path
- Path to modify
IOException
- if an ACL could not be removedpublic void setAcl(Path path, List<AclEntry> aclSpec) throws IOException
path
- Path to modifyaclSpec
- ListIOException
- if an ACL could not be modifiedpublic AclStatus getAclStatus(Path path) throws IOException
path
- Path to get
IOException
- if an ACL could not be readpublic int hashCode()
hashCode
in class Object
public boolean equals(Object other)
equals
in class Object
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |