public class HarFileSystem extends FileSystem
FileSystem.Statistics| Modifier and Type | Field and Description |
|---|---|
static int |
METADATA_CACHE_ENTRIES_DEFAULT |
static String |
METADATA_CACHE_ENTRIES_KEY |
static int |
VERSION |
DEFAULT_FS, FS_DEFAULT_NAME_KEY, SHUTDOWN_HOOK_PRIORITY, statistics, TRASH_PREFIX, USER_HOME_PREFIX| Constructor and Description |
|---|
HarFileSystem()
public construction of harfilesystem
|
HarFileSystem(FileSystem fs)
Constructor to create a HarFileSystem with an
underlying filesystem.
|
| Modifier and Type | Method and Description |
|---|---|
FSDataOutputStream |
append(Path f)
Append to an existing file (optional operation).
|
FSDataOutputStream |
append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
FSDataOutputStreamBuilder |
appendFile(Path path)
Create a Builder to append a file.
|
protected URI |
canonicalizeUri(URI uri)
Canonicalize the given URI.
|
protected void |
checkPath(Path path)
Check that a Path belongs to this FileSystem.
|
void |
close()
Close this FileSystem instance.
|
void |
completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
not implemented.
|
void |
copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path[] srcs,
Path dst)
The src files are on the local disk.
|
void |
copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path src,
Path dst)
not implemented.
|
void |
copyToLocalFile(boolean delSrc,
Path src,
Path dst)
copies the file in the har filesystem to a local file.
|
FSDataOutputStream |
create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStreamBuilder |
createFile(Path path)
Create a new FSDataOutputStreamBuilder for the file with path.
|
FSDataOutputStream |
createNonRecursive(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
boolean |
delete(Path f,
boolean recursive)
Not implemented.
|
protected URI |
getCanonicalUri()
Used for delegation token related functionality.
|
FileSystem[] |
getChildFileSystems()
Used for delegation token related functionality.
|
Configuration |
getConf()
Return the configuration used by this object.
|
long |
getDefaultBlockSize()
Return the number of bytes that large input files should be optimally
be split into to minimize I/O time.
|
long |
getDefaultBlockSize(Path f)
Return the number of bytes that large input files should be optimally
be split into to minimize I/O time.
|
short |
getDefaultReplication()
Get the default replication.
|
short |
getDefaultReplication(Path f)
Get the default replication for a path.
|
BlockLocation[] |
getFileBlockLocations(FileStatus file,
long start,
long len)
Get block locations from the underlying fs and fix their
offsets and lengths.
|
FileChecksum |
getFileChecksum(Path f,
long length)
Get the checksum of a file, from the beginning of the file till the
specific length.
|
FileStatus |
getFileStatus(Path f)
return the filestatus of files in har archive.
|
static int |
getHarHash(Path p)
the hash of the path p inside the filesystem
|
int |
getHarVersion() |
Path |
getHomeDirectory()
return the top level archive path.
|
Path |
getInitialWorkingDirectory()
Note: with the new FileContext class, getWorkingDirectory()
will be removed.
|
String |
getScheme()
Return the protocol scheme for the FileSystem.
|
FsServerDefaults |
getServerDefaults()
Return a set of server default configuration values.
|
FsServerDefaults |
getServerDefaults(Path f)
Return a set of server default configuration values.
|
FsStatus |
getStatus(Path p)
Returns a status object describing the use and capacity of the
filesystem.
|
URI |
getUri()
Returns the uri of this filesystem.
|
long |
getUsed()
Return the total size of all files in the filesystem.
|
long |
getUsed(Path path)
Return the total size of all files from a specified path.
|
Path |
getWorkingDirectory()
return the top level archive.
|
void |
initialize(URI name,
Configuration conf)
Initialize a Har filesystem per har archive.
|
FileStatus[] |
listStatus(Path f)
liststatus returns the children of a directory
after looking up the index files.
|
Path |
makeQualified(Path path)
Qualify a path to one which uses this FileSystem and, if relative,
made absolute.
|
boolean |
mkdirs(Path f,
FsPermission permission)
not implemented.
|
FSDataInputStream |
open(Path f,
int bufferSize)
Returns a har input stream which fakes end of
file.
|
boolean |
rename(Path src,
Path dst)
Renames Path src to Path dst.
|
Path |
resolvePath(Path p)
Return the fully-qualified path of path, resolving the path
through any symlinks or mount point.
|
void |
setOwner(Path p,
String username,
String groupname)
not implemented.
|
void |
setPermission(Path p,
FsPermission permission)
Not implemented.
|
boolean |
setReplication(Path src,
short replication)
Not implemented.
|
void |
setTimes(Path p,
long mtime,
long atime)
Set access time of a file.
|
void |
setWorkingDirectory(Path newDir)
Set the current working directory for the given FileSystem.
|
Path |
startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
not implemented.
|
boolean |
truncate(Path f,
long newLength)
Not implemented.
|
addDelegationTokens, append, areSymlinksEnabled, cancelDeleteOnExit, clearStatistics, closeAll, closeAllForUGI, concat, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, create, createNewFile, createNonRecursive, createNonRecursive, createSnapshot, createSnapshot, createSymlink, delete, deleteOnExit, deleteSnapshot, enableSymlinks, exists, fixRelativePart, get, get, get, getAclStatus, getAllStatistics, getAllStoragePolicies, getBlockSize, getCanonicalServiceName, getContentSummary, getDefaultPort, getDefaultUri, getFileBlockLocations, getFileChecksum, getFileLinkStatus, getFileSystemClass, getFSofPath, getGlobalStorageStatistics, getLength, getLinkTarget, getLocal, getName, getNamed, getQuotaUsage, getReplication, getStatistics, getStatistics, getStatus, getStoragePolicy, getStorageStatistics, getTrashRoot, getTrashRoots, getXAttr, getXAttrs, getXAttrs, globStatus, globStatus, isDirectory, isFile, listCorruptFileBlocks, listFiles, listLocatedStatus, listLocatedStatus, listStatus, listStatus, listStatus, listStatusIterator, listXAttrs, mkdirs, mkdirs, modifyAclEntries, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, newInstance, newInstance, newInstance, newInstanceLocal, open, primitiveCreate, primitiveMkdir, primitiveMkdir, printStatistics, processDeleteOnExit, removeAcl, removeAclEntries, removeDefaultAcl, removeXAttr, rename, renameSnapshot, resolveLink, setAcl, setDefaultUri, setDefaultUri, setStoragePolicy, setVerifyChecksum, setWriteChecksum, setXAttr, setXAttr, supportsSymlinks, unsetStoragePolicysetConfpublic static final String METADATA_CACHE_ENTRIES_KEY
public static final int METADATA_CACHE_ENTRIES_DEFAULT
public static final int VERSION
public HarFileSystem()
public HarFileSystem(FileSystem fs)
fs - underlying file systempublic String getScheme()
getScheme in class FileSystemharpublic void initialize(URI name, Configuration conf) throws IOException
initialize in class FileSystemname - a URI whose authority section names the host, port, etc.
for this FileSystemconf - the configurationIOException - on any failure to initialize this instance.public Configuration getConf()
ConfigurablegetConf in interface ConfigurablegetConf in class Configuredpublic int getHarVersion()
throws IOException
IOExceptionpublic Path getWorkingDirectory()
getWorkingDirectory in class FileSystempublic Path getInitialWorkingDirectory()
FileSystemgetInitialWorkingDirectory in class FileSystempublic FsStatus getStatus(Path p) throws IOException
FileSystemgetStatus in class FileSystemp - Path for which status should be obtained. null means
the default partition.IOException - see specific implementationprotected URI getCanonicalUri()
getCanonicalUri in class FileSystemFileSystem.canonicalizeUri(URI)protected URI canonicalizeUri(URI uri)
FileSystemFileSystem.getDefaultPort() returns a
default port.canonicalizeUri in class FileSystemNetUtils.getCanonicalUri(URI, int)public URI getUri()
getUri in class FileSystemprotected void checkPath(Path path)
FileSystemcheckPath in class FileSystempath - to checkpublic Path resolvePath(Path p) throws IOException
FileSystemresolvePath in class FileSystemp - path to be resolvedFileNotFoundException - if the path is not presentIOException - for any other errorpublic Path makeQualified(Path path)
FileSystemmakeQualified in class FileSystempath - to qualify.Path.makeQualified(URI, Path)public BlockLocation[] getFileBlockLocations(FileStatus file, long start, long len) throws IOException
getFileBlockLocations in class FileSystemfile - the input file status to get block locationsstart - the start of the desired range in the contained filelen - the length of the desired rangeIOExceptionpublic static int getHarHash(Path p)
p - the path in the harfilesystempublic FileStatus getFileStatus(Path f) throws IOException
getFileStatus in class FileSystemf - the path in har filesystemIOExceptionFileNotFoundException - when the path does not existpublic FileChecksum getFileChecksum(Path f, long length)
FileSystemgetFileChecksum in class FileSystemf - The file pathlength - The length of the file range for checksum calculationpublic FSDataInputStream open(Path f, int bufferSize) throws IOException
open in class FileSystemf - the file name to openbufferSize - the size of the buffer to be used.IOException - IO failurepublic FileSystem[] getChildFileSystems()
public FSDataOutputStream create(Path f, FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, Progressable progress) throws IOException
FileSystemcreate in class FileSystemf - the file name to openpermission - file permissionoverwrite - if a file with this name already exists, then if true,
the file will be overwritten, and if false an error will be thrown.bufferSize - the size of the buffer to be used.replication - required block replication for the file.blockSize - block sizeprogress - the progress reporterIOException - IO failureFileSystem.setPermission(Path, FsPermission)public FSDataOutputStream createNonRecursive(Path f, boolean overwrite, int bufferSize, short replication, long blockSize, Progressable progress) throws IOException
FileSystemcreateNonRecursive in class FileSystemf - the file name to openoverwrite - if a file with this name already exists, then if true,
the file will be overwritten, and if false an error will be thrown.bufferSize - the size of the buffer to be used.replication - required block replication for the file.blockSize - block sizeprogress - the progress reporterIOException - IO failureFileSystem.setPermission(Path, FsPermission)public FSDataOutputStream append(Path f, int bufferSize, Progressable progress) throws IOException
FileSystemappend in class FileSystemf - the existing file to be appended.bufferSize - the size of the buffer to be used.progress - for reporting progress if it is not null.IOException - IO failurepublic void close()
throws IOException
FileSystemFileSystem.deleteOnExit(Path), and remove this FS instance
from the cache, if cached.
After this operation, the outcome of any method call on this FileSystem
instance, or any input/output stream created by it is undefined.close in interface Closeableclose in interface AutoCloseableclose in class FileSystemIOException - IO failurepublic boolean setReplication(Path src, short replication) throws IOException
setReplication in class FileSystemsrc - file namereplication - new replicationIOExceptionpublic boolean rename(Path src, Path dst) throws IOException
FileSystemrename in class FileSystemsrc - path to be renameddst - new path after renameIOException - on failurepublic FSDataOutputStream append(Path f) throws IOException
FileSystemappend(f, getConf().getInt(IO_FILE_BUFFER_SIZE_KEY,
IO_FILE_BUFFER_SIZE_DEFAULT), null)append in class FileSystemf - the existing file to be appended.IOException - IO failurepublic boolean truncate(Path f, long newLength) throws IOException
truncate in class FileSystemf - The path to the file to be truncatednewLength - The size the file is to be truncated totrue if the file has been truncated to the desired
newLength and is immediately available to be reused for
write operations such as append, or
false if a background process of adjusting the length of
the last block has been started, and clients should wait for it to
complete before proceeding with further file updates.IOException - IO failurepublic boolean delete(Path f, boolean recursive) throws IOException
delete in class FileSystemf - the path to delete.recursive - if path is a directory and set to
true, the directory is deleted else throws an exception. In
case of a file the recursive can be set to either true or false.IOException - IO failurepublic FileStatus[] listStatus(Path f) throws IOException
listStatus in class FileSystemf - given pathFileNotFoundException - when the path does not existIOException - see specific implementationpublic Path getHomeDirectory()
getHomeDirectory in class FileSystempublic void setWorkingDirectory(Path newDir)
FileSystemsetWorkingDirectory in class FileSystemnewDir - Path of new working directorypublic boolean mkdirs(Path f, FsPermission permission) throws IOException
mkdirs in class FileSystemf - path to createpermission - to apply to fIOException - IO failurepublic void copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path src,
Path dst)
throws IOException
copyFromLocalFile in class FileSystemdelSrc - whether to delete the srcoverwrite - whether to overwrite an existing filesrc - pathdst - pathIOException - IO failurepublic void copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path[] srcs,
Path dst)
throws IOException
FileSystemcopyFromLocalFile in class FileSystemdelSrc - whether to delete the srcoverwrite - whether to overwrite an existing filesrcs - array of paths which are sourcedst - pathIOException - IO failurepublic void copyToLocalFile(boolean delSrc,
Path src,
Path dst)
throws IOException
copyToLocalFile in class FileSystemdelSrc - whether to delete the srcsrc - path src file in the remote filesystemdst - path local destinationIOException - IO failurepublic Path startLocalOutput(Path fsOutputFile, Path tmpLocalFile) throws IOException
startLocalOutput in class FileSystemfsOutputFile - path of output filetmpLocalFile - path of local tmp fileIOException - IO failurepublic void completeLocalOutput(Path fsOutputFile, Path tmpLocalFile) throws IOException
completeLocalOutput in class FileSystemfsOutputFile - path of output filetmpLocalFile - path to local tmp fileIOException - IO failurepublic void setOwner(Path p, String username, String groupname) throws IOException
setOwner in class FileSystemp - The pathusername - If it is null, the original username remains unchanged.groupname - If it is null, the original groupname remains unchanged.IOException - IO failurepublic void setTimes(Path p, long mtime, long atime) throws IOException
FileSystemsetTimes in class FileSystemp - The pathmtime - Set the modification time of this file.
The number of milliseconds since Jan 1, 1970.
A value of -1 means that this call should not set modification time.atime - Set the access time of this file.
The number of milliseconds since Jan 1, 1970.
A value of -1 means that this call should not set access time.IOException - IO failurepublic void setPermission(Path p, FsPermission permission) throws IOException
setPermission in class FileSystemp - The pathpermission - permissionIOException - IO failurepublic FsServerDefaults getServerDefaults() throws IOException
FileSystemgetServerDefaults in class FileSystemIOException - IO failurepublic FsServerDefaults getServerDefaults(Path f) throws IOException
FileSystemgetServerDefaults in class FileSystemf - path is used to identify an FS since an FS could have
another FS that it could be delegating the call toIOException - IO failurepublic long getUsed()
throws IOException
FileSystemgetUsed in class FileSystemIOException - IO failurepublic long getUsed(Path path) throws IOException
getUsed in class FileSystemIOException - IO failurepublic long getDefaultBlockSize()
FileSystemgetDefaultBlockSize in class FileSystempublic long getDefaultBlockSize(Path f)
FileSystemgetDefaultBlockSize in class FileSystemf - path of filepublic short getDefaultReplication()
FileSystemgetDefaultReplication in class FileSystempublic short getDefaultReplication(Path f)
FileSystemgetDefaultReplication in class FileSystemf - of the filepublic FSDataOutputStreamBuilder createFile(Path path)
FileSystemcreateFile in class FileSystempath - file pathpublic FSDataOutputStreamBuilder appendFile(Path path)
FileSystemappendFile in class FileSystempath - file path.FSDataOutputStreamBuilder to build file append request.Copyright © 2019 Apache Software Foundation. All Rights Reserved.