org.apache.hadoop.io.wrappedio

Class WrappedIO

java.lang.Object

org.apache.hadoop.io.wrappedio.WrappedIO

@InterfaceAudience.Public
 @InterfaceStability.Unstable
public final class WrappedIO
extends Object

Reflection-friendly access to APIs which are not available in some of the older Hadoop versions which libraries still compile against.

The intent is to avoid the need for complex reflection operations including wrapping of parameter classes, direct instantiation of new classes etc.

Method Summary

Modifier and Type	Method and Description
static List<Map.Entry<Path,String>>	bulkDelete_delete(FileSystem fs, Path base, Collection<Path> paths) Delete a list of files/objects.
static int	bulkDelete_pageSize(FileSystem fs, Path path) Get the maximum number of objects/files to delete in a single request.
static void	byteBufferPositionedReadable_readFully(InputStream in, long position, ByteBuffer buf) Delegate to ByteBufferPositionedReadable.read(long, ByteBuffer).
static boolean	byteBufferPositionedReadable_readFullyAvailable(InputStream in) Probe to see if the input stream is an instance of ByteBufferPositionedReadable.
static Path	fileSystem_getEnclosingRoot(FileSystem fs, Path path) Return path of the enclosing root for a given path.
static FSDataInputStream	fileSystem_openFile(FileSystem fs, Path path, String policy, FileStatus status, Long length, Map<String,String> options) OpenFile assistant, easy reflection-based access to FileSystem.openFile(Path) and blocks awaiting the operation completion.
static boolean	pathCapabilities_hasPathCapability(Object fs, Path path, String capability) Does a path have a given capability? Calls PathCapabilities.hasPathCapability(Path, String), mapping IOExceptions to false.
static boolean	streamCapabilities_hasCapability(Object object, String capability) Does an object implement StreamCapabilities and, if so, what is the result of the probe for the capability? Calls StreamCapabilities.hasCapability(String),

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

bulkDelete_pageSize

public static int bulkDelete_pageSize(FileSystem fs,
                                      Path path)

Get the maximum number of objects/files to delete in a single request.

Parameters:

fs - filesystem

path - path to delete under.

Returns:

a number greater than or equal to zero.

Throws:

UnsupportedOperationException - bulk delete under that path is not supported.

IllegalArgumentException - path not valid.

UncheckedIOException - if an IOE was raised.

bulkDelete_delete

public static List<Map.Entry<Path,String>> bulkDelete_delete(FileSystem fs,
                                                             Path base,
                                                             Collection<Path> paths)

Delete a list of files/objects.

• Files must be under the path provided in base.
• The size of the list must be equal to or less than the page size.
• Directories are not supported; the outcome of attempting to delete directories is undefined (ignored; undetected, listed as failures...).
• The operation is not atomic.
• The operation is treated as idempotent: network failures may trigger resubmission of the request -any new objects created under a path in the list may then be deleted.
• There is no guarantee that any parent directories exist after this call.

Parameters:

fs - filesystem

base - path to delete under.

paths - list of paths which must be absolute and under the base path.

Returns:

a list of all the paths which couldn't be deleted for a reason other than "not found" and any associated error message.

Throws:

UnsupportedOperationException - bulk delete under that path is not supported.

UncheckedIOException - if an IOE was raised.

IllegalArgumentException - if a path argument is invalid.

pathCapabilities_hasPathCapability

public static boolean pathCapabilities_hasPathCapability(Object fs,
                                                         Path path,
                                                         String capability)

Does a path have a given capability? Calls PathCapabilities.hasPathCapability(Path, String), mapping IOExceptions to false.

Parameters:

fs - filesystem

path - path to query the capability of.

capability - non-null, non-empty string to query the path for support.

Returns:

true if the capability is supported under that part of the FS. resolving paths or relaying the call.

Throws:

IllegalArgumentException - invalid arguments

streamCapabilities_hasCapability

public static boolean streamCapabilities_hasCapability(Object object,
                                                       String capability)

Does an object implement StreamCapabilities and, if so, what is the result of the probe for the capability? Calls StreamCapabilities.hasCapability(String),

Parameters:

object - object to probe

capability - capability string

Returns:

true iff the object implements StreamCapabilities and the capability is declared available.

fileSystem_openFile

@InterfaceStability.Stable
public static FSDataInputStream fileSystem_openFile(FileSystem fs,
                                                                               Path path,
                                                                               String policy,
                                                                               @Nullable
                                                                               FileStatus status,
                                                                               @Nullable
                                                                               Long length,
                                                                               @Nullable
                                                                               Map<String,String> options)

OpenFile assistant, easy reflection-based access to FileSystem.openFile(Path) and blocks awaiting the operation completion.

Parameters:

fs - filesystem

path - path

policy - read policy

status - optional file status

length - optional file length

options - nullable map of other options

Returns:

stream of the opened file

Throws:

UncheckedIOException - if an IOE was raised.

fileSystem_getEnclosingRoot

public static Path fileSystem_getEnclosingRoot(FileSystem fs,
                                               Path path)
                                        throws IOException

Return path of the enclosing root for a given path. The enclosing root path is a common ancestor that should be used for temp and staging dirs as well as within encryption zones and other restricted directories.

Parameters:

fs - filesystem

path - file path to find the enclosing root path for

Returns:

a path to the enclosing root

Throws:

IOException - early checks like failure to resolve path cause IO failures

byteBufferPositionedReadable_readFully

public static void byteBufferPositionedReadable_readFully(InputStream in,
                                                          long position,
                                                          ByteBuffer buf)

Delegate to ByteBufferPositionedReadable.read(long, ByteBuffer).

Parameters:

in - input stream

position - position within file

buf - the ByteBuffer to receive the results of the read operation. Note: that is the default behaviour of FSDataInputStream.readFully(long, ByteBuffer).

byteBufferPositionedReadable_readFullyAvailable

public static boolean byteBufferPositionedReadable_readFullyAvailable(InputStream in)

Probe to see if the input stream is an instance of ByteBufferPositionedReadable. If the stream is an FSDataInputStream, the wrapped stream is checked.

Parameters:

in - input stream

Returns:

true if the stream implements the interface (including a wrapped stream) and that it declares the stream capability.