Package org.apache.hadoop.util.functional

Class FutureIO

java.lang.Object

org.apache.hadoop.util.functional.FutureIO

@Public
@Unstable
public final class FutureIO
extends Object

Future IO Helper methods.

Contains methods promoted from FutureIOSupport because they are a key part of integrating async IO in application code.

One key feature is that the awaitFuture(Future) and awaitFuture(Future, long, TimeUnit) calls will extract and rethrow exceptions raised in the future's execution, including extracting the inner IOException of any UncheckedIOException raised in the future. This makes it somewhat easier to execute IOException-raising code inside futures.

Important: any CancellationException raised by the future is rethrown unchanged. This has been the implicit behavior since this code was first written, and is now explicitly documented.

Method Summary

Modifier and Type	Method	Description
static <T> List<T>	awaitAllFutures(Collection<Future<T>> collection)	Evaluates a collection of futures and returns their results as a list.
static <T> List<T>	awaitAllFutures(Collection<Future<T>> collection, Duration duration)	Evaluates a collection of futures and returns their results as a list, but only waits up to the specified timeout for each future to complete.
static <T> T	awaitFuture(Future<T> future)	Given a future, evaluate it.
static <T> T	awaitFuture(Future<T> future, long timeout, TimeUnit unit)	Given a future, evaluate it.
static <T> List<T>	cancelAllFuturesAndAwaitCompletion(Collection<Future<T>> collection, boolean interruptIfRunning, Duration duration)	Cancels a collection of futures and awaits the specified duration for their completion.
static <T> CompletableFuture<T>	eval(org.apache.hadoop.util.functional.CallableRaisingIOE<T> callable)	Evaluate a CallableRaisingIOE in the current thread, converting IOEs to RTEs and propagating.
static void	propagateOptions(FSBuilder<?,?> builder, Configuration conf, String prefix, boolean mandatory)	Propagate options to any builder, converting everything with the prefix to an option where, if there were 2+ dot-separated elements, it is converted to a schema.
static <T, U extends FSBuilder<T, U>> FSBuilder<T,U>	propagateOptions(FSBuilder<T,U> builder, Configuration conf, String optionalPrefix, String mandatoryPrefix)	Propagate options to any builder, converting everything with the prefix to an option where, if there were 2+ dot-separated elements, it is converted to a schema.
static <T> T	raiseInnerCause(CompletionException e)	Extract the cause of a completion failure and rethrow it if an IOE or RTE.
static <T> T	raiseInnerCause(ExecutionException e)	From the inner cause of an execution exception, extract the inner cause if it is an IOE or RTE.
static IOException	unwrapInnerException(Throwable e)	From the inner cause of an execution exception, extract the inner cause to an IOException, raising RuntimeExceptions and Errors immediately.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

awaitFuture

public static <T> T awaitFuture(Future<T> future)
                         throws InterruptedIOException,
IOException,
CancellationException,
RuntimeException

Given a future, evaluate it.

Any exception generated in the future is extracted and rethrown.

If this thread is interrupted while waiting for the future to complete, an InterruptedIOException is raised. However, if the future is cancelled, a CancellationException is raised in the {code Future.get()} call. This is passed up as is -so allowing the caller to distinguish between thread interruption (such as when speculative task execution is aborted) and future cancellation.

Type Parameters:

T - type of the result.

Parameters:

future - future to evaluate

Returns:

the result, if all went well.

Throws:

InterruptedIOException - waiting for future completion was interrupted

CancellationException - if the future itself was cancelled

IOException - if something went wrong

RuntimeException - any nested RTE thrown

awaitFuture

public static <T> T awaitFuture(Future<T> future,
 long timeout,
 TimeUnit unit)
                         throws InterruptedIOException,
IOException,
CancellationException,
RuntimeException,
TimeoutException

Given a future, evaluate it.

Any exception generated in the future is extracted and rethrown.

Type Parameters:

T - type of the result.

Parameters:

future - future to evaluate

timeout - timeout to wait.

unit - time unit.

Returns:

the result, if all went well.

Throws:

InterruptedIOException - waiting for future completion was interrupted

CancellationException - if the future itself was cancelled

IOException - if something went wrong

RuntimeException - any nested RTE thrown

TimeoutException - the future timed out.

awaitAllFutures

public static <T> List<T> awaitAllFutures(Collection<Future<T>> collection)
                                   throws InterruptedIOException,
IOException,
CancellationException,
RuntimeException

Evaluates a collection of futures and returns their results as a list.

This method blocks until all futures in the collection have completed. If any future throws an exception during its execution, this method extracts and rethrows that exception.

Type Parameters:

T - type of the result.

Parameters:

collection - collection of futures to be evaluated

Returns:

the list of future's result, if all went well.

Throws:

InterruptedIOException - waiting for future completion was interrupted

CancellationException - if the future itself was cancelled

IOException - if something went wrong

RuntimeException - any nested RTE thrown

awaitAllFutures

public static <T> List<T> awaitAllFutures(Collection<Future<T>> collection,
 Duration duration)
                                   throws InterruptedIOException,
IOException,
CancellationException,
RuntimeException,
TimeoutException

Evaluates a collection of futures and returns their results as a list, but only waits up to the specified timeout for each future to complete.

This method blocks until all futures in the collection have completed or the timeout expires, whichever happens first. If any future throws an exception during its execution, this method extracts and rethrows that exception.

Type Parameters:

T - type of the result.

Parameters:

collection - collection of futures to be evaluated

duration - timeout duration

Returns:

the list of future's result, if all went well.

Throws:

InterruptedIOException - waiting for future completion was interrupted

CancellationException - if the future itself was cancelled

IOException - if something went wrong

RuntimeException - any nested RTE thrown

TimeoutException - the future timed out.

cancelAllFuturesAndAwaitCompletion

public static <T> List<T> cancelAllFuturesAndAwaitCompletion(Collection<Future<T>> collection,
 boolean interruptIfRunning,
 Duration duration)

Cancels a collection of futures and awaits the specified duration for their completion.

This method blocks until all futures in the collection have completed or the timeout expires, whichever happens first. All exceptions thrown by the futures are ignored. as is any TimeoutException.

Type Parameters:

T - type of the result.

Parameters:

collection - collection of futures to be evaluated

interruptIfRunning - should the cancel interrupt any active futures?

duration - total timeout duration

Returns:

all futures which completed successfully.

raiseInnerCause

public static <T> T raiseInnerCause(ExecutionException e)
                             throws IOException

From the inner cause of an execution exception, extract the inner cause if it is an IOE or RTE. This will always raise an exception, either the inner IOException, an inner RuntimeException, or a new IOException wrapping the raised exception.

Type Parameters:

T - type of return value.

Parameters:

e - exception.

Returns:

nothing, ever.

Throws:

IOException - either the inner IOException, or a wrapper around any non-Runtime-Exception

RuntimeException - if that is the inner cause.

raiseInnerCause

public static <T> T raiseInnerCause(CompletionException e)
                             throws IOException

Extract the cause of a completion failure and rethrow it if an IOE or RTE.

Type Parameters:

T - type of return value.

Parameters:

e - exception.

Returns:

nothing, ever.

Throws:

IOException - either the inner IOException, or a wrapper around any non-Runtime-Exception

RuntimeException - if that is the inner cause.

unwrapInnerException

public static IOException unwrapInnerException(Throwable e)

From the inner cause of an execution exception, extract the inner cause to an IOException, raising RuntimeExceptions and Errors immediately.

1. If it is an IOE: Return.
2. If it is a UncheckedIOException: return the cause
3. Completion/Execution Exceptions: extract and repeat
4. If it is an RTE or Error: throw.
5. Any other type: wrap in an IOE

Recursively handles wrapped Execution and Completion Exceptions in case something very complicated has happened.

Parameters:

e - exception.

Returns:

an IOException extracted or built from the cause.

Throws:

RuntimeException - if that is the inner cause.

Error - if that is the inner cause.

propagateOptions

public static <T,
U extends FSBuilder<T,
U>>
FSBuilder<T,U> propagateOptions(FSBuilder<T,U> builder,
 Configuration conf,
 String optionalPrefix,
 String mandatoryPrefix)

Propagate options to any builder, converting everything with the prefix to an option where, if there were 2+ dot-separated elements, it is converted to a schema. See propagateOptions(FSBuilder, Configuration, String, boolean).

Type Parameters:

T - type of result

U - type of builder

Parameters:

builder - builder to modify

conf - configuration to read

optionalPrefix - prefix for optional settings

mandatoryPrefix - prefix for mandatory settings

Returns:

the builder passed in.

propagateOptions

public static void propagateOptions(FSBuilder<?,?> builder,
 Configuration conf,
 String prefix,
 boolean mandatory)

Propagate options to any builder, converting everything with the prefix to an option where, if there were 2+ dot-separated elements, it is converted to a schema.

   fs.example.s3a.option becomes "s3a.option"
   fs.example.fs.io.policy becomes "fs.io.policy"
   fs.example.something becomes "something"
 

Parameters:

builder - builder to modify

conf - configuration to read

prefix - prefix to scan/strip

mandatory - are the options to be mandatory or optional?

eval

public static <T> CompletableFuture<T> eval(org.apache.hadoop.util.functional.CallableRaisingIOE<T> callable)

Evaluate a CallableRaisingIOE in the current thread, converting IOEs to RTEs and propagating.

Type Parameters:

T - Return type.

Parameters:

callable - callable to invoke

Returns:

the evaluated result.

Throws:

UnsupportedOperationException - fail fast if unsupported

IllegalArgumentException - invalid argument