org.apache.hadoop.mapreduce

Class CryptoUtils

java.lang.Object

org.apache.hadoop.mapreduce.CryptoUtils

@InterfaceAudience.Private
 @InterfaceStability.Unstable
public class CryptoUtils
extends Object

This class provides utilities to make it easier to work with Cryptographic Streams. Specifically for dealing with encrypting intermediate data such MapReduce spill files.

Constructor Summary

Constructors

Constructor and Description
CryptoUtils()

Method Summary

Modifier and Type	Method and Description
static byte[]	createIV(org.apache.hadoop.conf.Configuration conf) This method creates and initializes an IV (Initialization Vector)
static int	cryptoPadding(org.apache.hadoop.conf.Configuration conf)
static boolean	isEncryptedSpillEnabled(org.apache.hadoop.conf.Configuration conf)
static org.apache.hadoop.fs.FSDataInputStream	wrapIfNecessary(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FSDataInputStream in) Wraps a given FSDataInputStream with a CryptoInputStream.
static org.apache.hadoop.fs.FSDataOutputStream	wrapIfNecessary(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FSDataOutputStream out) Wraps a given FSDataOutputStream with a CryptoOutputStream.
static org.apache.hadoop.fs.FSDataOutputStream	wrapIfNecessary(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FSDataOutputStream out, boolean closeOutputStream) Wraps a given FSDataOutputStream with a CryptoOutputStream.
static InputStream	wrapIfNecessary(org.apache.hadoop.conf.Configuration conf, InputStream in, long length) Wraps a given InputStream with a CryptoInputStream.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CryptoUtils

public CryptoUtils()

Method Detail

isEncryptedSpillEnabled

public static boolean isEncryptedSpillEnabled(org.apache.hadoop.conf.Configuration conf)

createIV

public static byte[] createIV(org.apache.hadoop.conf.Configuration conf)
                       throws IOException

This method creates and initializes an IV (Initialization Vector)

Parameters:

conf - configuration

Returns:

byte[] initialization vector

Throws:

IOException - exception in case of error

cryptoPadding

public static int cryptoPadding(org.apache.hadoop.conf.Configuration conf)
                         throws IOException

Throws:

IOException

wrapIfNecessary

public static org.apache.hadoop.fs.FSDataOutputStream wrapIfNecessary(org.apache.hadoop.conf.Configuration conf,
                                                                      org.apache.hadoop.fs.FSDataOutputStream out)
                                                               throws IOException

Wraps a given FSDataOutputStream with a CryptoOutputStream. The size of the data buffer required for the stream is specified by the "mapreduce.job.encrypted-intermediate-data.buffer.kb" Job configuration variable.

Parameters:

conf - configuration

out - given output stream

Returns:

FSDataOutputStream encrypted output stream if encryption is enabled; otherwise the given output stream itself

Throws:

IOException - exception in case of error

wrapIfNecessary

public static org.apache.hadoop.fs.FSDataOutputStream wrapIfNecessary(org.apache.hadoop.conf.Configuration conf,
                                                                      org.apache.hadoop.fs.FSDataOutputStream out,
                                                                      boolean closeOutputStream)
                                                               throws IOException

Wraps a given FSDataOutputStream with a CryptoOutputStream. The size of the data buffer required for the stream is specified by the "mapreduce.job.encrypted-intermediate-data.buffer.kb" Job configuration variable.

Parameters:

conf - configuration

out - given output stream

closeOutputStream - flag to indicate whether closing the wrapped stream will close the given output stream

Returns:

FSDataOutputStream encrypted output stream if encryption is enabled; otherwise the given output stream itself

Throws:

IOException - exception in case of error

wrapIfNecessary

public static InputStream wrapIfNecessary(org.apache.hadoop.conf.Configuration conf,
                                          InputStream in,
                                          long length)
                                   throws IOException

Wraps a given InputStream with a CryptoInputStream. The size of the data buffer required for the stream is specified by the "mapreduce.job.encrypted-intermediate-data.buffer.kb" Job configuration variable. If the value of 'length' is > -1, The InputStream is additionally wrapped in a LimitInputStream. CryptoStreams are late buffering in nature. This means they will always try to read ahead if they can. The LimitInputStream will ensure that the CryptoStream does not read past the provided length from the given Input Stream.

Parameters:

conf - configuration

in - given input stream

length - maximum number of bytes to read from the input stream

Returns:

InputStream encrypted input stream if encryption is enabled; otherwise the given input stream itself

Throws:

IOException - exception in case of error

wrapIfNecessary

public static org.apache.hadoop.fs.FSDataInputStream wrapIfNecessary(org.apache.hadoop.conf.Configuration conf,
                                                                     org.apache.hadoop.fs.FSDataInputStream in)
                                                              throws IOException

Wraps a given FSDataInputStream with a CryptoInputStream. The size of the data buffer required for the stream is specified by the "mapreduce.job.encrypted-intermediate-data.buffer.kb" Job configuration variable.

Parameters:

conf - configuration

in - given input stream

Returns:

FSDataInputStream encrypted input stream if encryption is enabled; otherwise the given input stream itself

Throws:

IOException - exception in case of error