org.apache.hadoop.io

Class WritableUtils

java.lang.Object

org.apache.hadoop.io.WritableUtils

@InterfaceAudience.Public
@InterfaceStability.Stable
public final class WritableUtils
extends Object

Constructor Summary

Constructors

Constructor and Description
WritableUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static <T extends Writable> T	clone(T orig, Configuration conf) Make a copy of a writable object using serialization to a buffer.
static void	cloneInto(Writable dst, Writable src) Deprecated. use ReflectionUtils.cloneInto instead.
static int	decodeVIntSize(byte value) Parse the first byte of a vint/vlong to determine the number of bytes
static void	displayByteArray(byte[] record)
static int	getVIntSize(long i) Get the encoded length if an integer is stored in a variable-length format
static boolean	isNegativeVInt(byte value) Given the first byte of a vint/vlong, determine the sign
static byte[]	readCompressedByteArray(DataInput in)
static String	readCompressedString(DataInput in)
static String[]	readCompressedStringArray(DataInput in)
static <T extends Enum<T>> T	readEnum(DataInput in, Class<T> enumType) Read an Enum value from DataInput, Enums are read and written using String values.
static String	readString(DataInput in)
static String[]	readStringArray(DataInput in)
static String	readStringSafely(DataInput in, int maxLength) Read a string, but check it for sanity.
static int	readVInt(DataInput stream) Reads a zero-compressed encoded integer from input stream and returns it.
static int	readVIntInRange(DataInput stream, int lower, int upper) Reads an integer from the input stream and returns it.
static long	readVLong(DataInput stream) Reads a zero-compressed encoded long from input stream and returns it.
static void	skipCompressedByteArray(DataInput in)
static void	skipFully(DataInput in, int len) Skip len number of bytes in input streamin
static byte[]	toByteArray(Writable... writables) Convert writables to a byte array
static int	writeCompressedByteArray(DataOutput out, byte[] bytes)
static int	writeCompressedString(DataOutput out, String s)
static void	writeCompressedStringArray(DataOutput out, String[] s)
static void	writeEnum(DataOutput out, Enum<?> enumVal) writes String value of enum to DataOutput.
static void	writeString(DataOutput out, String s)
static void	writeStringArray(DataOutput out, String[] s)
static void	writeVInt(DataOutput stream, int i) Serializes an integer to a binary stream with zero-compressed encoding.
static void	writeVLong(DataOutput stream, long i) Serializes a long to a binary stream with zero-compressed encoding.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WritableUtils

public WritableUtils()

Method Detail

readCompressedByteArray

public static byte[] readCompressedByteArray(DataInput in)
                                      throws IOException

Throws:

IOException

skipCompressedByteArray

public static void skipCompressedByteArray(DataInput in)
                                    throws IOException

Throws:

IOException

writeCompressedByteArray

public static int writeCompressedByteArray(DataOutput out,
                           byte[] bytes)
                                    throws IOException

Throws:

IOException

readCompressedString

public static String readCompressedString(DataInput in)
                                   throws IOException

Throws:

IOException

writeCompressedString

public static int writeCompressedString(DataOutput out,
                        String s)
                                 throws IOException

Throws:

IOException

writeString

public static void writeString(DataOutput out,
               String s)
                        throws IOException

Throws:

IOException

readString

public static String readString(DataInput in)
                         throws IOException

Throws:

IOException

writeStringArray

public static void writeStringArray(DataOutput out,
                    String[] s)
                             throws IOException

Throws:

IOException

writeCompressedStringArray

public static void writeCompressedStringArray(DataOutput out,
                              String[] s)
                                       throws IOException

Throws:

IOException

readStringArray

public static String[] readStringArray(DataInput in)
                                throws IOException

Throws:

IOException

readCompressedStringArray

public static String[] readCompressedStringArray(DataInput in)
                                          throws IOException

Throws:

IOException

displayByteArray

public static void displayByteArray(byte[] record)

clone

public static <T extends Writable> T clone(T orig,
                           Configuration conf)

Make a copy of a writable object using serialization to a buffer.

Parameters:

orig - The object to copy

Returns:

The copied object

cloneInto

@Deprecated
public static void cloneInto(Writable dst,
                        Writable src)
                      throws IOException

Deprecated. use ReflectionUtils.cloneInto instead.

Make a copy of the writable object using serialiation to a buffer

Parameters:

dst - the object to copy from

src - the object to copy into, which is destroyed

Throws:

IOException

writeVInt

public static void writeVInt(DataOutput stream,
             int i)
                      throws IOException

Serializes an integer to a binary stream with zero-compressed encoding. For -112 <= i <= 127, only one byte is used with the actual value. For other values of i, the first byte value indicates whether the integer is positive or negative, and the number of bytes that follow. If the first byte value v is between -113 and -116, the following integer is positive, with number of bytes that follow are -(v+112). If the first byte value v is between -121 and -124, the following integer is negative, with number of bytes that follow are -(v+120). Bytes are stored in the high-non-zero-byte-first order.

Parameters:

stream - Binary output stream

i - Integer to be serialized

Throws:

IOException

writeVLong

public static void writeVLong(DataOutput stream,
              long i)
                       throws IOException

Serializes a long to a binary stream with zero-compressed encoding. For -112 <= i <= 127, only one byte is used with the actual value. For other values of i, the first byte value indicates whether the long is positive or negative, and the number of bytes that follow. If the first byte value v is between -113 and -120, the following long is positive, with number of bytes that follow are -(v+112). If the first byte value v is between -121 and -128, the following long is negative, with number of bytes that follow are -(v+120). Bytes are stored in the high-non-zero-byte-first order.

Parameters:

stream - Binary output stream

i - Long to be serialized

Throws:

IOException

readVLong

public static long readVLong(DataInput stream)
                      throws IOException

Reads a zero-compressed encoded long from input stream and returns it.

Parameters:

stream - Binary input stream

Returns:

deserialized long from stream.

Throws:

IOException

readVInt

public static int readVInt(DataInput stream)
                    throws IOException

Reads a zero-compressed encoded integer from input stream and returns it.

Parameters:

stream - Binary input stream

Returns:

deserialized integer from stream.

Throws:

IOException

readVIntInRange

public static int readVIntInRange(DataInput stream,
                  int lower,
                  int upper)
                           throws IOException

Reads an integer from the input stream and returns it. This function validates that the integer is between [lower, upper], inclusive.

Parameters:

stream - Binary input stream

Returns:

deserialized integer from stream

Throws:

IOException

isNegativeVInt

public static boolean isNegativeVInt(byte value)

Given the first byte of a vint/vlong, determine the sign

Parameters:

value - the first byte

Returns:

is the value negative

decodeVIntSize

public static int decodeVIntSize(byte value)

Parse the first byte of a vint/vlong to determine the number of bytes

Parameters:

value - the first byte of the vint/vlong

Returns:

the total number of bytes (1 to 9)

getVIntSize

public static int getVIntSize(long i)

Get the encoded length if an integer is stored in a variable-length format

Returns:

the encoded length

readEnum

public static <T extends Enum<T>> T readEnum(DataInput in,
                             Class<T> enumType)
                               throws IOException

Read an Enum value from DataInput, Enums are read and written using String values.

Type Parameters:

T - Enum type

Parameters:

in - DataInput to read from

enumType - Class type of Enum

Returns:

Enum represented by String read from DataInput

Throws:

IOException

writeEnum

public static void writeEnum(DataOutput out,
             Enum<?> enumVal)
                      throws IOException

writes String value of enum to DataOutput.

Parameters:

out - Dataoutput stream

enumVal - enum value

Throws:

IOException

skipFully

public static void skipFully(DataInput in,
             int len)
                      throws IOException

Skip len number of bytes in input streamin

Parameters:

in - input stream

len - number of bytes to skip

Throws:

IOException - when skipped less number of bytes

toByteArray

public static byte[] toByteArray(Writable... writables)

Convert writables to a byte array

readStringSafely

public static String readStringSafely(DataInput in,
                      int maxLength)
                               throws IOException,
                                      IllegalArgumentException

Read a string, but check it for sanity. The format consists of a vint followed by the given number of bytes.

Parameters:

in - the stream to read from

maxLength - the largest acceptable length of the encoded string

Returns:

the bytes as a string

Throws:

IOException - if reading from the DataInput fails

IllegalArgumentException - if the encoded byte size for string is negative or larger than maxSize. Only the vint is read.