org.apache.hadoop.io
Class Text

java.lang.Object
  extended by org.apache.hadoop.io.BinaryComparable
      extended by org.apache.hadoop.io.Text
All Implemented Interfaces:
Comparable<BinaryComparable>, Writable, WritableComparable<BinaryComparable>

@Stringable
@InterfaceAudience.Public
@InterfaceStability.Stable
public class Text
extends BinaryComparable
implements WritableComparable<BinaryComparable>

This class stores text using standard UTF8 encoding. It provides methods to serialize, deserialize, and compare texts at byte level. The type of length is integer and is serialized using zero-compressed format.

In addition, it provides methods for string traversal without converting the byte array to a string.

Also includes utilities for serializing/deserialing a string, coding/decoding a string, checking if a byte array contains valid UTF8 code, calculating the length of an encoded string.


Field Summary
static int DEFAULT_MAX_LEN
           
 
Constructor Summary
Text()
           
Text(byte[] utf8)
          Construct from a byte array.
Text(String string)
          Construct from a string.
Text(Text utf8)
          Construct from another text.
 
Method Summary
 void append(byte[] utf8, int start, int len)
          Append a range of bytes to the end of the given text
static int bytesToCodePoint(ByteBuffer bytes)
          Returns the next code point at the current position in the buffer.
 int charAt(int position)
          Returns the Unicode Scalar Value (32-bit integer value) for the character at position.
 void clear()
          Clear the string to empty.
 byte[] copyBytes()
          Get a copy of the bytes that is exactly the length of the data.
static String decode(byte[] utf8)
          Converts the provided byte array to a String using the UTF-8 encoding.
static String decode(byte[] utf8, int start, int length)
           
static String decode(byte[] utf8, int start, int length, boolean replace)
          Converts the provided byte array to a String using the UTF-8 encoding.
static ByteBuffer encode(String string)
          Converts the provided String to bytes using the UTF-8 encoding.
static ByteBuffer encode(String string, boolean replace)
          Converts the provided String to bytes using the UTF-8 encoding.
 boolean equals(Object o)
          Returns true iff o is a Text with the same contents.
 int find(String what)
           
 int find(String what, int start)
          Finds any occurence of what in the backing buffer, starting as position start.
 byte[] getBytes()
          Returns the raw bytes; however, only data up to getLength() is valid.
 int getLength()
          Returns the number of bytes in the byte array
 int hashCode()
          Return a hash of the bytes returned from {#getBytes()}.
 void readFields(DataInput in)
          deserialize
 void readFields(DataInput in, int maxLength)
           
static String readString(DataInput in)
          Read a UTF8 encoded string from in
static String readString(DataInput in, int maxLength)
          Read a UTF8 encoded string with a maximum size
 void readWithKnownLength(DataInput in, int len)
          Read a Text object whose length is already known.
 void set(byte[] utf8)
          Set to a utf8 byte array
 void set(byte[] utf8, int start, int len)
          Set the Text to range of bytes
 void set(String string)
          Set to contain the contents of a string.
 void set(Text other)
          copy a text.
static void skip(DataInput in)
          Skips over one Text in the input.
 String toString()
          Convert text back to string
static int utf8Length(String string)
          For the given string, returns the number of UTF-8 bytes required to encode the string.
static void validateUTF8(byte[] utf8)
          Check if a byte array contains valid utf-8
static void validateUTF8(byte[] utf8, int start, int len)
          Check to see if a byte array is valid utf-8
 void write(DataOutput out)
          serialize write this object to out length uses zero-compressed encoding
 void write(DataOutput out, int maxLength)
           
static int writeString(DataOutput out, String s)
          Write a UTF8 encoded string to out
static int writeString(DataOutput out, String s, int maxLength)
          Write a UTF8 encoded string with a maximum size to out
 
Methods inherited from class org.apache.hadoop.io.BinaryComparable
compareTo, compareTo
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
 
Methods inherited from interface java.lang.Comparable
compareTo
 

Field Detail

DEFAULT_MAX_LEN

public static final int DEFAULT_MAX_LEN
See Also:
Constant Field Values
Constructor Detail

Text

public Text()

Text

public Text(String string)
Construct from a string.


Text

public Text(Text utf8)
Construct from another text.


Text

public Text(byte[] utf8)
Construct from a byte array.

Method Detail

copyBytes

public byte[] copyBytes()
Get a copy of the bytes that is exactly the length of the data. See getBytes() for faster access to the underlying array.


getBytes

public byte[] getBytes()
Returns the raw bytes; however, only data up to getLength() is valid. Please use copyBytes() if you need the returned array to be precisely the length of the data.

Specified by:
getBytes in class BinaryComparable

getLength

public int getLength()
Returns the number of bytes in the byte array

Specified by:
getLength in class BinaryComparable

charAt

public int charAt(int position)
Returns the Unicode Scalar Value (32-bit integer value) for the character at position. Note that this method avoids using the converter or doing String instantiation

Returns:
the Unicode scalar value at position or -1 if the position is invalid or points to a trailing byte

find

public int find(String what)

find

public int find(String what,
                int start)
Finds any occurence of what in the backing buffer, starting as position start. The starting position is measured in bytes and the return value is in terms of byte position in the buffer. The backing buffer is not converted to a string for this operation.

Returns:
byte position of the first occurence of the search string in the UTF-8 buffer or -1 if not found

set

public void set(String string)
Set to contain the contents of a string.


set

public void set(byte[] utf8)
Set to a utf8 byte array


set

public void set(Text other)
copy a text.


set

public void set(byte[] utf8,
                int start,
                int len)
Set the Text to range of bytes

Parameters:
utf8 - the data to copy from
start - the first position of the new string
len - the number of bytes of the new string

append

public void append(byte[] utf8,
                   int start,
                   int len)
Append a range of bytes to the end of the given text

Parameters:
utf8 - the data to copy from
start - the first position to append from utf8
len - the number of bytes to append

clear

public void clear()
Clear the string to empty. Note: For performance reasons, this call does not clear the underlying byte array that is retrievable via getBytes(). In order to free the byte-array memory, call set(byte[]) with an empty byte array (For example, new byte[0]).


toString

public String toString()
Convert text back to string

Overrides:
toString in class Object
See Also:
Object.toString()

readFields

public void readFields(DataInput in)
                throws IOException
deserialize

Specified by:
readFields in interface Writable
Parameters:
in - DataInput to deseriablize this object from.
Throws:
IOException

readFields

public void readFields(DataInput in,
                       int maxLength)
                throws IOException
Throws:
IOException

skip

public static void skip(DataInput in)
                 throws IOException
Skips over one Text in the input.

Throws:
IOException

readWithKnownLength

public void readWithKnownLength(DataInput in,
                                int len)
                         throws IOException
Read a Text object whose length is already known. This allows creating Text from a stream which uses a different serialization format.

Throws:
IOException

write

public void write(DataOutput out)
           throws IOException
serialize write this object to out length uses zero-compressed encoding

Specified by:
write in interface Writable
Parameters:
out - DataOuput to serialize this object into.
Throws:
IOException
See Also:
Writable.write(DataOutput)

write

public void write(DataOutput out,
                  int maxLength)
           throws IOException
Throws:
IOException

equals

public boolean equals(Object o)
Returns true iff o is a Text with the same contents.

Overrides:
equals in class BinaryComparable

hashCode

public int hashCode()
Description copied from class: BinaryComparable
Return a hash of the bytes returned from {#getBytes()}.

Overrides:
hashCode in class BinaryComparable
See Also:
WritableComparator.hashBytes(byte[],int)

decode

public static String decode(byte[] utf8)
                     throws CharacterCodingException
Converts the provided byte array to a String using the UTF-8 encoding. If the input is malformed, replace by a default value.

Throws:
CharacterCodingException

decode

public static String decode(byte[] utf8,
                            int start,
                            int length)
                     throws CharacterCodingException
Throws:
CharacterCodingException

decode

public static String decode(byte[] utf8,
                            int start,
                            int length,
                            boolean replace)
                     throws CharacterCodingException
Converts the provided byte array to a String using the UTF-8 encoding. If replace is true, then malformed input is replaced with the substitution character, which is U+FFFD. Otherwise the method throws a MalformedInputException.

Throws:
CharacterCodingException

encode

public static ByteBuffer encode(String string)
                         throws CharacterCodingException
Converts the provided String to bytes using the UTF-8 encoding. If the input is malformed, invalid chars are replaced by a default value.

Returns:
ByteBuffer: bytes stores at ByteBuffer.array() and length is ByteBuffer.limit()
Throws:
CharacterCodingException

encode

public static ByteBuffer encode(String string,
                                boolean replace)
                         throws CharacterCodingException
Converts the provided String to bytes using the UTF-8 encoding. If replace is true, then malformed input is replaced with the substitution character, which is U+FFFD. Otherwise the method throws a MalformedInputException.

Returns:
ByteBuffer: bytes stores at ByteBuffer.array() and length is ByteBuffer.limit()
Throws:
CharacterCodingException

readString

public static String readString(DataInput in)
                         throws IOException
Read a UTF8 encoded string from in

Throws:
IOException

readString

public static String readString(DataInput in,
                                int maxLength)
                         throws IOException
Read a UTF8 encoded string with a maximum size

Throws:
IOException

writeString

public static int writeString(DataOutput out,
                              String s)
                       throws IOException
Write a UTF8 encoded string to out

Throws:
IOException

writeString

public static int writeString(DataOutput out,
                              String s,
                              int maxLength)
                       throws IOException
Write a UTF8 encoded string with a maximum size to out

Throws:
IOException

validateUTF8

public static void validateUTF8(byte[] utf8)
                         throws MalformedInputException
Check if a byte array contains valid utf-8

Parameters:
utf8 - byte array
Throws:
MalformedInputException - if the byte array contains invalid utf-8

validateUTF8

public static void validateUTF8(byte[] utf8,
                                int start,
                                int len)
                         throws MalformedInputException
Check to see if a byte array is valid utf-8

Parameters:
utf8 - the array of bytes
start - the offset of the first byte in the array
len - the length of the byte sequence
Throws:
MalformedInputException - if the byte array contains invalid bytes

bytesToCodePoint

public static int bytesToCodePoint(ByteBuffer bytes)
Returns the next code point at the current position in the buffer. The buffer's position will be incremented. Any mark set on this buffer will be changed by this method!


utf8Length

public static int utf8Length(String string)
For the given string, returns the number of UTF-8 bytes required to encode the string.

Parameters:
string - text to encode
Returns:
number of UTF-8 bytes required to encode


Copyright © 2014 Apache Software Foundation. All Rights Reserved.