org.apache.hadoop.contrib.failmon
Class LocalStore

java.lang.Object
  extended by org.apache.hadoop.contrib.failmon.LocalStore

public class LocalStore
extends Object

This class takes care of the temporary local storage of gathered metrics before they get uploaded into HDFS. It writes Serialized Records as lines in a temporary file and then compresses and uploads it into HDFS.


Field Summary
static String COMPRESSION_SUFFIX
           
static char FIELD_SEPARATOR
           
static char RECORD_SEPARATOR
           
static int UPLOAD_INTERVAL
           
 
Constructor Summary
LocalStore()
          Create an instance of the class and read the configuration file to determine some output parameters.
 
Method Summary
 void close()
          Close the temporary local file
static void copyToHDFS(String localFile, String hdfsFile)
          Copy a local file to HDFS
 void insert(EventRecord er)
          Insert an EventRecord to the local storage, after it gets serialized and anonymized.
 void insert(EventRecord[] ers)
          Insert an array of EventRecords to the local storage, after they get serialized and anonymized.
static StringBuffer pack(SerializedRecord sr)
          Pack a SerializedRecord into an array of bytes
 void upload()
          Upload the local file store into HDFS, after it compressing it.
static void zipCompress(String filename)
          Compress a text file using the ZIP compressing algorithm.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

FIELD_SEPARATOR

public static final char FIELD_SEPARATOR
See Also:
Constant Field Values

RECORD_SEPARATOR

public static final char RECORD_SEPARATOR
See Also:
Constant Field Values

COMPRESSION_SUFFIX

public static final String COMPRESSION_SUFFIX
See Also:
Constant Field Values

UPLOAD_INTERVAL

public static final int UPLOAD_INTERVAL
See Also:
Constant Field Values
Constructor Detail

LocalStore

public LocalStore()
Create an instance of the class and read the configuration file to determine some output parameters. Then, initiate the structured needed for the buffered I/O (so that smal appends can be handled efficiently).

Method Detail

insert

public void insert(EventRecord er)
Insert an EventRecord to the local storage, after it gets serialized and anonymized.

Parameters:
er - the EventRecord to be inserted

insert

public void insert(EventRecord[] ers)
Insert an array of EventRecords to the local storage, after they get serialized and anonymized.

Parameters:
ers - the array of EventRecords to be inserted

pack

public static StringBuffer pack(SerializedRecord sr)
Pack a SerializedRecord into an array of bytes

Parameters:
sr - the SerializedRecord to be packed

upload

public void upload()
Upload the local file store into HDFS, after it compressing it. Then a new local file is created as a temporary record store.


zipCompress

public static void zipCompress(String filename)
                        throws IOException
Compress a text file using the ZIP compressing algorithm.

Parameters:
filename - the path to the file to be compressed
Throws:
IOException

copyToHDFS

public static void copyToHDFS(String localFile,
                              String hdfsFile)
                       throws IOException
Copy a local file to HDFS

Parameters:
localFile - the filename of the local file
hdfsFile - the HDFS filename to copy to
Throws:
IOException

close

public void close()
Close the temporary local file



Copyright © 2009 The Apache Software Foundation