Offline Edits Viewer is a tool to parse the Edits log file. The current processors are mostly useful for conversion between different formats, including XML which is human readable and easier to edit than native binary format.
The tool can parse the edits formats -18 (roughly Hadoop 0.19) and later. The tool operates on files only, it does not need Hadoop cluster to be running.
Input formats supported:
The Offline Edits Viewer provides several output processors (unless stated otherwise the output of the processor can be converted back to original edits file):
bash$ bin/hdfs oev -i edits -o edits.xml
| Flag | Description |
|---|---|
| [-i ; --inputFile] input file | Specify the input edits log file to process. Xml (case insensitive) extension means XML format otherwise binary format is assumed. Required. |
| [-o ; --outputFile] output file | Specify the output filename, if the specified output processor generates one. If the specified file already exists, it is silently overwritten. Required. |
| [-p ; --processor] processor | Specify the image processor to apply against the image file. Currently valid options are binary, xml (default) and stats. |
| [-v ; --verbose] | Print the input and output filenames and pipe output of processor to console as well as specified file. On extremely large files, this may increase processing time by an order of magnitude. |
| [-f ; --fix-txids] | Renumber the transaction IDs in the input, so that there are no gaps or invalid transaction IDs. |
| [-r ; --recover] | When reading binary edit logs, use recovery mode. This will give you the chance to skip corrupt parts of the edit log. |
| [-h ; --help] | Display the tool usage and help information and exit. |
In case there is some problem with hadoop cluster and the edits file is corrupted it is possible to save at least part of the edits file that is correct. This can be done by converting the binary edits to XML, edit it manually and then convert it back to binary. The most common problem is that the edits file is missing the closing record (record that has opCode -1). This should be recognized by the tool and the XML format should be properly closed.
If there is no closing record in the XML file you can add one after last correct record. Anything after the record with opCode -1 is ignored.
Example of a closing record (with opCode -1):
<RECORD>
<OPCODE>-1</OPCODE>
<DATA>
</DATA>
</RECORD>