org.apache.hadoop.mapreduce.lib.db

Class DBRecordReader<T extends DBWritable>

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<LongWritable,T>

org.apache.hadoop.mapreduce.lib.db.DBRecordReader<T>

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

DataDrivenDBRecordReader, MySQLDBRecordReader, OracleDBRecordReader

@InterfaceAudience.Public
 @InterfaceStability.Evolving
public class DBRecordReader<T extends DBWritable>
extends RecordReader<LongWritable,T>

A RecordReader that reads records from a SQL table. Emits LongWritables containing the record number as key and DBWritables as value.

Field Summary

Fields

Modifier and Type	Field and Description
protected PreparedStatement	statement

Constructor Summary

Constructors

Constructor and Description
DBRecordReader(org.apache.hadoop.mapreduce.lib.db.DBInputFormat.DBInputSplit split, Class<T> inputClass, Configuration conf, Connection conn, DBConfiguration dbConfig, String cond, String[] fields, String table)

Method Summary

Modifier and Type	Method and Description
void	close() Close the record reader.
T	createValue() Deprecated.
protected ResultSet	executeQuery(String query)
protected String	getConditions()
protected Connection	getConnection()
LongWritable	getCurrentKey() Get the current key
T	getCurrentValue() Get the current value.
protected DBConfiguration	getDBConf()
protected String[]	getFieldNames()
long	getPos() Deprecated.
float	getProgress() The current progress of the record reader through its data.
protected String	getSelectQuery() Returns the query for selecting the records, subclasses can override this for custom behaviour.
protected org.apache.hadoop.mapreduce.lib.db.DBInputFormat.DBInputSplit	getSplit()
protected PreparedStatement	getStatement()
protected String	getTableName()
void	initialize(InputSplit split, TaskAttemptContext context) Called once at initialization.
boolean	next(LongWritable key, T value) Deprecated. Use nextKeyValue()
boolean	nextKeyValue() Read the next key, value pair.
protected void	setStatement(PreparedStatement stmt)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

statement

protected PreparedStatement statement

Constructor Detail

DBRecordReader

public DBRecordReader(org.apache.hadoop.mapreduce.lib.db.DBInputFormat.DBInputSplit split,
                      Class<T> inputClass,
                      Configuration conf,
                      Connection conn,
                      DBConfiguration dbConfig,
                      String cond,
                      String[] fields,
                      String table)
               throws SQLException

Parameters:

split - The InputSplit to read data for

Throws:

SQLException

Method Detail

executeQuery

protected ResultSet executeQuery(String query)
                          throws SQLException

Throws:

SQLException

getSelectQuery

protected String getSelectQuery()

Returns the query for selecting the records, subclasses can override this for custom behaviour.

close

public void close()
           throws IOException

Close the record reader.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in class RecordReader<LongWritable,T extends DBWritable>

Throws:

IOException

initialize

public void initialize(InputSplit split,
                       TaskAttemptContext context)
                throws IOException,
                       InterruptedException

Description copied from class: RecordReader

Called once at initialization.

Specified by:

initialize in class RecordReader<LongWritable,T extends DBWritable>

Parameters:

split - the split that defines the range of records to read

context - the information about the task

Throws:

IOException

InterruptedException

getCurrentKey

public LongWritable getCurrentKey()

Get the current key

Specified by:

getCurrentKey in class RecordReader<LongWritable,T extends DBWritable>

Returns:

the current key or null if there is no current key

getCurrentValue

public T getCurrentValue()

Get the current value.

Specified by:

getCurrentValue in class RecordReader<LongWritable,T extends DBWritable>

Returns:

the object that was read

createValue

@Deprecated
public T createValue()

Deprecated.

getPos

@Deprecated
public long getPos()
                        throws IOException

Deprecated.

Throws:

IOException

next

@Deprecated
public boolean next(LongWritable key,
                                T value)
                         throws IOException

Deprecated. Use nextKeyValue()

Throws:

IOException

getProgress

public float getProgress()
                  throws IOException

The current progress of the record reader through its data.

Specified by:

getProgress in class RecordReader<LongWritable,T extends DBWritable>

Returns:

a number between 0.0 and 1.0 that is the fraction of the data read

Throws:

IOException

nextKeyValue

public boolean nextKeyValue()
                     throws IOException

Read the next key, value pair.

Specified by:

nextKeyValue in class RecordReader<LongWritable,T extends DBWritable>

Returns:

true if a key/value pair was read

Throws:

IOException

getSplit

protected org.apache.hadoop.mapreduce.lib.db.DBInputFormat.DBInputSplit getSplit()

getFieldNames

protected String[] getFieldNames()

getTableName

protected String getTableName()

getConditions

protected String getConditions()

getDBConf

protected DBConfiguration getDBConf()

getConnection

protected Connection getConnection()

getStatement

protected PreparedStatement getStatement()

setStatement

protected void setStatement(PreparedStatement stmt)