org.apache.hadoop.mapreduce.lib.join

Class OverrideRecordReader<K extends WritableComparable<?>,V extends Writable>

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<K,V>

org.apache.hadoop.mapreduce.lib.join.ComposableRecordReader<K,X>

org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader<K,V,V>

org.apache.hadoop.mapreduce.lib.join.MultiFilterRecordReader<K,V>

org.apache.hadoop.mapreduce.lib.join.OverrideRecordReader<K,V>

All Implemented Interfaces:

Closeable, AutoCloseable, Comparable<ComposableRecordReader<K,?>>, Configurable

@InterfaceAudience.Public
@InterfaceStability.Stable
public class OverrideRecordReader<K extends WritableComparable<?>,V extends Writable>
extends MultiFilterRecordReader<K,V>

Prefer the "rightmost" data source for this key. For example, override(S1,S2,S3) will prefer values from S3 over S2, and values from S2 over S1 for all keys emitted from all sources.

Field Summary

Fields inherited from class org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader

conf, jc, key, keyclass, kids, value

Method Summary

Methods

Modifier and Type	Method and Description
V	createValue() Create instance of value.
protected V	emit(TupleWritable dst) Emit the value with the highest position in the tuple.
protected void	fillJoinCollector(K iterkey) Instead of filling the JoinCollector with iterators from all data sources, fill only the rightmost for this key.

Methods inherited from class org.apache.hadoop.mapreduce.lib.join.MultiFilterRecordReader

combine, getDelegate, initialize, nextKeyValue

Methods inherited from class org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader

accept, add, close, compareTo, createKey, createTupleWritable, getComparator, getConf, getCurrentKey, getCurrentValue, getProgress, getRecordReaderQueue, hasNext, id, key, key, setConf, skip

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

emit

protected V emit(TupleWritable dst)

Emit the value with the highest position in the tuple.

Specified by:

emit in class MultiFilterRecordReader<K extends WritableComparable<?>,V extends Writable>

createValue

public V createValue()

Description copied from class: ComposableRecordReader

Create instance of value.

fillJoinCollector

protected void fillJoinCollector(K iterkey)
                          throws IOException,
                                 InterruptedException

Instead of filling the JoinCollector with iterators from all data sources, fill only the rightmost for this key. This not only saves space by discarding the other sources, but it also emits the number of key-value pairs in the preferred RecordReader instead of repeating that stream n times, where n is the cardinality of the cross product of the discarded streams for the given key.

Overrides:

fillJoinCollector in class CompositeRecordReader<K extends WritableComparable<?>,V extends Writable,V extends Writable>

Throws:

IOException

InterruptedException