org.apache.hadoop.mapred.join
Class OverrideRecordReader<K extends WritableComparable,V extends Writable>
java.lang.Object
org.apache.hadoop.mapred.join.CompositeRecordReader<K,V,V>
org.apache.hadoop.mapred.join.MultiFilterRecordReader<K,V>
org.apache.hadoop.mapred.join.OverrideRecordReader<K,V>
- All Implemented Interfaces:
- Comparable<ComposableRecordReader<K,?>>, Configurable, ComposableRecordReader<K,V>, RecordReader<K,V>
@InterfaceAudience.Public
@InterfaceStability.Stable
public class OverrideRecordReader<K extends WritableComparable,V extends Writable>
- extends MultiFilterRecordReader<K,V>
Prefer the "rightmost" data source for this key.
For example, override(S1,S2,S3) will prefer values
from S3 over S2, and values from S2 over S1 for all keys
emitted from all sources.
Method Summary |
protected V |
emit(TupleWritable dst)
Emit the value with the highest position in the tuple. |
protected void |
fillJoinCollector(K iterkey)
Instead of filling the JoinCollector with iterators from all
data sources, fill only the rightmost for this key. |
Methods inherited from class org.apache.hadoop.mapred.join.CompositeRecordReader |
accept, add, close, compareTo, createInternalValue, createKey, getComparator, getConf, getPos, getProgress, getRecordReaderQueue, hasNext, id, key, key, setConf, skip |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
emit
protected V emit(TupleWritable dst)
- Emit the value with the highest position in the tuple.
- Specified by:
emit
in class MultiFilterRecordReader<K extends WritableComparable,V extends Writable>
fillJoinCollector
protected void fillJoinCollector(K iterkey)
throws IOException
- Instead of filling the JoinCollector with iterators from all
data sources, fill only the rightmost for this key.
This not only saves space by discarding the other sources, but
it also emits the number of key-value pairs in the preferred
RecordReader instead of repeating that stream n times, where
n is the cardinality of the cross product of the discarded
streams for the given key.
- Overrides:
fillJoinCollector
in class CompositeRecordReader<K extends WritableComparable,V extends Writable,V extends Writable>
- Throws:
IOException
Copyright © 2014 Apache Software Foundation. All Rights Reserved.