Class OverrideRecordReader<K extends WritableComparable,V extends Writable>

  extended by org.apache.hadoop.mapred.join.CompositeRecordReader<K,V,V>
      extended by org.apache.hadoop.mapred.join.MultiFilterRecordReader<K,V>
          extended by org.apache.hadoop.mapred.join.OverrideRecordReader<K,V>
All Implemented Interfaces:
Comparable<ComposableRecordReader<K,?>>, Configurable, ComposableRecordReader<K,V>, RecordReader<K,V>

public class OverrideRecordReader<K extends WritableComparable,V extends Writable>
extends MultiFilterRecordReader<K,V>

Prefer the "rightmost" data source for this key. For example, override(S1,S2,S3) will prefer values from S3 over S2, and values from S2 over S1 for all keys emitted from all sources.

Nested Class Summary
Nested classes/interfaces inherited from class org.apache.hadoop.mapred.join.MultiFilterRecordReader
Field Summary
Fields inherited from class org.apache.hadoop.mapred.join.CompositeRecordReader
jc, kids
Method Summary
protected  V emit(TupleWritable dst)
          Emit the value with the highest position in the tuple.
protected  void fillJoinCollector(K iterkey)
          Instead of filling the JoinCollector with iterators from all data sources, fill only the rightmost for this key.
Methods inherited from class org.apache.hadoop.mapred.join.MultiFilterRecordReader
combine, createValue, getDelegate, next
Methods inherited from class org.apache.hadoop.mapred.join.CompositeRecordReader
accept, add, close, compareTo, createInternalValue, createKey, getComparator, getConf, getPos, getProgress, getRecordReaderQueue, hasNext, id, key, key, setConf, skip
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.hadoop.mapred.join.ComposableRecordReader
accept, hasNext, id, key, key, skip
Methods inherited from interface org.apache.hadoop.mapred.RecordReader
close, createKey, getPos, getProgress
Methods inherited from interface java.lang.Comparable

Method Detail


protected V emit(TupleWritable dst)
Emit the value with the highest position in the tuple.

Specified by:
emit in class MultiFilterRecordReader<K extends WritableComparable,V extends Writable>


protected void fillJoinCollector(K iterkey)
                          throws IOException
Instead of filling the JoinCollector with iterators from all data sources, fill only the rightmost for this key. This not only saves space by discarding the other sources, but it also emits the number of key-value pairs in the preferred RecordReader instead of repeating that stream n times, where n is the cardinality of the cross product of the discarded streams for the given key.

fillJoinCollector in class CompositeRecordReader<K extends WritableComparable,V extends Writable,V extends Writable>

Copyright © 2009 The Apache Software Foundation