org.apache.hadoop.mapred
Class SequenceFileInputFilter.MD5Filter

java.lang.Object
  extended by org.apache.hadoop.mapred.SequenceFileInputFilter.FilterBase
      extended by org.apache.hadoop.mapred.SequenceFileInputFilter.MD5Filter
All Implemented Interfaces:
Configurable, SequenceFileInputFilter.Filter
Enclosing class:
SequenceFileInputFilter<K,V>

public static class SequenceFileInputFilter.MD5Filter
extends SequenceFileInputFilter.FilterBase

This class returns a set of records by examing the MD5 digest of its key against a filtering frequency f. The filtering criteria is MD5(key) % f == 0.


Field Summary
static int MD5_LEN
           
 
Constructor Summary
SequenceFileInputFilter.MD5Filter()
           
 
Method Summary
 boolean accept(Object key)
          Filtering method If MD5(key) % frequency==0, return true; otherwise return false
 void setConf(Configuration conf)
          configure the filter according to configuration
static void setFrequency(Configuration conf, int frequency)
          set the filtering frequency in configuration
 
Methods inherited from class org.apache.hadoop.mapred.SequenceFileInputFilter.FilterBase
getConf
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

MD5_LEN

public static final int MD5_LEN
See Also:
Constant Field Values
Constructor Detail

SequenceFileInputFilter.MD5Filter

public SequenceFileInputFilter.MD5Filter()
Method Detail

setFrequency

public static void setFrequency(Configuration conf,
                                int frequency)
set the filtering frequency in configuration

Parameters:
conf - configuration
frequency - filtering frequency

setConf

public void setConf(Configuration conf)
configure the filter according to configuration

Parameters:
conf - configuration

accept

public boolean accept(Object key)
Filtering method If MD5(key) % frequency==0, return true; otherwise return false

Parameters:
key - record key
Returns:
true if a record is accepted; return false otherwise
See Also:
SequenceFileInputFilter.Filter.accept(Object)


Copyright © 2009 The Apache Software Foundation