|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.apache.hadoop.mapred.join.CompositeInputFormat<K>
public class CompositeInputFormat<K extends WritableComparable>
An InputFormat capable of performing joins over a set of data sources sorted and partitioned the same way.
A user may define new join types by setting the property
mapred.join.define.<ident> to a classname. In the expression
mapred.join.expr, the identifier will be assumed to be a
ComposableRecordReader.
mapred.join.keycomparator can be a classname used to compare keys
in the join.
,
JoinRecordReader
,
MultiFilterRecordReader
Constructor Summary | |
---|---|
CompositeInputFormat()
|
Method Summary | |
---|---|
protected void |
addDefaults()
Adds the default set of identifiers to the parser. |
static String |
compose(Class<? extends InputFormat> inf,
String path)
Convenience method for constructing composite formats. |
static String |
compose(String op,
Class<? extends InputFormat> inf,
Path... path)
Convenience method for constructing composite formats. |
static String |
compose(String op,
Class<? extends InputFormat> inf,
String... path)
Convenience method for constructing composite formats. |
ComposableRecordReader<K,TupleWritable> |
getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
Construct a CompositeRecordReader for the children of this InputFormat as defined in the init expression. |
InputSplit[] |
getSplits(JobConf job,
int numSplits)
Build a CompositeInputSplit from the child InputFormats by assigning the ith split from each child to the ith composite split. |
void |
setFormat(JobConf job)
Interpret a given string as a composite expression. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public CompositeInputFormat()
Method Detail |
---|
public void setFormat(JobConf job) throws IOException
func ::= <ident>([<func>,]*<func>)
func ::= tbl(<class>,"<path>")
class ::= @see java.lang.Class#forName(java.lang.String)
path ::= @see org.apache.hadoop.fs.Path#Path(java.lang.String)
Reads expression from the mapred.join.expr property and
user-supplied join types from mapred.join.define.<ident>
types. Paths supplied to tbl are given as input paths to the
InputFormat class listed.
IOException
compose(java.lang.String, java.lang.Class, java.lang.String...)
protected void addDefaults()
public InputSplit[] getSplits(JobConf job, int numSplits) throws IOException
getSplits
in interface InputFormat<K extends WritableComparable,TupleWritable>
job
- job configuration.numSplits
- the desired number of splits, a hint.
InputSplit
s for the job.
IOException
public ComposableRecordReader<K,TupleWritable> getRecordReader(InputSplit split, JobConf job, Reporter reporter) throws IOException
getRecordReader
in interface InputFormat<K extends WritableComparable,TupleWritable>
getRecordReader
in interface ComposableInputFormat<K extends WritableComparable,TupleWritable>
split
- the InputSplit
job
- the job that this split belongs to
RecordReader
IOException
public static String compose(Class<? extends InputFormat> inf, String path)
tbl(<inf>, <p>)
public static String compose(String op, Class<? extends InputFormat> inf, String... path)
<op>(tbl(<inf>,<p1>),tbl(<inf>,<p2>),...,tbl(<inf>,<pn>))
public static String compose(String op, Class<? extends InputFormat> inf, Path... path)
<op>(tbl(<inf>,<p1>),tbl(<inf>,<p2>),...,tbl(<inf>,<pn>))
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |