CLI MiniCluster
Purpose
Using the CLI MiniCluster, users can simply start and stop a single-node Hadoop cluster with a single command, and without the need to set any environment variables or manage configuration files. The CLI MiniCluster starts both a MapReduce and HDF clusters. This is useful for cases where users want to quickly experiment with a real Hadoop cluster or test non-Java programs that rely on significant Hadoop functionality.
Hadoop Tarball
To get a Hadoop distribution, download a recent stable release from one of the Apache Download Mirrors. Unpack the downloaded Hadoop distribution. In the distribution, edit the file conf/hadoop-env.sh to define at least JAVA_HOME to be the root of your Java installation.
Running the MiniCluster
From inside the root directory of the extracted tarball, you can start the CLI MiniCluster using the following command:
$ bin/hadoop jar hadoop-test-*.jar minicluster -jtport JT_PORT -nnport NN_PORT
In the example command above, JT_PORT and NN_PORT should be replaced by the user's choice of these port numbers. If not specified, random free ports will be used.
There are a number of command line arguments that the users can use to control which services to start, and to pass other configuration properties. The available command line arguments:
| Argument | Description |
|---|---|
| -D <property=value> | Options to pass into configuration object |
| -datanodes <arg> | How many datanodes to start (default 1) |
| -format | Format the DFS (default false) |
| -help | Prints option help. |
| -jhsport <arg> | JobHistoryServer port (default 0--we choose) |
| -namenode <arg> | URL of the namenode (default is either the DFS cluster or a temporary dir) |
| -nnport <arg> | NameNode port (default 0--we choose) |
| -nodemanagers <arg> | How many nodemanagers to start (default 1) |
| -nodfs | Don't start a mini DFS cluster |
| -nomr | Don't start a mini MR cluster |
| -rmport <arg> | ResourceManager port (default 0--we choose) |
| -writeConfig <path> | Save configuration to this XML file. |
| -writeDetails <path> | Write basic information to this JSON file. |
To display this full list of available arguments, the user can pass the -help argument to the above command.