Bigdata Hadoop - Cluster Management through CLUSH

Clush is an open source package that lets you use a parallel shell to simplify cluster management from the command line and shell. This topic covers how to install clush and run a utility that does cluster auditing for you. This step is extremely useful before you deploy your Hadoop cluster.

a.       Clush only needs to be installed on one node, e.g. the edge node or first node in the cluster. Clustershell is a package available for CentOS, Redhat and Ubuntu. e.g.

#yum --enablerepo=epel install clustershell

b.      Prerequisite: passwordless ssh to all nodes in the cluster for root or an account with sudo rights.

c.       Define at least one group (all) in /etc/clustershell/groups. e.g. all: node[1-9]

In our dev clusters, it is

#cd /etc/clustershell

#cat groups

all: node1.pravatsutar.com node2.pravatsutar.com node3.pravatsutar.com

d.      Test install/config with:

#clush -a date

Compare clush output with and without -b option:

#clush -a date and #clush -ab date

Clush Commands:

To copy to all the nodes

clush -a -c /home/clduser/mprrepo_bkp/jdk-7u51-linux-x64.rpm

To remove the files from the nodes

clush -g all rm -rf /home/clduser/mprrepo_bkp/jdk-7u51-linux-x64.rpm

If you want to install CLDB

clush -g cldb <mapr cldb rpm>

Use below command if you have any manual intervention to press YES

clush -g all yum -y install ganglia-gmond

To copy from one to another directory

clush -g all -c /etc/ganglia/gmond.conf --dest /etc/ganglia/

Uninstall all the rpm files from all the nodes

clush -g all yum -y remove mapr-core-3.1.0.23703.GA-1.x86_64

Command To erase the java

Clush –g all yum –y erase java

Command To delete the files

Clush –g all rm /test/test.txt

To append teradatahost names to our /etc/hosts file

clush -g all cat '/etc/terahosts >> /etc/hosts'

Where the Extract Files wants to place:
clush -g all tar -xf /usr/lib64/python2.6/numpy-1.9.1.tar.gz -C /usr/lib64/python2.6/

To Remove JAVA

Clush –g all -yum -y erase jdk

clush -g all yum -y install /home/clduser/jdk-7u51-linux-x64.rpm