Creating a Multi Node Cluster

Creating a Multi Node Cluster

I decided that I need to increase my mining power, so I setup two more nodes with the same hardware from my original post. The layout looks like this.

You only need to run geth on one node and connect your other ethminer instances (on node2 and node3) to your geth instance on (node1). That requires a little modification of the startup scripts to get everything connected.

Geth Start-Up

You need to tell geth to accept connections from remote hosts. The requires the following parameters.

 #!/usr/bin/env bash 
 echo "Starting geth" 
 screen -dmS geth /usr/bin/geth --rpc --rpcaddr "0.0.0.0" --rpcport 8545 --rpccorsdomain "*" 
 #Basic geth startup script 

--rpc enables the rpc console
--rpcaddr is the interface you want to listen on for conenctions. 0.0.0.0 is all interfaces
--rpcport is the port it will listen on 8545 is default
--rpccorsdomain is the domaint to accept request from

Ethminer Start-Up

To point your remote ethminers at your geth instance do the following in the startup script. You only need to do this on node2 and node3.

 #!/usr/bin/env bash 
 echo "Starting Ethminer" 
 screen -dmS ethminer /usr/bin/ethminer -G -F http://192.168.1.57:8545 
 #Ethminer startup script for remote connections 

-F puts ethminer into farm mode you need replace 192.168.1.57 with the IP of node where you are geth.

ethminer should start up as per normal. It will now just be getting work from the geth instance on node1.

{ Read More }

Create a passwordless SSH login to remote computers to automate launch of a zookeeper cluster

Create a passwordless SSH login to remote computers to automate launch of a zookeeper cluster

Passwordless SSH Login can be useful in situations, for example, when one likes to write a bash script that automatically setup or launch a zookeeper cluster. Suppose we want to ssh login to a computer at 192.168.2.4 without a password from a our client computer at 192.168.2.2, we can do so by creating a RSA pair of authentication key at 192.168.2.2 terminal:

> ssh-keygen -t rsa

In the terminal prompt, just press the ENTER key multiple times for each prompt. Once this is done, a public key rsa_id.pub is stored in the .ssh folder under the user root folder. Now ssh login to the 192.168.2.4 (password required at this point) to create the .ssh folder under the user root folder on that computer:

> ssh username@192.168.2.4 mkdir .ssh

The last step is to append the .ssh/rsa_id.pub file (which is the public key) from 192.168.2.2 to the .ssh/authorized_keys in 192.168.2.4 (password required at this point):

> cat .ssh/id_rsa.pub | ssh username@192.168.2.4 cat >> .ssh/authorized_keys

Now we can ssh login to 192.168.2.4 from the client 192.168.2.2 without password. For example, run the following command in the client 192.168.2.2 terminal:

> ssh username@192.168.2.4

Now we are ready to write a bash script to launch the zookeeper cluster which consists of the following networked computers:

192.168.2.1
192.168.2.3
192.168.2.4

First make sure that the zookeeper is setup in the cluster properly (follow instructions from this link: http://czcodezone.blogspot.sg/2014/11/setup-zookeeper-in-cluster.html and for storm at this link http://czcodezone.blogspot.sg/2014/11/setup-storm-in-cluster.html) and make sure the above steps of passwordless SSH login is repeated for 192.168.2.1 and 192.168.2.3 so that user can log into these two computers without password from the client 192.168.2.2. Now on the terminal of 192.168.2.2, create a start_zookeeper_cluster.sh script:

> touch start_zookeeper_cluster.sh
> gedit start_zookeeper_cluster.sh

In the start_zookeeper_cluster.sh that opens, enter the following lines:

ssh username@192.168.2.1 $ZK_HOME/bin/zkServer.sh start
ssh username@192.168.2.3 $ZK_HOME/bin/zkServer.sh start
ssh username@192.168.2.4 $ZK_HOME/bin/zkServer.sh start

Save and close the start_zookeeper_cluster.sh, and run the following command on 192.168.2.2 client terminal to make it executable:

> chmod +x ./start_zookeeper_cluster.sh

Now run the script to start the zookeeper cluster:

> ./start_zookeeper_cluster.sh

We can also write a stop_zookeeper_cluster.sh, start_storm_cluster.sh, stop_storm_cluster.sh in a similar way to automate the process.

{ Read More }