The following is a step by step guide on setting up IPMI management on a Rocks Cluster. It is based on several great posts on Rocks Mailing list and Rocks userguide presentation
Update: On Rocks Mailing list Philip Papadopoulos mentioned a Python Script that will automatically add IPMI interface by matching the last 2 octets to that of node: Script
1. Create the IPMI Network
rocks add network ipmi subnet=10.2.0.0 netmask=255.255.0.0 servedns=true
2. Add access on head node
The IPMI network is sharing the same vlan as data, so I will multihome the Frontend with a second IP on IPMI subnet
This adds it to Rocks config, but it appears to have no effect so I also ran it manually
rocks add host interface localhost iface=eth0:1 ip=10.2.1.1 subnet=ipmi name=ipmi-frontend
Manually:
ifcfg eth0:1 add 10.2.1.1/16
Add a script to make it start at reboot
Just copy /etc/sysconfig/network-scripts/ifcfg-eth0 to ifcfg-eth0:1 and change the device and IP address.
DEVICE=eth0:1 HWADDR=00:xx:xx:xx:xx:xx IPADDR=10.2.1.1 NETMASK=255.255.0.0 BOOTPROTO=static ONBOOT=yes MTU=1500
3. Set Password for IPMI
You can set a custom password for the IPMI system or use the default of admin
rocks set attr ipmi_password "password" shadow=yes
4. Add IPMI info for each node
Updated way:
Download the latest addInterface and ScarScript.py and run the following to add matching IP for each node.
If you are running Rocks 6.0/6.1 addInterface will not work for nodes whose card is not card eth0 (e.g. instead is em1). To fix this change line 67 to
"rocks list host interface %s | grep 'private ' | awk ' {print $4}'" %
for n in {0..6}; do
./addInterface compute-0-$n ipmi ipmi ipmi-0-$n
rocks set host interface channel compute-0-$n iface=ipmi channel=1;
done
Old Way:
node=0
for n in {254..245}; do
rocks add host interface compute-0-$node iface=ipmi ip=10.2.255.$n \
module=1 subnet=ipmi name=ipmi-0-$node;
rocks set host interface channel compute-0-$node iface=ipmi channel=1;
let node=$node+1;
done
Sync it
rocks sync host network
If you have every had DRAC set to use DHCP it needs to be set to static.
rocks run host "ipmitool lan set 1 ipsrc static" collate=yes
I also wanted to set the default gateway which is not set by the generated script and it will complain about network config if not set.
rocks run host "ipmitool lan set 1 defgw ipaddr 10.2.1.1" collate=yes
5. IPMI root Password
When I tried to connect using ipmitool and set password it reported NULL account disabled. To get around this is used the root account (id=2) and set a password for this.
rocks run host "ipmitool user set password 2 <password>"
6. Run the generated script on each node
rocks run host "sh /etc/sysconfig/ipmi"
7. Testing
ipmitool -I lan -H ipmi-0-0.ipmi -U root -P <password> chassis status
