Monday, December 30, 2013

[CC]IPMI over LAN on Ubuntu 12.04

Doing this on two of these machines:
System Information
Manufacturer: Supermicro
Product Name: X9DBL-3F/X9DBL-iF
In the IPMI section of the BIOS, set the IP/gateway/netmask addresses. (This can also be done from inside the Linux Kernel with `ipmitool` after loading the modules.) I left all the other IPMI settings as default.
After a reboot, visit the IP of the IPMI LAN device in a browser from a separate computer. I found everything worked except console redirection (SOL – Serial Over Lan,) as indeed this has to be setup inside the Linux Kernel.
These are the steps I took to make console redirection work:
1. The first thing I did was install the tools and setup the modules.
  • Install the needed software:
sudo apt-get install ipmitool freeipmi-tools
  • Find the information about the BMC (press ‘q’ to ‘quit’ less when done):
sudo ipmi-locate | less
  • I found the information needed in the first few lines of the output as seen here:
Probing KCS device using DMIDECODE... done IPMI Version: 2.0
 IPMI locate driver: DMIDECODE
 IPMI interface: KCS
 BMC driver device:
 BMC I/O base address: 0xCA2
 Register spacing: 1
  • Add the appropriate modules into the Kernel, for the ‘ipmi_si’ one use the information found with ipmi-locate like this:
sudo modprobe ipmi_si type=kcs ports=0xCA2 regspacings=1
sudo modprobe ipmi_devintf
sudo modprobe ipmi_msghandler
sudo modprobe ipmi_poweroff
sudo modprobe ipmi_watchdog
  • Now if you “ls -ahl /dev/ipmi*” you should see an ipmi0
2. Next, we need to setup a tty and tell grub to pass options to the Kernel command to redirect it’s console to that tty.
  • Make a copy of an existing tty and edit it. I’m using vim, but use whatever editor you like:
sudo cp /etc/init/tty1.conf /etc/init/ttyS1.conf
sudo vi /etc/init/ttyS1.conf
  • Edit the file to reflect ‘ttyS1′ instead of just ‘tty1′. Also edit the last line to use a baud rate of 115200, and tell it the type  ’vt100′. Mine looks like this when done:
# ttyS1 - getty
#
# This service maintains a getty on ttyS1 from the point the system is
# started until it is shut down again.

start on stopped rc RUNLEVEL=[2345]

stop on runlevel [!2345]

respawn
exec /sbin/getty -8 115200 ttyS1 vt100
  • We must pass “console=tty1 console=ttyS1,115200n8″ as options to the command in our boot loader that starts the kernel. Note: the order of options passed to the kernel is important, the last “console” entry defines the ‘default’ console for the kernel. For grub-pc open up /etc/default/grub with your favorite editor and with sudo permissions and change GRUB_CMDLINE_LINUX=”" line to look like this:
GRUB_CMDLINE_LINUX="console=tty1 console=ttyS1,115200n8"
  • Don’t forget to update grub after this: (I also changed ‘GRUB_HIDDEN_TIMEOUT_QUIET’ to false, and I uncommented the ‘GRUB_TERMINAL=console’ line.)
sudo update-grub
3. Now we’re pretty much ready to connect. The only problem I found was with the launch.jnlp file that the Supermicro webUI spits at you. I’ll show you the edits I made to make it work:
  • Again visit the IP pointed at your IPMI and login in to the webUI. Click the ‘Launch SOL’ button under the ‘Remote Control’ tab, choose to save the file instead of opening it in your browser’s web java application (‘IcedTea java web start’ in my case.)
  • Towards the beginning of the file, look for these lines:
<property name="jnlp.packEnabled" value="true"/>
<property name="jnlp.versionEnabled" value="true"/>
  • Copy those two lines and scroll down a little until you see the distribution you are running and paste them into the there. I did it for each distribution so I didn’t have to think about it. I ended up with entries looking like this:
<resources os="Linux" arch="x86_64">
  <property name="jnlp.packEnabled" value="true"/>
  <property name="jnlp.versionEnabled" value="true"/>
  <nativelib href="liblinux_x86_64.jar" download="eager" version="1.0.3"/>
</resources>
  • Lastly, scroll to the end of the file where you will see a list of “argument” xml tags. The one below the one that contains your IP probably contains a seemingly random string of letters, but it needs to be changed to the username you setup to login to the IPMI webUI, and the one below needs to be your password. Like this:
<argument>123.123.123.123</argument>
<argument>[username]</argument>
<argument>[password]</argument>
  • Save the file and run it and you should now be ready to sit in front of the machine remotely. Reboot the server to watch it boot up, offer you the opportunity to enter the BIOS, and then offer you a login prompt. :-)
P.S. Don’t forget to edit the factory installed users in your IPMI webUI and change their passwords and/or disable them!
EDIT 07/06/13:
Corrected the line:
GRUB_CMDLINE_LINUX="console=tty0 console=ttyS1,115200n8"
to:
GRUB_CMDLINE_LINUX="console=tty1 console=ttyS1,115200n8"
Also,
Corrected the line:
exec /sbin/getty -8 115200 tty1 vt100
to:
exec /sbin/getty -8 115200 ttyS1 vt100
Also,
Removed the lines from ttyS1.conf pertaining to ‘containers’:
and (
          not-container or
          container CONTAINER=lxc or
          container CONTAINER=lxc-libvirt)
Everything is running smoothly after this, even when networking is turned off in the kernel.