DataPower Lights Out Management with IPMI

IPMI is a special ethernet port of the DataPower appliance. It serves as the last resort for managing devices that have become unresponsive through traditional management ports, such as the WebGUI or SSH. When properly configured, it lets administrators remotely power cycle devices and provides access to the Command Line Interface (CLI). To save trips to the data center, this port should be enabled on a trusted management network.

Has your DataPower appliance become unresponsive? Sounds like trip to the data center is needed, either to power cycle the device or to modify some incorrect configuration. Did you remember to bring along your laptop, terminal software and the USB to RJ-45 adapter needed to physically connect to the malfunctioning  appliance? Did your access request to the data center get approved? Do you know where the rack is? Did you pack a sweater?

These are all questions that you had to answer when managing the old DataPower appliances (pre-7199 hardware) that became unresponsive to SSH or WebGUI connections. Luckily, there is a better solution that doesn’t receive a lot of publicity, but can save time and money.

DataPower offers a method of remotely managing all aspects of a physical appliance. This special port supports retrieving system information such as CPU usage or chassis temperature. It can control the power state of the appliance. It also provides a method for accessing the command line interface over a special LAN connection. DataPower refers to this feature as Serial over Lan (SoL). SoL provides access to the serial console even if the device physically powered off. All that is needed is that the ‘Intelligent Platform Management Interface’ (IPMI) is configured and power is connected to the device. You connect to this port via an IPMI v2.0 client.

In this article, we’ll talk about how to configure this port in DataPower, how to connect to it, provide examples and point out some of the concerns you need to be aware of before enabling it.

From Wikipedia:

“The Intelligent Platform Management Interface (IPMI) is a set of computer interface specifications for an autonomous computer subsystem that provides management and monitoring capabilities independently of the host system’s CPU, firmware (BIOS or UEFI) and operating system. IPMI defines a set of interfaces used by system administrators for out-of-band management of computer systems and monitoring of their operation. For example, IPMI provides a way to manage a computer that may be powered off or otherwise unresponsive by using a network connection to the hardware rather than to an operating system or login shell.”

Basically, IPMI is a method of working with systems that become unresponsive over their normal communication channels by implementing a hardware connection interface that bypasses the operating system. This new channel becomes functional by wiring the connection, configuring the service and connecting the device to power.

IPMI in DataPower is an independent LAN interface sharing the physical MGMT0 ethernet port. It needs its own network configuration (IP address and Gateway) separate from the MGMT0 interface used for system services like the WebGUI, SSH etc.

IPMI provides three primary services:

1. Polling of hardware state
2. Modification of hardware state
3. Establishing a Serial over LAN connection.

IPMI is configured using two objects: The IPMI LAN Channel and the IPMI Users objects. The LAN channel holds the service configuration and the IPMI Users holds the user account information for connecting clients. This account information is separate from any other user configuration.

The IPMI LAN channel is configured via the Network -> Management -> ‘IPMI LAN Channel‘ object. The object has the following configuration properties:

• Maximum Privilege Level:
  • ‘User’: Only read-only commands can be issued to the service, regardless of the connecting user privilege level.
  • ‘Operational’: Configuration modification commands are allowed.
  • ‘Callback’ is primarily used by third-party monitoring tools. This allows them to be notified when objects being monitored change without having to poll the port.
• Support Serial Over LAN: If enabled, then an IPMI v2 client can establish a connection to the CLI using the Remote Control and Management Protocol Plus (RCMP+).
• Serial-over-LAN Privilege Level: A separate privilege level to restrict operations over serial over LAN connections.
• IP Address and Gateway: The IP address and gateway of the LAN connection. You have to work with your networking team to ensure that they understand the nature of this port and provide a proper trusted network for the service.

The LAN Channel is now configured, but if we click on the ‘Allowed Users’ tab, we see that we haven’t setup any user accounts that can access the port. For that, we go to the IPMI User object. It has the following options:

• Name: The username to add.
• User ID: The IPMI user identifier. Each user must have a unique user id. A valid value is between 3 and 10.
• Password: The password for the user

We can return back to the ‘Allowed Users’ tab in the IPMI LAN Channel tab. For each user we set a privilege level, set if the user can access SoL and can limit the number of simultaneous sessions (0 means unlimited).

Now that IPMI is configured with an IP address an a user, we can connect to the device and issue commands.

IPMItool provides a simple command-line interface to IPMI-enabled devices. It issues commands to DataPower using the following format:

Additional information on the command can be accessed through help:

More detailed documentation on IPMItool can be found on its man page. IPMI is a widely adopted standard and not specific to just DataPower.

View the status of the chassis

Enable the location light on the appliance

This is useful for verifying the appliance when physically in the data center:

Turn the appliance on or off

Power cycle the appliance

Start a Serial over LAN connection

View a list of the sensors

View the System Event Log

This shows log events relevant to the IPMI interface (i.e. power cycles or firmware updates):

From the documentation, please be aware of connection priority when enabling Serial over LAN

If you enable serial over LAN support for the IPMI LAN channel, an IPMI user who connects through an IPMI 2.0 client has access to the serial console port.

• An IPMI user can connect through an IPMI 2.0 client independent of the state of the appliance. The only time that an IPMI user cannot connect to the appliance is when the appliance is disconnected from AC power.
• An IPMI user has higher priority than a user who is directly connected to the serial port.
• If there is a user who is directly connected to the serial port, the IPMI user usurps the current serial session and does not need to log in to the appliance. If there is no user who is directly connected to the serial port, the IPMI user must log in to the appliance.
• Because there can be only one serial user, the IPMI user will suspend the serial session of the user who is directly connected to the serial port until the remote IPMI user closes (deactivates) the serial session. When the IPMI user closes the serial session, the session for the user who is directly connected to the serial port resumes.
• If an IPMI user is connected to the appliance and you need to use the serial port, you must unplug the Ethernet cable from the MGT0 port and wait for the serial port to become available. This can take up to 20 minutes

Also note that:

• Ethernet port MGMT0 hosts the IPMI LAN adapter. This port should not be connected a network that services external internet traffic as you could expose the SoL capability to connections originating outside the firewall.
• For Virtual DataPower, there is no IPMI configuration. The tasks that IPMI allow are ones that the hypervisor provides.
• The IPMI user password configuration is stored internally on the BMC of the physical appliance and is not a part of the general DataPower configuration. This means that backups will not store this configuration.
• There can only be a maximum of 8 configured IPMI users.

Enabling IPMI lowers the cost of your DataPower Infrastructure. It removes the need to travel to the data center to perform low value tasks such as power cycling an unresponsive device. On top of that, it allows administrators complete access to the device configuration when normal management channels are non-functional (i.e. WebGUI or SSH). Caution needs to be taken to configure IPMI properly in your network. The MGMT0 port should only be wired to a trusted management network and the IPMI user credentials need to be secured. Even with these cautions, IPMI is an extremely valuable component of a well-managed DataPower environment and should be enabled where it can.