Flushing DataPower Probes using the Command Line Interface

3240114868_24a941a04c_o
Lost time is never found again
Benjamin Franklin

Executive Summary

Any task performed more than a handful of times should be scripted. In this example, we’ve replaced a manual process executed many times a day with a script that runs in under a second. This speeds up the testing and debugging cycle as the probes can be rapidly cleared and troubleshooting iterations can resume.

Introduction

In DataPower, there are three ways to modify the device’s configuration:

  • WebGUI
  • XML Management Interface
  • Command Line Interface (CLI)

The WebGUI is the most popular method of making configuration changes because it is the easiest but because it’s manual, it is also the most time-consuming. The XML Management interface requires properly formatted SOAP Envelopes which can be a pain to create. The Command Line Interface is available over SSH and allows changes to be made in an interactive environment that is feature rich but lacks user friendliness.

Manual Scenario

I have a project in the development and testing phase. They are using 5 DataPower proxies in 5 different domains. They’ve discovered that the something isn’t working as intended and need to have the debugging probes turned on to view the request and response messages sent to the services. In order to turn on the probes with the WebGUI, I have to:

  • Click on the Control Panel and wait for the page to load
  • Click on the Troubleshooting Icon and wait for the page to load
  • Click on the “Debug Probe” heading
  • Select the service and click Enable Probe and wait for the page to load
  • Click ok to dismiss the pop up dialog that the probe is now enabled

This in total these probes can be enabled in 2-3 minutes with expert clicking. The problem is that these 5 services are actually shared services used by other projects while during my project’s testing. Over time, the probes will be full of transactions from both sources. When I’m asked to check for a transaction, I will see a list of 25 identical entries and need to make 2-3 clicks to determine if that transaction is relevant.

To mitigate this issue, I always flush the probes right before the test runs. This minimizes the noise that other projects generate in the probes. This means that I have to re-run the 2-3 minute process before every test. You may run 10-20 individual tests before the problem is identified. This translates to 30-60 minutes spent on the no-value work of flushing probes. This slows down testing as testers need to wait for the refresh to complete.

Enable Probe via CLI Script

Enabling a probe is simply setting the debug-mode to on

Disable Probe via CLI Script

Disabling a probe is just setting the debug-mode to off.

Flush a Probe via CLI Script

Flushing a probe is simply disabling, saving the config to memory (via the exit command) and then enabling it

Comments

The script will work for an XML Firewall service (‘xmlfirewall’), but it can be modified for other services that support probes like:

  • Web Services Gateway: use ‘wsgw’
  • MultiProtocol Gateway: use ‘mpgw’
  • Web Application Firewall: use ‘web-application-firewall’
  • Web Token Service: use ‘web-token-service’

The number of entries saved in a probe is defined appliance wide with the debug-history command. By default, this will only allow 25 transactions to be stored in a First-In-First-Out (FIFO) queue. If you need more than 25, set the debug-mode to unbounded and this will allow 250 transactions. Please note that this consumes more memory and should only be used as needed.

Running The Script

The drawback of this script is that it isn’t dynamic. You need to know both the domain and the service name. Looping over the list of proxies in a domain  would require working outside the CLI interface with shell script logic. For my case, I didn’t need to take it that far. Projects are always touching a known set of services. I just take the block of code, duplicate it, add the specific service details and save it as a text file on my desktop.

On Windows, I use PuTTY to open an ssh connection to the CLI. Once I log into default domain, I simply copy and paste the script above in its entirety and DataPower will execute the commands. I tried to get PuTTY to accept commands from the file, but I couldn’t quickly figure it out.

On *nix, I ssh into the device as above but you could pipe the script into the SSH shell from the command line. This would require that you also embed the administrator username and password in your script, so be sure that you aren’t violating any security guidelines.

Conclusion

Enabling / Disabling probes manually is a waste of time. Use the script above and thank me for freeing up your time to perform higher value DataPower work such as identifying, debugging and resolving the issue at hand.

About the Author

Dan Zrobok

Twitter

Dan is the owner of Orange Specs Consulting, with over 14 years of experience working in enterprise systems integration. He is an advocate of the IBM DataPower Gateway platform and looks to improve environments that have embraced it. He also occasionally fights dragons with his three year old daughter Ruby, and newborn Clementine.

Share this Post