DataPower REST API Walkthrough

IBM DataPower Gateway firmware 7.2 introduced a light weight REST based configuration management API. This new API moves configuration management from the world of SOAP/XML messages into REST/JSON which are easier for today’s web based developers to consume. It’s good for retrieving object status or making small changes to the configuration but actions such as importing/exporting or backing up configuration are either non-existant or not currently implemented.

Overall, this a strong first step towards a comprehensive REST API and will mature over time.

IBM DataPower Gateway firmware version 7.2 introduces a new REST based configuration management API. This API begins to provide similar capabilities as the current XML Management Interface without the need to construct and consume SOAP/XML messages. It uses HTTP and JSON messages to retrieve and update DataPower configuration objects.

In this article, we will explain, enable and show sample requests that can be sent to the API.

With the XML Management interface, a SOAP envelope must be constructed, even for simple read requests. The overhead of constructing these envelopes is great and needlessly complicates scripting solutions.

The nice thing about a REST API is that the endpoint information exposed over HTTP GET can be accessed from a simple web browser request and reading the resulting JSON. This would be impossible with the XML Management Interface as a tool to generate SOAP requests would be needed.

With REST, HTTP protocol verbs are used to describe the intention of the request: 

• GET – retrieves information from the device.
• POST – creates a completely new object. If an object with the same name already exists, then the POST will fail with an error.
• PUT – updates an existing object or a property of an existing object.
• DELETE – removes an existing object or property of an existing object. If the target is a list, then the entire list will be removed.
• OPTIONS – retrieves the list of HTTP verbs supported by the endpoint. For example sending an OPTIONS request to ‘/mgmt’ will return an attribute ‘verb’ with value ‘GET’ because only HTTP GETs can be consumed by the endpoint.

Sending an invalid HTTP verb to an endpoint will result in an HTTP 501 with no error, or an HTTP 405 with an error object stating that the method is not allowed.

One of the interesting features of this REST API is that it is self describing. Sending GETs to the top-level URI returns a list of hrefs that point towards more refined requests. We’ll this behaviour later when a GET is sent to the ‘/mgmt’ endpoint. With the XML interface, you were required to read the WSDL and XSD contracts to discover the endpoints and objects that were supported.

The REST Management Interface is disabled by default. This ensures that a default environment is secure as possible. In order to enable it:

• Start in the default domain
• Expand Network -> Management -> REST Management Interface
• Set the Administrative State to Enabled
• Click Apply
• Click Save Config

By default, the service will listen for requests on all ethernet interfaces (0.0.0.0) on port 5554. It is recommended that the local address be set to a separate management-related network and that the port number be changed from the default.

In order to be authorized to use the management interface, a user account with administrator rights is required. The REST Management interface can be secured further via an IP addressed based access control list.

With the REST Management Interface enabled, an HTTP GET can be sent to the root management URI ‘/mgmt/’. This will return the URIs of the seven subsections of the API. They are:

• /mgmt/status/
• /mgmt/config/
• /mgmt/domains/config/
• /mgmt/filestore/
• /mgmt/actionqueue/
• /mgmt/metadata/
• /mgmt/types/

When invoking these URIs ensure that you keep the trailing slash. If it is missing then DataPower will return an error.

The following sections show some of the core pieces to the API sections along with examples of how to use them.

This endpoint provides access to read-only domain and device status objects.

• GET – /mgmt/status returns the list of supported status objects
  • This includes ActiveUsers, TCPTable, XMLNamesStatus and any other read-only object.
• GET – /mgmt/status/{domain}/{objectType} return the details of the {objectType} status object for the given {domain}.
  • To see the list of logged in users issue an HTTP GET to /mgmt/status/default/ActiveUsers.

This endpoint provides access to retrieve and update  modifiable configuration objects. These endpoints support GET, PUT, POST and DELETE depending on the specific URI being consumed.

• GET – /mgmt/config will return a list of objects that can be manipulated by this endpoint.
  • This include: AAAPolicy, WSGateway, XMLManager and pretty much any object with a user configuration.
• GET – /mgmt/config/{domain}/{objectType} returns the list of {objectType} objects for the {domain}.
  • To see the list of XML Managers in the default domains issue a GET request to /mgmt/config/default/XMLManager
• GET – /mgmt/config/{domain}/{objectType}/{objectName} returns the configuration properties of {objectType} with name {objectName} for the given {domain}.
  • To see the XML Manager named ‘default’ in the default domain issue an HTTP GET to /mgmt/config/default/XMLManager/default
• GET – /mgmt/config/{domain}/{objectType}/{objectName}/{propertyName} retrieves a specific scalar property from a given object.
  • To retrieve the XML Manager’s DocCacheSize value, issue an HTTP GET to /mgmt/config/default/XMLManager/default/DocCacheSize.
• POST – /mgmt/config/{domain}/{objectType} creates a new object of {objectType}. The body of the request contains the configuration details for the new object.
  • To create a new XML Manager in the default domain, issue a POST to /mgmt/config/default/XMLManager with the body: {“XMLManager” : {“name” : “DanNewXMLManager”,”mAdminState” : “enabled”, “UserSummary” : “Default XML-Manager”, “CacheSize” : 256, “SHA1Caching” : “on”, “StaticDocumentCalls” : “on”, “SearchResults” : “on”, “SupportTxWarn” : “off”, “ParserLimitsBytesScanned” : 4194304, “ParserLimitsElementDepth” : 512, “ParserLimitsAttributeCount” : 128, “ParserLimitsMaxNodeSize” : 33554432, “ParserLimitsExternalReferences” : “forbid”, “ParserLimitsMaxPrefixes” : 1024,”ParserLimitsMaxNamespaces” : 1024,”ParserLimitsMaxLocalNames” : 60000, “DocCacheMaxDocs” : 5000, “DocCacheSize” : 65556, “DocMaxWrites” : 32768, “UserAgent” : {“value”: “default”}}}
    • If the response was successful the status will be: “Configuration has been created.”
    • If you try and POST again, an error will occur: “Resource already exists.”

• DELETE – /mgmt/config/{domain}/{objectType}/{objectName} deletes the configuration of type {objectType} named {objectName} in the given {domain}
  • To delete the an XML Manager named ‘DanNewXMLManager’ issue a DELETE to /mgmt/config/default/XMLManager/DanNewXMLManager.
    • If the request is successful: “Configuration has been deleted.”
    • If the request could not be completed: “Resource not found.”
• PUT – /mgmt/config/{domain}/{objectType}/{objectName}/{propertyName} sets a specific value to a scalar proper for the given object.
  • To set the default XML Managers DocCacheSize value to 65556 issue a PUT to /mgmt/config/default/XMLManager/default/DocCacheSize with a body of {“DocCacheSize”:65556}.
    • If successful you will get back ‘”DocCacheSize”:”property has been updated.”‘

This appears to be a convenience URI for quickly grabbing the list of domains installed on the device.

• GET – /mgmt/domains/config retrieves the list of domains installed. Interestingly, the hrefs returned will point to a different subsection of the api:  /mgmt/config/default/Domain/{domain}.
• GET – /mgmt/config/default/Domain/{domain} retrieves the configuration of the domain object. This does not return the configuration objects contained inside the domain, just the settings related to a domain object.
• POST – /mgmt/config/default/Domain creates a new domain.
  • Send a POST to /mgmt/config/default/Domain with body of the request containing the domain object configuration details: {“Domain”:{“name”:”NEWDOMAIN”,”mAdminState”:”enabled”,”NeighborDomain”:{“value”:”default”,”href”:”/mgmt/config/default/Domain/default”},”FileMap”:{“CopyFrom”:”on”,”CopyTo”:”on”,”Delete”:”on”,”Display”:”on”,”Exec”:”on”,”Subdir”:”on”},”MonitoringMap”:{“Audit”:”off”,”Log”:”off”},”ConfigMode”:”local”,”ImportFormat”:”ZIP”,”LocalIPRewrite”:”on”,”MaxChkpoints”:3}}
• DELETE – /mgmt/config/default/Domain/{domain} deletes the {domain}.
  • To delete the domain ‘NEWDOMAIN’ issue an HTTP DELETE to /mgmt/config/default/Domain/NEWDOMAIN.

This will let you upload, modify and delete files. Note that the API will not return the contents of any file unless explicitly requested via a GET.

• GET – /mgmt/filestore/ returns a list of the sub-API that composes a unique file url: top, directory and files. Top is root of the file system (such as config, local or sharedcert). Directory and files are self explainatory.
• GET – /mgmt/filestore/{domain}/{filesystem_root} returns the immediate children of the folder but will not recurse down the hierarchy.
  •  To view the children of the local:// root, issue an HTTP GET to /mgmt/filestore/default/local.
    • NOTE: Don’t include the ‘:’. There seems to be a bug that allows it to be present, but the hrefs returned by the API will be invalid.
  • The contents of the file are not returned in these API calls.
• POST – /mgmt/filestore/{domain}/{filesystem_root}/[0..N optional_sub_folders] creates a new folder or file.
  • To create a folder called DANTEST in the local:// directory of the default domain issue an HTTP POST to /mgmt/filestore/ with body contents:     {“directory”: {“name”: “DANTEST”}}
    • If successful the response will be “Directory local:/DANTEST has been created.”
• DELETE – /mgmt/filestore/{domain}/{filesystem_root}/[0..N optional_sub_folders]/{folder} deletes a folder.
  • To delete the folder DANTEST in the default domain’s local:// directory issue an HTTP DELETE to /mgmt/filestore/default/local/DANTEST.
    • If successful the response will be “Directory local:///DANTEST has been deleted.”
• POST – /mgmt/filestore/{domain}/{filesystem_root}/[0..N optional_sub_folders] creates a new file in the specified location.
  • To create a few file in the DANTEST folder inside the local directory in the default domain issue an HTTP POST to /mgmt/filestore/default/local/DANTEST with body contents: {“file”: {“name”: “HelloWorld.txt”, “content”:”SGVsbG8gV29ybGQ=”}}
    • The content attribute is base64 encoded.
    • A successful response: “File local:/DANTEST/HelloWorld.txt has been created.”
• GET – /mgmt/filestore/{domain}/{filesystem_root}/[0..N optional_sub_folders]/{filename} returns the base64 encoded contents of the requested file.
  • To retrieve the HelloWorld.txt file in the DANTEST folder within the local root in the default domain issue an HTTP GET to /mgmt/filestore/default/local/DANTEST/HelloWorld.txt
    • “file”:”SGVsbG8gV29ybGQ=”

This lets you take ‘actions’. Actions are device updates that don’t necessarily map 1:1 with a configuration object. These actions include FlushStylesheetCache, SetLogLevel and (Generate)ErrorReport.

• GET – /mgmt/actionqueue/ returns the URI of the four subsections of this api: resource, operations, schema and metadata.
  • Sending a GET to /mgmt/actionqueue/{domain}/ returns a not implemented error.
• GET – /mgmt/actionqueue/{domain}/operations returns the list of actions that are supported by the endpoint. It’s a hodgepodge of refresh and flush actions in addition to extremely powerful actions like ExecConfig.
  • Each operation returns a schema and metadata href.
    • Sending a GET to /mgmt/actionqueue/default/operations/ExecConfig returns a ‘Not Implemented’ error
    • Sending a GET to /mgmt/metadata/default/operations/ExecConfig returns a description of the action, but not the details on how to use it from the REST API.

The API appears to be partially implemented and not well documented at this time.

This endpoint provides read-only metadata information about the objects and actions provided by the API.

• GET – /mgmt/metadata/ returns the URI formats for three subsections: object, property and operation
• GET – /mgmt/metadata/{domain}/{objectType} returns the metadata for the {objectType} in the {domain}.
  • Sending a GET to /mgmt/metadata/default/XMLManager returns the metadata for the XMLManager object.
• GET – /mgmt/metadata/{domain}/{objectType}/{propertyName} returns the metadata for the {objectType}’s {propertyName} in the {domain}
  • Send a GET to /mgmt/metadata/default/XMLManager/DocCacheMaxDocs returns the metadata for the DocCacheMaxDocs property of the XMLManager.
• GET – /mgmt/metadata/{domain}/operations/{operationName} returns the metadata for the given operation.
  • Send a GET to the /mgmt/metadata/default/operations/FlushStylesheetCache returns the metadata for Flushing the stylesheet cache of an XML Manager.

The data returned is formatted in a way that is very internal to the way that the devices stores the objects and will require trial and error to determine the proper JSON message format to successfully invoke the operations.

This read-only endpoint describes the internal data power object types that are used by the appliance. These types are exposed from the metadata endpoint.

• GET – /mgmt/types/ returns the list of types used by DataPower.
• GET – /mgmt/types/{domain}/{type} returns the information about the {type}
  • Send a GET to /mgmt/types/default/dmAAAPAuthorizeType to retrieve information about the ‘dmAAAPAuthorizeType’ type.

The API will almost always return a JSON array named ‘_links’ that contains hrefs to other URIs. One of these entries is ‘self’ that references back to the URI that was invoked by the client. This is valuable when you invoke one of the above seven API sections directly. You will get back a ‘_links’ object of all the other URIs that can be called.

For certain href properties, there is a reference to ‘{domain}’. This is a placeholder that represents that an actual domain should be used.

My overall opinion of the new API is positive. I like it for being able to quickly send GET requests to the endpoint from a web browser. I also like how easy it is to modify the configuration of objects in place via PUT/POST and to create/delete domains.

I do think that the API has some room to grow:

• I couldn’t find any way to import or export an entire domains worth of configuration. This will keep companies that heavily script their environments on the XML Management Interface.
• I couldn’t find a way to view an entire object and all of its dependencies. When using the /mgmt/config endpoint, it will only return the data contained directly by that object. When there is an object reference, the API will include an ‘href’ attribute with the URI that can be used to access the contents of that object. This means that if you would like to use the REST API to view the full configuration of an object, you need to write code that identifies these links, issues a new REST request to retrieve the object and then store and reference it in an external manner.
• The ‘action queue’, ‘metadata’ and ‘types’ need more design consideration and detailed documentation. The knowledge center itself refers the reader back to XSD definitions for ‘hints’ on how to use operations. Its not that helpful to point REST users back to complicated XSD and XML formats to understand the API.  I think that over time this will improve, but be aware that you will spend time trying to figure out how to use these endpoints successfully.
• Sometimes in the ‘_links’ list, there is a ‘doc’ element. I believe this is supposed to reference documentation about the object. At this time these URIs return ‘Not Implemented’.

The REST API is a great first step towards simplifying the managing of appliances through a scripting interface. The XML Management Interface was a pain to deal with due to WSDL contracts and XSD Data Definitions that needed SOAP Envelopes. The REST interface begins to tear down these technical roadblocks such that a simple web browser request (from someone with proper authority) can quickly view configuration without having to navigate the click-fest WebGUI.

As it stands now, thought,  enterprises should target its use only for simple tasks such as pulling device status or making small script updates, say to enable/disable probes. Anything larger either isn’t directly supported or will require time lost to trial and error.