Five Popular DataPower Patterns

15073670664_33357e764b_b

Summary

The DataPower Gateway supports a diverse ecosystem of solutions out of the box. Without guidance, this can be overwhelming when determining how to best leverage this technology within the bounds of the existing architecture. This article describes the following common patterns encountered at customer deployments:

  • Endpoint Security Pattern
  • Web Service / API Gateway Pattern
  • Transport Mediation Pattern
  • SOAP/XML to REST/JSON Pattern
  • Transformation and Aggregation Pattern

These patterns are woven together to provide enterprise standard capabilities designed to reduce the development cycle, centralize responsibility, increase quality and increase security. They allow enterprise service consumers to tightly focus on satisfying the business requirements and not waste effort repeatedly implementing low-level technological detail.

Introduction

The DataPower Gateway is the technological equivalent of a swiss army knife. There are many configuration choices that can be overwhelming at first glance. Do you choose to bring in the appliance to fill a niche role, or do you try and utilize every bell and whistle out of the gate? More importantly, how are other customers successfully leveraging these capabilities? Fortunately, there are common patterns that have emerged. Each of these patterns will be described, the business case presented and the related benefits of using DataPower will be discussed.

A pattern that isn’t contained in this list is not a statement of its quality or usefulness. It’s just a statement that all enterprises have the same basic needs and land up solving their problems in similar ways.

Endpoint Security Pattern

This is the most popular pattern.

Here, DataPower is a proxy placed in front of the service providers. A service consumer would then access the endpoint provided by the DataPower and DataPower would send the request to the backend service. Service providers are configured to only allow the device access to the service and service consumers are configured to only connect to DataPower.

When the traffic flows through the device, DataPower will:

  • Provide Access Control to the service provider, rejecting unauthorized connections.
  • Rate limit the incoming connection count such that the provider is not overwhelmed.
  • Provide industry standard transport layer security (TLS) over the consumer and provider connections.

Business Case

The volume of exposed services in the enterprise is growing. Each service requires its own access control and corresponding security configuration to meet the enterprise policy. This repeated configuration of the individual services cost time and money, slowing down projects. The lack of a single decision point for access control is making it difficult to track who is allowed access to what, or even whom to contact to be allowed access. This increases the costs of adding new service consumers, as the individual access control must to be modified in every individual service in every environment (DEV/UAT/QA/PROD). This change is also likely to span technologies, requiring more teams to be engaged on a project. All of this results in long turnaround times for a change that should conceptually be quick.

DataPower’s Enterprise Benefits 

  • Provides the enterprise book of record point for access control lists to service providers.
  • Is the first point of contact for all transactions in the enterprise is a device that supports the latest security standards (ex. TLS 1.2) and focuses on provider interoperability with many other existing standards (ex. LDAP)
  • Provides a proxy that can be placed in front of systems that don’t fully support the enterprise standard to bring them up to required levels. DataPower can do the heavy lifting of connecting to the service provider through alternative security models.
  • Standardizes the configuration for all service providers and consumers. Providers only need only to be configured to allow DataPower and Consumers only need to be configured to connect to DataPower.

WS / API Gateway Pattern

In this pattern, security is applied to inbound transactions from the internet and also to outbound transactions from the enterprise. This pattern is very similar to the Endpoint Security Pattern, but it has a few key differences:

  • The DataPower appliance is located on the edge of the enterprise network instead of being located internally.
  • DataPower is now accepting internet traffic destined for the service providers.
  • DataPower is being used as an outbound proxy to third-party vendor services.

Business Case

The enterprise needs to expose services to the internet in a secure manner. This could be in the form of a new externally consumed API or a set of SOAP services. Currently, holes are punched in the firewall and direct connections are made service providers as required. Every hole in the firewall is security liability, as it may connect to a server that could be insecure. These individual external access points then need to be monitored and audited to ensure that they meet the corporate security standard.

Also, the enterprise needs to consume services exposed by third-party vendors. Currently, each system makes its own outbound calls over the internet using its own security model requiring its own configuration settings. The maintenance of these security settings is getting expensive. It is found that these configurations are not always meeting the latest standards, which can cause sensitive data breach.

The enterprise requires a standard security model for securing these communications. Attempting to do it on a server by server basis is costly, complicated and prone to failure.

DataPower’s Enterprise Benefits

  • Is a single secure technology for traffic coming into and out of the enterprise.
  • Is the book of record for traffic coming into or out of the enterprise
  • Supports the latest security standards and the ability to communicate with a wide variety of systems
  • Reduces cost and complexity by reducing multiple security configuration points to one.

Transport Mediation Pattern

In this pattern, DataPower bridges the transports used by providers and consumers to access services. It can seamlessly accept transaction over HTTP and write a file via SFTP. It can pick up a message off of an MQ Queue and invoke a service provided over HTTPS.

Business Case

As the number of services increases, the technologies that host the services tends to increase. Multiple technologies results in services being exposed over multiple transports. The latest technology may use HTTPS, or they could be asynchronously exposed over MQ. Older technology may read files out of a directory via SFTP or consume IMS transactions over MQ to invoke services provided by the mainframe. Requiring service consumers to support all these transports is expensive as this boilerplate transport management code must be written, maintained and supported indefinitely. This utilitarian code has to be replicated per technology platform as well, turning into a large management effort.

DataPower’s Enterprise Benefits

  • Divorce the service consumer’s transport from the service providers transport. This can allow implementation of a new independent enterprise standard transport for service consumption.
  • Remove low value transport conversion utility code and configuration from your business services.

SOAP/XML to REST/JSON Pattern

With the shift towards JSON services, this pattern is gaining momentum. DataPower is used to convert the format of message payloads between JSON and SOAP.

Business Case

Businesses have already invested in exposing SOAP/XML services. Now they need to support clients that natively speak REST/JSON. They have three decisions:

  • Rewrite the service to natively speak JSON instead of SOAP.
  • Force every client to write JSON to SOAP conversion code.
  • Include the logic that converts between JSON and SOAP in a single technology.

Rewriting a service for something as minor as the input message format is a waste of money. There is little business value in how a message arrives to a service provider, the value is in processing the message.

Including conversion code in every client is a waste of effort and increases the complexity of every consumer. The code required to map between the two is not straightforward and tends to result in bugs. Even worse, the mapping effort gets duplicated for every service.

The business needs a centralized location for converting between these two messaging formats.

DataPower’s Enterprise Benefits

  • Provides a single enterprise-wide standard location for SOAP to JSON conversion.
  • Removes the low value conversion code from services. This allows them to remain tightly focused on business problem.
  • Avoids the cost of modifying the original SOAP service.

Transformation & Aggregation Pattern

This is another popular pattern. This pattern implements message transformation along with aggregation of the results of multiple service providers. The purpose of this pattern is to convert the input message supplied by the service consumer into the format of the service provider. This is repeated for up to N service providers, taking the responses and aggregating them into a single response to the client. This provides a single logical business service that is implemented as multiple technical service invocations, but keeps those details hidden to the consumer.

Business Case

The business is finding that there appears to be a relationship in development time and the number of services consumed. Each service consumed requires data mapping between provider and consumer. These maps have to be created repeatedly every time the provider service is called. One (poor) interim solution may have been the introduction of utility common code included into every project. This makes the situation even worse as now the utility code has to be tracked, maintained and serviced. Changing this utility code involves redeploying all the clients of the code, an additional large cost. To make matters worse, the utility code is growing larger and larger as more endpoints are exposed.

In addition, the business is finding the complexity of the data being integrated is increasing. There may be endpoints that support industry standard schemas like HL7 or SWIFT. These schemas tend to be verbose and it is complicated to construct a valid message format. It’s difficult to get these maps correct and the code implementing this mapping alone is dwarfing the rest of the service.

These situations justify the need of an enterprise mapping service. The utility logic of shovelling one data format into another can be offloaded to the expert team using expert technology designed to solve the problem.

DataPower’s Enterprise Benefits

  • Reduction in the Lines of Code (LoC) written by the service consumer. Less code means faster development, faster testing, less moving parts and cheaper cost.
  • Provides a core competency team in mapping data. This will reduce the costs of mapping as a specialized team will create lower cost and higher quality solutions.
  • Simplifying the designs of service consumers through endpoint aggregation.  Multiple service provider invocations can be woven into a single endpoint for the consumer.

Conclusion

These patterns provide functionality that can be leveraged across the entire enterprise. They remove costly low-value code from solutions and move that logic into a platform that was built for transforming large volumes of transactions. Environments with DataPower benefit from shorter development cycles, higher solution quality and benefit from a more secure architecture.

About The Author

Dan Zrobok

Twitter

Dan is the owner of Orange Specs Consulting, with over 14 years of experience working in enterprise systems integration. He is an advocate of the IBM DataPower Gateway platform and looks to improve environments that have embraced it. He also occasionally fights dragons with his three year old daughter Ruby, and newborn Clementine.

IF YOU'VE FOUND THIS INFORMATION USEFUL, PLEASE SHARE IT