In this blog, I’d like to share with you some of my observations evaluating some large scale enterprise SOA implementations.

Like many other struggling SOA initiatives, most start with the right motivations and full executive sponsorship to produce a modern architecture with obvious qualities: scalability, reliability, auditability, maintainability etc.

However, for SOA to deliver on these promises you need a more disciplined approach, and tooling to support the development in their effort to implement best practice patterns. On a recent snapshot of one of the companies, here is what I observed:

• Data is spread around multiple data stores and silo’d across regions. Management of duplicate data in both operational systems and data warehouses is not being done consistently and reliably.
• Various systems typically exchange data by building point-to-point interfaces, each with their own custom invocations to retrieve data from the various data sources (thousands of point-to-point data transfers).
• There is little or no reusability that is built into these independent processes. Almost all business logic, and policies are hard-coded.
• Transactions are not consistently logged or monitored.
• Business requirements are not consistently reflected in technical implementations, and business rules are not consistently enforced.

The risks facing these companies are not for the light-hearted:

• Realtime access to reliable information may not be possible.
• Change management is difficult and involves high risk, and high support costs.
• Unmanageability and cost explosion due to redundantly hard-coding business logic into services and policies.

While each industry has its own unique challenges, health care industry specifically is highly regulated as privacy of personal information is a major concern. Consistent authentication and authorization enforcement; auditability of meaningful use of the information, managing sensitive data appropriately are a few of the security requirements.
Despite the millions spent on SOA tooling from existing middle-ware vendors, many large companies are on their own in enforcing best practice SOA implementation patterns like schema, process, or policy centralization, using enterprise service bus effectively to de-couple their services, etc. Most existing tooling seemingly does wonders getting your first web service running in just under 5 minutes, but fail to provide any guidance on how to manage the complexity when hundreds of web services are deployed and are to interact with each other.  Data as well as service implementations are replicated to handle variations on use cases, to deal with versioning, and also to provide redundancy and failover mechanisms.

And since there is little reuse or architectural governance, consistent application of enterprise security policies across all services is extremely difficult.

Recent advances in model-driven software and ability to define custom domain specific languages (DSL) enables building SOA applications decleratively. A custom SOA framework and IDE which takes advantage of DSLs and model-driven software generation can:

• Capture business-centric security & compliance policies  in human readible terms.
• Enforce use of canonical data models,
• Track data lineage across transformations
• Generate runtime artifacts (read: code, and configuration)  automatically from these business level definitions and centrally manage them independent of where they are deployed.
  • For example, in health care,  code generators can be customized to suppport the audit requirements for HL7.
  • Translate declared policies and propagate them into the matching technical access and audit rules.

Furthermore, centralizing metadata about all IT assets can drastically simplify implementation of and enforce consistent use of enterprise wide utility services such as Logging, Auditing etc as well as enforce contextual enterprise security and policies across all services is required.

On my next blog series I will provide demonstrated examples on how domain specific languages and model driven software engineering can address these challenges.

Kagan Turgut