You are on page 1of 35

Guerrilla SOA

How to fight back when a vendor takes control of your


enterprise

Dr. Jim Webber


Service-Oriented Systems Practice Lead
http://jim.webber.name

© ThoughtWorks, 2006 1
Fundamental Premise

There are two things money cannot


buy:

• Love
(Lennon/McCartney)
• An SOA
(Webber)

© ThoughtWorks, 2006 2
Roadmap

• Enterprise Application Integration Approaches


• Enterprise Architecture, now and future
• The Appealing Rationale for ESB...
• Enterprise Architecture
• Realising SOA with Web Services
• What this means for you
• Conclusions
• Q&A

© ThoughtWorks, 2006 3
Enterprise Application Integration
Approaches
• Data integration
– Extract, transform, route, inject data
• Application level
– Re-use application APIs, or I/O mechanisms
• EAI implementation
– Queues etc
• Business domain tier
– Integration at the object level, as typified by CORBA, DCOM etc
• User interface
– Screen scraping, revamping, etc.
– Last resort, when an application offers no other hooks

© ThoughtWorks, 2006 4
To ESB or not to ESB, that is the question
• Product vendors are keen to provide product solution for
everything
– Or to supply “consultantware” solutions
• The Enterprise Service Bus is the latest incarnation of EAI
technology that supports a number of useful functions:
– Transformations; adapters; choreography; reliability; security etc
• Seems like a good idea...

© ThoughtWorks, 2006 5
Today’s Enterprise Architecture
Accounting Marketing

Product Development Support

© ThoughtWorks, 2006 6
How did we get here?
• Tactical decisions
• Time and technology pressures
• Path of least resistance for individual applications
• This is the thin end of the wedge, technical debt can only
increase from here
• Help!

© ThoughtWorks, 2006 7
Vendor Solutions Appear

• Business needs to
compete
– IT needs to be
responsive
• SOA gives IT a
business process focus
• Web Services are the
most sensible way to
implement SOA
• More proprietary
middleware is the
answer!
– 2 + 2 = See:
5 http://www.capeclear.com/technology/index.shtml
© ThoughtWorks, 2006 8
Integration Two Years Later
Accounting Marketing

Enterprise Service Bus


Product Development Support

© ThoughtWorks, 2006 9
Skeletons in the Closet...

Enterprise Service Bus

© ThoughtWorks, 2006 10
The Appealing Rationale for ESB...

• Perceived single framework for all integration needs


• Perceived simple connectivity between systems
• Some features for security, reliable delivery, etc.
• All you have to do is agree to lock yourself into a ESB and all
this can be yours...

© ThoughtWorks, 2006 11
...And the Reality

• The mess is swept under the carpet


– The spaghetti is still there, but it’s hidden inside a vendor box
• But the spaghetti is worse with an ESB
– Mixing business rules, transformations, QoS etc with connectors
• What if I wanted to remove or replace my current ESB platform?
– Vendor lock-in of the whole network!
– ESBs are proprietary, so no guarantees that the messages transmitted
across the bus are actually based on any open protocol
• Held to ransom by the ESB vendor!
– Cannot easily replace one ESB with another
– Can only easily integrate systems for which the ESB vendor provides
specific adaptors
– Or invest your money into extending their product

© ThoughtWorks, 2006 12
Intelligent Networks, Dumb Idea?

• Isn't this precisely what we're trying to get away from?


• Integration should happen on the wire by default, not inside
some server
• The ESB approach eschews the dumb network, smart
endpoint notion that underpins scalable, robust systems
• ESB vendors are the new telcos – telling us that smarts in the
network is for our own good
• But let’s see how ESBs play out over the longer term

© ThoughtWorks, 2006 13
Integration five years from now
Accounting Marketing

IT
Research

Enterprise Service Bus


Product Development Support

© ThoughtWorks, 2006 14
Integration ten years from now
Accounting Marketing

IT
Research

ESB
Product Development Support

© ThoughtWorks, 2006 15
How did this happen?
• Same old story:
– Tactical decisions
– Time and technology pressures
– Path of least resistance for individual applications
• Centralised ownership of the ESB sometimes is an inhibitor
– Too much effort to get on the bus, technically, politically
– Individuals always mean to redress hacked integrations
– But seldom do – it’s too hard when systems are live

© ThoughtWorks, 2006 16
Spaghetti is a fact of life
• Businesses change
• Processes change
• Applications change
• Integration changes
• Need an enterprise computing strategy that:
– Reflects the changing structure of the business;
– Is spaghetti-friendly;
– Commoditised;
– Robust, secure, dependable, etc.

© ThoughtWorks, 2006 17
Business-Led Integration
• ESBs integrate with whatever existing systems expose
– Green screen, web pages, CORBA objects, XML, etc
• Integration happens at a low level
– Mapping of bits and bytes of one variety onto bits and bytes of
another format
• This makes it hard to engage business in such projects
– Without business benefit no software has value
• Integration is currently opaque to the business
• Business must be involved in integration projects – not just
initiate them
– The integration domain must use the same vocabulary as the
business domain

© ThoughtWorks, 2006 18
Spaghetti-Oriented Architecture
• Fighting against spaghetti is usually unsuccessful
– This does not mean integration should be undertaken without
diligence!
• SOA is an approach which is spaghetti-agnostic
• Services are designed for integration with any consumer
– Integration is decentralised
• Result:
– Loosely coupled, re-usable services
– Focus on business-meaningful process orchestration

© ThoughtWorks, 2006 19
Building the Service-Oriented Enterprise
• SOAP becomes the ubiquitous transfer mechanism across the
enterprise (or Internet!)
• In effect, SOAP messages are the “EAI backbone”
– The underlying transport protocols are arbitrary
• Applications understand SOAP messages natively
– Development platforms have commoditised this
– True end to end integration, but maintains loose coupling
• In this context, existing ESB/EAI software becomes a toolkit for
implementing individual Web Services
• But integration happens at the SOAP level
– With business meaningful documents as the driving metaphor

© ThoughtWorks, 2006 20
Decentralised Integration

• The QoS functionality that a


Web Service requires is
implemented on a per-service
basis
– Not “one size fits all”
• Implement only those QoS
protocols that the service Transactions
currently needs Security
Reliable Delivery
– Push the integration Non-Repudiation
functionality to the edges
• SOAP + WS-Addressing
becomes the “bus”
• Incremental and autonomous
– Deliver high business-value
services first!

© ThoughtWorks, 2006 21
Application Integration...

• EAI/ESB frameworks are fine for application integration


– A framework for development of (distributed) applications
• Think of the EAI toolkit as a container for your application
– Application versus enterprise framework

Application Domain

Adapter Adapter Adapter


Bus
Choreography/Rules/Routing/Transformations

© ThoughtWorks, 2006 22
...and Composite Business Processes
• Processes across the enterprise consume and coordinate
lower-level applications
– Exposed via standards-based services

Application Domain Enterprise Process Domain

Adapter Adapter Adapter


Bus
Gateway
Choreography/Rules/Routing/Transformations

SOAP
Messaging,
WS-*
© ThoughtWorks, 2006 23
Metadata, Metadata, Metadata…

<mex…>

</mex>

<wsdl…>
<policy…>

</policy>

</wsdl>
<endpoint…>

</endpoint>

© ThoughtWorks, 2006 24
Policy and Contract
<wsdl…>
<policy…>
<security-policy>

</security-policy>
<transaction-policy>
<wsdl…>

<policy…>
</transaction-policy>

<reliability-policy>
</policy>


</reliability-policy>
</wsdl>

</policy>

</wsdl>
© ThoughtWorks, 2006 25
Proxy Generation
<wsdl…>
Consumer <policy…>
Implementation
<security-policy>

Web Services Client Stack

Proxy API
</security-policy>
<transaction-policy>
Security …
Handler
(WCF)

</transaction-policy>
<reliability-policy>
Tx Handler

</reliability-policy>
RM Handler …
</policy>

</wsdl>
© ThoughtWorks, 2006 26
End-to-End Messaging

s
Consumer Service

i
Implementation Implementation

t h IT
o S

Web Services Client Stack


Web Services Client Stack

Proxy API

W
Proxy API

n d h
a
Security

C
Handler

w i t Security
Handler

F
(WCF)
(WCF)

a y W C
d
Tx Handler Tx Handler

t o a
RM Handler n d RM Handler

Transport
© ThoughtWorks, 2006 27
“WS-Fabric”
Administrative
domain

Service

Service
Administrative

Service

Administrative
domain

domain
Service
network

Service
Service

SOAP messaging is the communication channel


for applications. The ESB (if it exists) is pushed
to the endpoints.
© ThoughtWorks, 2006 28
Same Old Architects

• Business people and application architects design business-centric


workflows which consume services
– Re-using the functionality already deployed into the service ecosystem
• Service architects and developers build services
– Using WS toolkits like WCF and Axis
• Enterprise architects influence QoS at the SOAP level...
– Using the WS-* specs
• ...and at the transport level
– Existing investments can form the underlay for SOA
• And undertake necessary governance roles

© ThoughtWorks, 2006 29
ESB or SOA?

• Investing in proprietary integration systems now is investing


in future legacy
• ESB is not the solution
– It’s oh-so 1990’s integration glue
• SOA is the solution
– Because it focuses on supporting business processes
• Web Services are a robust and commoditised platform for
SOA delivery

© ThoughtWorks, 2006 30
Conclusions

• SOA is the right integration architecture going forward


– SOA can be implemented incrementally
– Drive SOA from a business perspective
It looks like you’re
• Most valuable processes/applications/services first
trying to build an
– Commoditisation across the board SOA...

• Servers, developers, networking, re-use existing software, etc

• Migrating towards a successful SOA is not always easy


– Learning to build dependable SOAs can be difficult
– ESBs and Wizards cannot help – you need service-savvy geeks and
process-aware business people
• No centralised integration middleware needed
– Metadata, metadata, metadata!

© ThoughtWorks, 2006 31
Questions?

http://jim.webber.name

© ThoughtWorks, 2006 32
Large British Petroleum company
• Project: oil trade settlement system
• Duration: 18 months
• Problem: client tried to implement a message broker to
integrate with systems.
– A large degree of JavaScript was required to do what the tool could
not do.
– Implementation of the custom code was difficult to version control
and test because of the proprietary format.
– Specialist vendor skills were very expensive.
• Resolution: in the end, the client decide to abandon the
integration manager as too costly and difficult to maintain and
custom code was written to do the job.

© ThoughtWorks, 2006 33
Large Swiss bank
• Project: High net-worth portfolio management system and
trading system
• Duration: 2+ years (part of a larger programme of work)
• Problem: Client attempted to use a message broker as a hub to
transform messages to/from legacy system formats into the
canonical format.
– Resulted in large and complex number of mappings, all of which
were difficult to test, version control and deploy.
– It became increasingly difficult to manage changes to canonical
format and changes to the legacy system as these always required
a change and redeployment of the proprietary configuration files
for the message broker.
• Resolution: client moved the mapping to/from legacy formats
into custom written adapters, no longer requiring the message
broker
© ThoughtWorks, 2006 34
Large UK retailer
• Project: Supply chain management and direct sales integration
• Duration: project was 9 months
• Problem: Integration between front-end Web shop, warehouse
stock systems and suppliers’ supply chain and stock system
– Licensing was very expensive, testing and debugging was almost
impossible and maintenance was very complicated.
– Only 1% of messages going through the system required any kind
of manipulation, rendering the integration broker as overkill
• Resolution: After 6 months in production, the client abandoned
the proprietary integration broker

© ThoughtWorks, 2006 35

You might also like