Professional Documents
Culture Documents
© ThoughtWorks, 2006 1
Fundamental Premise
• Love
(Lennon/McCartney)
• An SOA
(Webber)
© ThoughtWorks, 2006 2
Roadmap
© ThoughtWorks, 2006 3
Enterprise Application Integration
Approaches
• Data integration
– Extract, transform, route, inject data
• Application level
– Re-use application APIs, or I/O mechanisms
• EAI implementation
– Queues etc
• Business domain tier
– Integration at the object level, as typified by CORBA, DCOM etc
• User interface
– Screen scraping, revamping, etc.
– Last resort, when an application offers no other hooks
© ThoughtWorks, 2006 4
To ESB or not to ESB, that is the question
• Product vendors are keen to provide product solution for
everything
– Or to supply “consultantware” solutions
• The Enterprise Service Bus is the latest incarnation of EAI
technology that supports a number of useful functions:
– Transformations; adapters; choreography; reliability; security etc
• Seems like a good idea...
© ThoughtWorks, 2006 5
Today’s Enterprise Architecture
Accounting Marketing
© ThoughtWorks, 2006 6
How did we get here?
• Tactical decisions
• Time and technology pressures
• Path of least resistance for individual applications
• This is the thin end of the wedge, technical debt can only
increase from here
• Help!
© ThoughtWorks, 2006 7
Vendor Solutions Appear
• Business needs to
compete
– IT needs to be
responsive
• SOA gives IT a
business process focus
• Web Services are the
most sensible way to
implement SOA
• More proprietary
middleware is the
answer!
– 2 + 2 = See:
5 http://www.capeclear.com/technology/index.shtml
© ThoughtWorks, 2006 8
Integration Two Years Later
Accounting Marketing
© ThoughtWorks, 2006 9
Skeletons in the Closet...
© ThoughtWorks, 2006 10
The Appealing Rationale for ESB...
© ThoughtWorks, 2006 11
...And the Reality
© ThoughtWorks, 2006 12
Intelligent Networks, Dumb Idea?
© ThoughtWorks, 2006 13
Integration five years from now
Accounting Marketing
IT
Research
© ThoughtWorks, 2006 14
Integration ten years from now
Accounting Marketing
IT
Research
ESB
Product Development Support
© ThoughtWorks, 2006 15
How did this happen?
• Same old story:
– Tactical decisions
– Time and technology pressures
– Path of least resistance for individual applications
• Centralised ownership of the ESB sometimes is an inhibitor
– Too much effort to get on the bus, technically, politically
– Individuals always mean to redress hacked integrations
– But seldom do – it’s too hard when systems are live
© ThoughtWorks, 2006 16
Spaghetti is a fact of life
• Businesses change
• Processes change
• Applications change
• Integration changes
• Need an enterprise computing strategy that:
– Reflects the changing structure of the business;
– Is spaghetti-friendly;
– Commoditised;
– Robust, secure, dependable, etc.
© ThoughtWorks, 2006 17
Business-Led Integration
• ESBs integrate with whatever existing systems expose
– Green screen, web pages, CORBA objects, XML, etc
• Integration happens at a low level
– Mapping of bits and bytes of one variety onto bits and bytes of
another format
• This makes it hard to engage business in such projects
– Without business benefit no software has value
• Integration is currently opaque to the business
• Business must be involved in integration projects – not just
initiate them
– The integration domain must use the same vocabulary as the
business domain
© ThoughtWorks, 2006 18
Spaghetti-Oriented Architecture
• Fighting against spaghetti is usually unsuccessful
– This does not mean integration should be undertaken without
diligence!
• SOA is an approach which is spaghetti-agnostic
• Services are designed for integration with any consumer
– Integration is decentralised
• Result:
– Loosely coupled, re-usable services
– Focus on business-meaningful process orchestration
© ThoughtWorks, 2006 19
Building the Service-Oriented Enterprise
• SOAP becomes the ubiquitous transfer mechanism across the
enterprise (or Internet!)
• In effect, SOAP messages are the “EAI backbone”
– The underlying transport protocols are arbitrary
• Applications understand SOAP messages natively
– Development platforms have commoditised this
– True end to end integration, but maintains loose coupling
• In this context, existing ESB/EAI software becomes a toolkit for
implementing individual Web Services
• But integration happens at the SOAP level
– With business meaningful documents as the driving metaphor
© ThoughtWorks, 2006 20
Decentralised Integration
© ThoughtWorks, 2006 21
Application Integration...
Application Domain
© ThoughtWorks, 2006 22
...and Composite Business Processes
• Processes across the enterprise consume and coordinate
lower-level applications
– Exposed via standards-based services
SOAP
Messaging,
WS-*
© ThoughtWorks, 2006 23
Metadata, Metadata, Metadata…
<mex…>
…
</mex>
<wsdl…>
<policy…>
…
</policy>
…
</wsdl>
<endpoint…>
…
</endpoint>
© ThoughtWorks, 2006 24
Policy and Contract
<wsdl…>
<policy…>
<security-policy>
…
</security-policy>
<transaction-policy>
<wsdl…>
…
<policy…>
</transaction-policy>
…
<reliability-policy>
</policy>
…
…
</reliability-policy>
</wsdl>
…
</policy>
…
</wsdl>
© ThoughtWorks, 2006 25
Proxy Generation
<wsdl…>
Consumer <policy…>
Implementation
<security-policy>
…
Web Services Client Stack
Proxy API
</security-policy>
<transaction-policy>
Security …
Handler
(WCF)
</transaction-policy>
<reliability-policy>
Tx Handler
…
</reliability-policy>
RM Handler …
</policy>
…
</wsdl>
© ThoughtWorks, 2006 26
End-to-End Messaging
s
Consumer Service
i
Implementation Implementation
t h IT
o S
Proxy API
W
Proxy API
n d h
a
Security
C
Handler
w i t Security
Handler
F
(WCF)
(WCF)
a y W C
d
Tx Handler Tx Handler
t o a
RM Handler n d RM Handler
Transport
© ThoughtWorks, 2006 27
“WS-Fabric”
Administrative
domain
Service
Service
Administrative
Service
Administrative
domain
domain
Service
network
Service
Service
© ThoughtWorks, 2006 29
ESB or SOA?
© ThoughtWorks, 2006 30
Conclusions
© ThoughtWorks, 2006 31
Questions?
http://jim.webber.name
© ThoughtWorks, 2006 32
Large British Petroleum company
• Project: oil trade settlement system
• Duration: 18 months
• Problem: client tried to implement a message broker to
integrate with systems.
– A large degree of JavaScript was required to do what the tool could
not do.
– Implementation of the custom code was difficult to version control
and test because of the proprietary format.
– Specialist vendor skills were very expensive.
• Resolution: in the end, the client decide to abandon the
integration manager as too costly and difficult to maintain and
custom code was written to do the job.
© ThoughtWorks, 2006 33
Large Swiss bank
• Project: High net-worth portfolio management system and
trading system
• Duration: 2+ years (part of a larger programme of work)
• Problem: Client attempted to use a message broker as a hub to
transform messages to/from legacy system formats into the
canonical format.
– Resulted in large and complex number of mappings, all of which
were difficult to test, version control and deploy.
– It became increasingly difficult to manage changes to canonical
format and changes to the legacy system as these always required
a change and redeployment of the proprietary configuration files
for the message broker.
• Resolution: client moved the mapping to/from legacy formats
into custom written adapters, no longer requiring the message
broker
© ThoughtWorks, 2006 34
Large UK retailer
• Project: Supply chain management and direct sales integration
• Duration: project was 9 months
• Problem: Integration between front-end Web shop, warehouse
stock systems and suppliers’ supply chain and stock system
– Licensing was very expensive, testing and debugging was almost
impossible and maintenance was very complicated.
– Only 1% of messages going through the system required any kind
of manipulation, rendering the integration broker as overkill
• Resolution: After 6 months in production, the client abandoned
the proprietary integration broker
© ThoughtWorks, 2006 35