Building Service Orientated Applications using Windows Communication Foundation White Paper

Prepared by Email address Web Address Telephone Number Facsimile Number

: Allan Rees : allan.rees@dariel.co.za : http://www.dariel.co.za : +27 11 715 3732 : +27 11 715 3737

Reg No: 2000/018350/07

Copyright 2009 Dariel Solutions (Proprietary Limited), All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior permission of the copyright holders

Building Service Orientated Applications Using WCF

1.
2.

Indexing
Table of contents
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. Indexing.............................................................................................................2 Table of contents...............................................................................................2 Table of Figures and Examples.........................................................................3 Revision History.................................................................................................4 Foreword............................................................................................................5 Executive Summary...........................................................................................6 The SOA Egg.....................................................................................................7 Nobody I talk to knows what to do, blogs just confuse me.................................8 A SOA model using WCF................................................................................10 Connectors, Managers, Engines and Gateways............................................11 Connector Tier...............................................................................................11 Business Logic Tier........................................................................................12 Data Access Layer.........................................................................................14 Utilities...........................................................................................................15 Data Contracts...............................................................................................16 Deployable.....................................................................................................16 Hosting...........................................................................................................16 Call patterns...................................................................................................17 Technical Requirements................................................................................18 Integrity .........................................................................................................18 Data Validation...............................................................................................22 Flexibility........................................................................................................24 Maintainability................................................................................................24 Performance..................................................................................................29 Reusability.....................................................................................................32 Scalability.......................................................................................................32 Security..........................................................................................................33 Authentication and Authorisation...................................................................33 Try it out.........................................................................................................36 References and Bibliography.........................................................................37

14 June 2011

Confidential

Page 2 of 37

Building Service Orientated Applications Using WCF

3.

Table of Figures and Examples

Figure 1 - SOA Egg...................................................................................................7 Figure 2 SOA Quality Spectrum..............................................................................8 Figure 3 SOA Model.............................................................................................11 Example 1 Manager example operation...............................................................12 Example 2 Engine example operation..................................................................14 Figure 4 Aggregate Root.......................................................................................15 Example 3 Using Proxy Helper.............................................................................17 Figure 5 - Transactions............................................................................................20 Figure 6 Transactions with publish/subscribe.......................................................21 Example 4 Setting up integrity..............................................................................22 Example 5 Data Validation....................................................................................24 Example 6 Instancing............................................................................................33 Example 7 = Security...............................................................................................34

14 June 2011

Confidential

Page 3 of 37

Building Service Orientated Applications Using WCF

4.

Revision History
Name Description Date Version

14 June 2011

Confidential

Page 4 of 37

Building Service Orientated Applications Using WCF

5.

Foreword

All truth passes through three stages:

First, it is ridiculed; second, it is violently opposed; and third, it is accepted as self-evident. - Arthur Schopenhauer (1788-1860)

I would like to thank Garrick Hensberg and Edrich Landsberg for their ideas, many of which have been included into this work.

14 June 2011

Confidential

Page 5 of 37

Building Service Orientated Applications Using WCF

6.

Executive Summary

SOA is a big business buzzword tossed into conversations at board meetings and at executive briefings. At this level, however, SOA really refers to connecting disparate systems across application, department, corporate, and even industry boundaries. This is the Big SOA concept, and this is the realm of the enterprise architect. This is the space of multimillion rand Service Bus applications, SAP systems and other wonderful products. But the fact still remains that a chain is a strong as its weakest link, if the systems hooked up to the top of the range Service Bus are not rock solid and can not be trusted to produce the correct results all the time, then the some of the true potential of the investment is lost. The path to Big SOA begins with a solid base and the ideas presented in this paper will provide you with the tools you need to achieve this. As businesses struggle to reap the rewards of their investment in a SOA approach, the importance of Little SOA is becoming lost in the marketing hype and scramble by vendors to sell you their version of service orientation. This paper offers practical advice for building Service Orientated Applications, using service oriented programming (SO) as an approach, that shows that every component can be a service while still maintaining the technical requirements that modern applications are required to exhibit and in most cases surpassing what many application frameworks offer to date. By building a solid Little SOA base, the platform is set at an enterprise level to realise the composition and reuse that is the value proposition of SOA, as without a Little SOA that is properly portioned, rock solid and composeable, attempts to realise more will fail. I hope to share some of the best practices, techniques and tools that I have learned from the last two years of applying WCF to the construction of service orientated applications and also to provide an insight to application architects on how to achieve the non functional specifications that many projects only play lip service to.

14 June 2011

Confidential

Page 6 of 37

Building Service Orientated Applications Using WCF

7.

The SOA Egg

It seems that everywhere one looks these days, its SOA this and SOA that, Service Orientation has become such a buzz word lately that the methodology is often lost in the marketing hype and scramble by vendors to sell you their version of service orientation. I call this the SOA egg, a farmer presented with an egg might envision a chicken, while a chef may see an omelette and a child a brightly painted Easter egg, SOA is no different, everyone has there own vision of what SOA is. [MSDN1]

Figure 1 - SOA Egg This paper will define a practical set of patterns for building Service Orientated Applications using service orientation as a methodology. These concepts are so deeply ingrained in this architectural approach that it shows that every component can be a service while still maintaining the technical requirements that modern applications are required to exhibit and in most cases surpassing what many application frameworks offer to date. It is important to note that in the context of this framework the term service is not synonymous with the term web service; I view the protocol that the service implements to be an implementation detail of the service not part of its architectural design. I advocate that most of the services detailed below be run as in-process services where it makes sense to do so. An In-Proc or in process call is one where the service resides in the same process as the client. I will be using Windows Communication Foundation (WCF) as an implementation technology, but the framework is technology agnostic and could be implemented in any development language with varying degrees of effort, to my mind WCF offers the most productive method of doing this to date.

14 June 2011

Confidential

Page 7 of 37

Building Service Orientated Applications Using WCF

8.

Nobody I talk to knows what to do, blogs just confuse me

Google the acronym SOA and 45 million results come back; clearly there is a lot of discussion going on around this topic. The trouble is that much of it is sales jargon and opinionated views that cover only a small subsection of the SOA landscape, one is hard pressed to find any practical advice on actually building service orientated applications above putting a web service in front of your application and calling it SOA, other advice pushes you down the slippery path of white box services and tightly coupled, unreliable and unmaintainable systems. I maintain that SOA is simply to our current knowledge the best and most productive way to build software. There is a spectrum of SOA goodness; this spectrum perhaps is a result of the path SOA has taken towards its current level of maturity. The problem is that many frameworks and developers have not reached the same destination yet.

Figure 2 SOA Quality Spectrum

Bad SOA or Just a Bunch of Web Services (JBOWS) is a relic of the SOA past where the focus was on tooling and technology rather than architecture and analysis, services are exposed somewhat arbitrarily. They do not contribute towards any defined broader architecture. Therefore service contracts will have an arbitrary level of stability. JBOWS refers to the scenario where random/unplanned functionality is exposed within an enterprise as Web services. It is a build it and they will come mentality. We usually tend to find this with IT led (rather than business led) SOA initiatives. Service orientated Integration is slightly better than JBOWS, with this approach; service contracts are centred on applications. They are expressed in terms of the

14 June 2011

Confidential

Page 8 of 37

Building Service Orientated Applications Using WCF application with which we are integrating. Consequently if the scope of the application changes, or the application itself is exchanged for another (such as exchanging an Onyx CRM with MS CRM), the service contract is very likely to change. So what about layered service models? In this case, we have atomic business tasks, business processes and data stores abstracted as services. These concepts are not at all stable. Businesses very often make changes in business processes that in turn require changes in how atomic business activities are defined and how data is represented. With this approach we are very likely to find ourselves changing our service contracts as business processes are updated. But what about business capabilities? Business capabilities are by their very nature incredibly stable. Although a retail organisation may make regular changes as to how it goes about inventory management, the fact is that it will always have an inventory management capability. Moreover, other capabilities within the enterprise don't care how inventory management is performed. They only care that it is performed. [Poole] With an understanding of a SOA Quality Spectrum the problem then is how one builds a service orientated application that has a firm grip on the technical requirements of; Integrity, Flexibility, Maintainability, Performance, Reusability, Scalability and Security, Usability is not considered as this will be a technical requirement of the client application. Any application framework also needs to sit at the right of the SOA quality spectrum, be as simple as possible to build by a team of individuals by giving them a set of guidelines of what goes where, thus eliminating many of the decisions that cause projects to loose their conceptual integrity before they are out of the development phase. The framework should build on all the lessons learned from the past eighty years of software engineering and incorporate the patterns and best practices the industry has distilled from this journey. Sounds like a tall order but it is entirely achievable in a vey productive and repeatable way.

14 June 2011

Confidential

Page 9 of 37

Building Service Orientated Applications Using WCF

9.

A SOA model using WCF

The model contains a small set of components from which a solution for any problem domain can be architected. Most of these components are services and each has a well defined architecture, they will differ only in implementation and composition between different projects and domains. They have a well defined interaction pattern and a clear set of guidelines of what their function is and their responsibility within the model making the implementers decision of what does where easier. They are; 1. Service Connector 2. Manager 3. Engine 4. Gateway 5. Agent 6. Capability Bunker

14 June 2011

Confidential

Page 10 of 37

Building Service Orientated Applications Using WCF Figure 3 SOA Model

10.
11.

Connectors, Managers, Engines and Gateways

Connector Tier

Service connectors should not contain any business logic. They should just delegate down to the business layer components to get the real work of the use cases done. The service connector exists to isolate the business logic from needing to know about the service boundary specifics. Service connectors form the boundary between client and the application. The service connector may implement the interface of the manager component. Thus the relationship particulars to any one client are encapsulated into a single layer, allowing the layers below to remain reusable and client agnostic thus when any set of managers is reused, they do not carry any of the restrictions of the older client, we would typically build a connector layer for the new client and enforce its application specific security demands in that layer. The connector layer has an added responsibility of inspecting the validation property of Data Contracts; this can loosely be considered a type of Design by Data Contract. In the scenario where a client calls a service connector, it is possible to have the Data Contracts validate the data that is bound to them, if you own the user interface (UI) components as well as the service connector it is possible to combine the two techniques to leverage the .Net framework to do client side validation on input, and then use the same mechanism to reject invalid input at the connector before any service operations have been invoked. The best way of using the validation is by both sides being .Net, but if some other technology calls the service, as soon as that call hits the connector and the WCF stack has translated the contract we essentially have a .Net contract again, and validation will continue as normal. So when something other than .Net calls the service, they wont have validation, but the service will always validate. This technique is valuable as it reduces the amount of error handling code needed in the layers below to protect against things like parameters exceeding their maximum length, rather the data contract assert it as incorrect and reject the call with a suitable error message, or flag it in the UI thus preventing the invocation of the business logic only to have an error thrown in the gateway. In a development where a Service Bus is being utilised this service layer may become optional if the Service Bus assumes the responsibility of resolving the

14 June 2011

Confidential

Page 11 of 37

Building Service Orientated Applications Using WCF application specific security requests and validation of the data flowing over the Bus.

12.

Business Logic Tier

The business layer components are broken into two main types: manager and engine components. Manager components are the code manifestation of a use case; they should bare a close resemblance to the implemented use case functionality. This aids system validation as it is easy to see what a system does by looking at the managers. Manager components are focused on implementing the workflow or business process for the use case to which it is aligned and determine the API exposed to its clients.

[OperationBehavior(TransactionScopeRequired = true)] Deduction CreateDeductionForBudgetItem(Deduction deduction, BudgetLineItem item) { using (var gateway = new ProxyHelper<BudgetLineItemGateway, IRepository<BudgetLineItem>>()) { gateway.Execute(g => g.Delete(item.Id)); } using (var gateway = new ProxyHelper<DeductionGateway, IRepository<Deduction>>()) { } } return gateway.Execute(g => g.Create(deduction));

Example 1 Manager example operation

Notice that in the above example the manager is controlling the flow of events, not implementing them. Managers can interact with gateways and engines directly, but never other managers, we will use an event driven architecture to achieve this to avoid the coupling and reduction in performance this introduces in a system. Managers are prime candidates for the introduction of workflow technology; this will allow a more visual programming experience, aiding maintenance and understanding of the system

14 June 2011

Confidential

Page 12 of 37

Building Service Orientated Applications Using WCF The engine components encapsulate the business rules that support the business process. This separation of managers and engines is intentional for maintenance reasons. Business rules are likely to change more frequently than the business processes. Keeping them isolated in their own component allows the developer to more easily locate and update the business rule code without needing to check in many locations throughout the architecture. The engine components can potentially be deployed and updated separately from the rest of the architecture to minimize any potential breakage with deploying updates. Engines encapsulate volatility, they can be thought of as a strategy around a concept. They contain the business rules and logic around the strategy that they encapsulate. They do validations and throw business exceptions. Business exceptions are exceptions where the client is able to modify their action and try again, this may be unnatural for many developers who are taught to Try...Catch...Throw exceptions whenever there is a chance of an exception occurring, this is a waste of effort in our opinion as nobody ever codes for all circumstances. When last did anyone write a disk crash exception? Rather let the framework catch, log and shield all application exceptions, as the client could not do anything about your server disk crash anyway, this way you have a log of what went wrong and where, and the client is not exposed to the inner workings of your service. As a side effect you also reduce the complexity of your code and increase the maintainability. Fault contracts are also not recommended at any layer of the model as you introduce coupling between your service and client, this is a bad coupling as they are coupled on exception type, if that type changes in the service, the client also needs to change. Rather throw only business exceptions, and when you do throw them, throw them as FaultException<T> with a message of a suitable nature so that the client can take corrective action and try again.
[OperationBehavior(TransactionScopeRequired = true)] void IClientEngine.ValidateModifyClientBusinessRule(Client client) { (this as IClientEngine).ValidateAddClientBusinessRule(client); } [OperationBehavior(TransactionScopeRequired = true)] void IClientEngine.ValidateDeleteClientBusinessRule(Client client) { bool hasSchedule = client.ActivationScheduleActive; if (hasSchedule) { bSchedule); } RaiseException(ExceptionMessages.ClientHasReha

14 June 2011

Confidential

Page 13 of 37

Building Service Orientated Applications Using WCF

}

Example 2 Engine example operation You should avoid creating references between engines and gateways, this increases code coupling. Where ever possible do the calls to fetch and save data from the manager and pass just the data to the engine to apply its strategy. 13. Data Access Layer

The data access components isolate the business layer from the details of data retrieval in the same way that the connector layer isolates the business layer from the details of exposing remotely accessible services. No data access code should reside outside the data access layer Gateways are responsible for the data access and contract transformation between the above layers and the data store. It is here that the impedance mismatch between schema and domain is resolved. An example would be a derived attribute on a contract, the underlying schema would not store the derived data it would be calculated and added to the contract during the transformation from schema to contract. This is not to be confused with the type of logic that is found in engines, engines exist to encapsulate business logic volatility, so that logic is gleaned from the business, the type of logic here is as a result of our choice of implementation and is unlikely to be as volatile as the engine logic. They follow a bounded context pattern [Evans], in a bounded context pattern each model has a context, the main idea is to define the scope of a model, to draw up the boundaries of its context, and to try to keep it unified. Gateways try to encapsulate their own representation of the domain as they need it to be for the interaction that they require with it. It is important to use an explicit name when naming your context to avoid potential naming conflicts later on. This eliminates the coupling and dependency problem of sharing a domain across every gateway in the application, they simply have their own context of the domain that is rich enough for them to do what they need to only. It is normal to have a gateway per data contract, for contracts that are composed of multiple contracts we still have a gateway for each contract; we simply interact with the composed contracts from the base or container contract, in data driven design terms think of this as an aggregate root. We use the term aggregates to describe the situation as the entire group is considered one unit with regard to data changes. Each aggregate has one root and it is the service

14 June 2011

Confidential

Page 14 of 37

Building Service Orientated Applications Using WCF with which the outside manager interacts, the manager holds only a reference to the root.

Figure 4 Aggregate Root An example could be a Client contract that has an employer property, in SOA we pass contracts as messages and not objects by reference, so employer is also a data contract. We would therefore have a ClientGateway and an EmployerGateway, but the call chain would be ClientManager to ClientGateway to EmployerGateway. We interact with the base contract Client and resolve the calls to the other contracts within the ClientGateway. An interesting observation here is how the concept of the domain has changed from object orientation to service orientation, one is no longer passing objects with state and behaviour, with service orientation you pass only the state as a message and delegate the behaviour to the service. 14. Utilities

Not depicted in the SOA Framework diagram are the utilities involved; this would include class libraries for shared data entity / data contract definitions (one per service), a logging library, a support library for providing diagnostics and end

14 June 2011

Confidential

Page 15 of 37

Building Service Orientated Applications Using WCF user support for deployed applications, and a Utilities library for shared types that dont belong in other libraries. 15. Data Contracts

All communication between components is via well defined data contracts; these follow a Data Transfer Object pattern [Fowler]. We strive to share data contracts where possible. These contracts have the responsibility of validating the data that is acceptable and setting a flag to indicate its acceptableness or lack thereof. 16. Deployable

The capability bunker is our unit of deployment, it translates to an msi file that packages all the related services together and allows an IT Professional to install it on the host server. The factoring of services into a capability bunker is a discipline in its own right, interested parties should read Roger Sessions works regarding this, I would suggest starting with Simple Architectures for Complex Enterprises before progressing to the Software Fortress book. 17. Hosting

A capability bunker can be seen as a piece of deployable business functionality, and within the capability bunker the service connector is the component that is hosted, as it forms the layer between the client and the implemented functionality. How then do we manage to have every class as a WCF service, but only host the connector layer? The technique is to utilize In-Proc services; we have a wrapper class that allows us to simulate the OO like programming model by instancing and hosting In-Proc the service we need to use at that particular time. This allows us to compose a sub system of interrelated services that all work together to realize the business functionality we are exposing via the service layer. As the connector layer component just implements the interface of the manager component, it can be said that we realize our systems use cases by composing the functionality out of a set of new or existing services, which brings us back to the essence and value proposition of SOA. It also then removes the administrative overhead of needing to host hundreds of different services and to manage their configurations. Only the functionality that is important to the business is hosted, and as a side effect system validation also becomes easy to achieve as there should be a close parity between the functionality exposed on the manager interface and the use case.

14 June 2011

Confidential

Page 16 of 37

Building Service Orientated Applications Using WCF Client IClientManager.ManageClient(OperationMode operationMode, Client client) {
using (var manager = new ProxyHelper<ClientManager,IClientManager>()) { return manager.Execute(m => m.ManageClient(operationMode, client)); }

} Example 3 Using Proxy Helper To reduce the dependency on the concrete implementation of ClientManager in the code above one could to use an Inversion of Control container to resolve the actual implementations for you. We have used Unity, but Windsor, StructureMap or Spring.Net would have the same functionality, just be careful around the instancing, as singletons could cause issues in the multithreaded environments implied by this design. 18. Call patterns

In Rogers Sessions book on Software Fortresses he is speaking to the enterprise architect, to the Big SOA paradigm, this paper seeks to drive these ideas down deeper into the enterprise and is looking at ways to apply them at the application or Little SOA level. Capability Bunkers are the Fortresses of the application world, they apply the concepts of simplicity at the application level, and the rules of how to factor unit cases into Bunkers will be explained later in this paper. Synchronous calls between managers, and by default Connectors, as they implement the managers interface, are forbidden, if a manager needs to call another manager they will use the publish/subscribe utility using an asynchronous queued protocol, we apply an event driven paradigm in the implementation of this. Within the Bunker, in Sessions terminology, intra-fortress or in our case intrabunker, synchronous calls and transactions are allowed. In this model calls between the managers and engines or manager and gateway can be synchronous and utilize transactions, but manager to manager calls are always asynchronous.

14 June 2011

Confidential

Page 17 of 37

Building Service Orientated Applications Using WCF

19.

Technical Requirements

The above set of components in themselves offer no help in meeting the technical requirements of; Integrity, Flexibility, Maintainability, Performance, Reusability, Scalability and Security. It is the interaction of the model and the technology used to construct it that will allow us to achieve the required non-functional aspects of our system. It can be said that architecture and the process used to build it are really just different representations of the same thing, and when the two are aligned one is able to build powerful systems very productively. So how then do we leverage the technology to help us achieve our required technical requirements with minimal effort, our chosen technology in this case being WCF, but it is equally achievable in Java with EJB 3 or the equivalent in other languages all that will differ is productivity. WCF can be said to be three minutes and three weeks, three weeks to think about how to do it, three minutes to do it, the beauty of WCF is that you almost dont use it. WCF moves the focus of development from implementation to design, as gives developers a fighting chance to deliver professional software in productive timeframes. Using the above mentioned model and patterns I will demonstrate the architecture and code implementation to achieve the required technical requirement, the level of adherence will depend on your systems requirements. The configuration below or a slight variation will suit 90% of most business applications. 20. Integrity

Integrity is the ability to detect and manage invalid data coming in to system as well as the imposition of complete transactions or rollbacks. Transactions We choose to adopt what is called a client/service transactional model in most cases; this is that if the consumer of the service presents our service a transaction we will join that one and allow the consumer to commit or rollback with our service as an atomic unit, possibly across a distributed system, otherwise we will make the Service Connector the root of all transactions. If no Isolation level is provided by the client we will assume the strictest Isolation level of Serializable, we have found this to be suitable in most systems except read intensive ones or in places where batch like functionality is required.

14 June 2011

Confidential

Page 18 of 37

Building Service Orientated Applications Using WCF In a complex system, transactions are essential for ensuring the integrity and consistency of the data modified by the call chains. Directly managing the transactions is error prone and extremely labour intensive. The transaction support in the .NET Framework and WCF make this considerably easier. Decisions still need to be made on where to apply transactional boundaries and when to span those transactions to other services. The decision includes not only what call chains to include in a transactional scope, but also what level of isolation to use for that transaction. The default in .NET is a Serializable transaction, which provides the maximum protection for consistent data resulting from a transaction. However, Serializable transactions also use the most locking at the database level to ensure that integrity so will have throughput impacts. Those impacts are generally best addressed by scaling out the application as needed for throughput rather than relaxing the isolation unless detailed analysis of each call chain is done by data access experts. Certain calling patterns, such as a read-only SELECT style call chain do not need to participate in a transaction. But if there is any potential for database modifications to be made as part of the overall business process of a call chain, then starting with a serializable transaction encapsulating all the activity of the call chain is the safest starting point. Through analysis, the decision to back off on the span of the transaction scope or the isolation level may be made if performance challenges dictate and the query pattern is permissive. These have to be treated on a case by case basis. Many raise a question of performance and locking issues due to this strict isolation level, research has found that in typical enterprise systems between 13% of users are concurrent at any one instance in time, 1% is normal 3% in a highly transactional system, for example, a system with 1500 users could expect 15 45 users to be concurrently accessing the system. This number is further reduced by assessing if they are all contending over the same resource at that instance in time. In instances where intensive reads are done on a resource before an edit is performed one may relax the Isolation level to ReadCommited, but this should be an architectural decision and not taken lightly. I recommend starting with Serializable, with the exception of select type call chains where a transaction is not required, but I would still recommend flowing the transaction at the interface level as in a SOA one can never be certain of the order and actual call chain that your service may participate in, so to be safe we flow the transaction trough to the next call to use if it needs too.

14 June 2011

Confidential

Page 19 of 37

Building Service Orientated Applications Using WCF The diagram below depicts an iDesign type model of the default model; the red line in this case represents the transaction boundary

Figure 5 - Transactions With this setting we enlist the entire logic unit of work into a single transaction that will enlist any transactional resource that it touches and if any exception occurs will trigger a rollback across the entire call chain even if the pieces are physically deployed on separate machines. This is distributed transaction is achieved by using the Distributed Transaction Controller packaged into the windows operating system. If publish/subscribe mechanisms are used the service calls will at least one transaction removed from the original causing a rollback to the senders queue.

14 June 2011

Confidential

Page 20 of 37

Building Service Orientated Applications Using WCF

Figure 6 Transactions with publish/subscribe

The settings for the Services Connectors are as follows 1. Turn on transaction flow in the bindings <netTcpBinding> <binding name="ReliableTcp" maxBufferSize="100000" maxReceivedMessageSize="100000" transactionFlow="true"> <readerQuotas maxArrayLength="100000" maxStringContentLength="100000"/> <reliableSession enabled="true"/> </binding> </netTcpBinding>

14 June 2011

Confidential

Page 21 of 37

Building Service Orientated Applications Using WCF 2. Set the transaction flow on the interface methods to Allowed [OperationContract] [TransactionFlow(TransactionFlowOption.Allowed)] void CaptureClientDetails(Client client); 3. Set the transaction scope required on the service implementation to true [OperationBehavior(TransactionScopeRequired = true)] void IClientManager.RequestEmployer(Employer employer) { } Example 4 Setting up integrity

The TransactionScopeRequired Attribute is not required for SELECT type call chains where no data is modified. If you want the transactions to flow between machines will need to turn on DTC (start >> run >>mmc >> component services >> computer >> my computer >> properties rt click MSDTC >> security configuration >> tick Network DTC Access >> Allow inbound >> allow outbound >> mutual authentication required >> ok >> ok Open port 135 (Port 135 is used by the RPC Endpoint Mapper.) Allow msdtc.exe (C:\WINDOWS\system32\msdtc.exe) through the firewall. If you are not on the same domain as your server change the authentication setting to none for development

21.

Data Validation

In order to validate invalid data we rely on an interface called IDataErrorInfo, we implement this interface on our data contracts, allowing them to become self validating data contracts. The usual use for this interface is to allow the user interface components to validate the data that needs to bind to a certain contract, thus the contract is able to encapsulate all the information it needs about what is valid data in its context. This method prevents the scattering of validation logic between the client and service. We extend this behaviour in two ways; Firstly we extend this validation to the connector by implementing a WCF behaviour called ServiceDataContractValidation to validate input that arrives at

14 June 2011

Confidential

Page 22 of 37

Building Service Orientated Applications Using WCF the connector service; or OperationDataContractValidation to do the same on a per method basis, IDataErrorInfo will contain any error messages prompted by the binding of incorrect data. To validate if valid data is being bound to our contract we will use our second extension, the DataMemberValidator will inspect the data bound to the data contract and compare it to the rules that where set up for valid data for that contract at design time by calling the Validate method. If any invalid data is bound to the contract a runtime the validator will flag it as invalid data. By using these two attributes together we thus create a mechanism we can use in UI controls to validate input against a particular contract at runtime and provide visual prompts, and reject any contracts that have invalid data bound to them at the service boundary rather that executing all the layers to have an exception thrown from the gateway because a sting was too long. Notice that IExtensibleDataObject is also implemented to allow for round trip versioning.
[DataContract(Namespace = "http://www.dariel.co.za/WCF/03/2009/01")] public class Client : IExtensibleDataObject, IDataErrorInfo [DataMember] [DataMemberValidator("Application Date", ValidatorConstraint.IsRequiredProperty)] public DateTime ApplicationDate { get; set; } [DataMember] [DataMemberValidator("Account Number", ValidatorConstraint.IsRequiredProperty, MaximumLength = 50)] public string AccountNumber { get; set; } protected string Validate(string columnName) { m_Error = String.Empty; MemberValidator<Client> validator = new MemberValidator<Client>(this); m_Error = validator.ValidateColumn(columnName); return m_Error; }

And on the Service Connector;

[OperationDataContractValidation] void IClientManager.ManageClient(OperationMode operationMode, Client client) { CheckArgument(client, "Client"); using (var manager = new ProxyHelper<ClientManager, IClientManager>()) { manager.Execute(m => m.ManageClient(operationMode, client)); } }

14 June 2011

Confidential

Page 23 of 37

Building Service Orientated Applications Using WCF Example 5 Data Validation 22. Flexibility

Flexibility is the ability of a system to adapt to varying environments and situations, and to cope with changes to business policies and rules. A flexible system is one that is easy to reconfigure or adapt in response to different user and system requirements. The intentional separation of concerns between the manger and engine components aids flexibility as only a very small piece of the system is affected when policies or rules change. Engines encapsulate rules, and mangers workflow, so if the how we do something changes, only managers change, if the rules around the process change, only engines change. As a general heuristic we say that engines and gateways should be reused. This means that the strategy and the data access should be written once and reused as much as possible. It is then possible to reconfigure the solution per use case (manager) by picking the engines and gateways required. This becomes productive when you need to provide new functionality, which is conceptually just more use cases, that use the existing data in new ways, you only need to write a new manager and reuse the gateways and engines. If you need to reuse a manager, it means you have two identical use cases, eliminate one and delegate functionality to the existing service, never copy and re-host a manager. This is the reason we introduced the concept of the aggregate root and a gateway per contract, so that even if you require a different view of the data you will be able to aggregate existing gateways to produce the contract that you require. 23. Maintainability

Maintainability is the ability of a system to undergo changes to its components, services, features, and interfaces as may be required when adding or changing the functionality, fixing errors, and meeting new business requirements. The maintainability of a system is dependent on its ability to manage complexity. In reality as it turns out, it is not complexity that causes maintenance headaches, but rather how the complexity is sequestered or partitioned. Complex code that is well partitioned and lives in one place only is easy to manage, and when a simpler method is found it is easy to remove and replace. Complex code that is scattered over many places with multiple dependencies to other pieces of dependent code equals maintenance nightmare.

14 June 2011

Confidential

Page 24 of 37

Building Service Orientated Applications Using WCF So the question is; how to partition our application so that we can avoid complexity from spreading. Luckily the answer lies in the realm of set theory and equivalence relations. In order to manage complexity we must first understand it, the complexity of a system is a function of the number of states in which a system can find its self. An example is a system (A) that has 3 dice, so the number of potential states is 63 or 216 potential states. Another system (B) with one dice has 6 1 or 6 potential states, if you where to make repeated guesses for both systems, you would be right, on average 36 times more often with B than you would be with system A because system B is less complex and easier to predict. With this basic model of complexity, we can gain some insight into how complexity can be better organised. Consider another two systems (C) and (D) both have three six sided dice, but in C all the dice are together as before, but in D the dice are divided into three partitions. Lets assume we can deal with the three partitions independently, in effect, three subsystems each like B. We know that the complexity of C is 216. The overall complexity of system D is the sum of the complexity of each partition (61 + 61 + 61) or 18. If you are not convinced of this imagine inspecting C and D for correctness in C you would need to examine 216 different states, checking each for correctness. In system D you would need to examine only 6 states in each partition.

14 June 2011

Confidential

Page 25 of 37

Building Service Orientated Applications Using WCF S = Sides, D = Dice S D SxD

(partitioned)

SD
(non-partitioned)

6 6 6 6 6 6 6 6 6

1 2 3 4 5 6 7 8 9

6 12 18 24 30 36 40 46 54

6 36 126 1296 7776 46656 279936 1679616 10077696

The above table demonstrates how much more complex a non-partitioned system of 9 dice is compared to a partitioned system containing the same number of dice. Ratio of 10,077,696 to 54, a lot! How then do we partition a software system so we too can gain the advantage from this property of the universe? A partition is a concept from set theory; a partition is a set of sub sets that divide a larger set so that all the items in the larger set live in one, and only one of the subsets. Like our dice example each die lives in one and only one bucket. One can also observe that in any set of items there are at least N possible ways to partition that set. In software systems it just isnt enough to just partition variables into different subsets; we need to find one that honours the dependencies between the variables. This is where equivalence relations become useful. An equivalence relation is a Boolean (true/false) binary relation that for any elements, say a, b, c in the same set the following three properties are always true: E(a,a) is always true reflexivity

14 June 2011

Confidential

Page 26 of 37

Building Service Orientated Applications Using WCF E(a,b) always implies E(b,a) symmetry E(a,b) and E(b, c) always implies E(a, c) transitivity Consider a small shop, all the items in the shop can be considered as being in the same set, if we pick the equivalence relation costs the same as lets see how this becomes useful. The shop stocks Cereal for R10.00, Pens for R10.00, Coke R10.00, and Notebooks R20.00. Reflexivity costs the same as (Pens, Pens) = true Symmetry - costs the same as (Cereal, Pens) = true and costs the same as (Pens, Cereal) = true Transitivity - costs the same as (Cereal, Pens) = true and costs the same as (Pens, Coke) = true and costs the same as (Cereal, Coke) = true And to check our logic costs the same as (Cereal, Notebooks) = false. Thus because for a particular equivalence relation every item in the set lives in only one partition we can use equivalence relations to formulate partitions. In formulating architecture the equivalence relation that is of most interest to us is synergy, two functions are synergistic when one requires the other to be effective, if you can imagine a situation when one can be used without the other they are said to be autonomous. In practice we take our system and pull out all the required functionality for example in a retail operation use cases could be: Calculate total cost, Calculate change, Charge credit card, Remove from stock, Stock report If we apply the synergy test we can we that Calculate total cost is synergistic with Calculate change, it is hard to imagine one without the other, in our retail set. At first glance Charge credit card looks synergistic with Calculate total cost, but remember symmetry, it is a two way street; it is possible to imagine a scenario where you would Calculate total cost without charging credit cards (cash payment

14 June 2011

Confidential

Page 27 of 37

Building Service Orientated Applications Using WCF for example), therefore Charge credit card is actually autonomous to Calculate total cost. Another property of equivalence relations is that although the inverse of an equivalence relation is never itself an equivalence relation it has some interesting mathematical properties of its own. If we denote E as an equivalence relation and its inverse as ~E, if E was used to generate a partition, the following could be said: If a and b are in the same equivalence class of E then ~E(a, b) is always false, and if a and b are in different equivalence classes of E then ~E(a, b) is always false. For example, costs the same as (Cereal, Pens) = true, think of this as E(a, b), and does not cost the same as (Cereal, Pens) = false, then this is ~E(a, b) because these two items are in the same equivalence class of items that cost ten rand. Similarly ~E(Pens, Notebooks) = true. This leads to a useful trick, as you can see it is quite difficult to generate such a set where does not cost the same is false for all elements within a set and true for all elements across sets, but if you realize that does not cost the same is the inverse of costs the same the exercise becomes trivial and you will automatically have the property of non-equivalence for does not cost the same. So why is all of this so important? Well in designing service orientated applications synergy is not a particularly important property, but its inverse autonomy is a very important one, so important it is included as a SOA tenant. So with this incredibly useful method to partition application functionality we also get the best possible partition and achieve the tenet of autonomy, our system is a small as it needs to be no smaller. The rules to formulating partitions can be summarised into five laws: 1. Partitions must be true partitions. a. Items live in one partition only, ever. 2. Partitions must be appropriate to the problem at hand. a. Partitions only minimise complexity when they are appropriate to the problem at hand, e.g. a clothing store organised by colour would have little value to customers looking for what they want. 3. The number of subsets must be appropriate.

14 June 2011

Confidential

Page 28 of 37

Building Service Orientated Applications Using WCF a. Studies show that there seems to be an optimum number of items in a subset, adding more subsets, thus reducing the number of items in each subset, has very little effect on complexity, but reducing the number of subsets, thus increasing the number of elements in each subset seems to add to complexity. The number seems to sit in the range 3 12, with 3 5 being optimal. 4. The size of the subsets must be roughly equal a. The size of the subsets and their importance in the overall partition must be roughly equivalent. 5. The interaction between the subsets must be minimal and well defined. a. A reduction in complexity is dependent on minimising both the number and nature of interactions between subsets of the partition. [Sessions] By reducing complexity and ensuring that our applications are well partitioned and autonomous we effectively sequester complexity and make our systems maintainable 24. Performance

Performance is an indication of the responsiveness of a system to execute any action within a given interval of time. The performance of any software is dependent on many factors, most of which have nothing to do with the framework and technology used to build it. Even the most performing software can still have a method that causes an endless loop or a live lock that causes a client to wait. A poor server or network could also be the cause of a loss of performance. The issue then is to ensure that neither the model nor the technology is the cause of any unacceptable loss of performance. The technology in this instance is WCF, the model advocates that every class is a service in order to leverage the built in plumbing that comes with WCF, but one often hears the question, what about performance? This is the wrong question to ask, as in reality nobody except developers even cares about performance as long as performance is adequate. So the actual question is, is the performance adequate? So lets weigh it all up with all classes as WCF services we automatically benefit from:

14 June 2011

Confidential

Page 29 of 37

Building Service Orientated Applications Using WCF 1. Encrypted calls 2. Authentication 3. Identity propagation 4. Authorization 5. Security audit 6. Transactional propagation 7. Transactional voting 8. Calls timeout 9. Reliability 10. Tracing and logging 11. Profiling and instrumentation 12. Instance management 13. Durability 14. Error masking 15. Fault isolation 16. Channel faulting 17. Buffering and throttling 18. Data version tolerance 19. Synchronization 20. Remotability 21. Queuing 22. Relay and discovery So what about performance, WCF is not free, but it is adequate for most applications, most applications will never notice it. Raw WCF with a single client calling to a single service with no work in it on a laptop provides some 10,000 calls per second, raw C# offers some 300,000 calls per second.

14 June 2011

Confidential

Page 30 of 37

Building Service Orientated Applications Using WCF To put the calls per second thing into perspective Amazon runs at about 150 calls per second. But raw WCF offers no value, in a real system with every aspect of WCF enabled, WCF can manage 100 calls per second on a laptop; in contrast, a C# application with all the equivalent aspects created as bespoke code and enabled manages a maximum of 10 calls per second. Strange you may think, what has happened? The WCF aspects underwent years of rigorous testing by some of the most devious minds in the industry and has been optimized and performance tweaked. So if you care about performance you have to use WCF as all its aspects have been performance tested against hundreds of different scenarios and perform better than the components you have written and only tested against the one or two scenarios that you knew about. There is a cost to using every WCF aspect; every aspect will cause certain degradation in performance, this is summed in the table below.

Aspect Encoding Process Boundary Authorization Authentication Instancing mode Reliability Performance counters Transactions Message Protection Logging and Tracing
[Lowy]

Cost (%) 0 0 0 0 0 10 25 66 85 95

Service guidelines tell us that services should be secure, so if message protection is mandatory, then you have security enabled and you should then also enable everything else as it will not affect your performance as you have already taken the hit for security. Only logging and tracing becomes optional, and that can always be turned on and off via configuration files if you need it.

14 June 2011

Confidential

Page 31 of 37

Building Service Orientated Applications Using WCF 25. Reusability

Reusability defines the capability for components and subsystems to be suitable for use in other applications and in other scenarios. Reusability minimizes the duplication of components and also the implementation time. In typical applications architects always strive for a degree of reuse with in project, but rarely realize reuse between different projects and never between projects that are written in different languages. This is the value proposition of SOA; with SOA we have changed the element of reuse from the class to the service, which does not sound significant, but with the move from a reference based call stack to passing messages reuse has changed from being the reuse of code to the delegation of functionality. The services that are being reused may never be included in you code base, but play just as import role in your system as those you have written just for that project. The model presented here follows this approach by not ever making any assumptions on how they will be called, every service has its own security, instancing and transactional requirements encoded into its design, thus allowing them to be called in any order by any other service without destroying the intent of their designer. A useful tool in managing reusability is the service repository, and those starting out writing service orientated applications should take a serious look at applications such a Managed Services Engine from Microsoft as a tool for creating and managing services. 26. Scalability

Scalability is the ability of a system to function well when there are changes to the load or demand. Typically, the system will be able to be extended over more powerful or more numerous servers as demand and load increase. Experience has shown us that when building WCF services, all roads lead to Per Call instancing, so by default we avoid state at all costs as state is the sworn enemy of scalability. All services should be per call and synchronized and any movement away from this is an executive decision. It applies to all levels of the architecture, service connector, manager, engine and gateway. As synchronized is the WCF default no attribute is required. This design gives us the safest and most scalable solution as default and allows for ease of development as there is no decision about how to set up any piece of the architecture.

14 June 2011

Confidential

Page 32 of 37

Building Service Orientated Applications Using WCF [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)] public class ClientManager : IClientManager [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)] public class ClientGateway : IRepository<Client>, ITranslate<Client, Model.Client>, IClientGateway [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)] public class BudgetEngine : IBudgetEngine Example 6 Instancing 27. Security

Security defines the ways that a system is protected from disclosure or loss of information, and the possibility of a successful malicious attack. A secure system aims to protect assets and prevent unauthorized modification of information. Security in layers is a very prudent approach when thinking about the type of security to incorporate into your application. I would say at minimum, every call to every service should be authenticated and every message to every service should be encrypted and signed to prevent tampering. All that remains is to decide how to authorise calls if needed, best advice is to turn security all the way up as default and only relax a setting if there is a reason to do so. This way as the designer of the system you have covered any liability you may have exposed yourself to by neglecting to include a security setting in your design. 28. Authentication and Authorisation

Authentication is the process for identifying the caller and verifying that they are who they say they are. Authorisation is the process of granting access to resources or operations based on that known identity. Users will have to authenticate with the UI to use it, a number of options exist including Windows integrated security, username/password authentication against a custom store, or using CardSpace and federated identities. Authentication should be performed at each process boundary crossing. This includes coming into the client applications, as well as entering each service and entering the database.

14 June 2011

Confidential

Page 33 of 37

Building Service Orientated Applications Using WCF Authentication at the service and client boundaries will likely be done with a mixture of ASP.NET and Window integrated security I. Client boundary - Connector Layer The code below is an example of user name/password authentication with role based security. I recommend using Juval Lowys declarative security framework as it abstracts all the plumbing of security into a well designed set of utilities. Note that the security setting is that of internet, but remember that this is just a classification and not based on the transport protocol and the same security setting could be applied to TCP calls. This type of security will be enforced on your applications boundary with its client; in this framework these are the Service Connector type components which are responsible for this. It is also possible to leverage the existing security frameworks that .Net has to offer to perform demands for a role on methods. An example of this is shown in the example below. /// <summary> /// Concrete implementation of IClientManager. /// </summary> [ErrorHandlerBehavior] [SecurityBehavior(ServiceSecurity.Internet, "TestServiceCert",UseAspNetProviders = true, ApplicationName = "DarielTest")] [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)] public class ClientService : IClientManager, IBudgetManager { /// <summary> /// Manages the client. /// </summary> /// <param name="operationMode">The operation mode.</param> /// <param name="client">The client.</param> [OperationBehavior(TransactionScopeRequired = true)] [PrincipalPermission(SecurityAction.Demand, Role = "DCO")] [PrincipalPermission(SecurityAction.Demand, Role ="DC_Principal")] [PrincipalPermission(SecurityAction.Demand, Role = "DC_Admin")] [PrincipalPermission(SecurityAction.Demand, Role = "DC_User")] [OperationDataContractValidation] Client IClientManager.ManageClient(OperationMode operationMode, Client client) Example 7 = Security

14 June 2011

Confidential

Page 34 of 37

Building Service Orientated Applications Using WCF II. Trusted Subsystem It is undesirable to allow the consumer of a service access to any backend resources, if such access where granted it would directly compromise the integrity of the resources and would lead to further implementation coupling. A solution to this problem is to use a pattern called the Trusted Sub system pattern where a service uses its own credentials to access resources. In order to do this we must ensure that the service performs the appropriate authorization and authentication on all requests that enter the subsystem. They also need to ensure that the caller is part of the trusted subsystem and not an upstream user trying to bypass access to the trusted subsystem. In this model we enforce the trust relationship by using X.509 certificates as well as client credentials. The type of credentials will depend on your chosen security mechanism. III. Calls between components. The calls between the service layer and the client have been well documented, but what about calls between the other components behind the service connector layer, how are they protected? For this we use a security scenario term as Intranet, as WCF to WCF services calling each other In-Proc, authorisation would be unnecessary as we would be authorising ourselves, remember the trusted sub system pattern, we use the services identity to call other resources, at this point we are inside the trusted subsystem. What is important at this level is that the messages are encrypted and signed; this is the default of the NetNamedPipeBinding we are using to construct our In-Proc service, so WCF both signs the message and encrypts its content providing integrity, privacy, and authenticity. This negates the risk of having the In-Proc windows service in the operating system compromised. IV. Authentication at the database boundary Every hosted service has a designated identity, and it is this identity that will be used to access the data store using Windows authorization. Thus it is possible for system administrators to limit access of any identity to a particular partition of the data store allowing for a very fine grained control if needed.

14 June 2011

Confidential

Page 35 of 37

Building Service Orientated Applications Using WCF

29.

Try it out

A sample project containing all the source code mentioned in this article will follow soon.

14 June 2011

Confidential

Page 36 of 37

Building Service Orientated Applications Using WCF

30.

References and Bibliography

[MSDN1] Chapter 1: Service Orientated Architecture (SOA); Corporation; http://msdn.microsoft.com/en-us/library/bb833022.aspx [Lowy] Programming WCF Services; Juval Lowy; OReilly; 2007

Microsoft

[Sessions] Simple Architectures for Complex Enterprises; Roger Sessions; Microsoft Press; 2008 [Evans] Domain Driven Design; Eric Evans; Addison-Wesley; 2003 [Fowler] Patterns of Enterprise Architecture; Martin Fowler; Addison-Wesley; 2002

14 June 2011

Confidential

Page 37 of 37