UNIT Ibala5656.webs.com/Unit1-middle-new.

doc New Syllabus Emergence of middleware-objects, web services-middleware elements-vendor architectureinteroperability-middleware in distributed applications-types of middleware-transaction oriented middleware-MOM-RPC Chapter 1 & 5 IT architecture It is a solution to the problem How do I structure my IT applications to best suit the business? Architecture identifies the components of a problem space, shows the relationships among them, and defines the terminology, rules, and constraints on the relationships and components. Main purpose is to try to impose order on chaos.

Fig. identifies various components of the architecture. Environment split into 3 parts, or tiers: 1. Presentation tier- Manages the interface with the various connections to the system. Access channels responsible for ensuring that information is delivered in the right way to whatever device is being used. 2. Application Tier Represents business logic. 3. Data Tier Various databases operate in the data tier. Integration Infrastructure Defines the h/w and s/w required to make the desired connections. A critical part of the s/w content is called middleware. Middleware It is a s/w that is necessary in practice to build distributed applications. 1

8 elements 1. Communication Link Communication link and protocol enable A and B to send data to each other through a physical communication link, either local or wide area 2 sets of protocols a. N/w protocol carry data over the link b. Middleware protocol handle dialog b/w A and B a. N/w protocol TCP/IP connection protocol Features No message loss No messages received in the wrong order No message corruption No message duplication UDP (User Datagram Protocol) Connection-less * Web Services standards do not specify a communication link standard. It runs over HTTP, which in turn uses TCP/IP. It can run over other networking protocol. 2. Middleware protocol 3 Scenarios a. Client/Server Each client initiates the dialogue, and there can be many clients to one server (pull technology) b. Peer-to-peer protocols Both sides are equal, and either one initiates a dialogue eg. TCP, E-mail and directory servers c. Push protocols Server initiates the messages. Ideal for broadcasting * SOAP (Simple Object Access Protocol) for Web services does not define any of these protocols

3. The programmatic interface It is a set of procedure calls used by a program to drive the middleware 3 dimensions a. Classification according to what entities are communicating 1. H/w-to-H/w communication eg. IP 2. Process-to-process communication eg. TCP 3. Object-to-object communication eg. IIOP b. Nature of the interface 2 categories 1. APIs (Application Programming Interface) Fixed set of procedure calls for using the middleware Has run-time flexibility It is for interpreters Many styles of interface o Message-based API sends and receives messages, with associated message types eg. MQSeries where message type is queue name o Command-language-based command is encoded into a language eg. SQL o Operation-call-based the operation call name of the server operation and its parameters are built up by a series of middleware procedure calls eg. COM+ 2. GPIs (Generated Programming Interfaces) Generate the interface from the component source or from a separated file written in an IDL Has compile-time flexibility It is for component builders c. Classification according to the impact on process threads control Blocked (Synchronous) thread stops until the reply arrives Unblocked (Asynchronous) client every now and then looks to see whether reply has arrived Event-based when the reply comes, an event is caused, waking up the client * Web services standard do not have a programmatic interface. Microsoft Visual Studio generates all files needed by ASP.NET to run web service. In C#, it designates some public methods [WebMethod]. Visual studio will generate relevant code to: Convert XML messages into calls on the method with parameter values set from the XML input data Convert the method result to XML output data Build a WSDL service description of the interface 4. Common Data Format It describes how the data should be structured so that both A and B understand it. One might use ASCII, other EBCDIC or UNICODE. One might have 16-bit, little-endian integers, other might use 32-bit, big-endian integers. For this, we use marshalling * In Web Services, XML is the universal data presentation standard 5. Server control 3

3 main tasks a. Process and thread control When the load is heavy, additional processes or threads may be started. When the load lightens, lessen number of processes or threads. When processes or threads are inactive, they need to be terminated. b. Resource Management For instance, database connection pooling c. Object Management Objects may be activated or deactivated * Web Services standards has nothing to say on server control 6. Naming and Directory Services Provide the means of locating the communicating elements, for example, for A to find B Naming service maps IP address to human understandable names eg. DNS Directory services provide a general facility for looking things up same as telephone directory * In Web Services, UDDI directories 7. Security It is concerned with ensuring that the communication b/w A and B is safe enough to meet requirements. It includes Encryption need support at protocol level Reliable identification of systems need support from server control functions Granting permission for access need support from dedicated security manager system * Web services have a number of security extensions 8. System Management It is concerned with the configuration, operation, fault management and performance of the environment. It keeps the whole thing working properly * Web Services standards have nothing to say about system management Why IT architecture is different from what we did before? Standalone applications each with its own presentation layer, business processing logic, and database is called silos

Ex. Company has 6 silos o Order Entry, Delivery, Warehouse, Billing, Accounts and Marketing Problems o Inconsistencies in the data o Difficulties in creating a single user view across multiple silos o Lack of business process integration Silo thinking is deeply embedded. Reasons include: o Departments dont want to lose power o Project management wants self-contained projects to control, often for budgetary reasons o Development methodologies are silo based o There is a fear of large integrated systems o There is a fear of changing large existing applications

Alternative Surround Architecture Organizations look for ways to gain the benefits of integration without the pain of touching the existing code. The idea is to surround the existing applications with a front end and a back end.

Recovery issues make the front-end hub much more complex In back end, data warehouse is much more effective if the data is clean. Otherwise the notion of taking data from multiple sources and reconstructing the correct form becomes unrealistic. Another Alternative Packages Organization cant run using only one package. Integration of packages needed. So problems exist with silo applications exist with packages also. Rewrite or evolve Given a decision to take an IT architectural approach, how can it be implemented? 2 options: rewrite or evolve. Rewriting because of technology-driven change such as DCE CORBA J2EE Web Services Reasons to be cautious about rewrites Existing application works Issues related to scalability, resiliency, and security may have been solved already Rewriting is expensive and lengthy, during which time the business cannot move forward Rewriting is risky

When should you rewrite? 1. When the technology is so old 2. Business functionality has changed so much * A key characteristic of a good architecture is that it is designed for evolution Chapter 2 Emergence of standard middleware Early Days / Networking evolution Networking dumb green-screen terminals attached to mainframes In 1969, DARPA (U.S. Department of Defenses Advanced Research Projects Agency) 4 node n/w called ARPANET In 1972, ARPANET - 50 node n/w (mainly universities and research sites)

Middleware for communication b/w companies in the same industry o SWIFT(Society for Worldwide Interbank Financial Telecommunication) for Interbank money transfers o IATA (International Air Transport Association) for airline industry for handling functions such as reservations and check-in In 1970s, IT vendors came with n/w architectures o IBMs System N/w Architecture (SNA) o Sperrys Distributed Communication Architecture (DCA) 7

o Burroughs N/w Architectute (BNA) o DECs Distributed N/w Architecture (DNA) Basic Services o File transfer o Remote printing o Terminal transfer o Remote file access Distributed applications eg. E-mail Open systems one standard that will create competition and drive down prices o Eg. Electrical interfaces (RS232), Networking protocols (X.25) o Standards Authority ISO (International Organization for Standardization) o Open Systems Interconnection (OSI) series of standards

o UNIX Advantages Cheap Unlimited supply of clever people If the system goes down, the only people complaining are students o TCP/IP Transmission Control Protocol is a connection-oriented transport standard for program-to-program communication over IP 8

OSI seven layer model Drawbacks Standardization process was too slow Standards were so complex OSI virtual terminal standard

So far, we have discussed n/w evolution. Building applications over n/w is the concern of middleware. Middleware Preliminaries Ease of use Location transparency Message delivery integrity messages should not be lost or duplicated Message format integrity messages should not be corrupted Language transparency Thread required for the run-time execution of code; it contains information like the position in the code file of the next executable instruction and the procedure call stack to return to the right place when the procedure is finished. Multithreading is running a process that has more than one thread, which makes it possible for more than one processor to work on a single process. Remote Procedure Calls (RPC) It is a way to access a remote service. The idea is that the syntax in the client (caller) and the server (the called) programs remain the same, just as if they were on the same machine. Eg. for RPC mechanisms Open N/w Computing (ONC) from Sun Microsystems Distributed Computing Environment (DCE) from Open S/w Foundation (OSF) RPC Architecture In RPC we write IDL which is similar to header file in C IDL generates stubs and server skeletons which are small chunks of C code that are compiled and linked to the client and server programs. Stub converts parameters into a string of bits and send the message over the n/w Skeleton takes the message, converts it back into parameters, and calls the server

 Marshalling the process of converting parameters to message. It handles different data formats. For instance, if the client uses 32-bit big-endian integers and server uses 64-bit small-endian integers, the marshalling s/w does the translation

10

Problem - multithreading Client side Client program is blocked when it is calling a remote procedure. So 2 threads are created. One for processing RPC and other for processing user input Server side Separate thread is needed for every client connection * Experience programmers avoid writing multithreading programs. Because of the problems in testing and finding the bugs. RPC Issues Issues that must be addressed: Marshalling: Parameters must be marshalled into a standard representation. Parameters consist of simple types (e.g., integers) and compound types (e.g., C structures or Pascal records). Moreover, because each type has its own representation, the types of the various parameters must be known to the modules that actually do the conversion. For example, 4 bytes of characters would be uninterpreted, while a 4-byte integer may need to the order of its bytes reversed. Semantics: Call-by-reference not possible: the client and server don't share an address space. That is, addresses referenced by the server correspond to data residing in the client's address space. One approach is to simulate call-by-reference using copy-restore. In copy-restore, call-byreference parameters are handled by sending a copy of the referenced data structure to the server, and on return replacing the client's copy with that modified by the server. 11

However, copy-restore doesn't work in all cases. For instance, if the same argument is passed twice, two copies will be made, and references through one parameter only changes one of the copies. Binding: How does the client know who to call, and where the service resides? The most flexible solution is to use dynamic binding and find the server at run time when the RPC is first made. The first time the client stub is invoked, it contacts a name server to determine the transport address at which the server resides. Transport protocol: What transport protocol should be used? Exception handling: How are errors handled?

Remote database access It provides the ability to read or write to a database that is physically on a different machine from the client program. How remote database access differs from RPC? It is a run-time, not a compile-time, interface. There is no IDL or equivalent. The procedure itself is typically written in a proprietary language, although many database vendors allow stored procedures to be written in Java. Technologies for remote database access ODBC (Open Database Connectivity) OLE DB (Object Linking and Embedding Database) ADO (Active Data Objects) ADO.NET JDBC (Java Database Connectivity) JDO (Java Data Objects) Distributed Transaction Processing A transaction usually means a sequence of information exchange and related work that is treated as a unit for the purposes of satisfying a request and for ensuring database integrity. For a transaction to be completed and database changes to make permanent, a transaction has to be completed in its entirety.

12

MOM Message Oriented Middleware Every DAD needs a MOM Motto of MOM Consortium DAD Distributed Application Development MOM Message-Oriented Middleware

MOM is a key piece of middleware that is absolutely essential for a class of client/server products. If the application can tolerate a certain level of time-independent responses, MOM provides the easiest path for creating enterprise and inter-enterprise client/server systems. This accumulates outgoing transactions in queue and do a bulk upload when a connection can be established with an office server.

MOM allows general purpose messages to be exchanged in a client/server system using message queues. Message queuing It is program-to-message queue. You can think of a message queue as a very fast mailbox since you can put a message in the box without the recipients being active. Provide an asynchronous communications protocol, meaning that the sender and receiver of the message do not need to interact with the message queue at the same time. Messages placed onto the queue are stored until the recipient retrieves them. 13

Key differences between Message queue and TCP/IP socket Queues have names Queues are independent of program If n/w goes down, messages can wait in the queue until the n/w comes up again Queues can be put on disk so that if the system goes down, queue is not lost Queue can be a resource manager and cooperate with a transaction manager Some message queue systems can cross networks of different types Advantage of message queuing powerful and simple idea efficient and has been used for applications that require sub-second response times. Disadvantage of message queuing no IDL no marshaling; the message is a string of bits, and is up to you to ensure that the sender and the receiver know the message layout. Two message queue software 1. MQSeries 2. MSMQ from Microsoft To put a message into the queue, a program does a Put; and to take a message out of the queue, the program does a Get. Middleware does the transfer of messages from queue to queue.

14

 If the n/w is down b/w client and server, messages cannot be queued

Message queuing products have lightweight versions, targeted at mobile workers using portable PCs or smaller devices

Advantages The primary advantage of a message-based communications protocol lies in its ability to store, route or transform messages in the process of delivery. Storage Most MOM systems provide persistent storage to back up the message transfer medium. This means that the sender and receiver do not need to connect to the network at the same time (asynchronous delivery). This becomes particularly useful when dealing with intermittent connections, such as unreliable networks, casual users or timed connections. It also means that should the receiver application fail for any reason, the senders can continue unaffected, as the messages they send will simply accumulate in the message store for later processing when the receiver restarts.

15

Routing MOM delivers another important advantage through its ability to route messages within the middleware layer itself. Taking things a step further, middleware messaging can deliver a single message to more than one recipient (broadcast or multicast). Transformation In a message-based middleware system, the recipient's message need not replicate the sender's message exactly. A MOM system with built-in intelligence can transform messages en-route to match the requirements of the sender or of the recipient. In conjunction with the routing and broadcast/multicast facilities, one application can send a message in its own native format, and two or more other applications may each receive a copy of the message in their own native format. Many modern MOM systems provide sophisticated message transformation (or mapping) tools which allow programmers to specify transformation rules applicable to a simple GUI drag-anddrop operation. Disadvantages The primary disadvantage of message oriented middleware stems from its requiring an extra component in the architecture, the message transfer agent. As with any system, adding another component can lead to reductions in performance and reliability, and can also make the system as a whole more difficult and expensive to maintain.

MOM Vs RPC Feature Metaphor Client/Server time relationship Client/Server Sequencing Style Post-office like

MOM Telephone like

RPC

Asynchronous. Clients and Servers may Synchronous. Clients and Servers operate at different times and speeds must run concurrently. Servers must keep up with clients. No fixed sequence Queued Servers must first come up before clients can talk to them. Call-Return Yes

Partners need to be No available Load-balancing Transactional Support Message Filtering Performance Asynchronous processing

Single queue can be used to implement Requires a separate TP Monitor. FIFO or priority based policy Yes (Some Products) Message Queue can participate in the commit synchronization Yes Yes. Queues and triggers are required No. Requires a Transactional RPC.

No Limited. Requires threads and tricky code for managing threads. 16

Slow. An intermediate hop is required Fast

Message queuing versus distributed transaction processing Distributed transaction processing The debit on account A and the credit on Account B are both done in one distributed transaction

Disadvantages Performance is degraded because of the overhead of sending additional messages for the two-phase commit If either system is down or the n/w b/w the systems is down the transaction cannot take place Message queuing It solves both the problems. Dotted line indicates that the message is not allowed to reach the second machine until the first transaction has committed 17

Flaw If destination transaction fails, money is taken out of one account and disappears. It fails A in ACID. Solution Reversal transaction

18

What is Middleware?

Middleware does not include the software that provides the actual service thats in the servers domain. It also does not include the user interface or the applications logic thats in the clients domain. It starts with the API set on the client side that is used to invoke a service, and it covers the transmission of the request over the network and the resulting response. Middleware divided into two types: (a) General Middleware (b) Service-Specific Middleware

(a) General Middleware

It is the substrate for most client/server interactions It includes the communication stacks, distributed directories, authentication stacks, distributed directories, authentication services, network time, remote procedure calls, and queuing services. Products that fall into the general middleware category include DCE, ONC+, NetWare, NamedPipes, LAN Server, LAN Manager, Vines, TCP/IP, APPC and NetBIOS. Message Oriented Middleware (MOM) products from Peerlogic, Covia, Message Express, System Strategies and IBM. These are depends on message queue system and increases portability, interoperability, flexibility.

(b) Service-Specific Middleware

It is need to accomplish a particular client/server type of service. This includes o Database-specific middleware such as ODBC, DRDA, EDA/SQL, SAG/CLI and Oracle Glue. o OLTP-specific middleware such as Tuxedos ATMI and /WS, Encinas Transactional RPC, and X/Opens TxRPC and XATMI o Groupware-specific middleware such as MAPI, VIM, VIC, SMTP and Lotus Notes Calls o Object-specific middleware such as OMGs CORBA and Microsofts Network OLE (or DCOM) o Internet-specific middleware such as HTTP, S-HTTP and SSL o System Management-specific middleware such as SNMP, CMIP and ORBs.

19