Professional Documents
Culture Documents
A) Physical Clocks
B) Lamport Clocks
C) Mutual Exclusion
1
Assignment Set – 1(ANSWER-1a)
Message passing is the paradigm of communication where messages are sent from a sender to one or more recipients. Forms of
messages include (remote) method invocation, signals, and data packets. When designing a message passing system several
choices are made:
Prominent theoretical foundations of concurrent computation, such as the Actor model and the process calculi are based on
message passing. Implementations of concurrent systems that use message passing can either have message passing as an
integral part of the language, or as a series of library calls from the language. Examples of the former include many distributed
object systems. Examples of the latter include Microkernel operating systems pass messages between one kernel and one or
more server blocks, and the Message Passing Interface used in high-performance computing.
Message passing systems and models
Distributed object and remote method invocation systems like ONC RPC, Corba, Java RMI, DCOM, SOAP, .NET Remoting,
CTOS, QNX Neutrino RTOS, OpenBinder, D-Bus and similar are message passing systems.
Message passing systems have been called "shared nothing" systems because the message passing abstraction hides underlying
state changes that may be used in the implementation of sending messages.
Message passing model based programming languages typically define messaging as the (usually asynchronous) sending
(usually by copy) of a data item to a communication endpoint (Actor, process, thread, socket, etc.). Such messaging is used in
Web Services by SOAP. This concept is the higher-level version of a datagram except that messages can be larger than a
packet and can optionally be made reliable, durable, secure, and/or transacted.
Messages are also commonly used in the same sense as a means of interprocess communication; the other common technique
being streams or pipes, in which data are sent as a sequence of elementary data items instead (the higher-level version of a
virtual circuit).
Examples of message passing style
#Actor model implementation
#Amorphous computing
#Flow-based programming
#SOAP (protocol)
Synchronous versus asynchronous message passing
Synchronous message passing systems require the sender and receiver to wait for each other to transfer the message. That is,
the sender will not continue until the receiver has received the message.
Synchronous communication has two advantages. The first advantage is that reasoning about the program can be simplified in
that there is a synchronisation point between sender and receiver on message transfer. The second advantage is that no
buffering is required. The message can always be stored on the receiving side, because the sender will not continue until the
receiver is ready.
Asynchronous message passing systems deliver a message from sender to receiver, without waiting for the receiver to be ready.
The advantage of asynchronous communication is that the sender and receiver can overlap their computation because they do
not wait for each other.
Synchronous communication can be built on top of asynchronous communication by ensuring that the sender always wait for
an acknowledgement message from the receiver before continuing.
The buffer required in asynchronous communication can cause problems when it is full. A decision has to be made whether to
block the sender or whether to discard future messages. If the sender is blocked, it may lead to an unexpected deadlock. If
messages are dropped, then communication is no longer reliable.
2
Assignment Set – 1(ANSWER-1b)
In computer science, a buffer is a region of memory used to temporarily hold data while it is being moved from one place to
another. Typically, the data is stored in a buffer as it is retrieved from an input device (such as a Mouse) or just before it is sent
to an output device (such as Speakers). However, a buffer may be used when moving data between processes within a
computer. This is comparable to buffers in telecommunication. Buffers can be implemented in either hardware or software, but
the vast majority of buffers are implemented in software. Buffers are typically used when there is a difference between the rate
at which data is received and the rate at which it can be processed, or in the case that these rates are variable, for example in a
printer spooler or in online video streaming.
A buffer often adjusts timing by implementing a queue (or FIFO) algorithm in memory, simultaneously writing data into the
queue at one rate and reading it at another rate.
Applications
Buffers are often used in conjunction with I/O to hardware, such as disk drives, sending or receiving data to or from a network,
or playing sound on a speaker. A line to a rollercoaster in an amusement park shares many similarities. People who ride the
coaster come in at an unknown and often variable pace, but the roller coaster will be able to load people in bursts (as a coaster
arrives and is loaded). The queue area acts as a buffer: a temporary space where those wishing to ride wait until the ride is
available. Buffers are usually used in a FIFO (first in, first out) method, outputting data in the order it arrived.
Buffer versus cache
A cache often also acts as a buffer, and vice versa. However, cache operates on the premise that the same data will be read from
it multiple times, that written data will soon be read, or that there is a good chance of multiple reads or writes to combine to
form a single larger block. Its sole purpose is to reduce accesses to the underlying slower storage. Cache is also usually an
abstraction layer that is designed to be invisible.
A 'Disk Cache' or 'File Cache' keeps statistics on the data contained within it and commits data within a time-out period in
write-back modes. A buffer does none of this.
A buffer is primarily used for input, output, and sometimes very temporary storage of data that is either enroute between other
media or data that may be modified in a non-sequential manner before it is written (or read) in a sequential manner.
Note that there are many different (often incompatible) technologies commonly used to accomplish this.
Message passing
An RPC is initiated by the client, which sends a request message to a known remote server to execute a specified procedure
with supplied parameters. The remote server sends a response to the client, and the application continues its process. There are
many variations and subtleties in various implementations, resulting in a variety of different (incompatible) RPC protocols.
While the server is processing the call, the client is blocked (it waits until the server has finished processing before resuming
execution).
An important difference between remote procedure calls and local calls is that remote calls can fail because of unpredictable
network problems. Also, callers generally must deal with such failures without knowing whether the remote procedure was
actually invoked. Idempotent procedures (those that have no additional effects if called more than once) are easily handled, but
enough difficulties remain that code to call remote procedures is often confined to carefully written low-level subsystems.
Sequence of events during a RPC
The client calls the Client stub. The call is a local procedure call, with parameters pushed on to the stack in the normal way.
3
The client stub packs the parameters into a message and makes a system call to send the message. Packing the parameters is
called marshalling.
The kernel sends the message from the client machine to the server machine.
The kernel passes the incoming packets to the server stub.
Finally, the server stub calls the server procedure. The reply traces the same steps in the reverse direction.
Distributed Shared Memory (DSM), also known as a distributed global address space (DGAS), is a concept in computer
science that refers to a wide class of software and hardware implementations, in which each node of a cluster has access to
shared memory in addition to each node's non-shared private memory.
Software DSM systems can be implemented in an operating system, or as a programming library. Software DSM systems
implemented in the operating system can be thought of as extensions of the underlying virtual memory architecture. Such
systems are transparent to the developer; which means that the underlying distributed memory is completely hidden from the
users. In contrast, Software DSM systems implemented at the library or language level are not transparent and developers
usually have to program differently. However, these systems offer a more portable approach to DSM system implementation.
Software DSM systems also have the flexibility to organize the shared memory region in different ways. The page based
approach organizes shared memory into pages of fixed size. In contrast, the object based approach organizes the shared
memory region as an abstract space for storing shareable objects of variable sizes. Another commonly seen implementation
uses a tuple space, in which the unit of sharing is a tuple.
Shared memory architecture may involve separating memory into shared parts distributed amongst nodes and main memory; or
distributing all memory between nodes. A coherence protocol, chosen in accordance with a consistency model, maintains
memory coherence.
Examples of such systems include:
Kerrighed
OpenSSI
MOSIX
Terracotta
TreadMarks
DIPC
Assignment Set – 1(ANSWER-3b)
Memory consistency model
Order in which memory operations will appear to execute
#What value can a read return?
Affects ease-of-programming and performance
An implementation of a memory consistency model is often stricter than the model would allow. For example, SC allows the
possibility of a read returning a value that hasn’t been written yet (see example discussed under 3.2 Sequential Consistency).
Clearly, no implementation will ever exhibit an execution with such a history. In general, it is often simpler to implement a
slightly stricter model than its definition would require. This is especially true for hardware realizations of shared memories
[AHJ91, GLL+90]
The memory consistency model of a shared-memory multiprocessor provides a formal specification of how the memory system
will appear to the programmer, eliminating the gap between the behavior expected by the programmer and the actual behavior
supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read in a
shared-memory programexecution. Intuitively, a read should return the value of the “last” write to the same memory location.
In uniprocessors, “last” is precisely defined by program order, i.e., the order in which memory operations appear in the
program. This is not the case in multiprocessors. For example, in Figure 1, the write and read of the Data field within a record
are not related by program order because they reside on two different processors. Nevertheless, an intuitive extension of the
uniprocessor model can be applied to the multiprocessor case. This model is called sequential consistency. Informally,
sequential consistency requires that all memory operations appear to execute one at a time, and the operations of a single
processor appear to execute in the order described by that processor’s program. Referring back to the program in Figure 1, this
model ensures that the reads of the data field within a dequeued record will return the new values written by processor P1.
Sequential consistency provides a simple and intuitive programming model. However, it disallows many hardware and
compiler optimizations that are possible in uniprocessors by enforcing a strict order among shared memory operations. For this
reason, a number of more relaxed memory consistency models have been proposed, including some that are supported by
4
commercially available architectures such as Digital Alpha, SPARC V8 and V9, and IBM PowerPC. Unfortunately, there has
been a vast variety of relaxed consistency models proposed in the literature that differ from one another in subtle but important
ways. Furthermore, the complex and non-uniform terminology that is used to describe these models makes it difficult to
understand and compare them. This variety and complexity also often leads to misconceptions about relaxed memory
consistency models.
5
for time T, where T is a parameter to be determined by the algorithm. During this waiting period, the clock process records the
time, according to its own clock, when the message was received. At the end of the waiting period, the clock process estimates
the skew of its clock with respect to each of the other nodes on the basis of the times at which it received resync messages. It
then computes a fault-tolerant average of the next resynchronization interval.
2. The global averaging algorithms differ mainly in the manner in which the fault-tolerant average of the estimated
skews is calculated. Two commonly used algorithms are: 1. The simplest algorithm is to take the average of the estimated
skews and use it as the correction for the local clock. However, to limit the impact of faulty clocks on the average value, the
estimated skew with respect to each node is compared against a threshold, and skews greater than the threshold are set to zero
before computing the average of the estimated skews. 2. In another algorithm, each node limits the impact of faulty clocks by
first discarding the m highest and m lowest estimated skews and then calculating the average of the remaining skews, which is
then used as the correction for the local clock. The value of m is usually decided based on the total number of clocks (nodes).
6
of work done on the problem of synchronizing physical clocks. We refer the reader to [4] for an intro- :) C/(t' - 0) = lim C,(t' -
181).
Examples of such resources are fine-grained flags, counters or queues, used to communicate between code that runs
concurrently, such as an application and its interrupt handlers. The synchronization of access to those resources is an acute
problem because a thread can be stopped or started at any time.
To illustrate: suppose a section of code is altering a piece of data over several program steps, when another thread, perhaps
triggered by some unpredictable event, starts executing. If this second thread reads from the same piece of data, the data, which
is in the process of being overwritten, is in an inconsistent and unpredictable state. If the second thread tries overwriting that
data, the ensuing state will probably be unrecoverable. These shared data being accessed by critical sections of code must,
therefore, be protected, so that other processes which read from or write to the chunk of data are excluded from running.
A mutex is also a common name for a program object that negotiates mutual exclusion among threads, also called a lock.
7
Stateful servers, on the other hand, do store session state. They may, therefore, keep track of which clients have opened which
files, current read and write pointers for files, which files have been locked by which clients, etc.
The main advantage of stateless servers is that they can easily recover from failure. Because there is no state that must be
restored, a failed server can simply restart after a crash and immediately provide services to clients as though nothing
happened. Furthermore, if clients crash the server is not stuck with abandoned opened or locked files. Another benefit is that
the server implementation remains simple because it does not have to implement the state accounting associated with opening,
closing, and locking of files.
The main advantage of Stateful servers, on the other hand, is that they can provide better performance for clients. Because
clients do not have to provide full file information every time they perform an operation, the size of messages to and from the
server can be significantly decreased. Likewise the server can make use of knowledge of access patterns to perform read-ahead
and do other optimizations. Stateful servers can also offer clients extra services such as file locking, and remember read and
write positions.
Replication
The main approach to improving the performance and fault tolerance of a DFS is to replicate its content. A replicating DFS
maintains multiple copies of files on different servers. This can prevent data loss, protect a system against down time of a
single server, and distribute the overall workload. There are three approaches to replication in a DFS:
1. Explicit replication: The client explicitly writes files to multiple servers. This approach requires explicit support from the
client and does not provide transparency.
2. Lazy file replication: The server automatically copies files to other servers after the files are written. Remote files are only
brought up to date when the files are sent to the server. How often this happens is up to the implementation and affects the
consistency of the file state.
3. Group file replication: write requests are simultaneously sent to a group of servers. This keeps all the replicas up to date, and
allows clients to read consistent file state from any replica.
Caching
Besides replication, caching is often used to improve the performance of a DFS. In a DFS, caching involves storing either a
whole file, or the results of file service operations. Caching can be performed at two locations: at the server and at the client.
Server-side caching makes use of file caching provided by the host operating system. This is transparent to the server and helps
to improve the server’s performance by reducing costly disk accesses.
Client-side caching comes in two flavours: on-disk caching, and in-memory caching. On-disk caching involves the creation of
(temporary) files on the client’s disk. These can either be complete files (as in the upload/download model) or they can contain
partial file state, attributes, etc. In-memory caching stores the results of requests in the client-machine’s memory. This can be
process-local (in the client process), in the kernel, or in a separate dedicated caching process. The issue of cache consistency in
DFS has obvious parallels to the consistency issue in shared memory systems, but there are other tradeoffs (for example, disk
access delays come into play, the granularity of sharing is different, sizes are different, etc.). Furthermore, because write-
through caches are too expensive to be useful, the consistency of caches will be weakened. This makes implementing Unix
semantics impossible. Approaches used in DFS caches include, delayed writes where writes are not propagated to the server
immediately, but in the background later on, and write-on-close where the server receives updates only after the file is closed.
Adding a delay to write-on-close has the benefit of avoiding superfluous writes if a file is deleted shortly after it has been
closed.
8
4. Handling communication between cooperating processes that have been separated as a result of process
migration.
On a single processor, multithreading generally occurs by time-division multiplexing (as in multitasking): the processor
switches between different threads. This context switching generally happens frequently enough that the user perceives the
threads or tasks as running at the same time. On a multiprocessor or multi-core system, the threads or tasks will actually run at
the same time, with each processor or core running a particular thread or task.
Many modern operating systems directly support both time-sliced and multiprocessor threading with a process scheduler. The
kernel of an operating system allows programmers to manipulate threads via the system call interface. Some implementations
are called a kernel thread, whereas a lightweight process (LWP) is a specific type of kernel thread that shares the same state and
information.
Programs can have user-space threads when threading with timers, signals, or other methods to interrupt their own execution,
performing a sort of ad-hoc time-slicing.