Synchronization (computer science)

In computer science, synchronization refers to one of

two distinct but related concepts: synchronization of
processes, and synchronization of data. Process synchro-
nization refers to the idea that multiple processes are to
join up or handshake at a certain point, in order to reach
an agreement or commit to a certain sequence of ac-
tion. Data synchronization refers to the idea of keep-
ing multiple copies of a dataset in coherence with one
another, or to maintain data integrity. Process synchro-
nization primitives are commonly used to implement data
synchronization.[1]
Figure 1: Three processes accessing a shared resource (critical
section) simultaneously.
1 The need for synchronization
proper synchronization techniques[2] are not applied, it
The need for synchronization does not arise merely in may cause a race condition where the values of variables
multi-processor systems but for any kind of concurrent may be unpredictable and vary depending on the timings
processes; even in single processor systems. Mentioned of context switches of the processes or threads.
below are some of the main needs for synchronization: For example, suppose that there are three processes
Forks and Joins: When a job arrives at a fork point, it is namely, 1, 2 and 3. All three of them are concurrently
split into N sub-jobs which are then serviced by n tasks. executing and they need to share a common resource
After being serviced, each sub-job waits until all other (critical section) as shown in Figure 1. Synchronization
sub-jobs are done processing. Then, they are joined again should be used here to avoid any conicts for accessing
and leave the system. Thus, in parallel programming, we this shared resource. Hence, when Process 1 and 2 both
require synchronization as all the parallel processes wait try to access that resource it should be assigned to only
for several other processes to occur. one process at a time. If it is assigned to Process 1, the
other process (Process 2) needs to wait until Process 1
Producer-Consumer: In a producer-consumer relation-
frees that resource (as shown in Figure 2).
ship, the consumer process is dependent on the producer
process till the necessary data has been produced. Another synchronization requirement which needs to be
considered is the order in which particular processes or
Exclusive use resources: When multiple processes are de-
threads should be executed. For example, we cannot
pendent on a resource and they need to access it at the
board a plane until we buy a ticket. Similarly, we can-
same time the operating system needs to ensure that only
not check emails without validating our credentials (i.e.,
one processor accesses it at a given point in time.This re-
user name and password). In the same way, an ATM will
duces concurrency.
not provide any service until we provide it with a correct
PIN.
Other than mutual exclusion, synchronization also deals
2 Thread or process synchroniza- with the following:
tion
deadlock, which occurs when many processes are
Thread synchronization is dened as a mechanism which waiting for a shared resource (critical section) which
ensures that two or more concurrent processes or threads is being held by some other process. In this case the
do not simultaneously execute some particular program processes just keep waiting and execute no further;
segment known as critical section. Processes access
to critical section is controlled by using synchroniza- starvation, which occurs when a process is waiting
tion techniques. When one thread starts executing the to enter the critical section but other processes mo-
critical section (serialized segment of the program) the nopolize the critical section and the rst process is
other thread should wait until the rst thread nishes. If forced to wait indenitely;

1
2 3 MINIMIZING SYNCHRONIZATION
Figure 2: A process accessing a shared resource if available,
based on some synchronization technique.
priority inversion, which occurs when a high prior-
ity process is in the critical section, it may be in-
terrupted by a medium priority process. This viola-
tion of priority rules can happen under certain cir-
cumstances and may lead to serious consequences in
real-time systems;
busy waiting, which occurs when a process fre-
quently polls to determine if it has access to a critical
section. This frequent polling robs processing time
from other processes.
3 Minimizing synchronization
One of the challenge for exascale algorithm design is
to minimize or reduce synchronization. Synchroniza-
tion takes more time than computation, especially in dis-
tributed computing. Reducing synchronization drew at-
tention from computer scientists for decades. Whereas
it becomes an increasingly signicant problem recently
as the gap between the improvement of computing and
latency increases. Experiments have shown that (global)
communications due to synchronization on a distributed
computers takes a dominated share in a sparse iterative
solver.[3] This problem is receiving increasing attention
after the emergence of a new benchmark metric,the High
Performance Conjugate Gradient(HPCG),[4] for ranking
the top 500 supercomputers.
3.1 Classic problems of synchronization
The following are some classic problems of synchroniza-
tion:
The ProducerConsumer Problem (also called The
Bounded Buer Problem);
The ReadersWriters Problem;
The Dining Philosophers Problem.
These problems are used to test nearly every newly pro-
posed synchronization scheme or primitive.
3.2 Hardware synchronization
Many systems provide hardware support for critical sec-
tion code.
A single processor or uniprocessor system could disable
interrupts by executing currently running code without
preemption, which is very inecient on multiprocessor
systems.[5] The key ability we require to implement syn-
chronization in a multiprocessor is a set of hardware
primitives with the ability to atomically read and modify
a memory location. Without such a capability, the cost
of building basic synchronization primitives will be too
high and will increase as the processor count increases.
There are a number of alternative formulations of the ba-
sic hardware primitives, all of which provide the abil-
ity to atomically read and modify a location, together
with some way to tell if the read and write were per-
formed atomically. These hardware primitives are the
basic building blocks that are used to build a wide variety
of user-level synchronization operations, including things
such as locks and barriers. In general, architects do not
expect users to employ the basic hardware primitives, but
instead expect that the primitives will be used by system
programmers to build a synchronization library, a pro-
cess that is often complex and tricky.[6] Many modern
hardware provides special atomic hardware instructions
by either test-and-set the memory word or compare-and-
swap contents of two memory words.
3.3 Synchronization strategies in pro-
gramming languages
In Java, to prevent thread interference and memory con-
sistency errors, blocks of code are wrapped into synchro-
nized (lock_object) sections. This forces any thread to ac-
quire the said lock object before it can execute the block.
The lock is automatically released when thread leaves the
block or enter the waiting state within the block. Any
variable updates, made by the thread in synchronized
block, become visible to other threads whenever those
other threads similarly acquires the lock.
3.5 Mathematical foundations 3

In addition to mutual exclusion and memory consistency, The barrier synchronization wait function for ith thread
Java synchronized blocks enable signaling, sending events can be represented as:
from those threads, which have acquired the lock and exe- (Wbarrier)i = f ((Tbarrier)i, (Rthread)i)
cute the code block to those which are waiting for the lock
within the block. This means that Java synchronized sec- Where Wbarrier is the wait time for a thread, Tbarrier
tions combine functionality of mutexes and events. Such is the number of threads has arrived, and Rthread is the
primitive is known as synchronization monitor. arrival rate of threads.[10]
Any object is ne to be used as a lock/monitor in Java. Experiments show that 34% of the total execution time is
[9]
The declaring object is implicitly implied as lock object spent in waiting for other slower threads.
when the whole method is marked with synchronized.
The .NET framework has synchronization primitives.
Synchronization is designed to be cooperative, demand-
ing that every thread or process follow the synchro- 3.4.3 Semaphores
nization mechanism before accessing protected resources
(critical section) for consistent results. In .NET, locking,
Main article: Semaphore_(programming)
signaling, lightweight synchronization types, spinwait and
interlocked operations are some of mechanisms related to
synchronization.[7] Semaphores are signalling mechanisms which can allow
one or more threads/processors to access a section. A
Semaphore has a ag which has a certain xed value as-
3.4 Implementation of Synchronization sociated with it and each time a thread wishes to access
the section, it decrements the ag. Similarly, when the
3.4.1 Spinlock thread leaves the section, the ag is incremented. If the
ag is zero, the thread cannot access the section and gets
Main article: Spinlock blocked if it chooses to wait.
Some semaphores would allow only one thread or process
Another eective way of implementing synchronization is in the code section. Such Semaphores are called binary
by using spinlocks. Before accessing any shared resource semaphore and are very similar to Mutex. Here, if the
or piece of code, every processor checks a ag. If the value of semaphore is 1, the thread is allowed to access
ag is reset, then the processor sets the ag and continues and if the value is 0, the access is denied.[11]
executing the thread. But, if the ag is set (locked), the
threads would keep spinning in a loop and keep checking
if the ag is set or not. But, spinlocks are eective only
if the ag is reset for lower cycles otherwise it can lead
to performance issues as it wastes many processor cycles 3.5 Mathematical foundations
waiting.[8]
Synchronization was originally a process-based concept
whereby a lock could be obtained on an object. Its pri-
3.4.2 Barriers
mary usage was in databases. There are two types of (le)
Main article: Barrier_(computer_science) lock; read-only and readwrite. Read-only locks may be
obtained by many processes or threads. Readerswriter
locks are exclusive, as they may only be used by a single
Barriers are simple to implement and provide good re- process/thread at a time.
sponsiveness. They are based on the concept of imple-
menting wait cycles to provide synchronization.Consider Although locks were derived for le databases, data is also
three threads running simultaneously, starting from bar- shared in memory between processes and threads. Some-
rier 1. After time t, thread1 reaches barrier 2 but it still times more than one object (or le) is locked at a time.
has to wait for thread 2 and 3 to reach barrier2 as it does If they are not locked simultaneously they can overlap,
not have the correct data. Once, all the threads reach bar- causing a deadlock exception.
rier 2 they all start again. After time t, thread 1 reaches Java and Ada only have exclusive locks because they are
barrier3 but it will have to wait for threads 2 and 3 and thread based and rely on the compare-and-swap processor
the correct data again. instruction.
Thus, in barrier synchronization of multiple threads there An abstract mathematical foundation for synchronization
will always be a few threads that will end up waiting for primitives is given by the history monoid. There are also
other threads as in the above example thread 1 keeps wait- many higher-level theoretical devices, such as process
ing for thread 2 and 3. This results in severe degradation calculi and Petri nets, which can be built on top of the
of the process performance.[9] history monoid.
4 4 DATA SYNCHRONIZATION

3.6 Synchronization examples condition variables;

Following are some synchronization examples with re- readerswriter locks;

spect to dierent platforms.[12]
spinlocks;

3.6.1 Synchronization in Windows barriers.

Windows provides:
4 Data synchronization
interrupt masks, which protect access to global re-
sources (critical section) on uniprocessor systems; Main article: Data synchronization
A distinctly dierent (but related) concept is that of data
spinlocks, which prevent, in multiprocessor systems,
spinlocking-thread from being preempted;

dispatchers, which act like mutexes, semaphores,

events, and timers.

3.6.2 Synchronization in Linux

Linux provides:

semaphores;

spinlocks;

readerswriter locks, for the longer section of codes

which are accessed very frequently but don't change
very often.

Enabling and disabling of kernel preemption replaced

spinlocks on uniprocessor systems. Prior to kernel ver-
sion 2.6, Linux disabled interrupt to implement short crit-
ical sections. Since version 2.6 and later, Linux is fully
preemptive.

3.6.3 Synchronization in Solaris

Figure 3: Changes from both server and client(s) are synchro-
Solaris provides: nized.

synchronization. This refers to the need to keep multiple

semaphores;
copies of a set of data coherent with one another or to
condition variables; maintain data integrity, Figure 3. For example, database
replication is used to keep multiple copies of data syn-
adaptive mutexes, binary semaphores that are imple- chronized with database servers that store data in dier-
mented dierently depending upon the conditions; ent locations.

readerswriter locks: Examples include:

turnstiles, queue of threads which are waiting on ac- File synchronization, such as syncing a hand-held
quired lock. MP3 player to a desktop computer;

Cluster le systems, which are le systems that

3.6.4 Pthreads synchronization
maintain data or indexes in a coherent fashion across
a whole computing cluster;
Pthreads is a platform-independent API that provides:
Cache coherency, maintaining multiple copies of
mutexes; data in sync across multiple caches;
 5

RAID, where data is written in a redundant fashion in the source system which captures the data, the security
across multiple disks, so that the loss of any one disk and information access privileges must be enforced on the
does not lead to a loss of data; target systems as well to prevent any potential misuse of
the information. This is a serious issue and particularly
Database replication, where copies of data on a when it comes for handling secret, condential and per-
database are kept in sync, despite possible large ge- sonal information. So because of the sensitivity and con-
ographical separation; dentiality, data transfer and all in-between information
must be encrypted.
Journaling, a technique used by many modern le
systems to make sure that le metadata are updated
on a disk in a coherent, consistent manner.
4.1.4 Data quality

4.1 Challenges in data synchronization Data quality is another serious constraint. For better
management and to maintain good quality of data, the
Some of the challenges which user may face in data syn- common practice is to store the data at one location and
chronization: share with dierent people and dierent systems and/or
applications from dierent locations. It helps in prevent-
ing inconsistencies in the data.
data formats complexity;

real-timeliness;
4.1.5 Performance
data security;
There are ve dierent phases involved in the data syn-
data quality; chronization process:
performance.
data extraction from the source (or master, or main)
system;
4.1.1 Data formats complexity
data transfer;
When we start doing something, the data we have usually
is in a very simple format. It varies with time as the orga- data transformation;
nization grows and evolves and results not only in building
a simple interface between the two applications (source data load to the target system.
and target), but also in a need to transform the data while
passing them to the target application. ETL (extraction Each of these steps is very critical. In case of large
transformation loading) tools can be very helpful at this amounts of data, the synchronization process needs to be
stage for managing data format complexities.. carefully planned and executed to avoid any negative im-
pact on performance.
4.1.2 Real-timeliness

This is an era of real-time systems. Customers want to 5 See also

see the current status of their order in e-shop, the current
status of a parcel deliverya real time parcel tracking
Futures and promises, synchronization mechanisms
, the current balance on their account, etc. This shows
in pure functional paradigms
the need of a real-time system, which is being updated
as well to enable smooth manufacturing process in real-
time, e.g., ordering material when enterprise is running
out stock, synchronizing customer orders with manufac- 6 References
turing process, etc. From real life, there exist so many
examples where real-time processing gives successful and [1] William, Stallings (2012). Operating Systems. Content
competitive advantage. Technologies, Inc.

[2] Gramoli, V. (2015). More than you ever wanted to

4.1.3 Data security
There are no xed rules and policies to enforce data secu-
rity. It may vary depending on the system which you are
using. Even though the security is maintained correctly
know about synchronization: Synchrobench, measuring the
impact of the synchronization on concurrent algorithms
(PDF). Proceedings of the 20th ACM SIGPLAN Sympo-
sium on Principles and Practice of Parallel Programming.
ACM. pp. 110.
6 7 EXTERNAL LINKS

[3] Shengxin, Zhu and Tongxiang Gu and Xingping Liu

(2014). Minimizing synchronizations in sparse iterative
solvers for distributed supercomputers. Computers &
Mathematics with Applications. ELSEVIER. 67 (1): 199
209.

[4] { { | url=http://hpcg-benchmark.org/ } }

[5] Silberschatz, Abraham; Gagne, Greg; Galvin, Peter Baer

(July 11, 2008). Chapter 6: Process Synchronization.
Operating System Concepts (Eighth ed.). John Wiley &
Sons. ISBN 978-0-470-12872-5.

[6] Hennessy, John L.; Patterson, David A. (September 30,

2011). Chapter 5: Thread-Level Parallelism. Computer
Architecture: A Quantitative Approach (Fifth ed.). Mor-
gan Kaufmann. ISBN 978-0-123-83872-8.

[7] Synchronization Primitives in .NET framework. MSDN,

The Microsoft Developer Network. Microsoft. Retrieved
23 November 2014.

[8] Massa, Anthony (2003). Embedded Software Develop-

ment with ECos. Pearson Education Inc. ISBN 0- 13-
035473-2.

[9] Meng, Chen, Pan, Yao, Wu, Jinglei, Tianzhou, Ping,

Jun. Minghui (2014). A speculative mechanism for
barrier sychronization. 2014 IEEE International Confer-
ence on High Performance Computing and Communica-
tions (HPCC), 2014 IEEE 6th International Symposium on
Cyberspace Safety and Security (CSS) and 2014 IEEE 11th
International Conference on Embedded Software and Sys-
tems (ICESS).

[10] M. M., Rahman (2012). Process synchronization in mul-

tiprocessor and multi-core processor. Informatics, Elec-
tronics & Vision (ICIEV), 2012 International Conference.
doi:10.1109/ICIEV.2012.6317471.

[11] Li, Yao, Qing, Carolyn (2003). Real-Time Concepts for

Embedded Systems. CMP Books. ISBN 1578201241.

[12] Silberschatz, Abraham; Gagne, Greg; Galvin, Peter Baer

(December 7, 2012). Chapter 5: Process Synchroniza-
tion. Operating System Concepts (Ninth ed.). John Wiley
& Sons. ISBN 978-1-118-06333-0.

Schneider, Fred B. (1997). On concurrent program-

ming. Springer-Verlag New York, Inc. ISBN 0-387-
94942-9.

7 External links
Anatomy of Linux synchronization methods at IBM
developerWorks

The Little Book of Semaphores, by Allen B. Downey

 7

8 Text and image sources, contributors, and licenses

8.1 Text
Synchronization (computer science) Source: https://en.wikipedia.org/wiki/Synchronization_(computer_science)?oldid=763260029
Contributors: Tregoweth, CesarB, Mac, Julesd, Jay, Tea2min, Vadmium, Mind the gap, Liao, Linas, Ruud Koot, Tckma, Yuriybrisk,
VKokielov, Jbcrail, Siddhant, Snarius, Procedure, CyberShadow, InverseHypercube, JorgePeixoto, Bluebot, Bazonka, Javalenok, Jon-
Harder, Tinctorius, Kalatash, Cydebot, Valodzka, Thijs!bot, Ferenczy, AllenDowney, Treppur, Remember the dot, 28bytes, Oshwah,
Wykypydya, SieBot, X-Fi6, Mochan Shrestha, El Pantera, Ggia, Czarko, VladN, XLinkBot, Rror, Galzigler, Deineka, Addbot, Ghet-
toblaster, Luckas-bot, Yobot, Materialscientist, Xqbot, DSisyphBot, Pomoxis, WSikUipeCdiKaS, Maggyero, Tom.Reding, Dinamik-bot,
Alph Bot, GoingBatty, RenamedUser01302013, ClueBot NG, Urul, Riceahmed, Troy.hester, BG19bot, Northamerica1000, Mquinson,
Dexbot, Davew123, Hrnen3000, DK1029, Filedelinkerbot, M.usman.14, FrobeniusNorm, M Santosh Dora, GreenC bot, Aakankshapatil,
Aashlesha94, Villagepig, Roeliedup and Anonymous: 79

8.2 Images
File:Commons-logo.svg Source: https://upload.wikimedia.org/wikipedia/en/4/4a/Commons-logo.svg License: PD Contributors: ? Origi-
nal artist: ?
File:Data_Synchronization.png Source: https://upload.wikimedia.org/wikipedia/commons/d/dc/Data_Synchronization.png License:
CC BY-SA 4.0 Contributors: Own work Original artist: M.usman.14
File:Folder_Hexagonal_Icon.svg Source: https://upload.wikimedia.org/wikipedia/en/4/48/Folder_Hexagonal_Icon.svg License: Cc-by-
sa-3.0 Contributors: ? Original artist: ?
File:Multiple_Processes_Accessing_the_shared_resource.png Source: https://upload.wikimedia.org/wikipedia/commons/a/a3/
Multiple_Processes_Accessing_the_shared_resource.png License: CC BY-SA 4.0 Contributors: Own work Original artist: M.usman.14
File:Question_book-new.svg Source: https://upload.wikimedia.org/wikipedia/en/9/99/Question_book-new.svg License: Cc-by-sa-3.0
Contributors:
Created from scratch in Adobe Illustrator. Based on Image:Question book.png created by User:Equazcion Original artist:
Tkgd2007
File:Shared_Resource_access_in_synchronization_environment.png Source: https://upload.wikimedia.org/wikipedia/commons/5/
50/Shared_Resource_access_in_synchronization_environment.png License: CC BY-SA 4.0 Contributors: Own work Original artist:
M.usman.14

8.3 Content license

Creative Commons Attribution-Share Alike 3.0