International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
741
Various Schemes of Load Balancing in Distributed Systems- A Review
Monika Kushwaha
Pranveer Singh Institute of Technology
Kanpur, U.P. (208020)
U.P.T.U., Lucknow
Abstract
Over the past three decade, parallel and distributed
computing witnessed major growth due to the declined
cost of hardware, advancement in communication
technology, explosive growth of internet and need to
solve large-scale problems. These systems often face the
problem of load imbalance, which can degrade the
performance of the system. Load balancing improves the
system performance by dividing the work load
effectively among the participating computers/nodes.
This paper presents various aspects of load balancing
and explains various load balancing approaches and
strategies and their respective algorithms.
Keywords: Load balancing, Thresholding, Migration,
Distributed Systems.
1. INTRODUCTION
Outstanding progress in computer technology has lead to
the rise of the demand of high speed processing and the
need of high scalability, availability and rapid response.
This resulted in use of parallel and distributed computing
systems where more than one processor processes the
job simultaneously. One of the main research issues in
parallel and distributed system is effective strategy to
distribute workload among multiple processors. Load
balancing is used for minimizing the response time,
maximizing the throughput, and to avoid the overload.
Load balancing is to ensure that every processor in the
system does approximately the same amount of work at
any point of time [2].
1.1 Load Balancing
Load balancing is the process of distribution or
redistribution of load among processor therefore
improving the performance of the system. In computing,
load balancing distributes workload across multiple
computing resources, such as computers, a computer
cluster, network links, or central processing units. Load
balancing aims to optimize resource use,
maximize throughput, minimize response time, and
avoid overload of any single resource [3].
www.ijsret.org
Volume 4, Issue 7, July 2015
Saurabh Gupta
Pranveer Singh Institute of Technology
Kanpur, U.P. (208020)
U.P.T.U., Lucknow
1.1.1 Need of Load Balancing
A distributed system contains number of processors
working independently with each other and linked by
communication channel. Some are not linked with any
communication channel. Each processor has an initial
load that is the amount of work to be performed, and
each may have a different processing capacity. The
workload has to be evenly distributed among all
processors based on their processing speed so that time
to execute all tasks gets minimized and idle time of each
processor can be reduced. This is why we need load
balancing.
Load imbalance is also a main problem in data parallel
applications and here also it mainly occurs due to the
uneven distribution of data among the various processors
in the system. Without good load distribution strategies
and techniques, we cannot aim to reach good speedup
and good efficiency.
1.1.2 Issues related to Load Balancing
The following issues are analyzed during load balancing:
a) In distributed environment the communication
channels are of finite bandwidth and the
processing units may be physically distant hence
load balancing need to take decision of whether to
allow task migration or not.
b) A computing job may not arbitrarily divisible
leading to certain constraints in dividing tasks.
c) Each job consists of several smaller tasks and each
of those tasks can have different execution times.
d) The load on each processor as well as on the
network can vary from time to time based on the
workload brought about by the users.
e) The processors capacity may be different from
each other in architecture, operation system, CPU
speed, memory size, and available disk space.
Taking above factors into consideration the load
balancing can be generalized into four basic steps:
a) Monitoring processor load and state
b) Exchanging load and state information between
processors
c) Calculating the new work distribution
d) Actual data movement
 International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
1.1.3 Policies of Load Balancing
A good load balancing algorithm is defined by some
basic policies [10] which are as follows:
Information policy: Specifies what workload
information to be collected, updated, When it is to be
collected and from where.
a) Triggering policy: Determines the appropriate
period to start a load balancing operation.
b) Resource type policy: Retrieve the task from most
overloaded resource or not.
c) Location policy: Uses the results of the resource
type policy to find a suitable partner for a server or
receiver
d) Selection policy: Defines the tasks which should be
migrated from busiest resources (source) to most
idle resources (receiver). A selection policy
considers several factors in selecting a task, for e.g.
transfer of small task will take less overhead.
1.1.4 Advantages of Load Balancing
Some major advantages of load balancing [5, 6] are as
follows:
a) It reduces the task waiting time.
b) It minimizes task response time.
c) It maximizes utilization of system resources.
d) It maximizes system throughput.
e) It improves reliability, and stability of the system.
f) It accommodates future modification.
g) Long starvation is avoided for small jobs.
h) In load balancing overall system performance is
enhance by improving the performance of each
node.
Because of the above benefits, the load balancing
strategies become a field of intensive research. As a
result, a large number of scheduling and load balancing
algorithms have been developed in the past several
years.
2. LOAD BALANCING APPROACHES
In load balancing there are two approaches and each
approach have their own algorithms. The two
approaches are as follows:
2.1 Static Load Balancing
In the Static load balancing approach, the load balancing
decisions are made deterministically or probabilistically
at compile time according to the performance of
computing nodes and remain constant during runtime.
Number of tasks in each node is fixed in this approach
[5]. Static load balancing methods are non-preemptive
i.e. once the load is allocated to the node it cannot be
transferred to another node.
www.ijsret.org
742
Volume 4, Issue 7, July 2015
Pros
1. It has minimum communication delays.
2. Algorithms are simple and easy to implement.
3. System overhead is minimized.
Cons
1. Task cannot be migrated while execution.
2. Overall performance of system decreased due to
load fluctuations.
3. Less useful when task have different execution
time and nodes are heterogeneous.
2.1.1 Algorithms of Static Load Balancing
Among various proposed algorithms major static load
balancing algorithms are as follows:
2.1.1.1 Round Robin Algorithm
Round Robin Algorithm (RRA) assigns tasks
sequentially and evenly to all the nodes. All tasks are
assigned to computing nodes based on Round Robin
order, meaning that computing nodes choosing is
performed in series and will be back to the first
computing node if the last computing node has been
reached [5]. Each node maintains its load index locally
independent of allocations from remote node.
Pros
1. Inter-process communication is not required.
2. Useful for jobs of equal processing time and
nodes of same capabilities.
Cons
1. Not useful when tasks have unequal processing
time.
2. Not useful when nodes have different capacities.
2.1.1.2 Randomized Algorithm
Randomized Algorithm (RA) uses random numbers in
selecting computing nodes for processing, without
having any information about the current or previous
load on the node. The computing nodes are selected
randomly following random numbers generated based on
a statistic distribution [5, 1].
Pros
1. It works well for particular special purpose
applications.
2. No inter process communication is required.
Cons
1. It is not considered elegant solution.
2. Maximum response time among all algorithms.
2.1.1.3 Central Manager Algorithm
In each step in Central Manager Algorithm (CMA),
central node selects slave node to be assigned a task.
 International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
Slave node having least load is being selected. The
central node maintains the load index of all slave nodes
connected to it. Whenever, load is changed, a message is
send by the slave nodes to the central node [2]. The load
manager makes load balancing decisions based on the
system load information, allowing the best decision
when of the process created [5, 2].
Pros
1. Performs well when dynamic activities are
created by different hosts.
Cons
1. CMA needs high level of inter process
communication.
2. Single point of failure.
2.1.1.4 Threshold Algorithm
In Threshold Algorithm (TA) tasks/processes are
assigned immediately upon creation to the computing
nodes (processors). Computing nodes for new processes
are selected locally without sending remote messages.
Each node keeps a private copy of the systems load
information. The load of a computing node can be
characterized by one of the three levels which are: under
loaded, medium and overloaded. Two threshold
parameters t_under and t_upper can be used to describe
these levels [5, 7]:
1. Under loaded: workload < t_under,
2. Medium : t_under workload t_upper,
3. Overloaded: workload > t_upper.
In the initial stage, all the computing nodes are
considered under loaded. Whenever the load state of a
computing node exceeds the load level threshold, then it
sends messages regarding the new load state to all of the
other computing nodes, regularly updating them so that
as the actual load state of the entire system can be known
to every node.
If the local state is not overloaded then the process is
allocated locally. Otherwise, a remote under loaded
processor is selected, and if no such host exists, the
process is also allocated locally.
Pros
1. It has low inter process communication.
2. Performance is improved because of large number of
local process allocations which decreases the
overhead of remote process allocations and remote
memory accesses.
Cons
Whenever all remote processors are overloaded then all
processes are allocated locally causing two major
problems:-
www.ijsret.org
743
Volume 4, Issue 7, July 2015
1. One overloaded processor may have much higher
load than other overloaded processors, causing
significant disturbance in load balancing.
2. Increased execution time of an application.
2.2 Dynamic Load Balancing
There are three general classes of problems where a
static load balancing is either impossible or can lead to
imbalance of load, problems are:
1. The first class consists of problems in which all
the tasks are available at the beginning of the
computation but the amount of time required by
each task is different.
2. The second class consists of problems in which
tasks are available at the beginning but as the
computation progresses, the amount of time
required by each task changes.
3. The third class consists of problems in which
tasks are not available at the beginning but are
generated dynamically.
In static load balancing too much information about task
and system is required before execution which is not
possible every time like in these three classes of
problems. So Dynamic load balancing was developed to
address these constraints.
Dynamic load balancing makes more informative load
balancing decisions during execution by the runtime
state information [1]. In dynamic load balancing
algorithms work load is distributed among the
processors at runtime. These algorithms monitor changes
on the system work load and redistribute the work
accordingly.
Pros
1. Dynamic load balancing works well for
heterogeneous systems.
2. Task can be redistributed to any processor while
run time hence overloading and under loading
problems become minimum.
3. It works well for task having different execution
time.
4. The system need not be aware of run-time
behavior of the applications before execution.
Cons
1. High communication over heads occurs and
becomes more when number of processors
increase.
2. Dynamic load balancing algorithms are complex
hence not very easy to implement.
3. System overhead increases because it is
preemptive.
 International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
2.2.1 Categories of Dynamic Load Balancing
In general, dynamic load-balancing algorithms can be
broadly categorized as follows:
a) Centralized or Decentralized
In centralized approach, only one processor acts as the
main or central controller (Master). Master node process
or holds the collection of tasks to be performed. Master
node has global view on the load information of all
nodes connected to it, and decides how to allocate jobs
to each of the nodes. And rest of the nodes act as slaves
[10]. Tasks are sent to the slave processes for execution.
It has advantage of ease of implementation, but suffers
from single point of failure.
In Decentralized approach, all nodes in the system are
involved in making the load balancing decision. The
state information is spread among all the nodes and
nodes are responsible in managing their own resources
or allocating tasks residing in their queues to other
nodes. It is more scalable and has better fault tolerance.
b) Cooperative or Non-cooperative
If a distributed load balancing algorithm is adopted, the
next issue that should be considered is whether the nodes
involved in the task scheduling process are working
cooperatively or independently (non-cooperatively). In
the non-cooperative approach, each node is independent
has autonomy over its own resource scheduling, that is,
decisions are made independently of the rest of the
system and therefore the node may migrate or allocate
tasks based on local performance.
In the cooperative scheme, each load balancing decision
maker (scheduler) has the responsibility to carry out its
own portion of the scheduling task, but all schedulers are
working toward a common system wide goal [5].
c) Adaptive or Non-Adaptive
Dynamic load balancing is said to be adaptive if
scheduled decisions take into consideration past and
current system performance and are affected by previous
decisions or changes in the environment.
It is said to be non-adaptive if parameters used in load
balancing remain the same regardless of system's past
behavior.
d) Sender/Receiver/Symmetrical Initiated
Distribution of task in dynamic load balancing can be
sender initiated, receiver initiated or symmetrical
initiated. In sender initiated strategy, Congested nodes
attempt to move work towards under-loaded nodes. In
receiver-initiated strategy, under-loaded nodes request
tasks to be sent to them from nodes with higher loads. In
the symmetric approach, both the under-loaded as well
as the overloaded nodes may initiate load transfers.
www.ijsret.org
744
Volume 4, Issue 7, July 2015
Algorithms of Dynamic Load Balancing
There are three basic dynamic load balancing algorithms
which are as following:
2.2.2.1 Central Queue Algorithm
Central Queue Algorithm [9] stores new activities and
unfulfilled requests in a cyclic FIFO queue on the main
host. Each new activity arriving at the queue manager is
inserted into the queue. Then, whenever a request for an
activity is received by the queue manager, it removes the
first activity from the queue and sends it to the requester.
If there are no ready activities in the queue, the request is
buffered, until a new activity is available.
When a processor load falls under the threshold, the
local load manager sends a request for a new activity to
the central load manager. The central load manager
answers the request immediately if a ready activity is
found in the process-request queue, or queues the
request until a new activity arrives. This is a centralized
initiated algorithm and need high communication among
nodes.
2.2.2.2 Local Queue Algorithm
This algorithm supports inter process migration. The
main concept in local queue algorithm is static allocation
of all new process with process migration initiated by
the host when its load falls under the predefined
minimum number of ready processes (threshold limit).
Initially, new processes created on the main host are
allocated on all under loaded hosts. From then on, all the
processes created on the main host and all other hosts are
allocated locally [9, 2].
When the local host gets under load it request for the
activities from the remote hosts. The remote hosts than
look up its local list for ready activities and compares the
local number of ready activities with the received
number. If the former is greater than the latter, then
some of the activities are passed on to the requestor host
and get the acknowledgement from the host.
This is a distributed co-operative algorithms and requires
inter-process communication but lesser as compared to
central queue algorithm.
2.2.2.3 Least Connection Algorithm
This algorithm decides the load distribution on the basis
of connections present on a node [13]. The load balancer
keeps track of the numbers of connections attached to
each node and selects the node with least number of
connections for load transfer. The number increases
when a new connection is established and decreases
when connection finishes or time out.
Least Connections methods work best in environments
where the nodes have similar capabilities. Load
imbalance may be caused when the tasks are all of
 International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
745
Volume 4, Issue 7, July 2015

different durations because Connection counting just
doesnt account for that scenario very well.
a) Nature: It tells whether algorithm is static or
dynamic.
b) Overhead: It is amount of overheads like inter-
process communication, migration of tasks etc.
involved while implementing the algorithm and
should be minimum.
c) Resource Utilization: It tells whether the
algorithm is able to utilize all the resources
optimally or not means less idle processors.
d) Process Migration: It tells when a system will
migrate its process. The algorithm is capable of
deciding when it should make changes of load
distribution during execution of process or not.
e) Fault Tolerant: It tells whether the algorithm can
work continuously in event of failure or not,
performance of algorithm is decreasing or not.
f) Response Time: It is time a distributed system
using a particular load balancing algorithm is
taking to respond and must be less.
g) Waiting Time: It is the time period spent waiting
in the ready queue and should be less.
h) Centralized or decentralized: It tells whether
algorithm is centralized or decentralized.
i) Cooperative: It defines the extent of
independence that each processor has in
concluding that how should it can use its own
resources.
j) Adaptability: It tells if algorithm can adapt to
changing situations.
2.3 Comparison on different Algorithms
The comparison of above given algorithms of static and
dynamic load balancing considering the following
parameters is as follows:

Table1: Comparative analysis of load balancing algorithm. D- Decentralized, C- centralized

Algorithms Roun Randomize Central Threshol Central Local Least

d d Manage d Queue Queue Connecti
Parameters Robi r on
n
Static static Static dynamic dynamic dynamic Dynamic
Nature
Low low Low high high high High
Overhead
Less less Less less less more More
Resource Utilization
No no No no no yes No
Process Migration
No no Yes no yes yes No
Fault Tolerant
less less Least less more more Less
Response Time
more more More more less less Less
Waiting Time
D D D D C C D
Centralized/Decentrali
zed
less less Less less more more More
Adaptability

3. Summary of Load Balancing Algorithms well as some other frequently used algorithms of both
There are many other dynamic and static algorithms. approaches.
Here we have summarized all the above algorithms as

www.ijsret.org
 International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
746
Volume 4, Issue 7, July 2015

Table 2: Summary of Load Balancing Algorithms

Approaches Algorithms Key features Advantages Disadvantages

Assigns task No inter-process Not useful if tasks
sequentially. communication have varying
Round Robin Chooses Simple and easy to execution time.
computing implement. Not useful if nodes
nodes serially. Minimum system have different
overhead. capacities.
Weighted Round Tasks assigned Improvement over Not useful if tasks
Robin according to round robin. have different
their weights. Useful for nodes execution time.
Static of different
capacities.

Nodes selected No inter-process Maximum

Randomized randomly. communication. response time.
Useful for special Uneven load
purpose distribution.
applications.
Master node Useful when High inter-process
Central Manager selects slave dynamic tasks are communication.
nodes. created. Bottleneck state
Least load node arises.
is selected.
If load > Minimize remote Increased
Threshold threshold limit process execution time.
task is assigned. allocations. Disturbed load
Minimize remote balancing.
memory accesses.
Stores new task Useful for Single point of
in cyclic FIFO heterogeneous failure.
Central Queue queue. nodes. No inter process
Has main host. Tasks assigned task migration.
Receiver after execution.
initiated
balancing.
Inter-process Less inter-process
Local Queue task migration if communication.
load < threshold
limit.
Load migrated Useful for nodes Not useful for
Dynamic Least Connection to node with of similar tasks of different
least capabilities. durations.
connections.
Task migrated to Less
Nearest neighbor immediate communication
neighbor nodes. delay.
Symmetric Decentralized

www.ijsret.org
 International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
747
Volume 4, Issue 7, July 2015

initiated load nature.

balancing.
Task migrated to Simple and easy to Communication
randomly implement. overheads.
selected node if Poor utilization of
load > threshold. system resources.
Random Load
information not
exchanged.
Nodes have a Minimized Algorithm is not
load vector. communication scalable.
Load vector overhead.
Probabilistic shared
periodically
with random
nodes.
Load Simple More
information implementation communication
exchanged. overhead.
Sender initiated
load balance.
Threshold Task is accepted
if node has load
> threshold.
Master node All resources are Single point of
Centralized keeps load utilized. failure.
Information and information. Efficient Not scalable.
Centralized Maser node performance
Decision takes load results.
balancing
decisions.
Master node Less inclusion of Not always up-to-
Centralized keeps load message date.
Information and information. information. Greater overhead
Distributed Load balancing Robust nature. occurs.
Decision decision is
distributed.
Each node takes No single point of More
Distributed own load failure. communication
Information and balancing Efficient overhead
Distributed decision. performance. sometime.
Decision Load
information is
distributed.

4. CONCLUSION emerged and load balancing is solution to such issues.

In this paper we have taken review of concepts about
load balancing and various load balancing strategies.
Computing resources are rapidly developing, and
growing uses of heterogeneous system with parallel or
distributed computing issue of load imbalance has
Here we presented that load balancing distributes the
load evenly among nodes, hence increasing overall
performance. Then we studied various policies which
should be considered while designing the load balancing
algorithm like information policy etc. Finally we

www.ijsret.org
 International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
discussed Static and Dynamic load balancing strategies
with their respective algorithms and concluded that each
strategy has their own pros and cons and there exists no
absolutely perfect balancing algorithm but one can use
depending on the need.
REFERENCES
[1] Mohsen and Hossein Delda, Balancing Load in a
Computational Grid Applying Adaptive, Intelligent
Colonies of Ants, Informatica 32 (2008) 327335.
[2] Sandeep Sharma, Sarabjit Singh, and Meenakshi
Sharma, Performance Analysis of Load Balancing
Algorithms, World Academy of Science, Engineering
and Technology, 2008.
[3]http://en.wikipedia.org/wiki/Load_balancing_%28co
mputing%29.
[4]https://devcentral.f5.com/articles/intro-to-load-
balancing-for-developers-ndash-the-algorithms.
[5] S. F. El-Zoghdy and S. Ghoniemy, A Survey of
Load Balancing In High-Performance Distributed
Computing Systems, International Journal of Advanced
Computing Research, Volume 1, 2014.
[6] Mr. Amitkumar S Manekar, Mr. Mukesh D
Poundekar, Prof. Hitesh Gupta and Prof. Malti Nagle,
A Pragmatic Study and Analysis of Load Balancing
Techniques In Parallel Computing, International
Journal of Engineering Research and Application, Vol.
2, Issue4, July-August 2012, pp.1914-1918.
[7] Daniel Grousa and Anthony T., Non-Cooperative
load balancing in distributed systems, Journal of
Parallel and Distributing Computing, 2005.
www.ijsret.org
748
Volume 4, Issue 7, July 2015
[8] P. A. Tijare and Dr. P. R. Deshmukh, Schemes for
Dynamic Load Balancing A Review, International
Journal of Advanced Research in Computer Science and
Software Engineering 3(6), June - 2013, pp. 688-692
[9] William Leinberger, George Karypis and Vipin
Kumar, "Load Balancing Across Near-Homogeneous
Multi-Resource Servers", 0-7695-0556- 2/00, 2000
IEEE.
[10] Mr.Gaurav Sharma and Jagjit Kaur Bhatia, A
review on different approaches for load balancing in
computational grid, Journal of Global Research in
Computer Science, Volume 4, No. 4, April 2013.
[11] Sherihan Abu Elenin, and Masato Kitakami,
Performance Analysis of Static Load Balancing in Grid,
International Journal of Electrical & Computer Sciences,
IJECS-IJENS, Vol: 11, No: 03,june 2011.
[12]http://www.loadbalancerblog.com/blog/2013/06/loa
d-balancing-scheduling-methods-explained.
[13] Suriya and Prashanth, Review of load balancing in
cloud computing, IJCS, vol.10 issue.1, Jan 2013.
[14] Javed hussain and Durgesh kumar mishra,
Analysis and investigation of nearest neighbor
algorithm for load balancing, International Journal of
Emerging Trends & Technology in Computer Science,
Volume 2, Issue 6, November December 2013.
[15] Sherihan Abu Elenin and Masato Kitakami,
Performance Analysis of Static Load Balancing in
Grid, International Journal of Electrical & Computer
Sciences, Vol: 11, No: 03, June 2011.