International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882

Volume 4, Issue 7, July 2015

Various Schemes of Load Balancing in Distributed Systems- A Review

Monika Kushwaha
Pranveer Singh Institute of Technology
Kanpur, U.P. (208020)
U.P.T.U., Lucknow

Saurabh Gupta
Pranveer Singh Institute of Technology
Kanpur, U.P. (208020)
U.P.T.U., Lucknow

Abstract
Over the past three decade, parallel and distributed
computing witnessed major growth due to the declined
cost of hardware, advancement in communication
technology, explosive growth of internet and need to
solve large-scale problems. These systems often face the
problem of load imbalance, which can degrade the
performance of the system. Load balancing improves the
system performance by dividing the work load
effectively among the participating computers/nodes.
This paper presents various aspects of load balancing
and explains various load balancing approaches and
strategies and their respective algorithms.
Keywords: Load balancing, Thresholding, Migration,
Distributed Systems.

1. INTRODUCTION
Outstanding progress in computer technology has lead to
the rise of the demand of high speed processing and the
need of high scalability, availability and rapid response.
This resulted in use of parallel and distributed computing
systems where more than one processor processes the
job simultaneously. One of the main research issues in
parallel and distributed system is effective strategy to
distribute workload among multiple processors. Load
balancing is used for minimizing the response time,
maximizing the throughput, and to avoid the overload.
Load balancing is to ensure that every processor in the
system does approximately the same amount of work at
any point of time [2].
1.1
Load Balancing
Load balancing is the process of distribution or
redistribution of load among processor therefore
improving the performance of the system. In computing,
load balancing distributes workload across multiple
computing resources, such as computers, a computer
cluster, network links, or central processing units. Load
balancing
aims
to
optimize
resource
use,
maximize throughput, minimize response time, and
avoid overload of any single resource [3].

1.1.1 Need of Load Balancing

A distributed system contains number of processors
working independently with each other and linked by
communication channel. Some are not linked with any
communication channel. Each processor has an initial
load that is the amount of work to be performed, and
each may have a different processing capacity. The
workload has to be evenly distributed among all
processors based on their processing speed so that time
to execute all tasks gets minimized and idle time of each
processor can be reduced. This is why we need load
balancing.
Load imbalance is also a main problem in data parallel
applications and here also it mainly occurs due to the
uneven distribution of data among the various processors
in the system. Without good load distribution strategies
and techniques, we cannot aim to reach good speedup
and good efficiency.
1.1.2 Issues related to Load Balancing
The following issues are analyzed during load balancing:
a) In distributed environment the communication
channels are of finite bandwidth and the
processing units may be physically distant hence
load balancing need to take decision of whether to
allow task migration or not.
b) A computing job may not arbitrarily divisible
leading to certain constraints in dividing tasks.
c) Each job consists of several smaller tasks and each
of those tasks can have different execution times.
d) The load on each processor as well as on the
network can vary from time to time based on the
workload brought about by the users.
e) The processors capacity may be different from
each other in architecture, operation system, CPU
speed, memory size, and available disk space.
Taking above factors into consideration the load
balancing can be generalized into four basic steps:
a) Monitoring processor load and state
b) Exchanging load and state information between
processors
c) Calculating the new work distribution
d) Actual data movement

www.ijsret.org

741

International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
Volume 4, Issue 7, July 2015

1.1.3 Policies of Load Balancing

A good load balancing algorithm is defined by some
basic policies [10] which are as follows:
Information policy: Specifies what workload
information to be collected, updated, When it is to be
collected and from where.
a) Triggering policy: Determines the appropriate
period to start a load balancing operation.
b) Resource type policy: Retrieve the task from most
overloaded resource or not.
c) Location policy: Uses the results of the resource
type policy to find a suitable partner for a server or
receiver
d) Selection policy: Defines the tasks which should be
migrated from busiest resources (source) to most
idle resources (receiver). A selection policy
considers several factors in selecting a task, for e.g.
transfer of small task will take less overhead.

1.1.4

Advantages of Load Balancing

Some major advantages of load balancing [5, 6] are as
follows:
a) It reduces the task waiting time.
b) It minimizes task response time.
c) It maximizes utilization of system resources.
d) It maximizes system throughput.
e) It improves reliability, and stability of the system.
f) It accommodates future modification.
g) Long starvation is avoided for small jobs.
h) In load balancing overall system performance is
enhance by improving the performance of each
node.
Because of the above benefits, the load balancing
strategies become a field of intensive research. As a
result, a large number of scheduling and load balancing
algorithms have been developed in the past several
years.

2. LOAD BALANCING APPROACHES

In load balancing there are two approaches and each
approach have their own algorithms. The two
approaches are as follows:
2.1 Static Load Balancing
In the Static load balancing approach, the load balancing
decisions are made deterministically or probabilistically
at compile time according to the performance of
computing nodes and remain constant during runtime.
Number of tasks in each node is fixed in this approach
[5]. Static load balancing methods are non-preemptive
i.e. once the load is allocated to the node it cannot be
transferred to another node.

Pros
1. It has minimum communication delays.
2. Algorithms are simple and easy to implement.
3. System overhead is minimized.
Cons
1. Task cannot be migrated while execution.
2. Overall performance of system decreased due to
load fluctuations.
3. Less useful when task have different execution
time and nodes are heterogeneous.
2.1.1 Algorithms of Static Load Balancing
Among various proposed algorithms major static load
balancing algorithms are as follows:
2.1.1.1 Round Robin Algorithm
Round Robin Algorithm (RRA) assigns tasks
sequentially and evenly to all the nodes. All tasks are
assigned to computing nodes based on Round Robin
order, meaning that computing nodes choosing is
performed in series and will be back to the first
computing node if the last computing node has been
reached [5]. Each node maintains its load index locally
independent of allocations from remote node.
Pros
1. Inter-process communication is not required.
2. Useful for jobs of equal processing time and
nodes of same capabilities.
Cons
1. Not useful when tasks have unequal processing
time.
2. Not useful when nodes have different capacities.
2.1.1.2 Randomized Algorithm
Randomized Algorithm (RA) uses random numbers in
selecting computing nodes for processing, without
having any information about the current or previous
load on the node. The computing nodes are selected
randomly following random numbers generated based on
a statistic distribution [5, 1].
Pros
1. It works well for particular special purpose
applications.
2. No inter process communication is required.
Cons
1. It is not considered elegant solution.
2. Maximum response time among all algorithms.
2.1.1.3 Central Manager Algorithm
In each step in Central Manager Algorithm (CMA),
central node selects slave node to be assigned a task.

www.ijsret.org

742

International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
Volume 4, Issue 7, July 2015

Slave node having least load is being selected. The

central node maintains the load index of all slave nodes
connected to it. Whenever, load is changed, a message is
send by the slave nodes to the central node [2]. The load
manager makes load balancing decisions based on the
system load information, allowing the best decision
when of the process created [5, 2].
Pros
1. Performs well when dynamic activities are
created by different hosts.
Cons
1. CMA needs high level of inter process
communication.
2. Single point of failure.
2.1.1.4 Threshold Algorithm
In Threshold Algorithm (TA) tasks/processes are
assigned immediately upon creation to the computing
nodes (processors). Computing nodes for new processes
are selected locally without sending remote messages.
Each node keeps a private copy of the systems load
information. The load of a computing node can be
characterized by one of the three levels which are: under
loaded, medium and overloaded. Two threshold
parameters t_under and t_upper can be used to describe
these levels [5, 7]:
1. Under loaded: workload < t_under,
2. Medium : t_under workload t_upper,
3. Overloaded: workload > t_upper.
In the initial stage, all the computing nodes are
considered under loaded. Whenever the load state of a
computing node exceeds the load level threshold, then it
sends messages regarding the new load state to all of the
other computing nodes, regularly updating them so that
as the actual load state of the entire system can be known
to every node.
If the local state is not overloaded then the process is
allocated locally. Otherwise, a remote under loaded
processor is selected, and if no such host exists, the
process is also allocated locally.
Pros
1. It has low inter process communication.
2. Performance is improved because of large number of
local process allocations which decreases the
overhead of remote process allocations and remote
memory accesses.
Cons
Whenever all remote processors are overloaded then all
processes are allocated locally causing two major
problems:-

1. One overloaded processor may have much higher

load than other overloaded processors, causing
significant disturbance in load balancing.
2. Increased execution time of an application.
2.2 Dynamic Load Balancing
There are three general classes of problems where a
static load balancing is either impossible or can lead to
imbalance of load, problems are:
1. The first class consists of problems in which all
the tasks are available at the beginning of the
computation but the amount of time required by
each task is different.
2. The second class consists of problems in which
tasks are available at the beginning but as the
computation progresses, the amount of time
required by each task changes.
3. The third class consists of problems in which
tasks are not available at the beginning but are
generated dynamically.
In static load balancing too much information about task
and system is required before execution which is not
possible every time like in these three classes of
problems. So Dynamic load balancing was developed to
address these constraints.
Dynamic load balancing makes more informative load
balancing decisions during execution by the runtime
state information [1]. In dynamic load balancing
algorithms work load is distributed among the
processors at runtime. These algorithms monitor changes
on the system work load and redistribute the work
accordingly.
Pros
1. Dynamic load balancing works well for
heterogeneous systems.
2. Task can be redistributed to any processor while
run time hence overloading and under loading
problems become minimum.
3. It works well for task having different execution
time.
4. The system need not be aware of run-time
behavior of the applications before execution.
Cons
1. High communication over heads occurs and
becomes more when number of processors
increase.
2. Dynamic load balancing algorithms are complex
hence not very easy to implement.
3. System overhead increases because it is
preemptive.

www.ijsret.org

743

International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
Volume 4, Issue 7, July 2015

2.2.1 Categories of Dynamic Load Balancing

In general, dynamic load-balancing algorithms can be
broadly categorized as follows:

Algorithms of Dynamic Load Balancing

There are three basic dynamic load balancing algorithms
which are as following:

a) Centralized or Decentralized
In centralized approach, only one processor acts as the
main or central controller (Master). Master node process
or holds the collection of tasks to be performed. Master
node has global view on the load information of all
nodes connected to it, and decides how to allocate jobs
to each of the nodes. And rest of the nodes act as slaves
[10]. Tasks are sent to the slave processes for execution.
It has advantage of ease of implementation, but suffers
from single point of failure.
In Decentralized approach, all nodes in the system are
involved in making the load balancing decision. The
state information is spread among all the nodes and
nodes are responsible in managing their own resources
or allocating tasks residing in their queues to other
nodes. It is more scalable and has better fault tolerance.

2.2.2.1 Central Queue Algorithm

Central Queue Algorithm [9] stores new activities and
unfulfilled requests in a cyclic FIFO queue on the main
host. Each new activity arriving at the queue manager is
inserted into the queue. Then, whenever a request for an
activity is received by the queue manager, it removes the
first activity from the queue and sends it to the requester.
If there are no ready activities in the queue, the request is
buffered, until a new activity is available.
When a processor load falls under the threshold, the
local load manager sends a request for a new activity to
the central load manager. The central load manager
answers the request immediately if a ready activity is
found in the process-request queue, or queues the
request until a new activity arrives. This is a centralized
initiated algorithm and need high communication among
nodes.

b) Cooperative or Non-cooperative
If a distributed load balancing algorithm is adopted, the
next issue that should be considered is whether the nodes
involved in the task scheduling process are working
cooperatively or independently (non-cooperatively). In
the non-cooperative approach, each node is independent
has autonomy over its own resource scheduling, that is,
decisions are made independently of the rest of the
system and therefore the node may migrate or allocate
tasks based on local performance.
In the cooperative scheme, each load balancing decision
maker (scheduler) has the responsibility to carry out its
own portion of the scheduling task, but all schedulers are
working toward a common system wide goal [5].
c) Adaptive or Non-Adaptive
Dynamic load balancing is said to be adaptive if
scheduled decisions take into consideration past and
current system performance and are affected by previous
decisions or changes in the environment.
It is said to be non-adaptive if parameters used in load
balancing remain the same regardless of system's past
behavior.
d) Sender/Receiver/Symmetrical Initiated
Distribution of task in dynamic load balancing can be
sender initiated, receiver initiated or symmetrical
initiated. In sender initiated strategy, Congested nodes
attempt to move work towards under-loaded nodes. In
receiver-initiated strategy, under-loaded nodes request
tasks to be sent to them from nodes with higher loads. In
the symmetric approach, both the under-loaded as well
as the overloaded nodes may initiate load transfers.

2.2.2.2 Local Queue Algorithm

This algorithm supports inter process migration. The
main concept in local queue algorithm is static allocation
of all new process with process migration initiated by
the host when its load falls under the predefined
minimum number of ready processes (threshold limit).
Initially, new processes created on the main host are
allocated on all under loaded hosts. From then on, all the
processes created on the main host and all other hosts are
allocated locally [9, 2].
When the local host gets under load it request for the
activities from the remote hosts. The remote hosts than
look up its local list for ready activities and compares the
local number of ready activities with the received
number. If the former is greater than the latter, then
some of the activities are passed on to the requestor host
and get the acknowledgement from the host.
This is a distributed co-operative algorithms and requires
inter-process communication but lesser as compared to
central queue algorithm.
2.2.2.3 Least Connection Algorithm
This algorithm decides the load distribution on the basis
of connections present on a node [13]. The load balancer
keeps track of the numbers of connections attached to
each node and selects the node with least number of
connections for load transfer. The number increases
when a new connection is established and decreases
when connection finishes or time out.
Least Connections methods work best in environments
where the nodes have similar capabilities. Load
imbalance may be caused when the tasks are all of

www.ijsret.org

744

International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
Volume 4, Issue 7, July 2015

different durations because Connection counting just

doesnt account for that scenario very well.
a) Nature: It tells whether algorithm is static or
dynamic.
b) Overhead: It is amount of overheads like interprocess communication, migration of tasks etc.
involved while implementing the algorithm and
should be minimum.
c) Resource Utilization: It tells whether the
algorithm is able to utilize all the resources
optimally or not means less idle processors.
d) Process Migration: It tells when a system will
migrate its process. The algorithm is capable of
deciding when it should make changes of load
distribution during execution of process or not.
e) Fault Tolerant: It tells whether the algorithm can
work continuously in event of failure or not,
performance of algorithm is decreasing or not.
f) Response Time: It is time a distributed system
using a particular load balancing algorithm is
taking to respond and must be less.

g) Waiting Time: It is the time period spent waiting

in the ready queue and should be less.
h) Centralized or decentralized: It tells whether
algorithm is centralized or decentralized.
i) Cooperative: It defines the extent of
independence that each processor has in
concluding that how should it can use its own
resources.
j) Adaptability: It tells if algorithm can adapt to
changing situations.
2.3
Comparison on different Algorithms
The comparison of above given algorithms of static and
dynamic load balancing considering the following
parameters is as follows:

Table1: Comparative analysis of load balancing algorithm. D- Decentralized, C- centralized

Algorithms
Parameters

Roun
d
Robi
n
Static

Randomize
d

Central
Manage
r

Threshol
d

Central
Queue

Local
Queue

Least
Connecti
on

static

Static

dynamic

dynamic

dynamic

Dynamic

Low

low

Low

high

high

high

High

Less

less

Less

less

less

more

More

No

no

No

no

no

yes

No

No

no

Yes

no

yes

yes

No

less

less

Least

less

more

more

Less

more

more

More

more

less

less

Less

less

less

Less

less

more

more

More

Nature
Overhead
Resource Utilization
Process Migration
Fault Tolerant
Response Time
Waiting Time
Centralized/Decentrali
zed
Adaptability

3. Summary of Load Balancing Algorithms

There are many other dynamic and static algorithms.
Here we have summarized all the above algorithms as

well as some other frequently used algorithms of both

approaches.

www.ijsret.org

745

International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
Volume 4, Issue 7, July 2015

Table 2: Summary of Load Balancing Algorithms

Approaches

Algorithms

Key features

Assigns
task
sequentially.
Chooses

computing
nodes serially.

Round Robin

Weighted Round
Robin

Tasks assigned
according
to
their weights.

Nodes selected
randomly.

Static

Randomized

Central Manager

Master
node
selects
slave
nodes.
Least load node
is selected.
If
load
>
threshold limit
task is assigned.

Disadvantages

No inter-process
communication
Simple and easy to

implement.
Minimum system
overhead.
Improvement over
round robin.
Useful for nodes
of
different
capacities.

Not useful if tasks

have
varying
execution time.
Not useful if nodes
have
different
capacities.
Not useful if tasks
have
different
execution time.

No inter-process
communication.
Useful for special
purpose
applications.
Useful
when
dynamic tasks are
created.

Maximum
response time.
Uneven
load
distribution.

Minimize remote
process
allocations.
Minimize remote
memory accesses.

High inter-process
communication.
Bottleneck state
arises.

Increased
execution time.
Disturbed
load
balancing.

Stores new task

in cyclic FIFO
queue.
Has main host.

Receiver
initiated
balancing.
Inter-process

task migration if
load < threshold
limit.

Useful
for
heterogeneous
nodes.

Tasks
assigned
after execution.

Single point of
failure.
No inter process
task migration.

Load migrated
to node with
least
connections.

Useful for nodes

of
similar
capabilities.

Task migrated to
immediate
neighbor nodes.
Symmetric

Less
communication
delay.
Decentralized

Threshold

Central Queue

Local Queue

Dynamic

Advantages

Least Connection

Nearest neighbor

www.ijsret.org

Less inter-process
communication.

Not useful for

tasks of different
durations.

746

International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
Volume 4, Issue 7, July 2015

Random

Probabilistic

Threshold

Centralized
Information and
Centralized
Decision

Centralized
Information and
Distributed
Decision

Distributed
Information and
Distributed
Decision

initiated
load
balancing.
Task migrated to
randomly
selected node if
load > threshold.
Load
information not
exchanged.
Nodes have a
load vector.
Load
vector
shared
periodically
with
random
nodes.
Load
information
exchanged.
Sender initiated
load balance.
Task is accepted
if node has load
> threshold.
Master
node
keeps
load
information.
Maser
node
takes
load
balancing
decisions.
Master
node
keeps
load
information.
Load balancing
decision
is
distributed.
Each node takes
own
load
balancing
decision.
Load
information is
distributed.

4. CONCLUSION
In this paper we have taken review of concepts about
load balancing and various load balancing strategies.
Computing resources are rapidly developing, and
growing uses of heterogeneous system with parallel or
distributed computing issue of load imbalance has

nature.

Simple and easy to

implement.

Communication
overheads.
Poor utilization of
system resources.

Minimized
communication
overhead.

Algorithm is not
scalable.

Simple
implementation

More
communication
overhead.

All resources are

utilized.
Efficient

performance
results.

Single point
failure.
Not scalable.

Less inclusion of
message
information.

Robust nature.

Not always up-todate.

Greater overhead
occurs.

No single point of
failure.
Efficient
performance.

More
communication
overhead
sometime.

of

emerged and load balancing is solution to such issues.

Here we presented that load balancing distributes the
load evenly among nodes, hence increasing overall
performance. Then we studied various policies which
should be considered while designing the load balancing
algorithm like information policy etc. Finally we

www.ijsret.org

747

International Journal of Scientific Research Engineering & Technology (IJSRET), ISSN 2278 0882
Volume 4, Issue 7, July 2015

discussed Static and Dynamic load balancing strategies

with their respective algorithms and concluded that each
strategy has their own pros and cons and there exists no
absolutely perfect balancing algorithm but one can use
depending on the need.

[8] P. A. Tijare and Dr. P. R. Deshmukh, Schemes for

Dynamic Load Balancing A Review, International
Journal of Advanced Research in Computer Science and
Software Engineering 3(6), June - 2013, pp. 688-692

[1] Mohsen and Hossein Delda, Balancing Load in a

Computational Grid Applying Adaptive, Intelligent
Colonies of Ants, Informatica 32 (2008) 327335.

[9] William Leinberger, George Karypis and Vipin

Kumar, "Load Balancing Across Near-Homogeneous
Multi-Resource Servers", 0-7695-0556- 2/00, 2000
IEEE.

[2] Sandeep Sharma, Sarabjit Singh, and Meenakshi

Sharma, Performance Analysis of Load Balancing
Algorithms, World Academy of Science, Engineering
and Technology, 2008.

[10] Mr.Gaurav Sharma and Jagjit Kaur Bhatia, A

review on different approaches for load balancing in
computational grid, Journal of Global Research in
Computer Science, Volume 4, No. 4, April 2013.

[3]http://en.wikipedia.org/wiki/Load_balancing_%28co
mputing%29.

[11] Sherihan Abu Elenin, and Masato Kitakami,

Performance Analysis of Static Load Balancing in Grid,
International Journal of Electrical & Computer Sciences,
IJECS-IJENS, Vol: 11, No: 03,june 2011.

REFERENCES

[4]https://devcentral.f5.com/articles/intro-to-loadbalancing-for-developers-ndash-the-algorithms.
[5] S. F. El-Zoghdy and S. Ghoniemy, A Survey of
Load Balancing In High-Performance Distributed
Computing Systems, International Journal of Advanced
Computing Research, Volume 1, 2014.
[6] Mr. Amitkumar S Manekar, Mr. Mukesh D
Poundekar, Prof. Hitesh Gupta and Prof. Malti Nagle,
A Pragmatic Study and Analysis of Load Balancing
Techniques In Parallel Computing, International
Journal of Engineering Research and Application, Vol.
2, Issue4, July-August 2012, pp.1914-1918.
[7] Daniel Grousa and Anthony T., Non-Cooperative
load balancing in distributed systems, Journal of
Parallel and Distributing Computing, 2005.

[12]http://www.loadbalancerblog.com/blog/2013/06/loa
d-balancing-scheduling-methods-explained.
[13] Suriya and Prashanth, Review of load balancing in
cloud computing, IJCS, vol.10 issue.1, Jan 2013.
[14] Javed hussain and Durgesh kumar mishra,
Analysis and investigation of nearest neighbor
algorithm for load balancing, International Journal of
Emerging Trends & Technology in Computer Science,
Volume 2, Issue 6, November December 2013.
[15] Sherihan Abu Elenin and Masato Kitakami,
Performance Analysis of Static Load Balancing in
Grid, International Journal of Electrical & Computer
Sciences, Vol: 11, No: 03, June 2011.

www.ijsret.org

748