Adaptive Link Recovery Scheme for Structured Peer-to-

peer Networks under Churn

{Seokhyun Kim,
Geunyoung Park}
Seoul National University
{shkim, gypark}@os.snu.ac.kr

Jiman Hong
*

Sungsil University
jiman@ssu.ac.kr


Yookun Cho
Seoul National University
ykcho@snu.ac.kr


ABSTRACT
The problem of churn in structured peer-to-peer (P2P) networks is
one of important research issues in this area. When churn,
continuous coming and going of participating nodes, is arisen,
search performance of structured P2Ps is degraded. Under high
churn rate, network topology is congested, so link recovery is
needed to maintain proper search performance. Generally periodic
link recovery is used to recover broken links by churn. In periodic
maintenance, link recovery period is important parameter. If it is
too high or low compared with churn rate, it is difficult to
guarantee consistent QoS. We propose an adaptive link recovery
scheme to assure consistent search performance of structured
P2Ps. In our scheme, churn rate is estimated by gossip protocol,
and then the expected number of failed links, N
f
, is derived from
it. Each node makes N
f
times of attempts to recover invalid links.
Simulation results show that our scheme guarantees consistent
QoS under various situations.
Categories and Subject Descriptors
C.2.4 [Computer-Communication Networks]: Distributed
Systems Distributed applications
General Terms
Algorithms, Management, Measurement, Performance, Reliability.
Keywords
Structured peer-to-peer, churn, link recovery, search performance.
1. INTRODUCTION
Structured peer-to-peer (P2P) networks have received a lot of
attention because of their good properties like efficiency, fault
tolerance and scalability. [1, 2, 3] These systems are also called as
distributed hash tables. (DHTs)
When DHTs are exposed to continuous joining and leaving of
nodes, i.e. node churning, good properties of DHTs cannot be
guaranteed. Useful properties of DHTs are satisfactory under
weakly dynamic environments. [4, 5, 6] In the presence of very
frequent joins and leaves of nodes, the consistency of routing
tables is quickly broken and it can leads to search performance
degradation, misrouting and possible partitioning of the system. [7,
8]
Generally periodic link recovery is used to handle churn. [8, 9] In
periodic maintenance, routing states of nodes are repaired
periodically. The period of link recovery is system parameter. If
link recoveries are performed too frequently compared with
churning rate, message overhead is increased. If the period of link
recovery is too long, the number of wrong links will increase and
search performance will be degraded.


To overcome shortcomings of periodic maintenance, we propose
an adaptive link recovery scheme for structured P2P networks.
Our scheme is composed of three steps. At first, leaving rate of
nodes is estimated with gossip protocol under high churning
environments. Only leaving rate is estimated, because joining rate
has no influence on link failures. With approximated node leaving
rate, the expected number of dead links, num
d
, is derived, and
num
d
trials for link recovery are performed at each recovery
period, i.e. num
d
links are recovered in every nodes at pi where p
is the recovery period and i1.
We use gossip protocol to estimate churn rate. As P2P networks
lack reliable infrastructures, any global and accurate statistic
information cannot be collected in P2P networks. Gossip protocol
is suitable to estimate node leaving rate of P2P networks because
it is simple, distributed and robust against networks dynamics. [10]
The next step is derivation of expected number of dead links from
the estimated node leaving rate. The number of dead links is
dependent on the geometry of underlying DHT. We select DKS
system [11] as underlying DHT because its nodes have regular
degree. Therefore, it is easy to derive the expected number of
dead links for DKS system. Finally, at each recovery period,
nodes make an attempt to recover dead links during the recovery
period, p.
Simulation results show that our methods show similar QoS for
various churn rate and link recovery periods. However, periodic
maintenance shows different QoS for different circumstances.
The rest of this paper is organized as follows. Section 2 provides
related work. In Section 3, gossip protocol that we use is
explained. The method for estimating expected number of failed
links is shown in Section 4. Simulation results are described in
Section 5 and conclusion is shown in Section 6.
2. RELATED WORKS
Ou et al. does various performance test under churn with
Kademlia as underlying DHT and Peer-to-Peer Protocol (P2PP) as
signaling protocol. [8] They use periodic link management
methods to maintain search performance of Kademlia. That
method is consisted of routing table exchange and random link



*
Corresponding Author
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
SAC11, March 21-25, 2011, TaiChung, Taiwan.
Copyright 2011 ACM 978-1-4503-0113-8/11/03$10.00.
update. A node exchange its routing table with other random node,
and by this method, latest information of P2P is spread to many
nodes. When the random link update algorithm is invoked, a node
selects one of its links randomly and sends keepAlive message to
it. The purpose of this is checking the validity of it.
Rhea et al. show that existing DHTs like Chord and Pastry have
the problem of churn by extensive simulation, and propose new
DHT, Bamboo for overcoming this problem. [9] They suggest
reactive and periodic algorithm to deal it and compares
performance difference of these two methods.
Ghodsi et al. [11] propose DHT, named DKS system, and explain
correct-on-change scheme. When a node is joined to or leaved
from DKS, it detects this change and reflects that to its routing
table. They prove that correct-on-change is invoked only when
some event is happened in it, and so it is more efficient than
periodic link renewal algorithms. However this technique is only
applied to DKS systems and other DHTs cannot use this directly.
We use gossip protocol in our proposal. Generally gossip protocol
is proceeded with next two steps in a node n. The first is a node
selection: n chooses another node m in P2P network. There can be
various gossip methods by ways of node selection. After node m
is picked, n and m exchange their information each other by
sending and receiving messages.
Kempe et al. [12] propose spatial gossip. In d-dimensional space
of real numbers, distance between two points, d is defined and
then, it is proved that message can be propagate by speed of
O(log1+d) by using their algorithm. The node for message swap
is selected by distance d in inverse polynomial way. Spatial gossip
has advantages of both uniform random and neighbor selection.
Structured P2P is appropriate to use spatial gossip because of
links built by measure of id-space and spread to wide area.
Jelasity et al [13] propose framework for gossip protocols. It is
hard to sample dynamic unstructured overlay networks. They
suggests solution to this problem and compare differences of
performance.
3. ESTIMATION OF NODE LEAVING
RATE WITH GOSSIP PROTOCOL
3.1 Underlying DHT
The underlying P2P network that is used for our method is DKS
system with k=2 and let it be called DKS
2
. Each node in DKS
2

manages links to other nodes that are apart by 2
i
hops from it. Let
n.lin
i
be the i
th
link of node n, and n.lin
i
.nd be the node directed by
i
th
link of n. Then we can obtain n.lin
i
.nd from i-1
th
link of n as
following, and figure 1 shows this link update process.
n.lin
i
.nd = (n.lin
i-1
.nd).lin
i-1
.nd
Each node in DKS
2
periodically runs link management procedure.
Firstly validity of successor node is checked. If it is unavailable,
backup node is selected from successor list.
1
Successor list is a set
of k consecutive nodes, where k is the system parameter. By using
successor list, DKS
2
is not partitioned with high probability.
Remaining links are updated sequentially, i.e. i
th
link is updated
and then (i+1)
th
link is updated. It can be shown that expected
time to update all nodes is O(log N) when every node has same
update period.

1
Successor list is proposed by Chord. [1]
3.2 Estimation of Node Leaving Rate
Gossip protocol is generally used for aggregation of observed data
in sensor networks, P2P overlays and other distributed systems.
Topologies of structured P2P networks are favorable to use gossip
protocols. For example, when average for certain statistic property
is estimated by gossip protocol, it can be done within of O(log N)
steps where N is the number of entire nodes.

Figure 1. Link update of DKS
2
We estimate node leaving rate,
l
, by gossip protocol. Node
failures are assumed not to be occurred. Every node notifies its
departure to successor node, and the successor writes the time of
node leaving in its log. Local
l
is calculated by this log, and local
values of
l
are mixed and converged by applying gossip protocol.
Node joining rate,
j
, is not our concern because performance
degradation of structured P2P is arisen mainly by node leaves or
failures.
The algorithms of local
l
calculation is shown in table 1, and the
notifyJoin which is used for setting initial
l
of new comers is also
shown in it.
Table 1. notifyLeaveand notifyJ oin
n.notifyLeave()
v = n.predecessor.
l

n.
l
= ( n.
l
+ v ) / 2
n.notifyJ oin()
n.predecessor.
l
= n.
l


Table 2. Push-pull gossip protocol
n.pushGossip()
randomly get m {n.links}
send n.
l
to m
n.pullGossip( m: node )
send n.
l
to m

n.processGossip(from:node)
if n = from then return
n.pullGossip(from)
n.
l
= ( n.
l
+ from.
l
) / 2

We use push-pull mechanism for gossip protocol. Node n pushes
its local
l
, n.
l
, to another node m, and after node m receives the
message n sent, m.
l
is pulled to n. Node n and m get new
l
as
following:
n.
l
= m.
l
= ( n.
l
+ m.
l
) / 2
Above push-pull is repeated in a node n when n.
l
is changed.
When node n invokes n.pushGossip, the node, m, that receives ns
push_gossip message is selected uniform randomly from ns link
table. The node m sends its m.
l
back to n, and node n and m
update their node leaving rate. Table 2 shows pseudo code for
push-pull gossip protocol.
4. ESTIMATION OF EXPECTED NUMBER
OF UNAVAILABLE LINKS
The node leaving rate obtained from the log of a node is local
information. These values are merged by gossip protocol and
gradually converge to average. Let
l
be the average of node
leaving rate. After sufficient communications are done by the
gossip protocol, each node can have the knowledge of average
node leaving rate,
l
.

Figure 2. Number of incoming and outgoing links of a node in
DKS
2

The probability that a link is dead can be derived from
l
. During
t, the number of dead nodes in DKS
2
is N
l
t because N
l
is the
node leaving rate of the entire P2P network. N is the total number
of nodes. We assume that node leaving rate and node joining rate
are same, so entire number of nodes are nearly same over time.
From above, the probability that a node is dead becomes
following:
Pr{a node is unavailable during t}
= number of leaving nodes during t / N
= N
l
t / N =
l
t
Whan a node leaves the DKS
2
, O(log N) links become unavailable,
because each node has O(log N) incoming and outgoing links.
Figure 2 shows this.
During t, the number of increased unavailable links can be
obtained as following:
The increase of unavailable links during t
= (the number of leaving nodes during t) (clog N)
= cN
l
t log N (where c is a const)
If the link update period of the DKS
2
is sufficiently short
compared with node leaving rate, every node has similar number
of links. In this case, unavailable links are distributed uniformly.
From above we can derive the number of unavailable links per
node during t as following:
The number of unavailable links per node during t
= cN
l
t log N / N
= c
l
t log N
Adaptive link recovery algorithm is run at every link update
period, p
lnk
, so the number of recovered links at each link update
time is c
l
p
lnk
log N.
In our adaptive link recovery algorithm, c is set to one because we
assume that link update period is sufficiently shorter than node
leaving rate, so the number of links in each node is ceiling(log N).
Table 3 shows our adaptive link recovery algorithm and simple
periodic link recovery algorithm.
Table 3. Two link recovery algorithms: adaptive and periodic
n.adaptiveRecovery()
n
r
=
l
p
lnk
log N
while n
r
>0 do
n.periodicRecovery()
n
r
= n
r
- 1
end
n.periodicRecovery()
i = rand(1, ceiling(log N))
// rand in [1, ceiling(log N)]
recover i
th
link
5. SIMULATION RESULTS
We make a discrete event simulator with c++. All simulations are
done with 1,000 nodes. Node leaving rate,
l
, is fixed as 0.1 and
various link recovery periods are tested, because important factor
is the ratio of
l
and the period of link recovery.
Two link recovery algorithms are compared here: the proposed
method and periodic link recovery scheme that are mentioned as
'adaptive' and 'periodic' in below figures. Experiments are done
with various link recovery periods.
Test scenario is as following. At first 1,000 nodes are inserted to
DKS
2
and links are built by its protocol, and then node churn is
starting. A node is joined or leaved for every link recovery period.
Probabilities of node join and leave are equal.
After 100,000 unit times of simulation are passed, search
performance is evaluated. 1,000 random keys are searched and
average hops for key search, average number of links and total
communication costs are measured.
5.1 Search Hop-counts
Our proposed adaptive method shows similar search performance
for all link recovery periods. However the performance of
periodic method is proportional to update periods. When link
recover period is long compared with churn rate, sufficient
number of links cannot be recovered in periodic method. In
adaptive method, however, expected number of unavailable links
is estimated and that number of link recovery is done. Therefore,
similar search performance can be maintained with the adaptive
method. Figure 3 shows this result.
5.2 Number of Recovered Links
The number of links recovered by the periodic method is inverse
proportional to link recovery periods. When link recovery period
is too long compared with churn rate, sufficient number of links
cannot be repaired, and search performance is degraded. Figure 4
shows this. The numbers of links recovered by the adaptive
method is consistent regardless of update periods.
5.3 Communication Costs
Communication costs of the periodic and the adaptive methods
are measured by total message size. The cost of the periodic
method is proportional to 1/p, where p is link recovery periods.
Therefore, the cost of the periodic method is largely increased
when link recovery period becomes smaller. (It shows the shape
of 1/x graph) When link recovery periods are in the range from 50
to 100, search hops, the number of recovered links and costs
become similar. When link recovery periods are greater than 100,
total cost of the periodic method is lower than that of adaptive
method. In this range, the periodic method does not recover
sufficient number of links, so search hops is increased and the
number of recovered links is decreased.
While the periodic method shows different QoS with various link
recovery periods, our adaptive method shows similar QoS for all
link update periods. Figure 5 shows this result.

Figure 3. Average search hop-counts


Figure 4. Number of recovered links
6. CONCLUSION
We propose adaptive link recovery scheme for structured P2P
networks. Our scheme estimates the rate of leaving nodes by the
push-pull gossip protocol which described in Section 3.2 and
derives the expected number of invalid links, N
f
, in each node. By
using this information, each node in P2P network can make N
f

times of link recovery trials. Our method shows regular QoS with
varying recovery period's length, i.e. it can find proper number of
link recoveries under various churning environments.

Figure 5. Costs measured by total message size
7. ACKNOWLEDGMENTS
This research was supported by Basic Science Research Program
through the National Research Foundation of Korea(NRF) funded
by the Ministry of Education, Science and Technology (KRF-
2008-314-D00335), by the MKE(The Ministry of Knowledge
Economy), Korea, under the ITRC(Information Technology
Research Center) support program supervised by the
NIPA(National IT Industry Promotion Agency)" (NIPA-2010-
(C1090-1021-0010)), and by the National Research Foundation of
Korea Grant funded by the Korean Government (2010-0028263)
8. REFERENCES
[1] Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., and
Balakrishnan, H. Chord: A scalable peer-to-peer lookup
service for internet applications. SIGCOMM Comput.
Commun. Rev. 31, 4 (Oct. 2001), 149-160.
[2] Ratnasamy, S., Francis, P., Handley, M., Karp, R., and
Shenker, S. 2001. A scalable content-addressable network.
In Proceedings of the 2001 Conference on Applications,
Technologies, Architectures, and Protocols For Computer
Communications (San Diego, California, United States).
SIGCOMM '01. ACM, New York, NY, 161-172.
[3] Rowstron, A., and Druschel, P. Pastry: Scalable,
Decentralized Object Location, and Routing for Large-Scale
Peer-to-Peer Systems. In Proceedings of the IFIP/ACM
International Conference on Distributed Systems Platforms
(MIDDLEWARE '01) (Heidelberg, Germany). Springer,
LNCS 2218., 2001, 329-350.
[4] Locher, T., Schmid, S. and Wattenhofer, R. eQuus: A
Provably Robust and Locality-Aware Peer-to-Peer System.
In Proceedings of the Sixth IEEE International Conference
on Peer-to-Peer Computing (P2P'06), vol., no., Sept. 2006, 3-
11.
[5] Anceaume, E., Ludinard, R., Ravoaja, A. and Brasileiro, F.
PeerCube: A Hypercube-Based P2P Overlay Robust against
Collusion and Churn. In Proceedings of the Second IEEE
International Conference on Self-Adaptive and Self-
Organizing Systems, 2008 (SASO '08) , vol., no., Oct. 2008,
15-24.
[6] Anceaume, E., Ludinard, R. and Sericola, B. Analytic Study
of the Impact of Churn in Cluster-Based Structured P2P
Overlays. In Proceedings of the IEEE International
Conference on Communications (ICC'10), vol., no., May
2010, 1-5.
[7] Li, X., Misra, J., and Plaxton, CG., Active and Concurrent
Topology Maintenance. In Proceedings of the 18th annual
Conrference on Distributed Computing (DISC '04),
(Trippenhuis, Amsterdam, Netherlands). Springer, LNCS
3274., 2004, 320-334.
[8] Rhea, S., Geels, D., Roscoe, T., and Kubiatowicz, J.
Handling churn in a DHT. In Proceedings of the Annual
Conference on USENIX Annual Technical
Conference (Boston, MA, June 27 - July 02, 2004). USENIX
Annual Technical Conference. USENIX Association,
Berkeley, CA, 2004, 10-10.
[9] Ou, Z., Harjula, E., Kassinen, O., and Ylianttila, M.
Performance Evaluation of a Kademlia-based
Communication-oriented P2P System Under Churn,
Computer Networks, Elsevier, Vol 54., 2010, 689-705.
[10] Shah, D. Gossip Algorithms. Found. Trends Netw. 3, 1 (Jan.
2009)
[11] Ghodsi, A., Alima, L. O., and Haridi, S. Low-Bandwidth
Topology Maintenance for Robustness in Structured Overlay
Networks, In Proceedings of the 38th Annual Hawaii
International Conference on System Sciences (HICSS'05),
(Big Island, Hawaii). IEEE Computer Society, Los Alamitos,
CA, USA, 2005, 302a.
[12] Kempe, D., Dobra, A. and Gehrke, J. Gossip-Based
Computation of Aggregate Information, In Proceedings of
Annual IEEE Symposium on Foundations of Computer
Science (FOCS'03). IEEE Computer Society, Los Alamitos,
CA, USA, 2003, 482.
[13] Jelasity, M., Voulgaris, S., Guerraoui, R., Kermarrec, A., and
van Steen, M. Gossip-based peer sampling. ACM Trans.
Comput. Syst. 25, 3 (Aug. 2007), 2007, 8.