2018 42nd IEEE International Conference on Computer Software & Applications
Predicting the breakability of blocking bug pairs
Hui Ding Wanwangying Ma Lin Chen Yuming Zhou
State Key Laboratory for Novel Software Technology
Nanjing University, China
dinghui85@gmail.com, wwyma@smail.nju.edu.cn, {lchen, zhouyuming, bwxu}@nju.edu.cn
Abstract—Software systems becomes increasingly complex for
the wide use of social-software-development platforms, such as
GitHub. Complicated inter-dependencies within ecosystems
impose new challenges in resolving the blocking bug pairs, in
which the upstream bugs prevent the downstream bugs to be
fixed. Generally, the downstream bugs cannot be fixed until the
upstream bugs are fixed in blocking bug pairs, which keeps the
downstream developers waiting for a long time. However,
previous research found that some blocking pairs could be
"broken" through a workaround, i.e., a temporary solution
proposed by the downstream developers before the upstream
bugs get fixed.
In this paper, we propose an approach to describe and predict
the breakability of the blocking bug pairs. Our goal is to help the
ecosystem developers to predict whether the given blocking bug
pair can be broken. We evaluate our approach on two real world
ecosystems: Mozilla Firefox and Netbeans, and have the
following two main findings. First, the participants within the
blocking bug pair and the vitality of the downstream bug have
much impact on the breakability. Second, we can build
breakability predication models with the accuracy of
approximate 80%. Moreover, for predicting unbreakable pairs
which affect the downstream projects seriously, our approach
achieves a precision over 92%. It can be used to effectively
remind the upstream developers to fix these bugs as quickly as
possible.
Keywords— blocking bugs; prediction; breakability
I. INTRODUCTION
Social coding sites, such as GitHub, help people around the
world to cooperate on software development and maintenance.
On social coding sites, projects and components with complex
dependencies between them form ecosystems. While bringing
convenience from software reuse, the impact of bugs in
upstream projects may reach the downstream projects through
their dependencies. The fix of some downstream bugs depends
on the fix of the bugs in their upstream projects. The two inter-
dependent bugs constitute a blocking bug pair.
Blocking bugs require more time to fix compared with non-
blocking bugs. In [1], they found that, developers have to pay
nearly two to three times longer to fix blocking bugs than to fix
non-blocking ones. Due to the huge impact of blocking bugs,
many researchers work on blocking bug prediction with bug
reports [1–4].
0730-3157/18/$31.00 ©2018 IEEE
DOI 10.1109/COMPSAC.2018.00035
Baowen Xu
However, even if we have identified the blocking pairs, it is
still difficult to fix them in some situations. First, the upstream
bug is of low severity for the upstream project so that the
upstream developers may not pay much attention to it. Then
the downstream projects probably under huge influence have to
wait for the unenergetic upstream. Second, the root cause of a
downstream bug is only a special case for the upstream bug.
The patch of the upstream bug is expected to solve all the
situations, and therefore takes more time to release. Third, the
upstream bug is not a runtime failure but an enhancement
requirement, for example the redesign of ill-formed APIs. In
the social coding process, developers often spend much time in
discussing the enhancement requirements to reach an
agreement. Fourth, the upstream bug is also blocked, resulting
in a complex blocking chain.
As mentioned above, to fix a downstream bug in a blocking
pair may be a long process. In [5], they found that some
downstream bugs were fixed before the dependent upstream
bugs in blocking pairs, which means the blocking pairs could
probably be broken. In their investigation, they found some
patterns how the downstream developers broke the blocking
pairs. However, it is usually difficult to know whether a
blocking pair is breakable or not. Thus an approach on
predicting the breakability of blocking bug pairs is necessary.
However little work has paid attention to this issue. The issue
of breakability prediction is important in two major aspects: on
one hand, downstream developers can choose to fix the
breakable bug without waiting for upstream developers, which
would save much time; on the other hand, upstream developers
should care more about the unbreakable bugs to avoid long
lasting impact on the downstream sides.
In this paper, we investigate the characteristics of blocking
pairs and attempt to build prediction models for breakability.
We conduct this study on two ecosystems: Mozilla Firefox and
Netbeans. We first collect and analyze bug reports of the two
subjects. Then we leverage machine learning algorithms to
build prediction models for breakability.
This paper makes the following contributions. First, we
investigate the effect of bug features on breakability
predication. Second, we propose an approach to predict the
breakability of blocking pairs in two large-scale ecosystems.
Then by applying various parameters, our model identifies
unbreakable blocking pairs for the upstream projects in order to
remind the developers to fix the bugs preferentially.
219
 The rest of this paper is organized as follows. Section II
describes related work. Section III introduces the preliminaries
and motivation. Section IV describes our research methodology
including the data source we used and the research questions.
Section V presents in detail the experimental results. We
propose further discussions on the practices of breaking
blocking pair in Section VI and examine the threats to validity
in Section VII. Finally, Section VIII concludes this paper.
II. RELATED WORK
A. Blocking bug prediction
Garcia and Shihab [1] proposed the first work about
blocking bug prediction. Based on 402962 bug reports from six
open source projects, they found that it took about two to three
times longer to fix blocking bugs than to fix non-blocking bugs.
They then extracted 14 different factors to build decision trees
to predict whether a bug would be a blocking bug. Their
models achieved 9-29% precision, 47-76% recall, and 15-42%
F-measure. They also found that the most important factors to
determine blocking bugs were the comments, comment-size,
number of developers in the CC list and the reporter’s
experience.
Work [2] extended Garcia and Shihab’s work by proposing
a novel method called ELBlocker. Considering the class
imbalance phenomenon in the bug reports, ELBlocker
leveraged ensemble learning techniques which combined
multiple classifiers built on different subsets of the training set.
Their approach significantly improved the F1-score of Garcia
and Shihab’s method, SMOTE, OSS, and Bagging by 14.69%,
23.36%, 30.98%, and 171.65%.
Our work concentrates on a specific feature of the blocking
bug pairs, i.e., the breakability, and uses learning-based
approach to predict whether a blocking bug pair is breakable or
not. If the pair could be broken, then the downstream
developers do not need to wait for an upstream fix. If the pair
could not be broken, the upstream developers are supposed to
put the upstream bug in high priority. Therefore, the results of
the breakability prediction will help the developers make
decisions when making the bug fix schedule, in order to
decrease the impact on the related projects, especially the
downstream ones.
B. Cross-project bugs
Blocking bugs refer to bugs prevent other bugs from being
fixed. Another special type of bug similar to blocking bugs are
cross-project bugs. If a bug in a project affects the projects
depending on it, then the bug is called a cross project bug. As
the development of software ecosystems, the number of cross-
project bugs has been increasing, which attracts the attention of
more and more researchers.
There have been some researchers focusing on the fixing
process of the cross-project bugs. Canfora et al. [6] investigated
Cross-System-Bug-Fixings (CSBFs) in FreeBSD and
OpenBSD kernels. They associated the occurrences of CSBFs
with social characteristics of contributors. By using social
network analysis, they found that cross-project bug fixings
mainly involved contributors with the highest degree,
220
betweenness, and brokerage level, as well as contributors that
changed the source code more than others did.
For the influence of inter-dependency, Bavota et al. [7]
found that the upstream projects strongly impact on
downstream projects when there were general dependencies
between them. Their study showed that a large number of
downstream code had to be modified when the upstream
project changed if the downstream project depended on the
upstream framework or general services.
Ma et al. [5] studied how developers fixed cross-project
correlated bug in the Github scientific Python ecosystem. They
leveraged the manual inspection and an online survey to
investigate how the developers track the root cause of cross-
project bugs and how they deal with cross-project bugs to
eliminate the bad impacts.
Ding et al. [8] extended Ma et al.’s work by focusing on the
workaround, which is commonly used by the downstream
developers when facing cross-project bugs. A workaround is a
temporary solution injected in the downstream code to bypass
an upstream error. Combining the manual inspection and
statistical comparisons, they found the workarounds and the
corresponding upstream fixes were significantly different in
code size and structure. They also identified four types of
common patterns of the downstream workarounds.
Our work focuses on blocking bug pairs of which the fix of
the upstream bug blocks the fix of the downstream one.
Different from the aforementioned studies which studied the
characteristics of the contributors or the practices of the
developers during the fixing process, our study attempt to
predict whether the blocked downstream bug can be fixed
before the upstream one, that is, whether the blocking pair is
breakable.
C. Prediction on other features or types of bugs
In recent years, a large number of researchers have
proposed prediction models to reduce the impact of bugs by
extracting data from bug reports. Some works focused on
certain features during bug fixing process, including the fixing
time [3,9–11] ,bug severity or priority [4,12,13] and the
characteristics of developers [14,15]. Other works concentrated
on some specific types of bugs, such as duplicate bugs [16,17],
re-open bugs [18,19], and performance bugs [20].
Our work puts our eyes on an emerging type of bugüü
blocking bugs, and predicts the breakability of them.
III. PRELIMINARIES & MOTIVATION
In this section, we first introduce the concept of
breakability. Then we provide the motivation to predict the
breakability.
A. Breakability
The concept of blocking bug has been well investigated by
previous works [1,2,6]. If the fix of one bug B1 relies on the fix
of another bug B2, the two bugs are called a blocking bug pair.
 TABLE I. DISTRIBUTION OF BREAKABLE BLOCKING BUG PAIRS
% Third, breakability prediction works in the stage after the
Bug Blocking Breakable Breakable
Ecosystems
reports pairs pairs in
Blocking
Mozilla
24456 11140 3733 33.51%
Firefox
Netbeans 108128 7760 2001 25.79%
The bug B2 refers to the upstream bug which is usually a
service provider, and the bug B1 refers to the downstream bug
which leverages the upstream service. Generally, B1 cannot be
fixed until B2 has been fixed. If B1 can be fixed before B2, we
say the blocking pair consisting of B1 and B2 is breakable. Ma
et al. investigated this phenomenon in [5], and found some
patterns how downstream developers broke the blocking pairs.
The definitions of “upstream” and “downstream” are
restricted to a certain blocking bug pair not the projects the
bugs belong to. Due to the inter-dependencies and interactions
within ecosystems, one project may be both the service
provider and API user for another project. There may be no
explicit “upstream” and “downstream” assignment for two
projects. For a blocking bug pair, the “upstream” and
“downstream” are clear and definite.
Breakable blocking pairs are not rare in ecosystems.
TABLE I. shows that nearly 34% of blocking pairs in Mozilla
Firefox and 26% in Netbeans are broken.
B. Motivation
Our study focuses on how to identify the breakable
blocking pairs. Given a series of blocking pairs, each
containing an upstream blocking bug and a downstream
blocked bug, our approach attempts to identify each pair as
breakable or unbreakable.
A number of research works have been proposed to predict
blocking bugs [1,2]. However, our study on breakability
prediction is significantly different from the blocking bug
prediction for the following reasons. First, breakable attribute
relates to a pair of related bugs while blocking attribute relates
to a single bug. One bug blocked by two upstream bugs is
expected to mark as blocked in blocking prediction, while in
breakability prediction, the blocked bug belongs to two
blocking pairs with two upstream bugs, and our approach is
expected to give the breakable attribute for each pair
respectively. Second, as the prediction objective is different,
the instances used are different. For a blocking pair, both
221
upstream and downstream information are used for prediction.
blocking relation is identified, while blocking bug prediction
works before the blocking relation is known.
The identification of breakable blocking pairs is helpful for
the downstream side to be quickly fixed. The requirements for
bug fixes are different between the upstream bug and the
downstream bug in a blocking pair. The upstream side as a
service provider would like to fix the bug perfectly in
consideration of all the cases, which may take a lot of time.
The downstream bug may be only a special case caused by the
upstream bug, and a quick fix for this case is preferred. For
example in Mozilla Firefox, the upstream bug #462222
blocked the downstream bug #528440. The upstream function
'nsIWindowMediator' returned wrong values at the time point
when a window was closed and not destroyed yet. This caused
"some intermittent failures". The downstream developers
solved this bug by adding special conditional check around the
upstream API in the year 2009, while the upstream bug was
fixed four years later in 2013. Another example is from
Netbeans, where the downstream bug #214254 was blocked by
the upstream bug #208919. In the upstream side, collisions
happened when the two modules defined a same name as a file
and a folder respectively. The comment of the downstream bug
#208919 said:
“OK, so the problem is that folder is defined first and then
another module defines file of the same name. What should
happen in this situation?”
The downstream bug #214254 was caused for the above
reason when the module entry is a file not a folder. This is a
special case of #208919, and the downstream developer fixed
the bug by “reverting the annotation based registration for
Folder”.
An upstream bug may block a number of downstream bugs,
forming many blocking pairs with the same upstream bug. If
most of the blocking pairs are unbreakable, the upstream bug
would cause huge impact on the ecosystem. The upstream bug
#1265429 in Mozilla Firefox blocked 12 downstream bugs and
all of the downstream bugs are unbreakable. The comment of
one downstream bug #1265729 is as follows:
“I have wanted to do this for a long time... it helps us avoid
a host of ugly hacks and makes our codebase make more
sense.”
The prediction on unbreakable blocking pairs helps the
upstream developers to find the bugs that blocked more
downstream bugs. These influential upstream bugs should be
fixed in a high priority.
 TABLE II.
Feature name Abbr.
Product product
Component component
Severity severity
Priority priority
Assignee asto
Bug reporter reporter
Number of the CC list ccnum
Number of comments cmtnum
Bug description size wnum
Hardware Platform rep_platf
Operating System op_sys
IV. RESEARCH METHODOLOGY
In this section, we first introduce the data source, and then
make a general analysis. Finally, we propose the research
questions and methods to investigate the questions.
A. Data Source
The investigated ecosystems are Mozilla Firefox 1 and
Netbeans2. Mozilla Firefox is an Internet browser and Netbeans
is a Java IDE. We collected the data from their Bugzilla
systems. We chose the bug reports that were already fixed, and
therefore the status in the reports are stable. As shown in
TABLE I., we got 24456 bug reports of Mozilla Firefox and
108128 bug reports of Netbeans. Then we analyzed each bug
report and extracted the blocking attribute. Through the bug
correlation information, we identified the blocking pairs. We
got 11140 blocking pairs from Mozilla Firefox and 7760 from
Netbeans. Finally, we marked the blocking pairs as breakable
or not.
The breakable information is not explicitly declared in bug
reports. We decided the breakable attribute by checking and
comparing the upstream and downstream bug activities.
Bugzilla provides detailed bug activities which are described as
lines of change records. Each line is a modification of the bug
report, containing five columns namely, Who, When, What,
Removed and Added. "Who" means the person that
implemented the change; "When" means at what time the
change happened; "What" means the field that was changed;
"Removed" and "Added" are the value change in this field.
To identify breakable pairs automatically, we leveraged the
time series of "fixed" status from the upstream and downstream
bug reports. If at one point, the downstream bug was marked as
"fixed" and the upstream one was not fixed yet, then this
blocking pair is breakable at that point. Fig. 1 shows a bug
activity segment of bug#462222 and bug#528440 in Mozilla
Firefox. The downstream bug#528440 was fixed at time 2009-
11-13, and the upstream bug#462222 was not fixed until 2013-
09-08. Obviously, this blocking pair is breakable.
Note that one blocking pair can be broken more than once
since the bug report might be reopened. In this condition, we
1 the circumstances, the upstream should pay more attention to
https://bugzilla.mozilla.org/
2
https://netbeans.org/bugzilla/
BUG FEATURES AND DESCRIPTIONS
Description
Product the bug located in.
Second-level categories below product.
Influence ot the bug.
The order to fix ranked by developers.
The person who manages and maintains the bug.
The person who reported the bug.
The number of people in the interesting list.
The number of comments of the bug.
The number of words in the bug description.
The hardware environment of the bug.
The operating system the bug observed on.
only consider the last broken point. After processing the bug
activities of blocking pairs, we got 3733 breakable pairs out of
11140 in Mozilla Firefox (33.5%) and 2001 breakable pairs out
of 7760 in Netbeans (25.8%).
Fig. 1. Bug activity segment of bug#462222 and bug#528550 in Firefox
B. Research Questions
The goal of this study is to predict the breakability of
blocking pairs. Based on a large data set collected from two
ecosystems, we attempt to answer the following questions:
RQ1: Which features are the best indicators for the
breakability?
There are various features in blocking pairs. Which features
can we leverage to predict the breakability? Are these features
of the same importance in prediction? We attempt to find the
answer.
RQ2: Can we build effective models to predict whether a
blocking pair can be broken?
The breakability is crucial for both the downstream and
upstream developers. Identifying the breakable blocking pair is
helpful for the downstream developers. For the downstream
side, with this message, they are able to fix the downstream
bugs without waiting for the fixes of the upstream bugs.
Although there are a considerable number of breakable pairs
for downstream developers to accelerate the fixing process, the
unbreakable pairs are still the majority. When the blocking pair
is unbreakable, the downstream can do nothing but wait. Under
the blocking pair.
222
C. Research Methods
1) Bug Features
To predict breakable pairs, we extracted various features
from both upstream and downstream bug reports. Both the
upstream and downstream bug reports are of the same
structures. The data fields are one to one corresponding. A
specified feature is of the same meaning to the bug itself, but
different to the pair. We chose 11 features from upstream and
downstream separately, and total 22 features for the pair. Note
that, for Mozilla Firefox the product feature is all "Firefox",
thus there are 20 valid features for it. We list each feature and
describe each one briefly in TABLE II.
After the features were extracted for upstream and
downstream bugs, we computed the information gain of each
feature with the breakable attribute. The evaluation of
information is as follow:
INFOGAIN(CLASS, ATTRIBUTE) = H(CLASS) - H(CLASS | ATTRIBUTE)
H(Class) is entropy of the class, in this study standing for
breakability. H(Class | Attribute) is the entropy of the class in
condition of the attribute. The change from the class without
any additional information to the class with a given attribute is
the information gain of the attribute on the class. Through the
information gain, we can observe the correlation between the
feature and the breakability. A greater information gain means
the feature is more related to the breakability.
The attributes with a large number of distinct values are
beneficial to information gain, such as the attribute “bug
reporters” which contains many distinct participants’ names.
To further investigate the attributes against the bias of
information gain, we resolved to information gain ratio. The
information gain ratio is calculated as follows:
GAINR(CLASS, ATTRIBUTE) = INFOGAIN(CLASS, ATTRIBUTE)/H(ATTRIBUTE)
H(Attribute) denotes entropy of the attribute. The
information gain ratio is the division of information gain and
entropy of the attribute. When there are a large number of
distinct values in the attribute, information gain ratio is
decreased by H(Attribute). With information gain ratio, we can
avoid the attribute to be over-fitted.
2) Prediction models for breakable pairs
We used Zero-R as our baseline model. We chose four
well-known machine learning classifiers for comparison,
namely: Naive Bayes, Bayes Networks, K-Nearest Neighbors
and Random Forests. Zero-R is a very simple classifier with
the prediction output only the majority class in the training set.
The Naive Bayes Classifier and the Bayes Networks Classifier
are both on the basis of conditional probability. The difference
is: Naive Bayes works under the assumption that features are
probability independent, while Bayes Networks does not
require this assumption. The K-Nearest Neighbor Classifier
calculates the distance from the unclassified instance to all the
instances in training set, and then consider k nearest neighbor
with the unclassified instance to decide the class. In this study,
we selected k=5 as the parameter. The Random Forests
223
Classifier is an ensemble learning approach. This approach
predicts with the majority vote of a series of decision trees.
To evaluate the efficiency of our prediction models, we
leveraged a confusion matrix. The confusion matrix records the
correct and incorrect results by a classifier. Each row
represents the instances predicted by the classifier (P and N)
and each column represents the actual class (T and F) they
belong to. For instance, a true positive (TP) represents a
blocking pair is breakable and is classified as breakable; a false
positive (FP) represents a blocking pair is unbreakable but is
classified as breakable; a true negative (TN) represents a
blocking pair is breakable but is classified as unbreakable; a
false negative (FN) represents a blocking pair is unbreakable
and is classified as unbreakable. The above four outcomes are
summarized in TABLE III.
TABLE III. THE CONFUSION MATRIX OF BREAKABLE PREDICTION
Actual class
breakable unbreakable
Predicated breakable TP FP
class unbreakable FN TN
With the values in the confusion matrix, we are able to
calculate the evaluation metrics for our prediction models. The
metrics we used are as follows:
1. Precision: The ratio of correctly classified breakable
pairs over all the pairs classified as breakable, and is calculated
as Pre = TP / (TP + FP).
2. Recall: The ratio of correctly classified breakable pairs
over all the actually breakable pairs, and is calculated as Rec =
TP / (TP+FN).
3. F-measure: Consider both the precision and recall, F-
measure is calculated as F-measure = 2*Pre*Rec / (Pre+Rec).
4. Roc Area: The ROC curve is created by plotting the
recall against the precision at various threshold levels. The area
under the ROC curve is between 0 and 1, and calculated by
integration.
5. Accuracy: The ratio between the number of correctly
classified pairs over total number of pairs, calculated by Acc =
(TP+TN) / (TP+FP+TN+FN).
An ideal prediction model is expected to achieve 100%
precision which means every pair classified as breakable is
actually breakable, and 100% recall which means every actual
breakable pair is classified as breakable. In this condition, the
F-measure, Roc Area, and Accuracy are all of the value 1. In
practice, it is difficult to build an ideal model and we attempt to
find an applicable one.
For prediction models, there is a trade-off between the
precision and recall. Since the unbreakable pairs are the
majority in the data set, there is room to increase the prediction
precision with the recall reduction in the acceptable range.
To increase the prediction precision on the unbreakable
pairs, we proposed a classifier based on Bayes Networks. This
classifier is able to classify with an assignable threshold from 0 TABLE IV. INFORMATION GAIN OF BUG FEATURES
to 1. The threshold is used to process the probability Mozilla Firefox Netbeans
distribution generated by the Bayes Network classifier. When Feature Info Gain Feature Info Gain
the threshold is 0.5, our classifier is the same as the Bayes reporter1 0.2913 reporter2 0.1863
Network classifier; when the threshold decreases from 0.5 to 0, asto2 0.19177 reporter1 0.16236
the classifier becomes stricter and eliminates more boundary reporter2 0.18799 asto2 0.10634
asto1 0.18024 component2 0.10213
instances to increase the precision; when the threshold cmtnum2 0.13666 asto1 0.08739
increases from 0.5 to 1, the classifier becomes looser and component1 0.08368 component1 0.0873
accept more boundary instances to increase the recall. For the component2 0.08113 cmtnum2 0.04586
upstream developers, we applied a lower threshold to improve ccnum2 0.06298 product2 0.04293
the precision. wnum1 0.03297 product1 0.03811
wnum2 0.02533 wnum2 0.02713
Since the recall will be reduced with the increase of op_sys1 0.02052 ccnum2 0.01698
precision, we cannot simply set the threshold to a very low severity1 0.01989 cmtnum1 0.01413
cmtnum1 0.01735 ccnum1 0.01078
value. We investigated the changing trends of the precision and ccnum1 0.0124 wnum1 0.01058
the recall with various thresholds. op_sys2 0.0062 severity1 0.00771
priority1 0.00469 op_sys2 0.00353
rep_platf1 0.00449 op_sys1 0.0032
priority2 0.0029 rep_platf2 0.00303
V. RESEARCH RESULTS rep_platf2 0.00194 priority1 0.00237
severity2 0.00107 priority2 0.0019
rep_platf1 0.00155
A. RQ1. Which features are the best indicators for the severity2 0.00132
breakability?
We performed information gain and information gain ratio
analysis on the collected data set. The results are shown in TABLE V. INFORMATION GAIN RATIO OF BUG FEATURES
TABLE IV and TABLE V. We use the suffix "1" and "2"
behind the feature names to distinguish between the features of Mozilla Firefox Netbeans
the upstream bugs ("1") and the downstream bugs ("2"). For Feature Gain Ratio Feature Gain Ratio
cmtnum2 0.04512 reporter2 0.02403
instance, "reporter1" denotes the reporter of the upstream bug reporter1 0.03784 reporter1 0.02156
and "component2" denotes the component where the asto2 0.0294 cmtnum2 0.02031
downstream bug happened. The second column of each subject severity1 0.02837 wnum2 0.01864
presents the information gain or information gain ratio value of ccnum2 0.02765 asto2 0.01675
the feature in first column. The features are ranked in asto1 0.02612 component2 0.01653
reporter2 0.02492 asto1 0.01346
descending order. The features on the top of the list are of
component1 0.01762 component1 0.01328
higher correlation with the breakability than the ones on the component2 0.01716 ccnum2 0.01231
bottom. wnum1 0.01677 product2 0.00992
cmtnum1 0.0137 severity1 0.00958
For the two subjects in our study, the rankings of the wnum2 0.01276 product1 0.00916
features are different due to the unique characteristics of each op_sys1 0.01125 cmtnum1 0.00817
ecosystem. However, from the ranking lists of the two software ccnum1 0.0089 ccnum1 0.00702
ecosystems, we can get some common conclusions. op_sys2 0.00366 wnum1 0.00694
priority1 0.00316 rep_platf2 0.00298
First, the participant-related attributes have much impact on rep_platf1 0.00281 op_sys2 0.00243
breakability. The reporters and assignees rank among the top priority2 0.00192 op_sys1 0.0022
severity2 0.00134 severity2 0.00168
30% in both lists, which shows that the features related with
rep_platf2 0.00125 priority1 0.00148
the developers play an important role in predicting the rep_platf1 0.00146
breakability. This is consistent with the results of Ma et al.’s priority2 0.00114
study [5]. The attribute of participants’ names contains many
distinct string values, which contributes to information gain information gain ratio analysis shows the number of
values. The four participant-related attributes are all ranked in downstream comments is an important factor of breakability.
top-5 in information gain analysis for each ecosystem. In gain
ratio analysis, the rankings of these attributes go down for the Third, different from our intuition, the severity and priority
distinct attribute values yet still in the top-7. Even though the of the downstream bug are of low importance for the
amount of distinct values, participant-related attributes are still breakability of blocking pairs in both subjects. These metrics
useful for breakability prediction. may contain unreliable data. We still need further research
works to investigate the reasons. Besides, operating system and
Second, the features indicating the vitality of the hardware platform are of small influence in Netbeans probably
downstream bugs are more influential than those of the because of the excellent multi-platform capabilities of Java.
upstream ones. The number of comments (cmtnum) and the
number of CC (ccnum) are two main indicators of the bug
vitality. In both subjects, the two indicators of the downstream
vitality rank higher than those of the upstream vitality. The

224
 TABLE VI. PERFORMANCE OF BREAKABILITY PREDICTION MODELS
Software Model Precision Recall F-measure Roc Area Accuracy
Zero-R 0.000 0.000 0.000 0.500 66.49 %
NaiveBayes 0.636 0.628 0.632 0.810 75.48 %
Mozilla Firefox BayesNet 0.629 0.729 0.676 0.831 76.54 %
kNNs 0.688 0.618 0.651 0.740 77.81 %
Random Forests 0.785 0.591 0.675 0.885 80.87 %
Zero-R 0.000 0.000 0.000 0.500 74.21 %
NaiveBayes 0.499 0.521 0.510 0.753 74.18 %
Netbeans BayesNet 0.488 0.583 0.531 0.764 73.45 %
kNNs 0.588 0.535 0.560 0.709 78.32 %
Random Forests 0.717 0.367 0.486 0.882 79.95 %

B. RQ2. Can we build effective models to predict if a blocking
pair can be broken?
To validate our models and to reduce the bias caused by
training set selection, we performed 10-fold cross-validation 10
times and use the average performance. For each validation,
our data set is divided into 10 parts, and each part conforms to
the distribution of the original data set. For each fold in ten,
one part is selected as the testing set and the other parts are
used as a training set. The model built on the training set is
validated on the testing set, and the evaluation metrics are
recorded. The average performance of each model is shown in
TABLE VI.
The Zero-R algorithm is the baseline model in our study. It
simply predicts all the instances as the majority class. The
proportion of breakable pairs is less than 50% in each subject,
and therefore Zero-R marks all the instances as unbreakable.
Thus, the precise, recall, and F-measure are all zeros.
The Naive Bayes model and the Bayes Network model are
based on the similar mechanism. The performance of the two
models are similar in precision, Roc Area, and accuracy. The
recall of the Bayes Network model is higher than that of all the
other models in this study including the Naïve Bayes. The
precision of the two Bayesian models is lower than that of
kNNs and Random Forests. In general, the two Bayesian
models perform well for the recall but poorly for the precision
when predicting breakable pairs.
For our data set, the kNNs model is balanced between the
precision and the recall. As for the precision, the kNNs model
outperforms the two Bayesian models but underperforms
Random Forests. Considering the recall, the kNNs model
performs better than Random Forests but worse than Bayes
Networks. No matter concerning the precision or the recall, the
kNNs model is not a bad choice.
Among the four classifiers we chose, the Random Forests
model performs best when considering the precision, Roc area,
and accuracy, but worst when considering the recall. Regarding
the precision and accuracy, the Random Forests model is no
doubt the best choice. For downstream developers, the
performance on precision is more important than recall for time
saving.
Generally, there is a trade-off between precision and recall.
For different purposes, the precise and the recall are of
different importance. If the downstream developers would like
to avoid spending time on the unbreakable bugs before the
upstream bugs is fixed, then a model of high precision is
preferred. In our study, the Random Forests model is the most
satisfied one in this case. The model of high recall is preferred
when the downstream developers would like to break the

Fig. 2. The performance of unbreakable prediction

225
 TABLE VII. PERFORMANCE OF UNBREAKABLE PREDICTION
Software Mozilla Firefox
Threshold Precision
0.50 (default) 0.852
0.40 0.860
0.30 0.869
0.20 0.878
0.10 0.890
0.05 0.903
0.01 0.927
blocking pairs as much as possible. The Bayes Network model
is best fit for this request in our study.
For the unbreakable prediction, we selected seven levels of
thresholds for our classifier to compare the performance
variation. The results are shown in TABLE VII. With the
decrease of the threshold, the precision improves gradually.
With the threshold value of 0.01, the precision is over 92% in
both subjects. This demonstrates our model provides a very
accurate suggestion of which blocking pairs are unbreakable.
With our model, the upstream developers are able to find the
bugs with huge impact on the downstream efficiently.
The two subjects differ in recall for the different proportion
of unbreakable pairs. Fig. 2 shows the trends of the precision
and the recall on decreasing thresholds. The increase of
precision reduces the recall at a larger rate in Netbeans than in
Mozilla Firefox. For different ecosystems, developers are
supposed to use different thresholds to balance the precision
and recall to better leverage our model.
Fig. 3. The downstream workaround for bug#528440 in Mozilla Firefox
VI. DISCUSSION
In this section, we discuss some findings about how
developers break the blocking bug pairs.
This study focused on the prediction of breakability, and
gave the answer to the question "whether a blocking pair is
breakable". Furthermore, we are also interested in how to break
the blocking pairs. To investigate these questions, three authors
manually inspected the bug reports and the code of fixes for the
broken blocking pairs.
Netbeans
Recall Precision Recall
0.784 0.845 0.787
0.760 0.852 0.759
0.738 0.858 0.730
0.707 0.867 0.692
0.658 0.880 0.626
0.612 0.891 0.555
0.506 0.924 0.410
The amount of broken blocking pairs is very large for
manually checking. There are 5734 broken pairs in our data set
which is difficult for the authors to check them all. For a
qualitative analysis, we chose 5% from the whole broken pairs
by random sampling. For some broken pairs, the code of fixes
could not be obtained due to the attachment invalidation or no
reference to the source code in bug reports. There broken pairs
were invalid for our manually checking and discarded. For the
valid pairs, the three authors first reviewed the bug reports and
code individually according to the same criteria. During this
procedure, they summarized and recorded the patterns of
breakability. And then they came together to discuss the
patterns and draw conclusions. By manually analyzing the bug
reports, we identify three kinds of practices of the developers.
First, if there is a workaround for the downstream bug, the
pair is obviously breakable. A workaround is a temporary patch
by the downstream and was investigated in [5]. Their study
showed the downstream usually offer a workaround to break
the blocking pair. The downstream workaround of bug#528440
in Mozilla Firefox is shown in Fig. 3. The function
“sss_getMostRecentBrowserWindow” should return the
reference of a window which is browsed most recently. The
upstream service “nsIWindowMediator” returned the reference
of a window before its destruction without checking the status
of the window. It crashed when the target window was closed
but not destroyed yet. The downstream developers fixed this
bug by adding special conditional checks around the return
values from the upstream side.
Second, the downstream developers may use the semi-
finished patch from the upstream side. The upstream
developers may raise multiple versions of patches before the
bug is finally fixed. Some early patches are not able to work
perfectly, but are enough for the downstream projects. Usually,
the downstream developers should be familiar with the
upstream code in order to use the upstream semi-finished
patches properly. During our inspection, we found that the
downstream participants often contributed to the upstream
projects as well in this case. For example, the downstream
bug#110302 was blocked by two upstream bugs, namely
bug#112464 and bug#114016, forming two blocking pairs of
the same downstream bug but different upstream bugs. The
three bugs are all about the IDE performance on processing
large files. The downstream bug is that the IDE was
unresponsive when loading large html files. In the upstream,
bug#112464 is about the editor redrawing function and
bug#114016 is about processing long lines in navigator.
Bug#114016 was fixed ahead of the downstream bug. With the
226
patch of bug#114016, the impact on the downstream side was
reduced and the comments from the downstream side is as
follows:
“After Vita's fix of issue #114016 the impression of the file
editing is much better. Now I can scroll like I want without a
high CPU load.”
In this case, the blocking pair breaks for the downstream
benefitting from the fix of another upstream bug. Although the
blocking pair consisting of bug#114016 and bug#110302 was
not broken, it helped the other blocking pair consisting of
bug#112464 and bug#110302 to break.
Third, the downstream developers may rewrite the buggy
upstream function and integrate the redesigned function into
the downstream code. Hence, the dependency is changed. In
our observation, this usually happened when the buggy
upstream function is loosely-coupled with the upstream project.
For example in Netbeans, the upstream bug#95534 blocks two
downstream bugs, namely bug#95531 and bug#95241. The two
downstream bugs were both caused by uncompleted metadata
model support from the upstream side. The downstream
developers released a series of functions that “covered most use
cases” and fixed the two bugs. The comment of bug#95531
said:
“Fixed temporarly.
Covered most use cases when ejb-jar.xml is missu(i)ng.”
And the comment of bug#95241 is:
“Fixed temporarly.
JAX WS service is generated when webservices.xml is
missing.”
Thus, the two blocking pairs were broken by integrating a
part of the upstream functions in themselves. This integrating
process reduced the inter-dependency between the upstream
models and the downstream models, and finally broke the
blocking pairs.
As mentioned above, the practices for breaking blocking
pairs were qualitatively analyzed by manual inspection. Due to
the large size of our data set, we were not able to inspect every
instance. In the future work, we plan to make an in-depth
investigation into this issue by leveraging nature language
processing and interviews/surveys with the developers.
VII. THREATS TO VALIDITY
Threats to internal validity. We identified breakable pairs
by checking the bug activities and comparing the fixing time of
the upstream and downstream bugs. This approach is correct
when the fixes were exactly recorded in bug activities. Besides,
in some pairs, the downstream bug was fixed only a short time
before the upstream one. In this case, we used a threshold for
the shortest time the downstream is earlier than upstream in a
breakable pair. In this study, the threshold is one day, which
means the downstream bug was fixed at least 24 hours earlier
than the upstream one in a breakable pair. We manually
227
checked a part of the instances by random sampling and did not
found the misjudged pairs by our approach.
Threats to external validity. This study investigated
18900 blocking pairs from 132584 bug reports in two
ecosystems. The two subjects differ in size, complexity,
developers, application field, and so on. All the bug data were
generated during the development process. However, our data
set may not be representative of all kinds of ecosystems.
Further study is needed to confirm our findings about the
breakability of blocking pairs for other ecosystems.
VIII. CONCLUSION AND FUTURE WORK
In this study, we studied the characteristics of breakable
blocking pairs, and proposed models to predict the breakability
of blocking pairs in two large-scale software ecosystems,
Mozilla Firefox and Netbeans. By performing information gain
analysis on the bug features, we found the developer related
features are most correlated with the breakability. In our
breakable prediction models, the Random Forests algorithm
achieved the precision of 78.5% and 71.7% for Mozilla Firefox
and Netbeans respectively. Meanwhile, since it is important for
the upstream developers to be reminded of the unbreakable
blocking pairs efficiently, we proposed a classifier with
variable thresholds to identify unbreakable pairs and achieved a
precision over 92%.
This study is a preliminary investigation on the breakability
of blocking pairs. There is still room for improvement in the
performance of our prediction models and we hope that this
work could inspire other researchers to find more efficient
models to predict the breakable pairs. In future work, we plan
to extend our work on more software ecosystems. We also plan
to perform a more in-depth analysis on developers' practices
for breaking blocking pairs and further develop breakable plan
generation tools to automatically suggest how to break a
blocking pair.
REFERENCES
[1] H. Valdivia Garcia, E. Shihab, Characterizing and predicting
blocking bugs in open source projects, Proc. 11th Work. Conf. Min.
Softw. Repos. - MSR 2014. (2014) 72–81.
doi:10.1145/2597073.2597099.
[2] X. Xia, D. Lo, E. Shihab, X. Wang, X. Yang, ELBlocker: Predicting
blocking bugs with ensemble imbalance learning, Inf. Softw.
Technol. 61 (2015) 93–106. doi:10.1016/j.infsof.2014.12.006.
[3] L.D. Panjer, Predicting Eclipse Bug Lifetimes, in: Fourth Int. Work.
Min. Softw. Repos. (MSR’07ICSE Work. 2007), IEEE, 2007: pp.
29–29. doi:10.1109/MSR.2007.25.
[4] A. Lamkanfi, S. Demeyer, Q.D. Soetens, T. Verdonck, Comparing
Mining Algorithms for Predicting the Severity of a Reported Bug,
in: 2011 15th Eur. Conf. Softw. Maint. Reengineering, IEEE, 2011:
pp. 249–258. doi:10.1109/CSMR.2011.31.
[5] W. Ma, L. Chen, X. Zhang, Y. Zhou, B. Xu, How do developers fix
cross-project correlated bugs? A case study on the GitHub scientific
python ecosystem, Proc. - 2017 IEEE/ACM 39th Int. Conf. Softw.
Eng. ICSE 2017. (2017) 381–392. doi:10.1109/ICSE.2017.42.
[6] G. Canfora, L. Cerulo, M. Cimitile, M. Di Penta, Social interactions
around cross-system bug fixings, in: Proceeding 8th Work. Conf.
Min. Softw. Repos. - MSR ’11, ACM Press, New York, New York,
USA, 2011: p. 143. doi:10.1145/1985441.1985463.
[7] G. Bavota, G. Canfora, M. Di Penta, R. Oliveto, S. Panichella, How
the Apache community upgrades dependencies: an evolutionary
 study, Empir. Softw. Eng. 20 (2015) 1275–1317.
doi:10.1007/s10664-014-9325-9.
[8] H. Ding, W. Ma, L. Chen, Y. Zhou, B. Xu, An empirical study on
downstream workarounds for cross-project bugs, 24th Asia-Pacific
Softw. Eng. Conf. (2017).
[9] L. Marks, Y. Zou, A.E. Hassan, Studying the fix-time for bugs in
large open source projects, in: Proc. 7th Int. Conf. Predict. Model.
Softw. Eng. - Promise ’11, ACM Press, New York, New York,
USA, 2011: pp. 1–8. doi:10.1145/2020390.2020401.
[10] C. Weiss, R. Premraj, T. Zimmermann, A. Zeller, How Long Will It
Take to Fix This Bug?, in: Fourth Int. Work. Min. Softw. Repos.
(MSR’07ICSE Work. 2007), IEEE, 2007: pp. 1–1.
doi:10.1109/MSR.2007.13.
[11] E. Giger, M. Pinzger, H. Gall, Predicting the fix time of bugs, in:
Proc. 2nd Int. Work. Recomm. Syst. Softw. Eng. - RSSE ’10, ACM
Press, New York, New York, USA, 2010: pp. 52–56.
doi:10.1145/1808920.1808933.
[12] M. Sharma, P. Bedi, K.K. Chaturvedi, V.B. Singh, Predicting the
priority of a reported bug using machine learning techniques and
cross project validation, in: 2012 12th Int. Conf. Intell. Syst. Des.
Appl., IEEE, 2012: pp. 539–545. doi:10.1109/ISDA.2012.6416595.
[13] A. Lamkanfi, S. Demeyer, E. Giger, B. Goethals, Predicting the
severity of a reported bug, in: 2010 7th IEEE Work. Conf. Min.
Softw. Repos. (MSR 2010), IEEE, 2010: pp. 1–10.
doi:10.1109/MSR.2010.5463284.
[14] D. Di Nucci, F. Palomba, G. De Rosa, G. Bavota, R. Oliveto, A. De
Lucia, A Developer Centered Bug Prediction Model, IEEE Trans.
Softw. Eng. (2017) 1–1. doi:10.1109/TSE.2017.2659747.
228
[15] M. Tufano, G. Bavota, D. Poshyvanyk, M. Di Penta, R. Oliveto, A.
De Lucia, An empirical study on developer-related factors
characterizing fix-inducing commits, J. Softw. Evol. Process. 29
(2017) e1797. doi:10.1002/smr.1797.
[16] X. Wang, L. Zhang, T. Xie, J. Anvik, J. Sun, An approach to
detecting duplicate bug reports using natural language and execution
information, in: Proc. 13th Int. Conf. Softw. Eng. - ICSE ’08, ACM
Press, New York, New York, USA, 2008: p. 461.
doi:10.1145/1368088.1368151.
[17] N. Jalbert, W. Weimer, Automated duplicate detection for bug
tracking systems, in: 2008 IEEE Int. Conf. Dependable Syst.
Networks With FTCS DCC, IEEE, 2008: pp. 52–61.
doi:10.1109/DSN.2008.4630070.
[18] E. Shihab, A. Ihara, Y. Kamei, W.M. Ibrahim, M. Ohira, B. Adams,
A.E. Hassan, K. Matsumoto, Studying re-opened bugs in open
source software, Empir. Softw. Eng. 18 (2013) 1005–1042.
doi:10.1007/s10664-012-9228-6.
[19] T. Zimmermann, N. Nagappan, P.J. Guo, B. Murphy,
Characterizing and predicting which bugs get reopened, in: 2012
34th Int. Conf. Softw. Eng., IEEE, 2012: pp. 1074–1083.
doi:10.1109/ICSE.2012.6227112.
[20] S. Zaman, B. Adams, A.E. Hassan, A qualitative study on
performance bugs, in: 2012 9th IEEE Work. Conf. Min. Softw.
Repos., IEEE, 2012: pp. 199–208. doi:10.1109/MSR.2012.6224281.