Professional Documents
Culture Documents
Chapter 3
Conceptual Framework
Bias is defined as the difference between the expected value of an estimator and the true
value of the parameter being estimated. An estimator or decision rule can be positive,
B ia s (θ$ ) = E [θ$ ] - θ
where θ$ is the estimator of the true value of the parameter θ and θ is the true value of
the parameter.
Accuracy is defined to be the measurement on how close the estimates to the true value.
Precision is defined to be the measurement on how close the estimates with one another.
Nonresponse is the failure to obtain valid response from a unit in the survey.
The types of nonresponse focus on the method in which the observations are nonresponse
values. Kalton (1983) stressed the importance to differentiate the types of nonresponse:
Unit (or Total) nonresponse takes place when no information was collected from a
sampling unit. There are many causes of this nonresponse, namely, the failure to contact
the respondent (not at home, moved or unit not being found), refused to participate,
inability of the unit to cooperate (might be due to an illness or a language barrier) or lost
questionnaires.
Item nonresponse, on the other hand, happens when the information is collected from a
unit is incomplete. There are many causes of item nonresponse, namely, refusal to answer
the question due to the lack of information necessarily needed by the informant, failure to
make the effort required to establish the information by retrieving it from his memory or
by consulting his records, refusal to give answers because the questions might be
rejected at an edit check on the grounds that it is inconsistent with other responses (may
include an inconsistency arising from a coding or punching error occurring in the transfer
Partial nonresponse is the failure to collect large sets of items for a responding unit. A
sampled unit fails to provide responses for the following reasons, namely, in one or more
waves of a panel survey, later phases of a multi-phase data collection procedure (e.g.
second visit of the FIES), and later items in the questionnaire after breaking off a
telephone interview. Other reasons include, data are unavailable after all possible
checking and follow-up, inconsistency of the responses that do not satisfy natural or
reasonable constraints known as edits which one or more items are designated as
Generated by Foxit PDF Creator © Foxit Software
http://www.foxitsoftware.com For evaluation only.
unacceptable and therefore are artificially missing, and similar causes given in Unit
(Total) Nonresponse. In this study, the researchers dealt with Partial Nonresponse
how missing data should be handled. There are three patterns of nonresponse namely
A missing data is said to be MCAR if the probability of having a missing value for Y is
unrelated to the value of Y itself or to any other variable in the data set. Data that are
MCAR reflect the highest degree of randomness and show no underlying reasons for
missing observations that can potentially lead to bias research findings (Musil, et al,
2002). Hence, the missing data is randomly distributed across all cases such that the
occurrence of missing data is independent to other variables in the data set. An example
of the MCAR pattern is when a sample unit in the survey fails to provide an answer to the
Another pattern of nonresponse is the MAR case. The missing data is considered to be
controlling for other variables in the analysis. This means that the likelihood of a case
Generated by Foxit PDF Creator © Foxit Software
http://www.foxitsoftware.com For evaluation only.
data set. An example of the MAR pattern is when a sampling unit fails to provide an
answer to the total monthly expenditure because the sampling unit is a male household.
The missing information about the total monthly expenditure is dependent on the gender
of the sampling unit and not on the total monthly expenditure itself.
Meanwhile, the NIN is regarded as the most problematic nonresponse pattern. When the
probability of missing data on Y is related to the value of Y and possibly to some other
variable Z even if other variables are controlled in the analysis, such case is termed as
NIN. NIN missing data have systematic, nonrandom factors underlying the occurrence of
the missing values that are not apparent or otherwise measured. NIN missing data are the
most problematic because of the effect in terms of generalizing research findings and
may potentially create bias parameter estimates, such as the means, standard deviations,
NIN pattern is when a sampling unit from the higher income groups fails to provide
information even if the gender of the unit is being controlled. Using the example in the
MAR pattern, however, the sampling unit did not also provide answer because he was a
high income earner. This is considered NIN since the sampling unit also depends on the
income group even if the gender of the unit was controlled. (Musil, et al., 2002)
These patterns are considered as an important assumption before any imputation takes
place. For an imputation procedure to work and achieve statistically acceptable and
reliable estimates, the pattern of nonresponse must either satisfy the MCAR or MAR
Generated by Foxit PDF Creator © Foxit Software
http://www.foxitsoftware.com For evaluation only.
assumption. For this study, the researchers created missing observations that satisfy the
MCAR assumption.
3.4 NR Bias
In most surveys, there is a large propensity of the post-analysis results to become invalid
due to the missing data. Missing data can be discarded, ignored or substituted through
nonresponse bias becomes a problem (Kalton, 1983). The effect of deleting the missing
Suppose the population is divided in two groups or strata. The population is divided in
two groups or strata, the first group consisting of all units in the population for which
units will be obtained if the units will be included in the sample (Respondents) and the
second group are those units for which no measurement will be obtained
(Nonrespondents).
To arrive at the proper estimation of the nonresponse bias, the following quantities are
defined:
Let R be the number of respondents and M (M stands for missing) be the number of
with replacement is drawn from each group. Under SRS, the following properties are :
Generated by Foxit PDF Creator © Foxit Software
http://www.foxitsoftware.com For evaluation only.
1 r 1 r
R−r 2
E[ yr ] = E[ ∑ yi ] = E[∑ yi ] = Y r and V ( yr ) = S , where S2 is
n i =1 n i =1 Rr
The corresponding sample quantities for the total number of respondents and
R M
nonrespondents are r and m respectively, with r + m = n. Let R = and M = be the
N N
r
proportions of respondents and nonrespondents in the population and let r = and
n
m
m= be the proportion of respondents and nonrespondents in the sample. The
n
and Y = Yr + Y m = RYr + M Y m , where Yr and Y r are the total and mean for respondents,
respectively and Ym and Y m are the same quantities for the nonrespondents, respectively.
(Kalton, 1983).
in two stages, first conditional on fixed r and then over different values of r, i.e. E [ y r ] =
E1E2 [ y r ] where E2 is the conditional expectation for fixed r and E1 is the expectation
E[ y r ] = E1[
∑ E (y2 ri )
] = E1[Y r ] = Y r
r
Hence, the bias of y r is given by:
B( y r ) = Y r − Y = M (Y r − Y m )
The equation above shows that y r is approximately unbiased for Y if either the
that for respondents, Y r . Since the survey analyst usually has no direct empirical
confidence that the bias is small is when the nonresponse rate is low. However, in
practice, even with moderate M , many survey results escape sizable biases because
In reducing nonresponse bias caused by missing data, there are many procedures that can
be applied and one of these procedures is imputation. In this study, imputation procedures
are applied to compensate for nonresponse and reduce bias to the estimates. Imputation is
Imputation is one of the many procedures that can be used to deal with nonresponse to
generate unbiased results. Imputation is the process of replacing a missing value, through
Generated by Foxit PDF Creator © Foxit Software
http://www.foxitsoftware.com For evaluation only.
Imputation has certain advantages. First, imputation methods help reduce biases in survey
estimates. Second, imputation makes analysis easier and the results are simpler to
present. Imputation does not make use of complex algorithms to estimate the population
parameters in the presence of missing data; hence, much processing time is saved. Lastly,
using imputation techniques can ensure consistency of results across analyses, a feature
On the other hand, imputation has also several disadvantages. There is no guarantee that
the results obtained after applying imputation methods will be less biased than those
based on the incomplete data set. There is a possibility that the biases from the results
using imputation could be greater. Hence, the use of imputation methods depends on the
suitability of the assumptions built into the imputation procedures used. Even if the biases
of univariate statistics are reduced, there is no assurance that the distribution of the data
and the relationships between variables will remain. More importantly, imputation is just
a fabrication of data. Many naive researchers falsely treat the imputed data as a complete
There are four Imputation Methods (IMs) applied in this study, namely, the Overall
imputation classes are needed to be defined in order to proceed in performing the IMs.
Imputation classes are stratification classes that divide the data into groups before
imputation takes place. The formation of imputation classes is very useful if the classes
are divided into homogeneous groups. That is, similar characteristics have some
propensity to provide same response. The variables used to define imputation classes are
observations, a group of observations coming from a variable with a response are used.
These records are called donors. The records with missing observations to be substituted
Problems might arise if the number of imputation classes is not formed with caution to
imputation methods that rely on them. The matching variable must have a definite
number of classes applied to each method. The larger the number of imputation classes,
the possibility of having fewer observations in one class increases. This can cause the
variance of the estimates under that class to increase. On the other hand, the smaller the
number of imputation class, the possibility of having more observations in that class
The mean imputation method is the process by which missing data is imputed by the
mean of the available units of the same imputation class to which it belongs (Cheng and
Generated by Foxit PDF Creator © Foxit Software
http://www.foxitsoftware.com For evaluation only.
Sy, 1999). One of the types of this method is the Overall Mean Imputation (OMI)
method. The OMI is one of the widely used methods in imputing for missing data. The
OMI method simply replaces each missing data by the overall mean of the available
(responding) units in the same population. The overall mean is given by:
∑y ri
y omi = i =1
= yr
r
where yomi is the mean of the entire sample of the responding units of the y-th variable
There are many advantages and disadvantages of this method. The advantage of using
this method is its universality. This means that it can be applied to any data set.
Moreover, this method does not require the use of imputation classes. Without imputation
classes, the method become easier to use and results are generated faster.
However, there are serious disadvantages of this method. Since missing values are
imputed by a single value, the distribution of the data becomes distorted (see Figure 1).
The distribution of the data becomes too peaked making it unsuitable in many post-
analysis. Second, it produces large biases and variances because it does not allow
variability in the imputation of missing values. Many related literatures stated that this
method is the least effective and thus recommended never to use this method.
Generated by Foxit PDF Creator © Foxit Software
http://www.foxitsoftware.com For evaluation only.
One of the most popular and widely known methods used is the Hot Deck Imputation
(HDI) method. The HDI method is the process by which the missing observations are
imputed by choosing a value from the set of available units. This value is either selected
neighbor hot deck). To perform this method, let Y be the variable that contains missing
data and X that has no missing data. In imputing for the missing data:
1. Find a set of categorical X variables that are highly associated with Y. The X
3. If there are cases that are missing within a particular cell in the table, select a case
from the set of available units from Y variable and impute the chosen Y value to
the missing value. In choosing for the imputation to be substituted to the missing
value, both of them must have similar or exactly the same characteristics.
Generated by Foxit PDF Creator © Foxit Software
http://www.foxitsoftware.com For evaluation only.
Cheng and Sy (1999) stated that the HDI method produces more accurate estimates with
the use of imputation classes. If the matching variables are closely associated with the
Example 1: Suppose that a survey is conducted with a sample of ten people. In the
survey, three people refused to provide their Grade Point Average (GPA) for the previous
term. Missing answer from each nonrespondent are replaced by a known value from a
responding unit who has similar characteristics such as sex, degree or course (Course),
Dean Lister (DL), Honor student in High School (HS2), and Hours of study classes
(HSC). Suppose the set of X matching variables that are highly associated to GPA are the
variables DL and HS2. Table 1 shows the data with imputed values. Values in parenthesis
are the imputed values that were randomly chosen in their respective imputation classes.
Like OMI, there are certain advantages in using this method. One major attraction of this
method cited by Kazemi (2005) is that imputed values are all actual values. More
importantly, the shape of the distribution is preserved. Since imputation classes are
On the other hand, it also has a set of disadvantages. First, in order to form imputation
distorted data set increases if the method used in imputing values to the missing
the donor record might be used repeatedly by the missing values causing the shape of the
distribution to get distorted. Third, the number of imputation classes must be limited to
ensure that all missing values will have a donor for each class.
As in MI and HDI methods, this procedure is one of the widely known used imputation
methods. The method of imputing missing values via the least-squares regression is
known to be the Regression Imputation (RI) method. There are many ways of creating
a regression to be used in imputing for the missing observations. The y-variable for
which imputations are needed is regressed on the auxiliary variable (x1, x2, ..., xp) for the
qualitative, the latter being incorporated into the regression model by means of dummy
variables. There are two basic types of the RI method: (a) Deterministic Regression
In comparing for the accuracy and efficiency of the RI method, it will be helpful if the RI
The use of the predicted value from the model given the values of the auxiliary values
that contains no missing data for the record with a missing response in the variable y is
called the Deterministic Regression Imputation (DRI). This method is seen as the
generalization of the mean imputation method. The model for DRI is given by:
p −1
yˆ k = βˆ 0 + ∑ βˆi X ik
i =1
where
Xik is the auxiliary variable that can either be a quantitative variable or a dummy
There are advantages and disadvantages of using DRI. DRI has the potential to produce
closer imputed value for the nonresponse observation. In order to make the method
effective by imputing a predicted value, which is near the actual value, a high R2 is
needed. Though this method has the potential to make closer imputed values, this method
is a time-consuming operation and often times unrealistic to consider its application for
Using the DRI can also underestimate the variance of the estimates. It can also distort the
distribution of the data. One major disadvantage of this method is that it can produce out-
The use of the predicted value from the deterministic regression model has similar
it, an estimated residual is added to the predicted value. The use of this predicted value
plus some type of randomly chosen estimated residual is called the Stochastic
p −1
yˆ k = βˆ 0 + ∑ βˆi X ik + eˆk
i =1
where
$y
k is the predicted value for the k-th nonresponding unit to be imputed
Xik is the auxiliary variable that can either be a quantitative variable or a dummy
There are various ways in which this could be done depending on the assumptions made
1. Assume that the errors are homoscedastic and normally distributed, N (0, σ e ). Then
2
2
σ e2 could be estimated by the residual variance from the regression, s e and the residual
2. Assume that the errors are heteroscedastic and normally distributed, with σ ej2 being
the residual variance in some group j. Estimate the σ ej2 by s ej2 , and choose a residual for a
2
recipient in group j from N (0, s ej ).
3. Assume that the residual all come from the same, unspecified distribution. Then
estimate ŷ k by ŷ k + êk , where ê k is the estimated residual for a random chosen donor.
4. The assumption in (3) accepts the linearity and additivity of the model. If there are
doubts about these assumptions, it may be better to take not a random-chosen donor but
instead one close to the recipient in terms of his x-values. In the limit, if a donor with the
same set of x-values is found, this procedure reduces to assigning that donor’s y-value to
the recipient.
Generated by Foxit PDF Creator © Foxit Software
http://www.foxitsoftware.com For evaluation only.
There are advantages and disadvantages in using SRI. Similar to DRI, this method can
produce imputed values that are near to the nonresponse observation if the model has a
high R2. This method is also a time-consuming operation and often times unrealistic to
consider its application for all the items with missing values in a survey. This method can
also produce out-of-range values other than the predicted value without the added
residual. It is possible under SRI that after adding the residual to the deterministic