Operations Research and Applications : An International Journal (ORAJ), Vol.4, No.

3/4, November 2017

RESIDUALS AND INFLUENCE IN NONLINEAR

REGRESSION FOR REPEATED MEASUREMENT DATA
Munsir Ali, Yu Feng, Ali choo, Zamir Ali
Nanjing University of Science and Technology, P.R. China

ABSTRACT
All observations dont have equal significance in regression analysis. Diagnostics of observations is an
important aspect of model building. In this paper, we use diagnostics method to detect residuals and influential
points in nonlinear regression for repeated measurement data. Cook distance and Gauss newton method have
been proposed to identify the outliers in nonlinear regression analysis and parameter estimation. Most of these
techniques based on graphical representations of residuals, hat matrix and case deletion measures. The results
show us detection of single and multiple outliers cases in repeated measurement data. We use these techniques
to explore performance of residuals and influence in nonlinear regression model.

KEY WORDS:
Hat matrix, Cook distance, Residuals, Nonlinear regression models. Mathematics Subject Classification:
62J20,62J02, 62G05,62J05,62J99.

1. INTRODUCTION
Data containing of repeated measurements hold on each of number of individuals appear frequently
in biomedical and biological implementations. This kind of modeling data generally implies
characterization of the relationship among the measured response of y , measurement factor, or
covariate x [11]. In many implementations, the relationship between y and x is nonlinear in
unknown parameters of attention.

The expression of repeated measurement on an individual requires definite care in marking the
random variation in the data. It is important to recognize random variation among
measurements within a given individual and random variation among the individuals.
Inferential methods assist these different variance components in the framework of a proper
hierarchical statistical model. When the relationship between x and y in the unknown
parameters is linear, the framework is that of the classical linear mixed effects model [10].
In this case, Bayesian inferential method is provided satisfactory hierarchical linear model
[14]. There is a substantial literature about hierarchical linear model, McCulloch, Casella,
and Searle (1992). Linear modeling methods for repeated measurement data are quite
advanced and developed, and well recorded in statistical literature, Crowder and Hand
(1990), Lindsey(1993), and Diggle, Liang and Zenger(1994).
In this particular work, we aim to indicate residuals data points in nonlinear regression for repeated
measurement data and parameter estimation. We use Cook.distacne and Gauss newton method, and
we also explore some useful examples for parameter estimation and Outliers detection. The
organization of this paper is given as; in section 2, we give some models and parameter estimation;
section 3 deals with the diagnostics methods in case of single and multiple Outliers detection by
DOI: 10.5121/oraj.2017.4401 1
 Operations Research and Applications : An International Journal (ORAJ), Vol.4, No.3/4, November 2017
scatterplots and parameter estimation with some applicable examples while section 4 concludes the
paper.

2. THE MODEL AND THE PARAMETER ESTIMATION

We introduce hierarchal nonlinear model that forms the fundamental inferential methods and discuss
the available techniques for the analysis of repeated measurement data. In the linear case, intra and
inter individual variation can assist within the two stages model. The first stage characterizes by a
nonlinear regression model with a model for individual covariance structure, and inters individual
variability represent in the second stage through individual specific regression parameters.
Let y ij denote the jth response, j = 1,.., ni for ith individual, i = 1,.., m, taken at a set of
m
conditions sum up by the vector of covariates x ij , so that a sum of N = i =1 ni response have
been observed. The vector x ij includes variables.

Suppose that, for individual i , the jth response obey the model.

yij = f ( xij , i ) + eij (1)

Where e ij is a random errorexpression considering unreliability in the response, given the ith
individual, with E (eij i ) = 0 Getting the response and errors for the ith individual into the
( n i 1) vectors y i = [ y i1 ,......, y ini ] ' , and e i = [e i1 ,...., e ini ] ' , respectively, and interpreting the
( ni 1) vector.

yi = f ( xi , i ) + ei , (2)
where E (ei i ) = 0 .
The model given in (1) and (2) describes the organizing and random variation association with
measurement on the ith individual.

If for nonlinear regression i ~ N (0, i ) , then y on the parameter of score function L ( )

observation information matrix L ( ) and fisher information matrix I ( ) respectively.
Computational of nonlinear least square estimates need to use the iterative numerical algorithm.
^
L ( ) = 0 , we may use Taylor expansion at point 0
^ ^ ^ ^
L ( ) = L ( 0 ) + L ( )( 0 ) + o 0 =0

i +1 = i + [ L ( i )] 1 L ( i ), i = 1,2,..... (3)

Until i +1 i < , is an advance fixed value. Gauss newton method has some important
properties.

2
 Operations Research and Applications : An International Journal (ORAJ), Vol.4, No.3/4, November 2017

3. STATISTICAL DIAGNOSTICS FOR NONLINEAR MODELS WITH REPEATED

MEASUREMENT DATA
In statistics, Cook's distance is an often used to estimate the influential points of a data [12].Data
points with huge residuals (outliers) and/or high leverage may misrepresent the outcome and
accuracy of a regression.

^ ^ ^ ^
^ ( (ij ) )T (U TU )( ( ij ) ) (4)
Dij = Dij (U U , p 2 ) =
T
^
2
p
f ( x, ) ^ ^
Where U = , Cook distance gives squared distance from to ( i ) relative to the fixed

^
geometry of U TU . The values of Di (U T U , p 2 ) can be converted to a familiar probability scale
by comparing calculated values to the F ( p, n p) distribution.

Cook distance in multiple cases:

^ ^ ^ ^

T
^
2
( (i) )(U T U )( (i ) ) (5)
D i = D i (U U , p ) = ^
2
p
Di Can be expressed in multidimensional analogues of the ri , and vii . The results are obtained by
^ ^
first expressing ( i ) as a function of :
^
( i ) = (U (Ti )U ( i ) ) 1U (Ti )Y( i ) = (U T U U iT U i ) 1 ( X T Y X iT Yi ) (6)

The inverse of (6)

^
( i ) = [(U T U ) 1 + (U T U ) 1U iT ( I Vi ) 1U i (U T U ) 1 ][U T Y X iT Yi ]
^ ^
= (U T U ) 1U iT [ ( I Vi ) 1 X i + ( I + ( I Vi ) 1Vi )Yi ] (7)
^ ^
( i ) = (U T U ) 1U iT ( I Vi ) 1 ei (8)

Substituting into (6) ahead to the form:

eiT ( I Vi ) 1Vi ( I Vi )1 ei
Di = ^
(9)
2
p
Single case Cook distance:
^ I ^ ^ ^
( ij ) = + [ I ( ij ) ( )]1 L ij ( ) (10)

In this case, I ( ) = U T 1U , and L ( ) = U T 1Ue

3
 Operations Research and Applications : An International Journal (ORAJ), Vol.4, No.3/4, November 2017
Replacing into (4), we get the form
Dij = [U ijT ij1U ij ]1U ijT ij1eij (11)

Multiple cases Cook distance

^ ^ ^ ^
( i ) = + [ I ( i ) ( )]1 L ( i ) ( )
Substituting into (6), this form gets

Di = [U (Ti ) (i1)U ( i ) ]1U (i ) 1U ( i ) e(i ) (12)

Example 1:
We observe the data in table I that taken from a study reported by Kwan et al. (1876) of the
pharmacokinetics of indomethacin following bolus intravenous injection of the same dose in
six human volunteers, for each subject plasma concentrations of indomethacin were
measured at 11 times intervals regarding from 15 to 8 hours post-injection[11].

Table. i: plasma concentrations ( g / ml ) following intravenous injection of indomethacin for six human

We consider two examples to calculate Gauss newton method

y = 1 exp(2 x) + 3 exp(4 x), 1 ,......, 4 > 0, (13)

We examine Gauss Newton method

i+1 = i +[UT 1U]1UT 1e

f ( )
Using MATLABs convention for representing Jacobin matrix U which is equal to
U=

where = In a known case, and e = y f ( ) ,
We chose initial values of , 0 = [0.7, 0.6, 0.54, 0.5] , after 5 iterations we obtained
^
= [0.75, 0.65, 0.50, 0.45] .which is satisfied under condition i +1 i < 10 4 .

4
 Operations Research and Applications : An International Journal (ORAJ), Vol.4, No.3/4, November 2017
Example 2:

We consider another example to compute Gauss newton method.

The result of estimation of the parameters in based on 11 responses for the fifth subjects are given in
Table I. Using Matlab to calculate G-N method and get parameter estimations.
We choose initial values,

0 = [1.0000,1.2000,-1.1000,-1.2000] ,
then use Gauss newton method to estimate the values of . After 5 iterations, we obtained
= [1.2715 , 1.0408, -1.2327, -1.5069] , and we satisfied under this condition
i +1 i < 10 4
.

Example.3.
f ( )
U=
We consider Table I, we focus on fifth subject to detect single case outlier. Where ,
= In and e is unobservable error y f ( ) .

Fig.1. Scatter plot for the table I (fifth individual) under model (11).

In the above scatterplot, we obtained cooks distance and found outlier in a set of predicted
values. First observation of our data set is an outlier which is indicated in (figure.1).

5
 Operations Research and Applications : An International Journal (ORAJ), Vol.4, No.3/4, November 2017

Example.4
We consider another example to detect multiple outliers cases.

Figure. 2. Scatter plot for the table I under model (12).

We obtained cooks. Distance and found four values that fall far from other data points. So
we consider these (23, 56, 45, 12) points outliers in 66 observations data. The outliers are
designated in (figure.2) cooks distance plot.

4. CONCLUSION:
It is well understood that all observations of a data set dont play the same role in the result
of regression analysis. For example, the character of the regression line maybe determine by
only a few observations, while most of the data is somewhat ignored. Such observations that
highly influence the results of the analysis are called influential observations.It is important,
for many causes, to be able to detect influential observations. In this paper, we established
Gauss newton method for parameter estimation and as well we extended rebut version of
Cook. Distance in single and multiple cases to detect outliers data points for repeated
measurement data.

REFERENCES:

[1] Ayinde, K., Lukman, A.F. and Arowolo, O. (2015) Robust Regression Diagnostics of Influential
Observations in Linear Regression Model. Open Journal of Statistics,vol.5, pp273-283.
[2] Altman, N. & Krzywinski, M.(2016) Analyzing outliers influential or nuisance. .Nature methods,
vol.13, pp281-282.
[3] Law, M. & Jackson, D. (2017) Residual plot for linear models with censored outcome data: A refined
method for visualizing residual uncertainty. Communication in statistics simulation and computation,vol.
46, pp3159-3171.
[4] Cook, R.D and Tsai, C.L. (1985)Residual in nonlinear regression, Biometrika, vol. 72, No.1, pp23-29.
6
 Operations Research and Applications : An International Journal (ORAJ), Vol.4, No.3/4, November 2017

[5] Cook R.D. (1979)Influence observations in linear regression, J.Amer.statist.Assoc, vol.74, pp169-74.
[6] Cook R.D, and presscot. (1981)Approximation significance levels for detecting outlier in linear
regression, Technometrics, vol.23,pp59-64.
[7] Ellenberg, J.H. (1976)Testing of a single outlier from a general regression model, Biometrics, vol. 32,
pp637-45.
[8] Vonesh, E.F. (1992)Nonlinear models for the analysis of longitudinal data, Statistics in medicine, vol.
11, pp1929-1954.
[9] Solomon P.J. and cox D.R. (1992)Nonlinear components for variance models, Biometrikka,vol. 79,
pp1-11.
[10] Cook R.D. (1979)Influence observation in liner regression, J.Am.statist.assoc,vol. 74, pp169-174.
[11] Diggle, P. J. (1988)An approach to the analysis of repeated measurements, Biometrics, vol. 44, pp959-
971.
[12] PREGIBON, D. (1981) Logistic regression diagnostics, Annual of statistics, vol.9, pp705-724.
[13] Anscombe, F.J. (1961) Examination of residuals, Proc.fouth Berkeley symp vol. 1, pp1-36.
[14] MARIE DAIDIAN and DAVID M.GILTINAN .march. (1995) Nonlinear models for repeated
measurement data.

AUTHOR

Munsir Ali, school of science, department of statistics Nanjing University of science and
technology, P.R china.