Home

Search

Collections

Journals

About

Contact us

My IOPscience

Least-squares analysis of time series data and its application to two-way satellite time and frequency transfer measurements

This article has been downloaded from IOPscience. Please scroll down to see the full text article. 2003 Metrologia 40 S342 (http://iopscience.iop.org/0026-1394/40/3/314) View the table of contents for this issue, or go to the journal homepage for more

Download details: IP Address: 200.55.133.202 The article was downloaded on 27/05/2013 at 20:50

Please note that terms and conditions apply.

INSTITUTE OF PHYSICS PUBLISHING Metrologia 40 (2003) S342S347

METROLOGIA PII: S0026-1394(03)61411-8

Least-squares analysis of time series data and its application to two-way satellite time and frequency transfer measurements
P M Harris, J A Davis, M G Cox and S L Shemar
National Physical Laboratory, Queens Road, Teddington, Middlesex TW11 0LW, UK

Received 24 April 2002 Published 5 June 2003 Online at stacks.iop.org/Met/40/S342 Abstract Two-way satellite time and frequency transfer (TWSTFT) has been used operationally by the international time and frequency community for several years. This paper describes analysis techniques being developed at the National Physical Laboratory for the processing of TWSTFT measurements. The measurements are modelled in terms of phase and normalized frequency offsets together with random errors considered to be a linear combination of the well-known noise types of white phase modulation, white frequency modulation and random walk frequency modulation encountered in time and frequency measurements. A method is described for solving the model to provide estimates of the phase and normalized frequency offsets, together with their associated uncertainties, at any measurement epoch. The random errors associated with the measurements are rst characterized through the use of second-difference statistics. An analysis of the second-difference statistics is undertaken to estimate a covariance matrix for the time series of measurements. Finally, with the covariance matrix so obtained, generalized least-squares regression is used to provide unbiased and efcient estimates of the required offset parameters.

1. Introduction
Two-way satellite time and frequency transfer (TWSTFT) [1] has been used operationally by primary timing laboratories for several years. TWSTFT links between laboratories have proved to be both stable and reliable, and consequently TWSTFT measurements are used for several of the main links in the computation of International Atomic Time (TAI). This paper describes analysis techniques being developed at the National Physical Laboratory for the processing of TWSTFT measurements. The overall aim of the study is to be able to determine at any measurement epoch estimates of the phase and normalized frequency offsets, together with their uncertainties, between two clocks or timescales being compared using TWSTFT. We describe an approach, based on generalized leastsquares (GLS) regression, for obtaining estimates of the phase and normalized frequency offsets from a uniformly spaced
0026-1394/03/SS0342+06$30.00

time series of phase measurements assuming knowledge of the random noise processes underlying the measurements (section 2). We indicate, again focusing on a uniformly spaced time series, how the analysis of second-difference statistics (Allan variances) can provide estimates of the parameters used to describe the underlying noise processes (section 3). In practice, the noise parameters are estimated rst, and are subsequently used to obtain estimates of the offset parameters. We present the results of applying this two-stage process to simulated time series of uniformly spaced measurements (section 4). For many TWSTFT links, measurements are made only on Mondays, Wednesdays and Fridays. The measurements constitute a time series that is unevenly spaced (with a two-, two-, three-day spacing) but consistent (the spacing pattern is repeated). The modications to the analysis necessary to deal with the consistent but uneven spacing of TWSTFT data are presented (section 5).
Printed in the UK

2003 BIPM and IOP Publishing Ltd

S342

Least-squares analysis of time series data

2. Estimating the phase and normalized frequency offset parameters

Let x(t) denote the phase difference, as a function of time t , between two clocks and {xi : i = 1, . . . , m} a time series of measurements of x made at uniformly spaced times {ti : i = 1, . . . , m}. Suppose x(t) is capable of being modelled in terms of a phase offset p0 and a normalized frequency offset f0 , that is, (1) x(t) = p0 + f0 t and the measurements can be modelled as x i = p0 + f 0 t i + e i , where ei denotes a random error. Using matrix notation, x = X p + e, where x1 x2 x = . , . . xm p0 p= , f0 t1 t2 , . . . (3) (2)

a lower triangular matrix. The covariance matrix V (e) for noise arising as a linear combination of (independent) WPM, WFM and RWFM is then expressed as
2 2 2 V (e) = WPM VWPM + WFM VWFM + RWFM VRWFM ,

(5)

2 2 2 where WPM , WFM and RWFM denote the variances of the white noise process underlying each noise type. A GLS analysis or GaussMarkov regression [2] may be applied to obtain estimates of the phase offset p0 and the normalized frequency offset f0 together with their associated uncertainties. The estimates determined are the solution to the problem

minp (x Xp)T V 1 (x Xp),

V = V (e),

(6)

1 1 X = . . .

and are optimal in the sense of being the estimates of minimum variance from the class of linear unbiased estimates of p0 and f0 [2]. A measure of the consistency of the input data with the tted model dened by the solution to this problem is the residual sum of squares (RSS), given by RSS = (x Xp)T V 1 (x Xp), (7)

1 tm e1 e2 e = . . . . em

We suppose that the elements of e have zero expectation (E(e) = 0) and, for the moment, a known covariance matrix V (e) [2]. The elements ei are taken as a linear combination of (independent) samples of known noise types. For simplicity of presentation we restrict the analysis to the well-known noise types of white phase modulation (WPM), white frequency modulation (WFM), and random walk frequency modulation (RWFM). These noise types are characterized below. Provided other noise types, such as icker phase modulation (FPM) and icker frequency modulation (FFM), can be characterized similarly, the extension to these noise types is possible, but is not analytically straightforward. We do not address FPM and FFM, with the understanding that no new fundamental concepts need be considered. We express V in terms of known covariance matrices that relate to each noise type and parameters describing the magnitude of each noise type are present. Each noise type is characterized by a transformation of a white noise process. The covariance matrices V for standardized WPM, WFM and RWFM, for which the transformed white noise process has unit standard deviation, are given, respectively, by VWPM = I, VWFM = W W T , VRWFM = (W 2 )(W 2 )T , (4)

evaluated at the solution to (6), which follows a chisquared distribution with = m 2 degrees of freedom [2, appendix B]. The results of this analysis for the examples of (pure) WPM, WFM and RWFM are given in table 1. For ease of presentation of results, we suppose ti = i0 with i = m/2, . . . , +m/2 (m even) for WPM and i = 0, . . . , m for WFM and RWFM. Other cases can also be handled straightforwardly. The results show that for WFM and RWFM the estimate p0 of the phase offset at t = 0 is the measurement x0 , whereas for WPM it is the mean of the measured phase offset values. Furthermore, for WFM and RWFM the estimate f0 of the normalized frequency offset is the gradient of the chord joining the rst and last measured values (for WFM) and the rst and second values (for RWFM). The law of propagation of uncertainty [4] is applied to the model dened by the GLS regression and the statistical model (5) for the measurements to provide the standard uncertainties of the estimates p0 and f0 and their covariance. (For the cases considered in table 1, the estimates p0 and f0 are uncorrelated for WPM and WFM, but correlated for RWFM.) The estimates of p0 and f0 , together with the model (1), can be used to predict the phase difference x at any given time t . A further application of the law of propagation of uncertainty permits the standard uncertainty of any quantity derived from p0 and f0 , such as a predicted phase difference, to be evaluated.

3. Estimating the noise parameters

We now consider the problem of obtaining estimates of the 2 2 2 noise parameters WPM , WFM and RWFM from a time series {xi : i = 1, . . . , m} of phase difference measurements. With these estimates we can use (5) to construct an estimate of the covariance matrix V (e) for the measurements, and then proceed as described in section 2 to obtain estimates of the phase and normalized frequency offsets. S343

where I is the identity matrix and W denes the summation operator [3, appendix A]: 1 0 ... 0 . 1 1 . . . . . , W = . . . . . . . . . 0 1 ... 1 1
Metrologia, 40 (2003) S342S347

P M Harris et al

Table 1. Results of GLS regression for the examples of (pure) WPM, WFM and RWFM. Noise type WPM x = (xm/2 , . . . , xm/2 )T WFM x = (x0 , . . . , xm )T RWFM x = (x0 , . . . , xm )T x0 x0 Estimate of p0
m/2 j =m/2

Standard uncertainty of p0 WPM m+1 WFM RWFM

Estimate of f0
m/2 j =m/2 j 0 xj m/2 2 2 j =m/2 j 0

Standard uncertainty of f0 WPM

m/2 j =m/2 2 j 2 0

xj

m+1

xm x0 m0 x1 x0 0

WFM 0 m

RWFM 2 0

We begin by forming second-differences of the measurements, namely, zi (n) = xi 2xi +n + xi +2n , i = 1, . . . , m 2n = M, (8)

where = n0 (n < m/2) is the averaging time, to give the time series {zi : i = 1, . . . , M }. Because the model is linear and the measurements are uniformly spaced (in time) the time series of values zi is independent of the (unknown) phase and normalized frequency offsets of the original time series. Moreover, it follows from the characterizations (4) of the noise types present in the original time series that the time series of values zi is stationary, that is, the variances var(zi ) and covariances cov(zi , zi +j ) are independent of the index i . We then calculate the variance statistic v(n) given by v(n) = 1 M
M 2 zi (n), i =1

(9)

which is related to the (commonly calculated) Allan variance AVAR( ) by 1 AVAR( ) = 2 v(n). 2 An analysis of the variance statistic v(n), also referred to as the maximal-overlap estimator [5, 6], yields expressions for the expectation of v(n), E [v(n)] = and the elements cov[v(n1 ), v(n2 )] = 2 M1 M2
M1 M2

Figure 1. Values of the expectation of variance statistics v(n) against n0 for pure WPM, WFM and RWFM and various choices of n. For each noise type the values of v(n) are scaled by a common multiplicative constant such that the scaled value of v(32) is unity.

1 M

M 2 E [zi (n)] =: R0 (n) i =1

(10)

{E [zi (n1 )zj (n2 )]}2

i =1 j =1

(11) of the covariance matrix V (v ) for a vector v of such statistics. The expressions are given in terms of the covariances of the elements of the stationary series {zi : i = 1, . . . , M } as well as, perhaps more conveniently for computation, in terms of the autocovariance function for that series [6]. Figure 1 shows values for the expectation of v(n) calculated from (10) for time series of pure WPM, WFM and RWFM consisting of m = 250 measurements and n = 1, 2, . . . , 32. (In this gure the values of v(n) are scaled by a common multiplicative constant such that the scaled value of v (32) is unity and, without any loss of generality, 0 is set equal S344

Figure 2. Values of variance statistics v(n) against n0 for a mixture of WPM, WFM and RWFM and various choices of n.

to 1 s.) Figure 2 shows values for the variance statistic v(n) calculated from (9) for a time series simulated to be a mixture 2 2 of WPM, WFM and RWFM, with WPM = 1, WFM = 0.5625 2 and RWFM = 0.01, using the above values of m and n. It is clear from gure 1 that the statistic v(n) can be used to distinguish between the three noise types in the same way that
Metrologia, 40 (2003) S342S347

Least-squares analysis of time series data

the Allan variance can be used for this purpose. However, gure 2 suggests that the variances and covariances of the statistic v(n) given by (11) are important for an understanding of how values of v(n) are related to the functions for the individual noise types shown in gure 1. It follows from the expressions (10) and (11) that values v(nk ) of the second-difference variance statistic (9) calculated from the time series {xi : i = 1, . . . , m} of phase measurements may be described by the model v(nk ) = E [v(nk )] + fk , where
2 WPM 2 WFM E [v(nk )] = WPM R0 (nk ) + WFM R0 (nk ) 2 RWFM +RWFM R0 (nk )

noise parameters. That approach is to solve a weighted least-squares problem that accounts for the variances of the calculated variance statistics, but not their covariances. Also, in that approach, estimates of the variances are established using information, for example, about the distributions of the variance statistics, which are approximately chi-squared.

4. Results for simulated data

In this section we present the results of applying to simulated data the approaches described in section 3 to obtain estimates of the noise parameters. We then present the results of using the values so obtained to determine estimates of the phase and normalized frequency offsets by solving the GLS regression problem described in section 2. A thousand different time series {(i0 , xi ): i = 1, . . . , m} of phase measurements are generated with the following parameters:
2 = 1, WPM p0 = 0 , 2 2 WFM = 0.5625, RWFM = 0.01, f0 = 0, 0 = 1, m = 256.

and the elements fk of f have zero expectation (E( f ) = 0) and covariance matrix V (v ). Using matrix notation the model equations are written as
v = R q + f,
WPM (nk ), where the k th row of R contains the values R0 WFM RWFM 2 2 2 R0 (nk ) and R0 (nk ), and q = (WPM , WFM , RWFM )T . This suggests (as for the estimation of offset parameters described in section 2) that estimates of the noise parameters 2 2 2 WPM , WFM and RWFM might be obtained by solving a GLS problem of the form

minq (v R q)T V 1 (v R q),

V = V ( f ).

(12)

2 , For each time series, estimates of the noise parameters WPM 2 2 WFM and RWFM are obtained using approaches A1 and A2 (section 3) and a benchmark approach (B) that involves solving the GLS regression problem (12) using a covariance matrix V ( f ) constructed using the above true values for the noise parameters. These three approaches are implemented using seven variance statistics v(nk ) corresponding to the averaging times

However, the difculty with this approach is that the covariance matrix V ( f ) for the calculated values of the variance statistics 2 2 v(n) depends on the unknown noise parameters WPM , WFM 2 and RWFM . We compare two approaches to solving the GLS problem given above: A1. Replace V ( f ) by the identity matrix, that is, ignore the variances and covariances of the values v(n), and solve (12) to obtain estimates of the noise parameters. A2. Given initial estimates of the noise parameters, construct an approximation to V ( f ) using (11), and solve (12) to obtain new estimates of the noise parameters. Repeat the process until the change in successive estimates of the noise parameters is small. In section 4, the results of the comparison are given for simulated data. We note that the ideas described here are similar to an approach [7] where a number of different variances (Allan variance, modied Allan variance, etc) evaluated for different integration (averaging) times are used to estimate

= nk 0 ,

nk {1, 2, 4, 8, 16, 32, 64}.

For each approach, the 1000 estimates of each noise parameter are summarized by their means and standard deviations. These statistics provide information about the bias (the expected departure from the true value) and efciency (the spread) of the estimates obtained using each approach. The results are given in table 2. The results suggest that, for all three approaches, there is no signicant bias in the estimates returned. However, approach A1, in which the variances and covariances of the values v(n) are ignored, gives inefcient estimates of the noise parameters, whereas approaches A2 and B have comparable efciencies. In fact, approximately 64% of the covariance 2 2 , WFM matrices V (e) constructed using the estimates of WPM 2 and RWFM returned by approach A1 are not positive denite, and hence not valid covariance matrices. For approaches A2 and B, the corresponding gure is 0.5%. Finally, for about 1% of the 1000 simulated time series, the (iterative) approach A2 fails to converge within a xed number (20) of iterations.

Table 2. Summary statistics for the estimates of the noise parameters obtained from algorithms A1, A2 and B applied to 1000 simulated time series.
2 WPM (=1) 2 WFM (=0.5625) 2 RWFM (=0.01)

Algorithm A1 A2 B

Mean 1.005 0.993 0.993

Standard deviation 3.870 0.169 0.165

Mean 0.546 0.556 0.558

Standard deviation 2.791 0.204 0.197

Mean 0.010 0.011 0.010

Standard deviation 0.012 0.006 0.006

Metrologia, 40 (2003) S342S347

S345

P M Harris et al

Table 3. Summary statistics for the estimates of phase and normalized frequency offsets obtained from algorithms GLS, A2 and B applied to 1000 simulated time series. p0 (=0) Algorithm GLS A2 B Mean 0.008 0.008 0.015 Standard deviation 1.065 1.057 1.067 Mean 0.001 0.003 0.002 f0 (=0) Standard deviation 0.307 0.320 0.320 Mean 252.4 254.3 254.4 RSS (=254) Standard deviation 21.9 3.3 3.4

Using the estimates of the noise parameters obtained for each time series from the approaches A2 and B, an estimate of the covariance matrix V (e) for the time series is constructed using (5), and the GLS regression problem (6) is solved to give estimates of the phase and normalized frequency offsets, p0 and f0 , respectively. Approach A1 is not considered because such a large proportion of the resulting covariance matrices V (e) are not valid. The benchmark approach (GLS) for this analysis is to solve the GLS regression problem (6) using a covariance matrix V (e) constructed using the above true values for the noise parameters. For each approach, the 1000 estimates of each offset and the corresponding value of the RSS measure (7) are summarized by their mean and standard deviation. The results are given in table 3. The results suggest that for all three approaches there is no signicant bias in the estimates returned. In addition, the efciencies of the estimates are also comparable. For all three approaches the mean of the RSS values is close to the expected value of 254 (= m 2). Furthermore, for the GLS approach, the standard deviation of the RSS values is close to the expected value of 22.5 ( 2 (m 2) ). It is not yet understood why the standard deviations of the values of RSS for approaches A2 and B, which make use of estimates of the noise parameters, are appreciably smaller than expected.

and the time series xtw given by xtw = S x contains only the observed measurements made on Mondays, Wednesdays and Fridays. (If it is required to consider a number of TWSTFT measurements that do not form a whole number w of weeks, it is necessary only to omit one or two of the nal rows of S .) Applying the transformation S to the model equations (3) gives xtw = Xtw p + etw , where Xtw = SX with E(etw ) = SE(e) = 0 and V (etw ) = SV (e)S T . and etw = S e

Consequently, given an estimate of V (e) (as in section 2) we obtain estimates of the phase and normalized frequency offsets by solving a GLS problem of the form minp (xtw Xtw p)T V 1 (xtw Xtw p), V = V (etw )

5. Analysis of TWSTFT measurements

For many TWSTFT links, measurements are made only on Mondays, Wednesdays and Fridays. The measurements constitute a time series that is unevenly spaced (with a two-, two-, three-day spacing) but consistent (the spacing pattern is repeated). We indicate here the modications to the analysis described in sections 2 and 3 necessary to deal with the consistent but uneven spacing of TWSTFT data. Suppose x denotes a time series {xi : i = 1, . . . , m} of phase measurements made daily, that is, corresponding to uniformly spaced times ti = i0 with 0 = 1 day, and with m = 7w, that is, consisting of w-weeks worth of data. Let S dene a transformation from the uninterrupted time series x to a time series containing only the observed values. Assuming that the rst measurement x1 is made on a Monday, the transformation S for TWSTFT measurements takes the form Sw Sw S= , .. . 1 0 0 0 0 0 0 Sw = 0 0 1 0 0 0 0 , 0 0 0 0 1 0 0 S346

(cf expression (6)). To obtain an estimate of V (e) it is necessary to determine 2 2 2 , WFM and RWFM estimates of the noise parameters WPM from the time series xtw of observed measurements. If we can construct second-difference operators, which when applied to xtw give a time series of values zi that are (a) independent of the (unknown) phase and normalized frequency offsets of the original time series, and (b) stationary, then we can apply the approach described in section 3. We calculate variance statistics v(n) in terms of such operators and model the statistics in terms of the noise parameters 2 2 2 , WFM and RWFM we wish to determine. A family WPM of second-difference operators for use on unevenly but consistently spaced measurements has been proposed [8]. Initial investigations regarding the performance on simulated data of these operators combined with the approach of section 3 have been favourable.

6. Summary
In this paper we have considered the least-squares analysis of measurements of the phase difference between two clocks or timescales with the aim of determining estimates at any epoch of the phase offset between the clocks or timescales. We have considered this analysis in two parts. First, assuming knowledge of the noise processes underlying the
Metrologia, 40 (2003) S342S347

Least-squares analysis of time series data

measurements, we have solved a GLS regression problem to obtain unbiased and efcient estimates of the offsets. Second, we have described an approach, based on the analysis of second-difference variance statistics, to estimate the parameters describing the noise processes. We have presented the results of applying these ideas to simulated data. Other simulations, not described here, suggest that the results are improved most noticeably as seconddifference variance statistics based on longer averaging times (greater n) are included and, to a lesser extent, as intermediate values of n and other variance statistics (such as modied Allan variance) are included. Furthermore, the ideas would seem to extend quite naturally to third-difference statistics, providing the possibility of analysing measurements modelled to include a linear frequency drift component. Although the presentation has focused on uniformly spaced time series of phase measurements, we have indicated the modications to the approaches described, which are necessary to address the uneven but consistent spacing of TWSTFT data. In particular, the second-difference variance statistics designed for such data, and described in the companion paper [8], would appear to provide a means of addressing this issue. Finally, it is expected that the methods described in this paper have wider applications than just TWSTFT, including GPS carrier-phase and GPS common-view measurements, and can be generalized to

multiple time series, such as those found in redundant time transfer links.

References
[1] Kirchner D 1991 Two-way time transfer via communication satellites Proc. IEEE 79 98390 [2] Mardia K V, Kent J T and Bibby J M 1979 Multivariate Analysis (London: Academic) [3] Tavella P and Leonardi M 1998 Noise characterization of irregularly spaced data 12th EFTF (1012 March, 1998) pp 20914 [4] Guide to the Expression of Uncertainty in Measurement 2nd edn (Geneva, Switzerland: International Organization for Standardization) [5] Howe D A, Allan D W and Barnes J A 1981 Properties of Signal Sources and Measurement Methods: Proc. 35th Annual Symposium on Frequency Control pp A1A47; also in NIST Technical Note 1337, March 1990, pp TN-14TN-55 [6] Greenhall C A 1991 Recipes for degrees of freedom of frequency stability estimators IEEE Trans. Instrum. Meas. 40 9949 [7] Vernotte F, Lantz E, Groslambert J and Gagnepain J J 1993 Oscillator noise analysis: multivariance measurement IEEE Trans. Instrum. Meas. 42 34250 [8] Davis J A, Harris P M, Shemar S L and Cox M G 2003 The characterization of regular but unevenly spaced TWSTFT data using second difference statistics Metrologia 40 S3128

Metrologia, 40 (2003) S342S347

S347