You are on page 1of 29

Prepared by

Francis Chiew and Lionel Siriwardena


CRC for Catchment Hydrology

www.toolkit.net.au/trend

USER GUIDE
Document History
Date Author Revision Description of Change
February 2004 Francis Chiew 0.1 Basic User Guide for Modelling School
June 2004 Robert Argent 1.0 Creation of User Guide from Notes
First version of User Guide. This applies to
July 2004 Francis Chiew 1.0.2b
version 1.02b (beta) of the Trend software
February 2005 Francis Chiew 1.0.2 Final release

Copyright notice
© CRC for Catchment Hydrology, Australia 2005

Legal Information
To the extent permitted by law, the CRC for Catchment Hydrology (including its employees and consultants)
accepts no responsibility and excludes all liability whatsoever in respect of any person's use or reliance on
this publication or any part of it.
Acknowledgements

TREND is a product from the CRC for Catchment Hydrology’s (CRCCH) Climate Variability Program. The
scientific development and testing were carried out by Francis Chiew1 and Lionel Siriwardena1 and the
software was developed by Sylvain Arene2 and Joel Rahman3.

The Product Manager of TREND is Francis Chiew.

1
University of Melbourne
2
University of Melbourne and Griffith University
3
CSIRO Land and Water
TREND – User Guide

CONTENTS

1 Introduction...........................................................1
1.1 The user guide.............................................................................................. 1
1.2 TREND......................................................................................................... 1
1.2.1 Overview .................................................................................... 1
1.2.2 Features ..................................................................................... 2
1.2.3 Audience .................................................................................... 2
1.2.4 Limitations and cautionary notes.................................................... 2
1.3 Related documents ........................................................................................ 2
1.4 Data requirements......................................................................................... 3
1.4.1 Input data................................................................................... 3
1.4.2 Calculated results ........................................................................ 3
1.5 References and training.................................................................................. 3

2 Installation ............................................................4
2.1 Technical specifications.................................................................................. 4
2.2 Licence agreement ........................................................................................ 4
2.3 Folders ........................................................................................................ 5
2.4 Installation ................................................................................................... 5
2.4.1 Framework installation ................................................................. 5
2.4.2 Stand-alone PC ........................................................................... 5
2.5 Uninstalling TREND ....................................................................................... 6

3 Using TREND.........................................................7
3.1 Getting started.............................................................................................. 7
3.2 Data Input screen.......................................................................................... 7
3.3 Options screen ............................................................................................. 9
3.3.1 Test selection panel...................................................................... 9
3.3.2 Resampling panel ........................................................................ 9
3.3.3 Period panel ............................................................................... 9
3.3.4 Display panel ............................................................................ 10
3.3.5 Running the tests........................................................................ 10
3.4 Results screen ............................................................................................. 10
3.4.1 Test display panel ...................................................................... 10

i
3.4.2 Summary statistics panel..............................................................10
3.4.3 Test result panel .........................................................................10
3.5 Saving test results.........................................................................................12

4 Description of statistical tests ...............................14


4.1 Statistical testing ..........................................................................................14
4.1.1 Basic concepts ...........................................................................14
4.1.2 Significance level........................................................................15
4.1.3 Resampling analysis to estimate significance level...........................15
4.1.4 Parametric and non-parametric tests .............................................15
4.2 Description of statistical tests in TREND...........................................................16
4.2.1 Mann-Kendall Test......................................................................16
4.2.2 Spearman’s Rho Test...................................................................17
4.2.3 Linear Regression Test .................................................................17
4.2.4 Distribution Free CUSUM Test......................................................18
4.2.5 Cumulative Deviation Test ...........................................................18
4.2.6 Worsley Likelihood Ratio Test .......................................................19
4.2.7 Rank-Sum Test ...........................................................................20
4.2.8 Student’s t Test ...........................................................................20
4.2.9 Median Crossing Test..................................................................21
4.2.10 Turning Points Test......................................................................21
4.2.11 Rank Difference Test ...................................................................21
4.2.12 Autocorrelation Test....................................................................22

5 References...........................................................23

ii
Introduction

1 Introduction
TREND is designed to facilitate statistical testing for trend, change and randomness in
hydrological and other time series data. TREND has 12 statistical tests, based on the
WMO/UNESCO Expert Workshop on Trend/Change Detection and on the CRC for
Catchment Hydrology publication Hydrological Recipes.

1.1 The user guide


The User Guide provides basic instructions for using TREND. It also provides an overview of
statistical testing and descriptions of the 12 statistical tests in TREND.

1.2 TREND

1.2.1 Overview
TREND has 12 statistical tests that can be used to test for trend, change and randomness in
hydrological and other time series data:
• Mann-Kendall (non-parametric test for trend)
• Spearman’s Rho (non-parametric test for trend)
• Linear Regression (parametric test for trend)
• Distribution-Free CUSUM (non-parametric test for step jump in mean)
• Cumulative Deviation (parametric test for step jump in mean)
• Worsley Likelihood Ratio (parametric test for step jump in mean)
• Rank-Sum (non-parametric test for difference in median from two data periods)
• Student’s t (parametric test for difference in mean from two data periods)
• Median Crossing (non-parametric test for randomness)
• Turning Points (non-parametric test for randomness)
• Rank Difference (non-parametric test for randomness)
• Autocorrelation (parametric test for randomness).

1
Trend User Guide

1.2.2 Features
• TREND allows easy statistical testing using different tests
• TREND supports various time series data input formats
• TREND provides simple statement of test result
• TREND displays test statistic and critical values for various statistical significance levels
• TREND performs resampling analysis to determine critical test statistic values
• TREND allows easy retrieval of test results.

1.2.3 Audience
TREND is designed for hydrologists, environmental scientists, consultants and researchers to
facilitate statistical testing for trend, change and randomness in time series data.

1.2.4 Limitations and cautionary notes


TREND performs the statistical tests only on annual time series data (see Section 3.2 on how
you can modify the data file to use TREND for non-annual time series data).
The statistical tests in TREND are only valid if the time series data is not serially correlated.
Most (but not all) time-series data with time steps shorter than the annual time step are
serially correlated. The Autocorrelation test (one of the tests in TREND) can be used to test if
the time series data is serially correlated.
Users are strongly encouraged to carry out an exploratory data analysis (EDA) before using
TREND. EDA involves using graphs to explore, understand and present data. EDA allows
much greater appreciation of the features in the data than summary statistics or statistical
significance level. Outliers and obvious errors in the data can also be detected through
EDA. A well-conducted EDA may eliminate the need for a formal statistical analysis. At the
very least, the user should view a time series plot of the data (with a trend line fitted to the
data) before using TREND.
Users should have a good understanding of the statistical tests and assumptions (read
Chapter 4). Users should also note that a statistical test provides evidence, not proof (if a
trend/change is detected, the reason for the trend/change must be investigated). Statistical
significance is not the same as importance (e.g., a change may be detected, but the size of
the change may be so small that it is of little importance).

1.3 Related documents

TrendTests.xls
The statistical tests in TREND are relatively easy to understand and the user can gain a good
appreciation of the tests by following the descriptions in Chapter 4 and the examples in the
Excel spreadsheet TrendTests.xls. TrendTests.xls can be downloaded from Documentation in
the TREND homepage (www.toolkit.net.au/trend).

TrendWorkshop.ppt
TrendWorkshop.ppt is a powerpoint presentation used for the CRC for Catchment Hydrology
Modelling School. It presents an overview of the need for analysing data for trend/change,
exploratory data analysis, basic concepts in statistical testing and types of statistical tests.

2
Introduction

TrendWorkshop.ppt can be downloaded from Documentation in the TREND homepage


(www.toolkit.net.au/trend).

1.4 Data requirements

1.4.1 Input data


TREND requires a continuous time series as input data.
Various standard TIME/toolkit time series data formats are supported by TREND.
TREND performs the statistical tests on annual time series data (i.e., TREND will convert daily
and monthly time series data into an annual time series before carrying out the statistical
testing on the annual time series - see Section 3.2 on how you can modify the data file to use
TREND for non-annual time series data).

1.4.2 Calculated results


TREND displays as an output the value of the test statistic, the critical values of the test
statistic at α = 0.01, α = 0.05 and α = 0.1 significance levels, and a statement of the test
result, for all the statistical tests selected by the user.

1.5 References and training


TREND is a product developed by the CRC for Catchment Hydrology (CRCCH).
As a part of product delivery, the CRCCH runs training workshops. Details of workshops are
posted on the Toolkit website (www.toolkit.net.au) in the news and events sections.
The main references for TREND are:
Kundzewicz, Z.W. and Robson, A. (Editors) (2000) Detecting Trend and Other Changes in
Hydrological Data. World Climate Program – Water, WMO/UNESCO, WCDMP-45,
WMO/TD 1013, Geneva, 157 pp.
Grayson, R.B., Argent, R.M., Nathan, R.J., McMahon, T.A. and Mein, R. (1996)
Hydrological Recipes: Estimation Techniques in Australian Hydrology. Cooperative Research
Centre for Catchment Hydrology, Australia, 125 pp.

3
Trend User Guide

2 Installation
2.1 Technical specifications
Type of Machine Intel based PC with CD-ROM drive
Processor Minimum: 133 MHz Pentium class processor
Recommended: 400 MHz or faster (particularly when
resampling analysis is used)
Memory Minimum: 128 MB of RAM
Recommended: 512 MB
Disk Space Little (< 100 MB)
Operating System Windows 2000
Windows XP Professional
Windows NT®4.0 + Service Pack 6a
Windows® Server 2003
Other Supporting Software To run TREND, you will also need (see Section 2.4.1 on how
you can install these):
- .NET Framework 1.1 Redistributable or later; and
- Visual J#.NET Redistributable Package 1.1

The person installing the software must have local administrator access on the computer.
Ideally, you should also have the relevant permissions to install and uninstall software.

Note If you are using a networked computer, then inform your system
administrator before applying any service packs or updates, as there
may be other applications on your computer that could be affected by
these updates.

2.2 Licence agreement


A licence agreement is part of the installation procedure. You must acknowledge that you
have read, understood and agree to be bound by the TREND software licence agreement to
be able to proceed with the installation.

4
Installation

2.3 Folders
TREND can be downloaded from the toolkit website www.toolkit.net.au/trend.
The installation of TREND (see Section 2.4.2) will create a directory that contains:
• The TREND software (default path c:\Program Files\Toolkit\TREND)
• Sample data file (default path c: \Program Files\Toolkit\TREND\Data)

2.4 Installation

2.4.1 Framework installation

.NET Framework
The .NET Framework is a component of the Microsoft Windows® operating system used to
build and run Windows-based applications. TREND is built using the .NET Framework and
consequently requires that the .NET Framework Redistributable framework be installed prior
to running. The .NET Framework Redistributable will install the .NET Framework onto your
machine and is downloadable from
www.msdn.microsoft.com/netframework/downloads/redist.aspx.
You can check to see if you already have the .NET Framework installed by clicking Start on
your Windows desktop, selecting Settings > Control Panel, and then double clicking the
Add/Remove Programs icon. When that window appears, scroll through the list of
applications. If you see Microsoft .NET Framework 1.1 listed, the latest version is already
installed and you do not need to install it again.

Visual J#.NET
Visual J#.NET is an extension to .NET to allow the execution of components written in the J#
programming language. The Visual J#.NET Redistributable Package 1.1 must be installed
after the installation of the .NET Framework. It is available from
www.msdn.microsoft.com/vjsharp/downloads/howtoget/default.aspx.
You can check to see if you already have the Visual J#.NET Framework installed by clicking
Start on your Windows desktop, selecting Settings > Control Panel, and then double clicking
the Add/Remove Programs icon. When that window appears, scroll through the list of
applications. If you see Microsoft Visual J# .NET Redistributable Package 1.1 listed, the
latest version is already installed and you do not need to install it again.

2.4.2 Stand-alone PC
Unzipping the file downloaded from the Toolkit website (TREND_1.0.2.zip) would produce
three files. To install TREND, double-click Setup.Exe, which will initiate an install wizard,
which leads you through the install procedure. The TREND software and sample data file will
be installed to a default directory (c:\Program Files\Toolkit\TREND) or to a directory that you
specify.
When TREND is successfully installed, a TREND short-cut icon will appear on the Desktop.
You can run the TREND software by double clicking this icon.

5
Trend User Guide

2.5 Uninstalling TREND


To uninstall TREND, click Start on your Windows desktop, select Settings > Control Panel,
and then double click the Add/Remove Programs icon. When that window appears, scroll
through the list of applications to locate TREND. Select TREND and click the Remove button.

6
Using TREND

3 Using TREND
3.1 Getting started
To run TREND, double click the TREND icon on the desktop.
The TREND introduction splash screen will appear for approximately 5 seconds.
The Data Input screen will then appear (see Figure 3.1).

Figure 3.1 Data Input screen (before loading data)

3.2 Data Input screen


A continuous time series data is required to run TREND.
To load data, click the Select button (see Figure 3.1)
Select a data file using the Open dialogue box, by clicking on the appropriate data file, and
then clicking the Open button (see Figure 3.2).

7
Trend User Guide

[Example input data file can be found in c:\Program Files\Toolkit\TREND\Data]


The data will be loaded and an annual time series plot will be displayed (see Figure 3.3).
Click Next to continue.

Note TREND performs the statistical tests only on the annual time series data.
TREND will convert daily and monthly time series data files into an
annual time series before carrying out the statistical testing on the
annual time series. Input data files must be continuous and complete
(daily and monthly data files must have complete years of records).

To use TREND on a non-annual time series, you need to create an


“annual” data file and relate your time series data to Years 1, 2, 3 and
so on (using data input formats supported by TREND – see
www.toolkit.net.au or example data file in c:\Program
Files\Toolkit\TREND\Data). Most non-annual time series data will be
serially correlated, and will therefore violate a key assumption in the
statistical tests in TREND.

Figure 3.2 Dialogue box for loading data

8
Using TREND

Figure 3.3 Data Input screen (after loading data)

3.3 Options screen


There are four Panels in the Options screen (see Figure 3.4). Once you have attended to
these Panels, click Next to continue.

3.3.1 Test selection panel


Select/check the statistical tests that you wish to run. It is okay to leave all 12 tests selected
as TREND runs very quickly (unless the resampling analysis option is used).

3.3.2 Resampling panel


Select/check the resampling box if resampling analysis is required to estimate the critical test
statistic values (see Section 4.1.3 for a description of resampling analysis).
Then enter the number of samples for the resampling analysis (1000 samples are
recommended to obtain a good estimate of the critical test statistic value for significance
level of α = 0.05).
TREND may take some time to run with resampling analysis.

3.3.3 Period panel


Select a “period separation year” by entering the year directly or by moving the arrow below
the time series plot.
The period separation is used only in the Rank Sum and Student’s t tests, which test whether
the means/medians in the earlier part of the data (from the first year of data up to the year
before the separation year) and the later part (from the separation year to the final year of
data) of the data are different.

9
Trend User Guide

3.3.4 Display panel


The “accuracy” relates to the display, and not how TREND computes the statistics. Use the
default value of 3.

3.3.5 Running the tests


After attending to these four Panels, click Next to run the statistical tests. A Result screen will
appear when TREND has completed running the statistical tests.

Figure 3.4 Options screen

3.4 Results screen

3.4.1 Test display panel


The result from each statistical test is presented separately.
Choose the statistical test for which you want the results to be displayed (see Figure 3.5).

3.4.2 Summary statistics panel


This panel displays the length of the annual time series data (number of years), and the
mean, median and standard deviation of the time series data.

3.4.3 Test result panel


This panel displays the result for the statistical test that you have selected in the Test display
panel.

10
Using TREND

TREND displays the value of the test statistic (e.g., z-statistic in the Mann-Kendall Test in
Figure 3.5), the critical values of the test statistic at significance levels of α = 0.01, α = 0.05
and α = 0.1, and a statement of the test result.
If the user has requested for resampling analysis, TREND also displays the critical values of
the test statistic obtained from the resampling analysis (see Figure 3.6), and compares the
test statistic against these critical values to determine the test result.
TREND also displays other relevant statistics (where appropriate).
For some tests, graphs of the time series data or time series of the test statistic can be viewed
by clicking the More button.

Note TREND gives results for a two-sided tail test. That is, TREND compares
the test statistic value against critical test statistic values for a two-sided
tail. TREND therefore tests for trend/change in any direction, rather
than for trend/change in a pre-specified direction (e.g., increasing
trend).

Figure 3.5 Results screen

11
Trend User Guide

Figure 3.6 Results screen (with resampling analysis)

3.5 Saving test results


The results can be exported to an output file by clicking the Save button (see Figure 3.5). In
the Save As dialogue box that appears on the screen, enter a file name and select an
appropriate location for the file, and click the Save button.
The results are best viewed in an Excel spreadsheet, either by opening the file using Excel, or
saving the file as a .csv file.
In the output file (see Figure 3.7), the first column lists the statistical tests, the second column
gives the test statistic for each test, the next three columns give the critical values of the test
statistic for significance levels of α = 0.1, α = 0.05 and α = 0.01 (from standard statistical
table), the next three columns give the critical values of the test statistic for significance levels
of α = 0.1, α = 0.05 and α = 0.01 (from resampling analysis) (only if resampling analysis
option is selected), and the last column gives the test result (NS means not significant at α =
0.1; S means statistically significant, with the significance level shown in brackets).

12
Using TREND

Figure 3.7 Output file summarising results from statistical testing

13
Trend User Guide

4 Description of
statistical tests
4.1 Statistical testing
This section provides a succinct description of the basic concepts in statistical testing that are
relevant to TREND. The reader can to refer to Kundzewicz and Robson (2000) for a more
detailed description of statistical testing for trend/change, and to standard text books on
statistics for more detailed information.

4.1.1 Basic concepts

Hypothesis
The starting point of a statistical test is to define a null hypothesis (H0) and an alternative
hypothesis (H1). For example, to test for trend in a time series, H0 would be that there is no
trend in the data, and H1 would be that there is an increasing or decreasing trend.

Test statistic
The test statistic is a means of comparing H0 and H1. It is a numerical value calculated from
the data series that is being tested.

Significance level
The significance level is a means of measuring whether the test statistic is very different from
the (critical) values that would typically occur under H0.

Power and errors


There are two possible types of errors. Type I error is when H0 is incorrectly rejected. Type II
error is when H0 is accepted when H1 is true. A test with low Type II error is said to be
powerful.

14
Description of statistical tests

4.1.2 Significance level


The significance level (α) is a means of measuring whether the test statistic is very different
from values that would typically occur under H0.
Specifically, the significance level is the probability of a test statistic value as extreme as, or
more extreme than the observed value assuming no trend/change (H0). For example, for α
= 0 .05, the critical test statistic value is the value that would be exceeded by 5% of test
statistic values obtained from randomly generated data. If the test statistic value is greater
than the critical test statistic value, H0 is rejected.
The significance level is therefore the probability that a test detects a trend/change (reject H0)
when none is present (Type I error).
A possible interpretation of the significance level might be:
α > 0.1 little evidence against H0
0.05 < α < 0.1 possible evidence against H0
0.01 < α < 0.05 strong evidence against H0
α < 0.01 very strong evidence against H0.
For most traditional statistical methods, critical test statistic values for various significance
levels can be looked up in statistical tables or calculated from simple formulas, provided that
the test assumptions are satisfied. Where test assumptions are violated, resampling methods
can be used to estimate the significance level of a test statistic (see Section 4.1.3).
For detecting trend/change of any direction, the critical test statistic value at α/2 is used (two-
sided tail). For detecting trend/change in a pre-specified direction (e.g., an increasing
trend), the critical test statistic value at α is used (one-sided tail). TREND gives results for a
two-sided tail test.

4.1.3 Resampling analysis to estimate significance


level
Resampling analysis is a robust method for estimating the significance level of a test statistic.
It is particularly useful when the test assumptions are violated.
In resampling analysis, the original time series (input data) is resampled to provide many
replicates of time series data of equal length as the original data. The time series data for
each replicate is obtained by randomly selecting data value from any year in the original time
series continuously until a time series of equal length as the original data is constructed. In
TREND, the data are resampled with replacement (bootstrapping method), i.e., a replicate
series may contain more than one of some values in the original series and none of other
values.
The test statistic value of the original time series data can then be compared with the test
statistic values of the generated data (replicates) to estimate the significance level. For
example, if the test statistic value of the original data is greater than the 950th highest test
statistic value from 1000 replicates, H0 is rejected at α = 0.05 (i.e., a trend/change is
detected, with a 5% probability that this trend/change is incorrectly detected).
Therefore, the critical test statistic values for significance levels of α = 0.1, α = 0.05 and α
= 0.01, are the 90th, 95th and 99th percentile values respectively of test statistic values from
the generated (resampled) time series.

4.1.4 Parametric and non-parametric tests


Parametric tests assume that the time series data and the errors (deviations from the trend)
follow a particular distribution (usually normal distribution). Parametric tests are useful as
they also quantify the change in the data (e.g., magnitude of change in the mean or gradient
of the trend). Parametric tests are generally more powerful that non-parametric tests. Where

15
Trend User Guide

the assumption of normally distributed data is violated, resampling analysis can be used to
estimate the significance level or critical test statistic values for various significance levels.
Non-parametric tests are generally distribution-free. They detect trend/change, but do not
quantify the size of the trend/change. They are very useful because most hydrologic time
series data are not normally distributed.

4.2 Description of statistical tests in TREND


TREND has 12 statistical tests that can be used to test for trend, change and randomness in
hydrological and other time series data:
• Mann-Kendall (non-parametric test for trend)
• Spearman’s Rho (non-parametric test for trend)
• Linear Regression (parametric test for trend)
• Distribution-Free CUSUM (non-parametric test for step jump in mean)
• Cumulative Deviation (parametric test for step jump in mean)
• Worsley Likelihood Ratio (parametric test for step jump in mean)
• Rank-Sum (non-parametric test for difference in median from two data periods)
• Student’s t (parametric test for difference in mean from two data periods)
• Median Crossing (non-parametric test for randomness)
• Turning Points (non-parametric test for randomness)
• Rank Difference (non-parametric test for randomness)
• Autocorrelation (parametric test for randomness).
The null hypothesis H0 for the tests for trend is that there is no trend in the data.
The null hypothesis H0 for the tests for step jump in mean/median is that there is no step
jump in the mean/median.
The null hypothesis H0 for the test for difference in means/medians is that there is no
difference in the means/medians between two data periods.
The null hypothesis H0 for the tests for randomness is that the data come from a random
process.
TREND provides results for a two-sided tail test, therefore, where critical test statistic values
are given in the following descriptions, they are for a two-tailed test.

Note The statistical tests in TREND are relatively easy to understand and the
user can gain a good appreciation of the tests by following the
descriptions below and the examples in the Excel spreadsheet
TrendTests.xls. TrendTests.xls can be downloaded from Documentation
in the TREND homepage (www.toolkit.net.au/trend).

4.2.1 Mann-Kendall Test


This method tests whether there is a trend in the time series data. It is a non-parametric test.
The n time series values (X1, X2, X3, ….., Xn) are replaced by their relative ranks (R1, R2, R3,
….., Rn) (starting at 1 for the lowest up to n).
The test statistic S is:

16
Description of statistical tests

n −1 n
S = ∑ ∑
i =1
[
j=i +1
sgn (Rj – Ri) ]

where sgn(x) = 1 for x > 0


sgn(x) = 0 for x = 0
sgn(x) = -1 for x < 0
If the null hypothesis Ho is true, then S is approximately normally distributed with:
µ = 0
σ = n (n – 1) (2n + 5) / 18
The z-statistic is therefore (critical test statistic values for various significance levels can be
obtained from normal probability tables):
z = | S | / σ0.5
A positive value of S indicates that there is an increasing trend and vice versa.

4.2.2 Spearman’s Rho Test


This is a rank-based test that determines whether the correlation between two variables is
significant. In trend analysis, one variable is taken as the time itself (years) and the other as
the corresponding time series data.
Like the Mann-Kendall Test, the n time series values are replaced by their ranks.
The test statistic ρs is the correlation coefficient, which is obtained in the same way as the
usual sample correlation coefficient, but using ranks:
ρs = Sxy / (Sx Sy)0.5
n
where Sx = ∑i =1
(xi – x )2

n
Sy = ∑i =1
(yi – y )2

n
Sxy = ∑
i =1
(xi – x ) (yi – y )

and xi (time), yi (variable of interest), x and y refer to the ranks ( x , y , Sx and Sy have the
same value in a trend analysis).

For large samples, the quantity ρs n −1 is approximately normally distributed with mean
of 0 and variance of 1 (critical test statistic values for various significance levels can be
obtained from normal probability tables).

4.2.3 Linear Regression Test


This is a parametric test that assumes that the data are normally distributed. It tests whether
there is a linear trend by examining the relationship between time (x) and the variable of
interest (y).
The regression gradient is estimated by:

17
Trend User Guide

∑ (x
i =1
i − x )( y i − y)
b = n

∑ (x
i =1
i − x) 2

and the intercept is estimated as:

a = y - b x
The test statistic S is:
S = b/σ
n
12∑ ( y i − a − bx i )
i =1
where σ =
n (n − 2)(n 2 − 1)
The test statistic S follows a Student-t distribution with n-2 degrees of freedom under the null
hypothesis (critical test statistic values for various significance levels can be obtained from
Student’s t statistic tables).
The linear regression test assumes that the data are normally distributed and that the errors
(deviations from the trend) are independent and follows the same normal distribution with
zero mean.

4.2.4 Distribution Free CUSUM Test


This method tests whether the means in two parts of a record are different (for an unknown
time of change). It is a non-parametric test (distribution free).
Given a time series data (x1, x2, x3, ….., xn), the test statistic is defined as:
k
Vk = ∑ sgn( x i − x median ) k = 1,2,3,…..,n
i =1

where sgn(x) = 1 for x > 0


sgn(x) = 0 for x = 0
sgn(x) = -1 for x < 0
xmedian is the median value of the xi data set.

The distribution of Vk follows the Kolmogorov-Smirnov two-sample statistic (KS = (2/n)


max|Vk|) with the critical values of max|Vk| given by:
α = 0.10 1.22√n
α = 0.05 1.36√n
α = 0.01 1.63√n
A negative value of Vk indicates that the latter part of the record has a higher mean than the
earlier part and vice versa.

4.2.5 Cumulative Deviation Test


This method tests whether the means in two parts of a record are different (for an unknown
time of change). The test assumes that the data are normally distributed.
The purpose of this test is to detect a change in the mean of a time series after m
observations:

18
Description of statistical tests

E(xi) = µ i = 1,2,3,…..,m
E(xi) = µ + ∆ i = m+1, m+2, ….., n
where µ is the mean prior to the change and ∆ is the change in the mean.
The cumulative deviations from the means are calculated as:
k
So* = 0 Sk* = ∑ (x
i =1
i − x) k = 1,2,3,…..,n

and the rescaled adjusted partial sums are obtained by dividing the Sk* values by the standard
deviation:
Sk** = Sk* / Dx
n
(x i − x) 2
D x
2
= ∑
i =1 n
The test statistic Q is:
Q = max | Sk**|
and is calculated for each year, with the highest value indicating the change point.
Critical values of Q/√n are given in the table below. A negative value of Sk* indicates that the
latter part of the record has a higher mean than the earlier part and vice versa.

Table of critical values of Q/√n

N Q/√n at significance level


α = 0.10 α = 0.05 α = 0.01
10 1.05 1.14 1.29
20 1.10 1.22 1.42
30 1.12 1.24 1.46
40 1.13 1.26 1.50
50 1.14 1.27 1.52
100 1.17 1.29 1.55
∝ 1.22 1.36 1.63

4.2.6 Worsley Likelihood Ratio Test


This method tests whether the means in two parts of a record are different (for an unknown
time of change). The test assumes that the data are normally distributed. It is similar to the
Cumulative Deviation Test but weights the values of Sk* depending on their position in the
time series.
Zk* = [k (n – k)]-0.5 Sk*
Zk** = Zk* / Dx
The test statistic W is:

(n − 2) 0.5 V
W=
(1 − V 2 ) 0.5
where V = max | Zk**|

19
Trend User Guide

Critical values of W are given in the table below. A negative value of W indicates that the
latter part of the record has a higher mean than the earlier part and vice versa.

Table of critical values of W

N W at significance level
α = 0.10 α = 0.05 α = 0.01
10 3.14 3.66 4.93
15 2.97 3.36 4.32
20 2.90 3.28 4.13
25 2.89 3.23 3.94
30 2.86 3.19 3.86
35 2.88 3.21 3.87
40 2.88 3.17 3.77
45 2.86 3.18 3.79
50 2.87 3.16 3.79

4.2.7 Rank-Sum Test


This method tests whether the medians in two different periods are different. It is a non-
parametric test.
To compute the rank-sum test statistic:
Rank all the data, from 1 (smallest) to N (largest). In the case of ties (equal data values), use
the average of ranks;
Compute a statistic S as the sum of ranks of the observations in the smaller group (the
number of observations in the smaller group is denoted as n, and the number of
observations in the larger group is denoted as m); and
Compute the theoretical mean and standard deviation of S under Ho for the entire sample
µ = n (N + 1) / 2
σ = [n m (N + 1) / 12]0.5
The standardised form of the test statistic Zrs is computed as:
Zrs = (S – 0.5 – µ) / σ if S > µ
Zrs = 0 if S = µ
Zrs = |S + 0.5 – µ| / σ if S < µ
Zrs is approximately normally distributed, and the critical test statistic values for various
significance levels can be obtained from normal probability tables.

4.2.8 Student’s t Test


This method tests whether the means in two different periods are different. The test assumes
that the data are normally distributed.
The Student’s t test statistic t is (critical test statistic values for various significance levels can
be obtained from Student’s t statistic tables):

20
Description of statistical tests

( x − y)
t=
1 1
S +
n m
where x and y are the means of the first and second periods respectively, and m and n are
the number of observations in the first and second periods respectively, and S is the sample
standard deviation (of the entire m and n observations).

4.2.9 Median Crossing Test


The n time series values are replaced by 0 if xi < xmedian and by 1 if xi > xmedian.
If the time series data come from a random process, then m (the number of times 0 is
followed by 1 or 1 is followed by 0) is approximately normally distributed with:
µ = (n – 1) / 2
σ = (n – 1) / 4
The z-statistic is therefore (critical test statistic values for various significance levels can be
obtained from normal probability tables):
z = | (m – µ) | / σ0.5

4.2.10 Turning Points Test


The n time series values are assigned 1 if xi-1 < xi > xi+1 or xi-1 > xi < xi+1, otherwise they are
assigned as 0.
The number of times 1 appears (m*) is approximately normally distributed with:
µ = 2 (n – 2) / 3
σ = (16n – 29) / 90
The z-statistic is therefore (critical test statistic values for various significance levels can be
obtained from normal probability tables):
z = | m* – µ | / σ0.5

4.2.11 Rank Difference Test


The n time series values are replaced by their relative ranks starting at 1 for the lowest up to
n.
The statistic U is the sum of the absolute rank differences between successive ranks:
n
U = ∑R
i =2
i − R i −1

For large n, U is normally distributed with:


µ = (n + 1) (n – 1) / 3
σ = (n – 2) (n + 1) (4n – 7) / 90
The z-statistic is therefore (critical test statistic values for various significance levels can be
obtained from normal probability tables):
z = | U – µ | / σ0.5

21
Trend User Guide

4.2.12 Autocorrelation Test


The lag-one autocorrelation coefficient is calculated as:

⎡ n −1 ⎤
⎢∑ ( x i − x )( x i +1 − x )⎥
r1 =
⎣ i =1 ⎦
⎡ n
2⎤
⎢∑ ( x i − x ) ⎥
⎣ i =1 ⎦
If the time series data come from a random process, then the expected value and variance of
r1 are:
E(r1) = - 1 / n
Var (r1) = (n3 – 3n2 + 4) / [n2 (n2 – 1) ]
The z-statistic is therefore (critical test statistic values for various significance levels can be
obtained from normal probability tables):
z = | r1 – E(r1) | / Var(r1)0.5

22
References

5 References
Kundzewicz, Z.W. and Robson, A. (Editors) (2000) Detecting Trend and Other Changes in
Hydrological Data. World Climate Program – Water, WMO/UNESCO, WCDMP-
45, WMO/TD 1013, Geneva, 157 pp.
Grayson, R.B., Argent, R.M., Nathan, R.J., McMahon, T.A. and Mein, R. (1996)
Hydrological Recipes: Estimation Techniques in Australian Hydrology. Cooperative
Research Centre for Catchment Hydrology, Australia, 125 pp.

23

You might also like