Design of Experiments and

Experimental Data Processing

Course plan

October 9, 2008

Fall 2008
Prof. Ivan Kalaykov
Room: T-1224, Tel. 019-303625, ivan.kalaykov@oru.se

1 Course objectives
In order to discover certain properties of a particular process or system, the investigators perform ex-
periments or series of tests, in which the input variables of a process or system are changed purposefully
such that from the observed output response the reasons for these changes could be identified.
The goals of this course are to introduce the participants to the terminology, use and some under-
lying statistical principles of experimental design with particular emphasis on the correct analysis of
data arising from designed experiments. We will discuss several experimental designs, their advantages
and disadvantages, estimation of treatment effects, and significance testing. Computer examples and
assignments will complement the lectures. We will emphasize the data analytic and statistical aspects
of the design of experiments in this course rather than the philosophy of science. The topics covered
should be useful to students at the Masters and Ph.D. level or practitioners who might be involved in
the design of experiments or the collection, analysis and interpretation of data from designed experi-
ments. This includes persons studying or working in business, economics, the life and health sciences,
engineering, social sciences, mathematics, and statistics. Even a good statistical methods course cen
be considered as a prerequisite, the course will introduce basic statistics knowledge for those who did
not follow such a course.
For various designs of experiments, the participants should be proficient in generating relevant hy-
potheses to answer research questions, selecting appropriate multilevel and/or multi-variable statistical
models, computing descriptive and inferential statistics, and effectively communicating these results
through interpretation and presentation of results. By the end of the course they are expected to be
able to:
• Explain the difference between descriptive and inferential statistics and provide examples of both;
• Address assumptions and limitations of statistical models;
• Formulate and apply hypothesis testing;
• Compute required sample sizes for different effect sizes;
• Conduct both parametric and non-parametric statistical analyses; models will include factorial
ANOVA, randomized block designs, linear regression, repeated measures models;
• Assess whether or not the results statistically support the original research question;
• Communicate interpretation of statistical results, both written and orally.

1
2 Course contents
1. Statistics in engineering and science. Strategy of experimentation and the role of statistics.
Basic principles and typical applications. Populations and samples. Parameters and statistics.
Mathematical and statistical modeling. Guidelines for designing experiments.
2. Fundamentals of statistical inference. Traditional summary statistics. Statistical inference.
Probability concepts. Interval estimation. Statistical tolerance intervals. Tests of statistical hy-
potheses. Sample size and power. Inferences on means and standard deviations. Inferences on
two populations or processes using independent pairs of correlated data values. Inferences on two
populations or processes using data from independent samples. Comparing standard deviations
from several populations.
3. Statistical principles in experimental design Experimental-design terminology. Common
design problems (masking factor effects, uncontrolled factors, erroneous principles of efficiency,
one-factor-at-a-time testing). Selecting a statistical design (consideration of objectives, factor
effects, precision and efficiency, randomization).
4. Factorial experiments in completely randomized designs Factorial experiments. Interac-
tions. Calculation of factor effects. Graphical assessment of factor effects.
5. Analysis of completely randomized designs. Balanced multifactor experiments (fixed factor
effects, analysis-of-variance models, analysis-of-variance tables). Parameter estimation. Sta-
tistical tests (tests on individual parameters, F-tests for factor effects). Multiple and graphical
comparisons.
6. Fractional factorial experiments. Confounding of factor effects. Design resolution. Two-level
fractional factorial experiments. Three-level fractional factorial experiments. Combined two- and
three-level fractional factorials. Sequential experimentation (screening experiments, designing a
sequence of experiments).
7. Analysis of fractional factorial experiments. General approach for the analysis of data
from unbalanced experiments. Analysis for data from unbalanced designs, from two-level and
three-level fractional factorial experiments and experiments with combinations of factors having
two and three levels. Analysis of screening experiments.
8. Experiments in randomized block designs Controlling experimental variability. Complete
block designs. Incomplete block designs (two-level and three-level factorial experiments, balanced
incomplete block designs). Latin-square and crossover designs.
9. Analysis of designs with random factor levels. Random factor effects. Variance-component
estimation. Analysis of data from block designs. Latin-square and crossover design.
10. Nested designs. Crossed and nested factors. Hierarchically nested designs. Split-plot designs.
Restricted randomization.
11. Special designs for process improvement. Assessing quality performance. Statistical designs
for process improvement (Taguchi’s robust product design approach, an integrated approach).
12. Analysis of nested designs and designs for process improvement. Hierarchically nested
designs. Split-plot designs. Gage repeatability and reproducibility designs. Signal-to-noise ratios.
13. Linear regression with one predictor variable. Uses and misuses of regression Strategy for
a comprehensive regression analysis. Scatterplot smoothing. Least-squares estimation. Inference.
14. Linear regression with several predictor variables. Least squares estimation. Inference.
Interactions among quantitative predictor variables. Polynomial model fits.
15. Linear regression with factors and covariates as predictors. Recoding categorical predic-
tors and factors. Analysis of covariance for completely randomized designs and for randomized
complete block designs.

2
 16. Designs and analyses for fitting response surfaces. Uses of response-surface methodol-
ogy. Locating an appropriate experimental region. Designs for fitting response surfaces. Fitting
response-surface models.
17. Model assessment. Outlier detection. Evaluating model assumptions. Model respecification.
18. Variable selection techniques. Comparing fitted models. All-possible-subset comparisons.
Stepwise selection methods. Collinear effects.

3 Homework Assignments
Homework will be due every Wednesday following the week of assignment. At the beginning of each
class on Wednesday, a student (randomly selected) will present the homework solutions to the rest of
the class. This presentation will count towards the participation grade. Assignments are individual,
unless otherwise said. One assignment is a Case Study, a paper that you are to analyze from a statistics
viewpoint. The analysis is not a formal research report, therefore short text in bullets is fine. It should
address whether or not the general DOE was appropriate, were the appropriate statistical tests used
(and if not, what should have been used), were the results presented in a clear and concise manner,
and what was the overall contribution of the statistical analysis to the main theme of the paper.

4 Small Project
During the course, the participants will work on a small project, in which they will develop a hypoth-
esis, design of experiment(s), conduct the experiment(s), analyze the results, and communicate the
conclusions through both a written report in research paper format as well as in a presentation at the
final week of the course. Participants are not required to focus on human-computer interaction, and
are encouraged to relate this assignment to a current research project. They have to submit a proposal
for the project within the second week of the course. Those who do not submit a proposal will receive
a task from the teacher. Participants can work in pairs on the project, and are encouraged to do so for
easier data collection. However, each one have to submit an individual report, where the data analysis
and conclusions have to be produced by individual efforts.

5 Examination
The final grade of the course will be based on three components: the written report on the small project,
the oral presentation of the small project and the homework assingments. The following grades apply:
VG, G and U.

6 Software
While statistical software will not be a requirement in this class, it is strongly advised that each student
become familiar with the package of their choice. During the lectures and seminars, illustrations of
various methods will be given in MATLAB and Statistics Toolbox1 . Recommended alternatives include:
SPSS2 , S+3 , Minitab4 , MathCAD5, SIMCA6 .
1
http://www.mathworks.com/
2
http://www.spss.com/statistics/
3
http://www.insightful.com/products/splus/default.asp
4
http://www.minitab.com/products/minitab/default.aspx?home=m1
5
http://www.ptc.com/products/mathcad/data-analysis-extension-pack
6
http://www.umetrics.com/

3
7 Literature
Required text:
[1] Robert L. Mason, Richard F. Gunst and James L. Hess, Statistical Design and Analysis of Experi-
ments (with Applications to Engineering and Science), Second Edition, 2003, Wiley-Interscience, ISBN
0-471-37216-1.

Other useful texts:

[2] Douglas C. Montgomery, Design and Analysis of Experiments, Sixth Edition, 2005, John Wiley and
Sons, ISBN 0-471-48735-X or 0-471-66159-7.

[3] Angela Dean and Daniel Voss, Design and Analysis of Experiments, 1999, Springer, ISBN 0-387-
98561-1.

[4] Murray R. Spiegel, Schaum’s Outline of Theory and Problems of Probability and Statistics, 1998,
McGraw-Hill, ISBN 0-07-060220-4.

A lot of useful information can be found on the Wikipedia web-site http://en.wikipedia.org/. Other
books containing the same or similar topics can be also used. Consult the teacher for the appropriate
ones.

4
8 Schedule

Date Topic Reading Assignment

05-11-2008 Statistics in engineering and science. Fundamentals Ch.1, Ch.2 [1] Homework1
Wednesday of statistical inference.
07-11-2008 Inferences on means and standard deviations. Sta- Ch.3, Ch.4 [1] Homework2
Friday tistical principles in experimental design.
12-11-2008 Factorial experiments in completely randomized Ch.5, Ch.6 [1] Homework3
Wednesday designs. Analysis of completely randomized de- Project
signs. proposed
14-11-2008 Fractional factorial experiments. Analysis of frac- Ch.7, Ch.8 [1] Homework4
Friday tional factorial experiments. Project
assigned
19-11-2008 Experiments in randomized block designs. Analysis Ch.9, Ch.10 [1] Homework5
Wednesday of designs with random factor levels.
21-11-2008 Nested designs. Special designs for process im- Ch.11, Ch.12, Homework6
Friday provement. Analysis of nested designs and designs Ch.13 [1]
for process improvement.
26-11-2008 Linear regression with one predictor variable. Lin- Ch.14, Ch.15 Homework7
Wednesday ear regression with several predictor variables. [1]
28-11-2008 Linear regression with factors and covariates as pre- Ch.16 [1] Homework8
Friday dictors.
03-12-2008 Designs and analyses for fitting response surfaces. Ch.17 [1] Homework9
Wednesday
05-12-2008 Model assessment. Variable selection techniques. Ch.18 [1] Homework10
Friday
10-12-2008 No lecture, teacher available for consultations.
Wednesday
12-12-2008 No lecture, teacher available for consultations.
Friday
17-12-2008 Seminar - project presentations None None
Wednesday
19-12-2008 Seminar - project presentations None None
Friday