ST102 HIGHLIGHTS !+!−1 ! !!

Unordered (‡) = 0!when!! < 0 !!

! ! ! − ! ! !!
! ! = ! ! !for!! = 0,1,2 … ! Conditions:)! ! ≥ 0!and! ! ! !" = 1
(2) Descriptive statistics !!
1!when%! > !
Birthday problem
Sample mean ! ! ≠! !=! ⟹! !=! =0
Key properties of DRV CDF
!!
!! !! !! !
!−! ! Ordered%outcomes%w/o%replacement !
!= = = !! !(!! ) ! ! = = ! ! !is#strictly#increasing:#!! < !! ,!then!! !! < !(!! ) ! ! = ! ! = !′(!)
! !! !!! !! Ordered%outcomes%with%replacement !"
lim ! ! = 0 , lim ! ! = 1
Sample median ! = 365$days, ! = Number'of'people !→!! !→!! CRV CDF

For$any!!! < !! , ! !! < ! < !! = ! !! − !(!! ) !

!! + !!!! Independence ! ! =! !≤! = !(!) !", ∀!
! !
!!" = ! !!! !or# !!
! 2 ! ⊥⊥ !, ! ! ∩ ! = ! ! !(!) DRV expected value, ! ! !or!!

!
Median:(! ! = 0.5
Sample standard deviation ! !

! !! = !(!! ) = ! !! … !(!! ) ! ! = !! ! !! + ⋯ + !! ! !! = !! ! !! Interquartile+range:+!!" − !!"

! ! ! !!!
! ! ! ! ! !!! !!!
!!! !! − ! !!! !!
− !! !!! !! !!− !!
!= = = ! (+)$skew:$Mean > Median,!(!)"skew:"Mean > Median
!−1 !−1 !−1 ! !∩! ! ! ! !
! !! = = = !(!) ! ! ! = !!! ! !! + ⋯ + !!! ! !! = !!! !(!! )
! ! ! ! Pareto distribution
Sample variance !!!

Conditional probability 0!when!! < !

!
!
! ! ! ! ! ! DRV variance and standard deviation, !"# ! !or!! ! !
!!!!! − ! !!! !! − !! !!! !! !! − !! ! ! = !!
!! = = = !when!! ≥ !, {!, !} > 0
!−1 !−1 !−1 ! !∩! ! ! !!!
! !! =
! ! !"# ! = ! ! ! − ! ! !
= [!! − !(!)]! ! !!
Linear transformation of random variable 1 !!! ! > 0, ! !!! > 0! ⟹ !! ! > 0
!!!
Chain rule of conditional probabilities !! !!
(!!! + !) ! !! + !"
!= = = !! + ! = !{ ! − ! ! !
} !! ! 1
! ! ! !!! 2 ! !" = !! ! −!! !!!! !"
! !!! −!
! ! ! !! ∩ !! ∩ !! … ∩ !! = ! !! ! !! !! !
!! !
! !
!!! !"! + ! − !! − ! !! !!! !! − ! !!! !!!
!!! = = = ! ! !!! != ! !! − ! ! ! = [!! − !(!)]! !(!! ) !! !
!−1 !−1 = ! !! !!
= −! ! −! !! = 1
!
!!! −!
= ! !! ! !! !! ! !! !! ∩ !! … !(!! |!!!! ∩ … !! )
(3) Introduction to probability theory
0!when!! < !
Total probability formula Linear transformation of random variable
Key properties of set operators ! ! = ! !
1− when ≥ !
! ! !
! !" + ! = ! !"(!) + ! !(!) = !" ! + !
Distributive*laws*I:*! ∩ ! ∪ ! = (! ∩ !) ∪ (! ∩ !) ! ! = ! ! ∩ !! = ! ! !! !(!! ) !"
!!! !!! ! ! =
Distributive*laws*II:*! ∪ ! ∩ ! = (! ∪ !) ∩ (! ∪ !) !"# !" + ! = ! !" + ! − !" ! − ! ! !−1
E.g.!! ! = ! ! ∩ ! + (! ! ∩ ! ! ! = ! ! ! ! ! + (!(!|! ! )!(! ! ) !
De#Morgan' s"laws"I:! ! ∩ ! !
= !! ∪ ! ! = ! !! ! − ! ! !
= !! ! ! − ! ! !
= ! ! !"#(!) ! !
!"# ! =
Bayes’ theorem !−1 !−2
De#Morgan's#laws#II:# ! ∪ ! !
= !! ∩ ! ! Constants:(! ! = !, !"# ! = !" ! = 0
! ! ∩ !! ! ! !! ! !! CRV expected value, ! ! !or!!
Partitions and pairwise disjointedness ! !! ! = ⟹ ! , ! ∈ {1 … !} DRV moment generating function (MGF)
! ! !!! ! ! !! ! !! !!
! ! ! = !" ! !"
!! ∩ !! = ∅!and!! !! = ! (!! ) (4) Discrete random variables (DRV) (Stepwise) !! ! = ! ! !" = ! !" !(!)
!!

!!! !!
Probabilities of equally likely outcomes DRV probability density function (PDF) ! !! = ! ! ! ! !"
! !!
Number'of'desired'outcomes !!! ! = !! ! ⟹ !!! 0 = !(! ! )
Conditions:)! ! = ! = ! ! ≥ 0!and! ! !! = 1 !"
! Outcome = !!∈ ! CRV variance and standard deviation, !"# ! !or!! !
Total&number&of&outcomes&in&sample
!
! ! !for!! = 0,1,2 … ! !!!! ! = !!! ! ⟹ !!!! 0 = !(! ! ) !!
Combinatorial counting rules ! ! = !" !"# ! = ! ! ! − ! ! !
= !−! ! !
! ! !"
0!otherwise !!
! !
!"# ! = ! ! −! ! = !!!! 0 − [!!! 0 ] !
With replacement W/O replacement DRV cumulative distribution function (CDF)
!!

!
!! (4) Continuous random variables (CRV) (Continuous) != ! !! − ! ! ! = !−! ! !! ! !"
Ordered !
!−! ! ! ! = ! ! ≤ ! , ! ∈ ℝ!and!! ! = !(!! ) !!

!! ∈!,!! !! CRV PDF

! !
1 2
 ! !
CRV MGF !! − 1 !"# ! = ! !
!"# ! = !! + !! ~!" !! + !! ⟹ !! ~!" !!
12
!! !!! !!! !!! !
‡ !!! ! = ! !"! !
!! ! = ! ! !" = ! !" !(!)!" ! ! (1 − ! !" )
‡ !!! ! = Continuous uniform distribution
!! !(1 − ! ! ) 1 !!! !
!
1 ! > 0, ! !! ! > 0 ⟹ ! ! > 0!
! 1 ! 2!
1 !! > 0, ! ! > 0
!!! ! = !! ! ⟹ !!! 0 = !(! ! ) ! ! = ! − ! ,!for!! ≤ ! ≤ !
!" !!
! 0!otherwise
1 (‡) 2 !(!) !" = 1
!"# ! = ! ! ! − ! ! !
= !!!! 0 − [!!! 0 ]! 2 ! =1 !!
! !
0!for!! < !
!!! !−!
Summary ! ! =! !≤! = !(!) !" = !for!! ≤ ! ≤ ! Linear combinations of the normal distribution
! !−!
Bernoulli distribution, !~Bernoulli(!) 1!for!! > !
DRV CRV 1 !! ± !~!(!! ± !! , !!! + !!! )
!!for!! = 1 !+!
!""#$
! ! = 1 − !!for!! = 0 ! ! =
!""#$ 2 2 !!" ± !~!(!" ± !, ! ! ! ! )
! ! !! ! !! !" ! !" 0!otherwise
!"#$% !
!"#$% !−! 3 !!" ± !"~!(!!! ± !!! , ! ! !!! + ! ! !!! )
0!for!! < 0 !"# ! =
12
! ! = 1 − !!for!0 ≤ ! < 1 !!!
!"# ! ! !! − ! ! !
! !! − ! ! !
1!for!! ≥ 1 Standard normal, = !~!(0,1! )
! !" − ! !" !
‡ !!! ! =
! ! =! !(! − !) 1 !!
!""#$ !""#$
! ! = ! ! ! , ∀! ∈ ℝ
!! ! !" !" 2!
! !(!) ! !(!)!" !"# ! = !(1 − !) 1 !! − ! > 0! ⟹ ! ! > 0
!"#$% !
!"#$% 1 !!
!! ! = 1 − ! + ! ! !
!
1 ! ! !−! ! ! =! ! = ! ! ! !"
2 ! !" = = =1 !! 2!
(5) Common distributions of RV ! !−! !−! ! !−!
1 !! > 0, 1 − ! > 0 ⟹ ! ! > 0 ! ! =0
Binomial distribution, !~!(!, !), from (3) Exponential distribution, from (4)
!
! ! !"# ! = 1
2 ! ! !! = ! + 1 − ! = 1 !!"
! ! = !
! 1 − ! !!! !for!! = 0,1,2 … ! ! ! = !! !for!! > 0, ! > 0
!!! 0!for!! ≤ 0 !−! !−! !−! !−!
0!otherwise ! !<!<! =! <!< =! −!
! ! ! !
Poisson distribution, !~!"(!) 0!for!! ≤ 0
0!when!! < 0 ! ! =
!
! ! 1 − ! !!" !for!! > 0 Poisson approximation of the binomial distribution
! ! =
! 1 − ! !!! !for$ ! = 0,1,2 … ! ! !! !!
!!! ! !for!! = 0,1,2 … 1
! ! = !! ! ! = !~! !, ! ⟹ !" ≤ 5 ⟹ !~!"(!" = !)
0!otherwise !
! ! = !"
Normal approximation of the binomial distribution
! 1
!"# ! = !"(1 − !) !! !"# ! =
! !!
,!for!! = 0,1,2 … !! !~! !, ! ⟹ !" ≥ 5!and!! 1 − ! ≥ 5 …
! ! = !!
!!!
!! ! = ! ! ! + 1 − ! !
!
0!otherwise !! ! = … ⟹ !~! !", !" 1 − !
!−!
1 !! ≥ 0, 1 − ! ≥ 0 ⟹ ! ! ≥ 0 ! ! =!
1 !! > 0, ! !!" > 0 ⟹ ! ! > 0 CC1: !(! ≤ !) ≈ ! ! ≤ ! + 0.5
! CC2: !(! < !) ≈ ! ! < ! − 0.5
! ! !!! ! ! !"# ! = !
2 ! ! 1−! = !+ 1−! =1 =1 !! !! CC3: !(! ≥ !) ≈ ! ! ≥ ! − 0.5
!
CC4: !(! > !) ≈ ! ! > ! + 0.5
!!!
!! ! = ! ! ! ! !! 2 !! !!" !" = −1 −!! !!" !" = −[! !!" ]!!
! =1
!! !
Discrete uniform distribution (‡) Normal approximation of the Poisson distribution
1 !! !! > 0, !! ≥ 0, !! ≥ 0 ⟹ ! ! ≥ 0
1 ‡ !Memoryless:*! ! > ! + ! ! > ! = ! ! > ! , !, ! ≥ 0
!~!" ! ⟹ ! > 10 ⟹ !~!(!, !)
! ! = ! ! = ! = ! ,!∀! = 1,2 … !
! !
! !! !! !!
2 ! = ! !! = ! !! ! ! = 1 Normal distribution, !~!(!, ! ! )
0!otherwise !! !! ! ! = ! = ! ! − 0.5 < ! < ! + 0.5 !by#CC
!!! !!!
1 !!! !
0!for!! < 0 !
(6) Multivariate random variables
! !
! ! ! = ! !! ! , ∀! ∈ℝ
1 ! ! 2!
! ! = ,!∀! = 1,2 … ! Note:! ≈ ! ! !(Taylor(series(approximation)
! !! Discrete joint probability function
!!! !!! !
1 !!! !
!
1!for!! > ! ! ! = ! !! ! !"
Linear combinations of the Poisson distribution !! ! 2! ! !, ! = !(! = ! ∩ ! = !) ≡ !!" (!, !)
! + 1!
! ! = 0.5!~!"(0.5!) ! ! =!
2

! !
3 4
 ! ! !
Method of moments estimator (MME)
Conditions:)! !, ! ≥ 0!∀ !, ! !and! ! !, ! = 1 Discrete(&(independent:(! !! ∩ !! … !! = !(!! ) (23) Point estimation II
!!! !!! !!! Sample moment Population moment
!
1 ! Standard error
Discrete univariate marginal distribution !! = !! = ! ⟹ ! ! ! =!
Continuous(&(independent:!! !! ∩ !! … !! = !(!! ) ! !!!
1 ! !
! !!! !! = !!! ⟹ ! ! + ! ! ! ! ! = ! ! + !! !" ! =
! !
!! ! = ! = ! ! = !, ! , ! ∈ {!! , !! , … , !! } !!!

!!! Independent random variables are also uncorrelated

⋮ ⋮ Maximum likelihood estimator (MLE)
!! ie.$!"# !, ! = 0!and!!"## !, ! = 0 1 !
!
!! = !!! ! !!
! ! = !!! (!) ! !!! ! ! = ! !! , … , !! ; ! = !(!! )
!!!! Joint distributions of random variables
!!!
! ! Let k denote the number of parameters in the random
!
distribution (ie. Normal = 2, Poisson = 1). Use integrals ! ! = !"# !
! !" = !" !!" !, ! , !" ∈ {0, … , !} ! !! !! + ! = !! ! !! + !
for continuous variables.
!!! !!!
!"!! !" !
Maximize(log!likelihood(function:( =0
E.g.!! ! ± ! = ! ! ± ! ! !!! = ! ! = ! !"
Discrete conditional distribution
! !
! ! 1 1 ! = ! = !(!)
!(! = ! ∩ ! = !) E.g.$! ! ! = ! ! !and!! ! ! =! ! ! ! !! = ! ! ! − ! ! !
= !!! − ! ! = !! − ! !
!!|! !! =! !=!!=! = … ! !
!(! = !) ! !
!!! !!!
!!" = ! ! = ℎ(!)
!"# !! !! + ! = !!! !"# !! + 2 !! !! !"#(!! , !! ) Unbiased sample variance
!!" (!, !) !!! !!! !!! MLE for non-differentiable (ie. Discrete) distributions
…≡
!! (!) !
!
E.g.!!"# ! ± ! = !"# ! + !"# ! ± 2!"#(!, !) ! = !! ! = !!!!"#
!! !−1
!!!! !!" !, ! !! !
Conditions:!!!|! ! ! ≥ 0!and! = ! ! ! ! Invariance property of MLE
!! ! !! ! 1 1 !
!!! !!
!
Independent:!!"# !! !! + ! = !!! !"# !! !! = !! − ! !
= !!! −
!−1 !−1 !
Covariance: Measure of association/dependence !!! !!! !!! !!! If!!!is#the#MLE#for!!,"then"! ! !is#the#MLE#for#!(!)

E.g.!!"# ! ± ! = !"# ! + !"# ! ,!as!!"# !, ! = 0 ! ! !

= ! !! − ! !!! =! ! !
−! ! !
!"# !, ! = !"# !, ! = !{ ! − ! ! ! − ! ! }… ie.$Solve$for$!!first&and&sub!! ! !in
! !
… = ! !" − ! ! !(!) !! MLE utilizes information about the entire population
Independent:!! !! !! = !! ! !! ⟹ ! ! ! − !"# ! + ! ! !
= ! ! + !! − + !!
! distribution. It is often more efficient/accurate than
!!! !!!
Properties of covariance MME/LSE which uses E(X) and Var(X).
!!
E.g.!! !" = ! ! !(!) ! !! − ! !! = > 0 ⟹ Biased
!"# !, ! = ! !! − ! ! ! ! = !"#(!) ! (24) Point estimation III
(7) Sampling distribution
!"# !, ! = ! !" − ! ! ! ! = !" ! − !" ! = 0 Mean square error (MSE) MLE with unknown parameters
Sampling distribution
!"# !" + !, !" + ! = ⋯ = !"#$%(!, !) Bias! ! = ! ! − !(!) = ! ! − ! ! !! , !! = !"# !! , !!
Population*distribution:*!~! !, ! !
Correlation: Measure of linear association ⟹! ! =!⇔! !! !! = ! !! !! = ! !" !! !" !!
= 0, =0
!! !!! !!!
!"#(!, !) Sampling)distribution:)!~! !,
!"## !, ! = !"## !, ! = ! 1
!"# ! !"#(!) Where! !! = =1 Fisher information
!
Chi square distribution, t-distribution – refer to (26)
Positive(coorelation:*!"## !, ! ≈ +1 ! ! !!
MSE = ! !−! = Bias ! + !"#(!) Central(limit(theorem:(When!!!is#large,!!~! !,
Central limit theorem !
Negative(coorelation:(!"## !, ! ≈ −1
!! !!
If ! ≥ 30 , then any random sample from any ⟹ !"# ! = ⇔ !"# !! !! = !"# !! !!! = 1
Not$coorelated:$!"## !, ! ≈ 0 ! ! When!!!is#large,!!~! !,
distribution with sample mean ! and sample variance !" !
! ! is asymptotically normally distributed: 1 1
Independence Where! !!! = = !!
!! ! ! ! !"#(!; !)
!! ! ! =− ! !; ! !!"
!!" (!, !) ie.! ≈ !~! !, !! !
Dependent:!!!|! !! = ≠ !! (!) ! Consistent(estimator:( lim MSE = 0 !!
!! (!) !→!

F-distribution – refer to (30) (25) Interval estimation

Independent:!!!" !, ! = !! (!)!! (!) Bias ! = ! ! − ! = 0

(22) Point estimation I Confidence interval

!!
!"# ! = = MSE(!)
!
! !
5 6
 Min.%length%of%interval,%max.%coverage%probability ! !−!
! ! ! !−! ! − !! Perform'a'one!tailed'T'test'at!!!level$of$significance
= = ~!!!! Apply%a%T%test,%! ! ≥ !! = ! ! ≥ ~!!!!
!
!!!! ! − 1 !! ! !/ !
!−!
! ! ≤ !!/! =! !−1 !!
!−1 ! − !! !! + !! − 2 ! − ! − (!! − !! )
! ! !< × ~!!!!!
⟹ 2! ! ≥ = 2(!!! ≤ ! ≤ !!! ) = ! 1 1
"T!distribution*with*k*degrees*of*freedom" !/ ! + !!! (!! − 1)
!!/! ! !!/! ! !! !!
⟹! !− ≤!≤!+ =!
! ! ! ! = 0,!for!! > 1 If#! ≤ !,!reject&!! .!Otherwise)do)not)reject)!!
!!
! ! !known:& ! − !!/! !" ! , ! + !!/! !" ! ~!(0,1) ! (Normal variance test) ! !< ~!!!!
!"# ! = ,!for!! > 2 !!
!−2
!!/! ! !!/! ! !! :!! ! = !!! !vs#!! : ! ! > !!! !
⟹! !− ≤!≤!+ =! (Confidence interval)
! ! (27/28) Hypothesis testing
Perform'a'one!tailed'! ! test$at$!!level$of$significance
! !
! ! !unknown:' ! − !!/! !" ! , ! + !!/! !" ! ~!!!! (One-tailed normal test) ! − ! − !!/!,!!! , ! − ! + !!/!,!!!
!! ! !
!
Apply%a%chi!square'test,'! !!!! > ! !(! − 1) = !
MLE-based interval !! :!! = !! !vs#!! :!! > !! !!
Two normal means with !!! , !!! known but unequal,
!! ≠ !!
Perform'a'one!tailed'Z'test'at!!!level$of$significance If#! ≤ !,!reject&!! .!Otherwise)do)not)reject)!!
!!/! !!/!
!− ,! + !! :!!! = !! − !! = 0!vs#!! :!!! = !! − !! < 0
! − !! Type I error: Rejecting the null hypothesis when it is
!" ! !" ! Apply%a%Z%test,%! ! ≥ !! = ! ! ≥ ~!(0,1) actually not false
!/ !

(26) ! ! distribution and t distribution ⟹ ! = !! !(Type%I%error) = ! ! − ! − (!! − !! )

! !< ~!(0,1)
Chi-squared ! ! distribution If#! ≤ !,!reject&!! .!Otherwise)do)not)reject)!! Type II error: Not rejecting the null hypothesis when it !!! !!!
+
is false !! !!
Let!!! , ! ∈ 1,2. , … !!be#!~!(0,1)! Vice versa for “smaller than” equality
! Type%II%error = ! (Confidence interval)
!
(Two-tailed normal test)
!= !!! + !!! +⋯+ !!! = !!! !Upper%bound%initial%reject − !True
!=! !< !!! !!! !!! !!!
!!! !! :!! = !! !vs#!! :!! ≠ ! !! !/ ! ! − ! − !!/! + , ! − ! + !!/! +
!! !! !! !!
Then!!~!!! !("! ! !distribution*with*k*degrees*of*freedom") Perform'a'two!tailed'Z'test'at!!!level$of$significance Decision made
!! not rejected !! rejected Two normal means with !!! , !!! unknown but equal,
!
! − !! !! 1−! !
!! ≠ !!
Apply%a%Z%test,%! ! ≥ !! = ! ! ≥ ~!(0,1) True state
! ! = ! !!! = !" ! ! = ! !"# ! + ! ! !
=! !! ! 1−!
!/ !
!!!
!! :!!! = !! − !! = 0!vs#!! :!!! = !! − !! < 0
! − !! Power of a test: Probability of rejecting H0 when H1 is
!"# ! = !"#$ ! ! = ![! ! ! − ! ! ! ! ] = 2! ⟹ 2! ! ≥ = 2!! = ! true
!/ ! Perform'a'one!tailed'T'test'at!!!level$of$significance
If!!! !are$independent$random$variables$and!!! ~!!! Determinants for power include: True difference (larger
If#! ≤ !,!reject&!! .!Otherwise)do)not)reject)!! differences in μx-μy, higher power), population !! + !! − 2 ! − ! − (!! − !! )
! != ×
variance (smaller population variance, higher power), 1 1 !!! !! − 1 + !!! (!! − 1)
!! + !! + ⋯ + !! = !!! = !
!~!!!!!⋯!! (One-tailed T test) +
knowledge of population variance (known, higher !! !!
!!! power), sample size (larger n, higher power), design of
!! :!! = !! !vs#!! :!! > !! experiment (paired design, higher power)
! ! !values'can'only'be'positive !~!!!!!!!!
Perform'a'one!tailed'T'test'at!!!level$of$significance (29) Inference for two normal samples
Confidence interval for ! ! !!! ! − 1 + !!! (! − 1) !!! + !!!
If!!! = !! , ! ! = =
! − !! 2(! − 1) 2
Apply%a%T%test,%! ! ≥ !! = ! ! ≥ ~!!!! Two normal means with paired observations, !!! , !!!
! − 1 !! ! − 1 !! ! !/ ! unknown, !! = !! (Unrelated, independent samples)
≤ !! ≤ ~!!!! !! ! − 1 !!! ! − 1 !!!
!!!!/! !!/! …= + !
~!!(!!!)
⟹ !!! ≤ ! ≤ !!! ! − ! = !~ !! , !!! 2 !−1 !! !!
T-distribution
If#! ≤ !,!reject&!! .!Otherwise)do)not)reject)!! !! = !! − !! (Confidence interval)
Let!!~! 0,1 !and!!~!!!
!
(Two-tailed T test) 1 1 1
! !!! = !! − !! − !(! − !) + !2! !! − 1 + !2! (!! − 1)
~!! !−1 !! !!
!! :!! = !! !vs#!! :!! ≠ ! !! !!! ! − ! ± !!/!,!!!!!!!
! !! + !! − 2
!
Perform'a'two!tailed'T'test'at!!!level$of$significance !! :!!! = !! − !! = 0!vs#!! :!!! = !! − !! < 0

! !
7 8
(30) Inference for correlation coefficients and Identifying intervals ! for continuous distributions
!
variances Perform'a'two!tailed'T'test'at'!!level$of$significance !!" − !!"
! !
! !(!!!)(!!!) = !
~!(!!!)(!!!)
Number'of'intervals = !ie.$(!! ≥ 5) !!"
Correlation coefficient !!! 5
! ! > ! ~!!,!!!!,!!!!
!! 100
!"#(!, !) Test if probability of A from X is the same as B from Y
! = !"## !, ! = !"## !, ! = Interval)! = %
! 5
!"# ! !"#(!) Do not write |F| or F !A"from"X + !B"from"Y
!=
! − !! !
!{ ! − ! ! !−! ! } (31) Goodness of fit tests = !!!!.!…
⟹ !
! !! − ! ! ! [! ! ! − ! ! ! ] !!! = !!!!
Test if the distribution fits the data
(32) Contingency tables
Sample correlation coefficient !! : !!" = !!" !vs#!! : !!" ≠ !!"
Category Test if two distributions are independent
1 2 Total
! (Discrete) Perform'a'one!tailed'! ! !test$at$!!level$of$significance
!!!(!! − !)(!! − !)
!=
Category !!"
!
!! − ! ! !
!! − ! ! (−!! , !! ] (−!! , !! ] Total !!" − !!"
!
!!! !!! (Continuous) Category 1 ! !
! !!"!!!!!! = ~!(!!!)(!!!)
1 2 Total !!"
! ! !! !! !! !! = !
!" − !" − !!! !!! !!" !!!
⟹ ! = 1
(33) Introduction to linear regression analysis
! ! ! !
! − 1 ! !!! !!! !! !! !! 1 Category
!! − !! − 2 !!" !!! !!!
! ! 2 Linear regression model
!! !! !! !! = !! = ! Total !!! !!! !
! ≥1
!! = !! + !! !! + !!
Test if 2 samples are correlated !! − !! !! − !! !! − !! !! − !! !!!
!!! = × !!! !! !!
! (!! − !)(!! − !) !! !! −
!! = = !
!! :!! = 0!vs#!! :!! < 0 !! − !! !
!! − !! !
!! − !! ! !! − !! ! ! !! !
(!! − !) !!! −
!! !! !! !! !!" !
Perform'a'one!tailed'T'test'at'!!level$of$significance Category 1
!! !! − !!!
Always'aggregate'!! ≥ 5 1 2 Total ⟹
!−2 !!! − !! !
! !<! ~!!!! 1 !!! !!" !!!
1 − !! !! = !! !!
Category !! = ! − !! !
2 !!" !!! !!!
2
F-distribution (All parameters in PDF known) Total !!! !!! ! Normal linear regression model

Let!!~!!! !and!!~!!! !! : !! = !! = !!! !vs#!! : !! ≠ !! = !!! ! !! = !! + !! !! ⟹ ! ! = !! + !! !

!!" − !!"
!
!/! Perform'a'one!tailed'! !test$at$!!level$of$significance Category 1 !"# !! = ! !
= !~!!,!
!/! !
1 2 Total
!
!! − !! ! ! !! = 0
! !!,!!! = ~!!,!!! 1 !!! − !!! !!" − !!" 0
"F!distribution*with*degrees*of*freedom*(p,k)" !! Category
2 !!" − !!" !!! − !!! 0 !"# !! = ! !!! = ! !
2
! (Not all parameters in PDF known ie. Parameters are Total 0 0 0
! ! = ,"for!! > 2
!−2 estimated from the sample data) ! !! = !!
!
!!" − !!"
!
2! (! + ! − 2) !! : !! = !! = !!! !vs#!! : !! ≠ !! = !!! !!
!"# ! = !!" !"# !! =
! ! − 2 ! (! − 4) !! − ! !
Perform'a'one!tailed'! ! !test$at$!!level$of$significance Category 1
!!
!/! !
= ! !! ~!!,! !! − !! ! 1 2 Total !" !! =
!/! !
! !!,!!!!! = !
~!!,!!!!! !! − ! !
!! !!! − !!! !
!!" − !!" !
1
Test if 2 variances are equal !!! !!"
! !! = !!
Where%! = Number'of'unknown'parameters Category !!" − !!" !
!!! − !!! !
2
!!! !!! 2 !!" !!!
!! !!!
!! :! ! = 1!vs#!! : ! > 1 !! !(!! ) !!" − !!"
!
!"# !! = ×
!! !! E.g.$! = ! ! = =! Total ! !! − ! !
! !!"
Perform'a'one!tailed'T'test'at'!!level$of$significance !
E.g.$Poisson$PDF$with$unknown!!!in#5#categories~!!!!!! !! : !!" = !!! !!! !vs!!! :!!!" ≠ !!! !!! ! !!! 1 !!
!" !! = =! +
!!! ! ! !! − ! ! ! !! − ! !
! ! > ! ~!!,!!!!,!!!! E.g.$Normal$CDF$with$unknown!!, ! ! !in#7#categories~!!!!!! Perform'a'one!tailed'! ! !test$at$!!level$of$significance
!!
! !
9 10
 !
1 !
!! = !! − !! − !! !! !
!−2 Regression)SS Residual)SS
!!! != = 1−
Total&SS Total&SS
(34) Linear regression: Statistical inference
! ! represents'%'of'data'closest'to'the'best'fit'line
(Confidence interval for !! )
Residual)SS
!! − !! !,!!! !" !! , !! + !!/!,!!! !"(!! ) !−2
!!"# = 1−
Total&SS
(Confidence interval for !! ) !−1

!! − !! !,!!! !" !! , !! + !!/!,!!! !"(!! ) (36) Prediction and diagnostics

Test if the slope of a regression model is non-zero Confidence interval for ! ! = !(!)

!! : !! = !!vs!!! :!!! ≠ ! ! ! ! = ! !! + ! !! ! = !! + !! !

Perform'a'one!tailed'T'test'at'!!level$of$significance !! !! − ! !
!"# ! ! = × !
! !! − !
!! − !
! != ~!!!!
!"(!! ) 1 !−! !
1 − !!interval: !! ± !!/!,!!! ! ! +
! !! − ! !
Residual analysis

!
!! = !! − !! + !! !! 1 !
Where%! ! = !! − !! − !! !!
!−2
Plot%!! !against'!! !should'yield'no'discernible!pattern !!!

(35) Linear regression analysis with minitab The confidence interval only covers the true
expectation ! ! = !(!). It does not span the entire !
ANOVA
Predictive interval for ! − ! !
!

Total&SS = !! − ! ! ! !−! ! = ! − !! + !! ! = 0
!!!
!! !! − ! !
! !"# ! − ! ! = !! + × !
! !! − !
Regression)SS = !!! !! − ! !

!!!
1 !−! !
! 1 − !!interval: !! ± !!/!,!!! ! ! 1 + +
! ! !! − ! !
Residual)SS = Total&SS − Regression)SS = !! − !! − !! !!
!!!

! Predictive interval is longer than the confidence

1 ! Residual)SS interval, as it spans the entire !
!! = !! − !! − !! !! =
!−2 !−2
!!!

!
Regression)SS (! − 2)!!! !!!! !! − ! !
!!
!= = =
Residual)SS !! − !! − !! !!
!
!" !!
!−2

F-test for slope of a regression model is non-zero

!! : !! = 0!vs!!! :!!! ≠ 0

Perform'a'one!tailed'F'test'at'!!level$of$significance

!
!!
! != ~!!,!!!
!" !!

Regression correlation coefficient

!
11