Professional Documents
Culture Documents
Definitions:
1 𝑥 𝑥 ⋯ 𝑥 𝑥
𝑦
⎡1 𝑥 𝑥 ⋯ 𝑥 ⎤ ⎡𝑥 ⎤
𝒚≡ ⋮ 𝑿≡⎢
⎢⋮ ⎥
⎥ = [𝟏 𝒙 𝒙 ⋯ 𝒙 ] where 𝒙 ≡ ⎢ ⋮ ⎥
𝑦 ⋮ ⋮ ⋱ ⋮
× ⎣1 𝑥 𝑥 ⋯ 𝑥 ⎦ ⎣𝑥 ⎦ ×
×( + )
𝛽 𝑢
⎡𝛽 ⎤ ⎡𝑢 ⎤
𝜷≡⎢
⎢⋮ ⎥
⎥ 𝒖≡⎢ ⋮ ⎥
⎣𝛽 ⎦( ⎣𝑢 ⎦ ×
+ )×
𝑆 𝜷̂ ≡ 𝑢̂ = 𝒖 𝒖 = 𝒚 − 𝑿𝜷 ̂ 𝒚 − 𝑿𝜷 ̂ = 𝒚 − 𝜷 ̂ 𝑿 𝒚 − 𝑿𝜷 ̂
=
= 𝒚 𝒚 − 𝒚 𝑿𝜷 ̂ − 𝜷 ̂ 𝑿 𝒚 + 𝜷 ̂ 𝑿 𝑿𝜷.̂
The expressions in red are equal scalars. We can thus write them together as −2𝒚 𝑿𝜷 ̂ and re-express 𝑆 𝜷 ̂ :
𝑆 𝜷 ̂ = 𝒚 𝒚 − 2𝒚 𝑿𝜷 ̂ + 𝜷 ̂ 𝑿 𝑿𝜷.̂
𝜕𝑆 𝜷 ̂
=𝟎 ×( + ) ⟹ −2𝒚 𝑿 + 2𝜷 ̂ 𝑿 𝑿 = 𝟎 ×( + ) .
𝜕𝜷 ̂
[Note: By rules of matrix differentiation, 𝜕 −2𝒚 𝑿𝜷 ̂ /𝜕𝜷 ̂ = −2𝒚 𝑿 and 𝜕(𝜷 ̂ 𝑿 𝑿𝜷)/𝜕𝜷
̂ ̂ = 2𝜷 ̂ 𝑿 𝑿.]
𝜷 ̂ 𝑿 𝑿 = 𝒚 𝑿.
𝑿 𝑿𝜷 ̂ = 𝑿 𝒚.
𝜷 ̂ = (𝑿 𝑿)− 𝑿 𝒚 ∎