0 views

Uploaded by Richard Pines

- 07056_2_19_maths
- W4 Revision Maths F4
- 150156895 Introduction to Finite Element Vibration Analysis PDF
- Initial Bounds on the Relative Magnitude of Ratio Averages
- Sample Calculations
- 408 Variance Sol
- 18.112-John B. Conway Functions of One Complex Variable II Graduate Texts in Mathematics Pt. 2 1995
- Solutions of Shakuntala Devi Book
- 682871.pdf
- Math_121_April_2009
- MMSE Equalizer Part 2.pdf
- Math C2 2008 June Question
- p256.pdf
- C3 June 2014
- Book Announcements
- Final ECSE_Class Schedule_Fall 2018-
- 10.1007_s10489-018-1178-2.pdf
- UT Dallas Syllabus for stat3360.501 06s taught by Grigory Kramer (gkramer)
- Scribd 1 - Copy (5)
- Procedure for Pricing Bermudans and Callable Swaps

You are on page 1of 2

distributions by ρ0 ( x ) = p( x |Y = 0), ρ1 ( x ) = p( x |Y = 1). The Bayes optimal classifier is

(

1, ρ1 ( x ) > ρ0 ( x )

f ∗ (x) =

0, otherwise.

A plug-in classifier is (

1, ρe1 ( x ) > ρe0 ( x )

f (x) =

0, otherwise.

We will use the following upper bound for the classification error. For a proof see next page.

Lemma 1.

1 1

Z Z

E[1{ f ( X ) 6= Y }] − E[1{ f ∗ ( X ) 6= Y }] ≤ |ρ0 ( x ) − ρe0 ( x )| + |ρ1 ( x ) − ρe1 ( x )|.

2 2

Now, our classifier first estimates θ, and then approximates the class conditional distributions using

θ̂. Therefore, ρe0 ( x ) is the density of N (−θ̂, σ2 I) and ρe1 ( x ) is the density of N (θ̂, σ2 I). Using Pinsker’s

inequality we have

s

1 Dkl N (θ, σ2 I)kN (θ̂, σ2 I) kθ − θ̂ k2

Z

|ρ0 ( x ) − ρe0 ( x )|dx ≤ =

2 2 2σ

Similarly,

1 kθ − θ̂ k2

Z

|ρ1 ( x ) − ρe1 ( x )|dx ≤

2 2σ

Therefore, we obtain

kθ − θ̂ (( X1 , Y1 ), . . . , ( Xn , Yn ))k2

E[1{ f ( X ) 6= Y }] − E[1{ f ∗ ( X ) 6= Y }] ≤ E(X1 ,Y1 ),...,(X1 ,Yn ) . (1)

2σ

σ2 I

Now we have E[θ̂ ] = θ and Cov[θ̂ ] = n .

dσ2

= tr(E[((θ̂ − θ )(θ̂ − θ )T )]) = tr(σ2 I) =

n

By Jensen’s inequality we get r

d

q

E[kθ̂ − θ k2 ] ≤ E[kθ̂ − θ k22 ] =σ (2)

n

Combining (1) and (2) we obtain

r

∗ 1 d

E[1{ f ( X ) 6= Y }] − E[1{ f ( X ) 6= Y }] ≤

2 n

and finally using the Bayes error that we computed in class we have

r

σ2 1 d

E[1{ f ( X ) 6= Y }] ≤ +

kθ k22 2 n

1

Lemma 2.

1 1

Z Z

E[1{ f ( X ) 6= Y }] − E[1{ f ∗ ( X ) 6= Y }] ≤ |ρ0 ( x ) − ρe0 ( x )| + |ρ1 ( x ) − ρe1 ( x )|.

2 2

Proof. For any classifier f we have

1 1

Z Z

E[1{ f ( X ) 6= Y }] = 1{ f ( x ) = 1}ρ0 ( x )dx + 1{ f ( x ) = 0}ρ1 ( x )dx

2 2

1 1

Z Z

= 1{ f ( x ) = 1}ρ0 ( x )dx + (1 − 1{ f ( x ) = 1})ρ1 ( x )dx

2 2

1 1 1

Z Z

= + 1{ f ( x ) = 1}ρ0 ( x )dx − 1{ f ( x ) = 1})ρ1 ( x )dx

2 2 2

1 1

Z

= + 1{ f ( x ) = 1}(ρ0 ( x ) − ρ1 ( x ))dx

2 2

Therefore, for the classifiers f , f ∗ we have that

1

Z

E[1{ f ( X ) 6= Y }] − E[1{ f ∗ ( X ) 6= Y }] = (1{ f ( x ) = 1} − 1{ f ∗ ( x ) = 1})(ρ0 ( x ) − ρ1 ( x ))dx

2

Now we prove that

and ρe0 ( x ) < ρe1 ( x ). Therefore, the LHS of inequality (3) is

- 07056_2_19_mathsUploaded bycaspross
- W4 Revision Maths F4Uploaded bybidarihassan
- 150156895 Introduction to Finite Element Vibration Analysis PDFUploaded byHonorato Maria B. Ana
- Initial Bounds on the Relative Magnitude of Ratio AveragesUploaded byTI Journals Publishing
- Sample CalculationsUploaded byLaura Andrews
- 408 Variance SolUploaded byLi Nguyen
- 18.112-John B. Conway Functions of One Complex Variable II Graduate Texts in Mathematics Pt. 2 1995Uploaded bySandys Nunes
- Solutions of Shakuntala Devi BookUploaded byAyush Jain
- 682871.pdfUploaded byJOY JOY GASGA
- Math_121_April_2009Uploaded byzengrz
- MMSE Equalizer Part 2.pdfUploaded byfateruler51
- Math C2 2008 June QuestionUploaded byChristy Yeung
- p256.pdfUploaded by123Super
- C3 June 2014Uploaded byDon Wilson
- Book AnnouncementsUploaded byRitesh Singh
- Final ECSE_Class Schedule_Fall 2018-Uploaded byShakil
- 10.1007_s10489-018-1178-2.pdfUploaded byRituparna Chutia
- UT Dallas Syllabus for stat3360.501 06s taught by Grigory Kramer (gkramer)Uploaded byUT Dallas Provost's Technology Group
- Scribd 1 - Copy (5)Uploaded by-Nicky Khoo-
- Procedure for Pricing Bermudans and Callable SwapsUploaded byalexandergir
- 04 Exponential PoissonUploaded byFadila Nastiti
- Dr Edward CraneUploaded byDelger Amar
- Tclw CalculatorUploaded bywlexxx
- scholarschallenge4Uploaded byapi-255402067
- Trial Math SpmUploaded byJamzury Rasid
- CKP 10 ClosureUploaded byPraveen Avupati
- ma3215_hw2_soln.pdfUploaded byPei Jing
- Arithmetic CodingUploaded byRakesh Inani
- Folio Addmath 2014(Shared)Uploaded byKing_Huzaifah
- pretest-110-b.pdfUploaded byrayza miral

- packing listUploaded byapi-411840618
- Adding a Node to a 10g RAC Cluster.pdfUploaded byPraveen Kumar
- STO Configuration StepsUploaded bybeema1977
- A Philosophy of Cinematic ArtUploaded bybluemlein
- Anestesi propofol jurnalUploaded byShankara Pillai
- Momentum Collision ImpulseUploaded byYasin Dereli
- Job Ad AssignmentUploaded byRussell Reeves
- Genocide of One - Kazuaki Takano.epubUploaded bydarry
- A Crime is Different From a TortUploaded byMissPardis
- MCB Internship ReportUploaded byAdeelBaig
- 41. International Endodontic Journal.pdfUploaded byRonald Huanca Cano
- Purification of ProteinsUploaded byArrianne Jaye Mata
- Industrial ApplicationUploaded byJustin Marc Estipona
- So No FusionUploaded byVincent J. Cataldi
- Science FictionUploaded byEdgar Mora Quiros
- Professional Ethics Question BankUploaded byJayagopal Jai
- Neuron Communication LectureUploaded byJennifer Isom Schmidtke, Ph.D.
- glen rose 525 cornerstone listUploaded byapi-262868841
- ssntd5Uploaded byamyounis
- Federal ruling blocking casino in East WindsorUploaded byPatrick Johnson
- Vol. 4 No.4Uploaded byjulialeo
- Pottery Analysis: A Sourcebook. by Prudence RiceUploaded byJulian Trovero
- jan 12 if the rosholt house walls could talkUploaded byapi-190819948
- 991111_EUS教學(6)重點式急診心臟超音波之應用Uploaded byjuice119
- Raas BuiltUploaded bySingaravelu Subramanian
- The Greek TriumvirateUploaded byGene
- Nelson_Framing Effects on Values, Emotions and AttitudesUploaded byDiana Bonilla
- Heptameron M. Od Navare i Cveće ZlaUploaded byAleksandra Đurđević
- AFGE Local 1858 and Army Aviation and Missile Command 3-4-03Uploaded byE Frank Cornelius
- 28.8 Vina S s35.246 Piya TanUploaded bypancakhanda