0 views

Uploaded by Loges Waran

- 10 Chapter 1
- f1
- Network Anomaly Detection and Visualization using Combined PCA and Adaptive Filtering
- Houseofbots.com
- Music genre classification based on local feature selection using a self-adaptive harmony search algorithm
- Literature Assessment for Pose-Invariant Face Recognition
- Use of Gait Energy Image in Implementation of Real Time Video Surveillance System
- Chapter 1
- Machine_Learning_for_Direct_Marketing_Response_Mod.pdf
- Water_Fraud_REPORT.docx
- Puspita_2019_J._Phys.__Conf._Ser._1196_012073
- Multi-Level Dimensionality Reduction Methods Using Feature Selection and Feature Extraction
- svm-2
- gps
- Machine Learning
- CS6659_AI_IA 3
- 200032-Statistics for Business-Final Exam-SPRING 2018
- East North Central Ft Workers
- Week 3 Example 1.6 Mean variance stdev grouped data.xlsx
- CS273 ML StockPricePrediction

You are on page 1of 27

INTRODUCTION

TO

MACHINE

LEARNING

3RD EDITION

ETHEM ALPAYDIN

© The MIT Press, 2014

alpaydin@boun.edu.tr

http://www.cmpe.boun.edu.tr/~ethem/i2ml3e

CHAPTER 13:

KERNEL MACHINES

Kernel Machines

3

first

Define the discriminant in terms of support vectors

The use of kernel functions, application-specific

measures of similarity

No need to represent instances as vectors

Convex optimization problems with a unique solution

Optimal Separating Hyperplane

4

if C1

X x , r t where r

t

t t t 1 x

1 if x t

C2

find w and w0 such that

w T xt w0 1 for r t 1

w T xt w0 1 for r t 1

which can be rewritten as

r t w T xt w0 1

Margin

5

on either side

Distance of x to the hyperplane is w x w0

T t

w

r t w T xt w0

We require , t

w

1 2

2

Margin

6

min w subject to r t w T xt w 0 1, t

1 2

2

Lp w t r t w T xt w 0 1

N

1 2

2 t 1

w r w x w 0 t

N N

1 2 t t T t

2 t 1 t 1

Lp N

0 w t r t xt

w t 1

Lp N

0 t r t 0

w 0 t 1

7

Ld w w w T t r t xt w0 t r t t

1 T

2 t t t

w w t

1 T

2 t

r r x x t

1 t s t s t T s

2 t s t

subject to t r t 0 and t 0, t

t

Most αt are 0 and only a small number have αt >0; they are

the support vectors

8

Soft Margin Hyperplane

9

r t wT x t w0 1 t

Soft error

t

t

New primal is

1

2

2

Lp w C t t t t r t wT x t w0 1 t t t t

10

Hinge Loss

11

0 if y t r t 1

Lhinge(y , r )

t t

1 y t t

r otherwise

n-SVM

12

1 1

min w - n t

2

2 N t

subject to

r t w T xt w 0 t , t 0, 0

Ld r r x x

1 N t s t s t T s

2 t 1 s

subject to

1

t r

t t

0 ,0 t

,

N t

t

n

Kernel Trick

13

z = φ(x) g(z)=wTz

g(x)=wT φ(x)

The SVM solution

w t r t z t t r t φxt

t t

T t t

φx

t T

gx t r t K xt , x

t

Vectorial Kernels

14

Polynomials of degree q:

K x , x x x 1

t T t q

K x, y xT y 1

2

x1y1 x 2 y 2 12

1 2 x1y1 2 x 2 y 2 2 x1 x 2 y1y 2 x12 y12 x 22 y 22

x 1, 2 x1 , 2 x 2 , 2 x1 x 2 , x , x 2

1

2 T

2

Vectorial Kernels

15

Radial-basis functions:

xt x 2

K xt , x exp

2s 2

Defining kernels

16

Kernel “engineering”

Defining good measures of similarity

String kernels, graph kernels, image kernels, ...

Empirical kernel map: Define a set of templates mi

and score function s(x,mi)

(xt)=[s(xt,m1), s(xt,m2),..., s(xt,mM)]

and

K(x,xt)= (x)T (xt)

Multiple Kernel Learning

17

K x, y K1 x, y K 2 x, y

K x, y K x, y

1 2

m

K x, y i K i x, y

i 1

t s r t r s i K i xt , x s

1

Ld t

t 2 t s i

t i

t i

Multiclass Kernel Machines

18

1-vs-all

Pairwise separation

Error-Correcting Output Codes (section 17.5)

Single multiclass optimization

1 K

min w i C it

2

2 i 1 i t

subject to

w zt T xt w zt 0 w i T xt wi 0 2 it , i z t , it 0

SVM for Regression

19

f(x)=wTx+w0

Use the є-sensitive error function

if r t f xt

e r , f x t

t t 0

r f x t

otherwis e

min w C t t

1 2

2

t

r t w T x w0 t

w x w r

T

0

t

t

t , t 0

20

Kernel Regression

21

Kernel Machines for Ranking

22

but at least +1 unit margin.

Linear case:

1

min w i C it

2

2 t

subject to

w T xu w T xv 1 t , t : r u r v , it 0

One-Class Kernel Machines

23

min R 2 C t

t

subject to

x t a R 2 t , t 0

Ld x x r r x x

N

t t T s t s t s t T s

t t 1 s

subject to

0 t C , t 1

t

24

Large Margin Nearest Neighbor

25

D(xi, xj)=(xi-xj)TM(xi-xj)

For three instances i, j, and l, where i and j are of

the same class and l different, we require

D(xi, xl) > D(xi, xj)+1

and if this is not satisfied, we have a slack for the

difference and we learn M to minimize the sum of

such slacks over all i,j,l triples (j and l being one of k

neighbors of i, over all i)

Learning a Distance Measure

26

similar approach where M=LTL and learns L

Kernel Dimensionality Reduction

27

PCA on the

kernel matrix

(equal to

canonical PCA

with a linear

kernel)

Kernel LDA, CCA

- 10 Chapter 1Uploaded byIbrahim Yahaya
- f1Uploaded bySrinivas Ambala
- Network Anomaly Detection and Visualization using Combined PCA and Adaptive FilteringUploaded byijcsis
- Houseofbots.comUploaded byabbas91
- Music genre classification based on local feature selection using a self-adaptive harmony search algorithmUploaded byAfonso Palandri
- Literature Assessment for Pose-Invariant Face RecognitionUploaded byijsret
- Use of Gait Energy Image in Implementation of Real Time Video Surveillance SystemUploaded byInternational Organization of Scientific Research (IOSR)
- Chapter 1Uploaded byHimanshu Nimje
- Machine_Learning_for_Direct_Marketing_Response_Mod.pdfUploaded byTomás Vallejo Mora
- Water_Fraud_REPORT.docxUploaded byjyothibg
- Puspita_2019_J._Phys.__Conf._Ser._1196_012073Uploaded byPokerto Java
- Multi-Level Dimensionality Reduction Methods Using Feature Selection and Feature ExtractionUploaded byAdam Hansen
- svm-2Uploaded byFlavio L M Barboza
- gpsUploaded byOmar Awale
- Machine LearningUploaded byNishantRaj
- CS6659_AI_IA 3Uploaded byjanu13
- 200032-Statistics for Business-Final Exam-SPRING 2018Uploaded byAdam Rae
- East North Central Ft WorkersUploaded byUma Shankar
- Week 3 Example 1.6 Mean variance stdev grouped data.xlsxUploaded byRIFKI ARIWARDI
- CS273 ML StockPricePredictionUploaded byEvenCheng
- Aisc 2016Uploaded byCS & IT
- Lecture 01Uploaded byAditya Sreekar
- Build a Neural Network in Python _ EnlightUploaded byPedro Elias Romero Nieto
- Neural networkUploaded byAshish Tiwari
- beta manag.xlsUploaded byDinesh Moorthy
- FIXED LAMPIRAN.docUploaded byAdityaRamadhana
- Factor Analysis GurvinderUploaded bygurvinder12
- WatsonExplorer WatsonKnowledgeStudio Integration v1101Uploaded byekatonb
- Football betting secrets.pdfUploaded byJulius Ogowewo
- shortestPathKernel_IEUploaded byNguyen T A Nhon

- CTS,CLS,CLR.pdfUploaded byLoges Waran
- dsame_chap1-introduction.pdfUploaded byLoges Waran
- MICAI2015_EFIM_High_Utility_Itemset_Mining.pdfUploaded byLoges Waran
- DATA vivaUploaded byLoges Waran
- Introduction to Artificial IntelligenceUploaded byLoges Waran
- Class and ObjectUploaded byLoges Waran
- New Text DocumentUploaded byLoges Waran
- Tata Infotech netUploaded bygeethikachoudhary
- WK4 - BitStuffingUploaded byLoges Waran
- Inter M Board E210882Uploaded byFrankmorel

- Computer Architecture - Memory SystemUploaded byamit_coolbuddy20
- Invitation Letter BnhsUploaded byjohn paulo gaa
- Experimental investigation of the thermal performances of an extensive green roof in the Mediterranean areaUploaded bySantiago Sostenible
- Birth PlanUploaded byjessestew
- Sdi011.Manual.ver1.05Uploaded byMaximchick
- businees environment conceptsUploaded bysadafahmed007
- Alex a Deo Resume 2018Uploaded byAnonymous NYMlovW
- réservation system for HotelUploaded bymichael
- Change Bank Details FormUploaded byMustafa Bapai
- Prevention of MoneyUploaded byRajesh Govardhan
- R4G-20-STE-CON-COS-208Uploaded bySantoshYadav
- unit 6 exam 2ESO burlington bookUploaded byBenilde Bastida Duran
- Deep Photo StyleUploaded byIamIN
- Guide to Dairy Farming 2Uploaded byBaka Yaro
- The Carrot and Stick ApproachUploaded byJustynaBorowska
- EM -I LAB 2008 REGUploaded byJeevi Tha
- FireNet Series Battery Calculator V3.057Uploaded bymujahid_islam85
- 830-03983-03Uploaded bysurvieGuide
- Nokia E52 Service manual L1&L2Uploaded byJuhanaTuomisto
- Camaras Ahd GenesisUploaded byDairo Salgado
- 9781783984909_Blender_3D_Basics_Beginner's_Guide_Second_Edition_Sample_ChapterUploaded byPackt Publishing
- Carbaryl a Pesticide Causes Reproductive Toxicity in Albino Rats 2161 0681.1000126Uploaded byjhika1304
- HRUploaded by1l0veu
- 128100 Paper Based Exam DocumentUploaded byYorymao
- 01-ArduinoIntroUploaded byranas837
- Gender EnglischUploaded byLoreto Vidal
- Car Park & Landscape DesignUploaded byAdlan Zahriman
- HARI Resume.doc_(1) - CopyUploaded byshivamawasthi7
- The chemistry of spicesUploaded bylordregulus
- Name of drug.docxUploaded byAngelica Mercado Sirot