1

Chapter 2 Intelligent Agents

CS 461 Artificial Intelligence Pinar Duygulu Bil ent !ni"ersity# Spring 2008
Sli$es are %&stly a$apte$ fr&% AI'A

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

(utline
) Agents an$ en"ir&n%ents ) *ati&nality ) P+AS ,Perf&r%ance %easure# +n"ir&n%ent# Actuat&rs# Sens&rs) +n"ir&n%ent types ) Agent types

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

Agents
) An agent is anything that can /e "ie0e$ as
percei"ing its en"ir&n%ent thr&ugh sens&rs an$ acting up&n that en"ir&n%ent thr&ugh actuat&rs Assu%pti&n1 +"ery agent can percei"e its &0n acti&ns ,/ut n&t al0ays the effects-

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

Agents
) 2u%an agent1
eyes# ears# an$ &ther &rgans f&r sens&rs3 han$s#legs# %&uth# an$ &ther /&$y parts f&r actuat&rs

) *&/&tic agent1
ca%eras an$ infrare$ range fin$ers f&r sens&rs3 "ari&us %&t&rs f&r actuat&rs

) A s&ft0are agent1
4eystr& es# file c&ntents# recei"e$ net0&r pac ages as sens&rs Displays &n the screen# files# sent net0&r pac ets as actuat&rs

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

Agents an$ en"ir&n%ents

) Percept1 agent6s perceptual input at any gi"en instant ) Percept se7uence1 c&%plete hist&ry &f e"erything the agent has e"er percei"e$ ) An agent6s ch&ice &f acti&n at any gi"en instant can $epen$ &n the entire percept se7uence &/ser"e$ t& $ate ) An agent6s /eha"i&r is $escri/e$ /y the agent functi&n 0hich %aps fr&% percept hist&ries t& acti&ns1 8f 1 P * A 9 ) :e can i%agine ta/ulating the agent functi&n that $escri/es any gi"en agent ,+;ternal characteri<ati&n) Internally# the agent functi&n 0ill /e i%ple%ente$ /y an agent pr&gra% 0hich runs &n the physical architecture t& pr&$uce f ) agent = architecture > pr&gra%
CS461 Artificial Intelligence Pinar Duygulu Spring 2008

?acuu%@cleaner 0&rl$
) A0& l&cati&ns1 A an$ B ) Percepts1 l&cati&n an$ c&ntents# eBgB# 8A#Dirty9 ) Acti&ns1 Left# Right# Suck# NoOp
Percept se7uence 8A#Clean9 8A# Dirty9 8B#Clean9 8B#Dirty9 8A#Clean9#8A#Clean9 8A#Clean9#8A#Dirty9 D 8A#Clean9#8ABClean9#8A#Clean9 8A#Clean9#8A#Clean9#8A#Clean9 Acti&ns *ight Suc Ceft Suc *ight Suc D *ight Suc

(ne si%ple functi&n is 1 if the current s7uare is $irty then suc # &ther0ise %&"e t& the &ther s7uare
CS461 Artificial Intelligence Pinar Duygulu Spring 2008

*ati&nal agents
) An agent sh&ul$ stri"e t& F$& the right thingF# /ase$ &n 0hat it can percei"e an$ the acti&ns it can perf&r%B ) Ahe right acti&n is the &ne that 0ill cause the agent t& /e %&st successful ) Perf&r%ance %easure1 An &/Gecti"e criteri&n f&r success &f an agentHs /eha"i&r ) +BgB# perf&r%ance %easure &f a "acuu%@cleaner agent c&ul$ /e a%&unt &f $irt cleane$ up# a%&unt &f ti%e ta en# a%&unt &f electricity c&nsu%e$# a%&unt &f n&ise generate$# etcB ) As a general rule# it is /etter t& $esign perf&r%ance %easures acc&r$ing t& 0hat &ne actually 0ants in the en"ir&n%entB *ather than acc&r$ing t& h&0 &ne thin s the agent sh&ul$ /eha"e ,a%&unt &f $irt cleane$ "s a clean fl&&r) A %&re suita/le %easure 0&ul$ re0ar$ the agent f&r ha"ing a clean fl&&r

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

*ati&nality
) :hat is rati&nal at any gi"en ti%e $epen$s &n f&ur things
Ahe perf&r%ance %easure that $efines the criteri&n &f success Ahe agent6s pri&r n&0le$ge &f the en"ir&n%ent Ahe acti&ns that the agent can perf&r% Ahe agent6s percept se7uence t& $ate

*ati&nal Agent1 I&r each p&ssi/le percept se7uence# a rati&nal agent sh&ul$ select an acti&n that is e;pecte$ t& %a;i%i<e its perf&r%ance %easure# gi"en the e"i$ence pr&"i$e$ /y the percept se7uence an$ 0hate"er /uilt@in n&0le$ge the agent hasB

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

?acuu% cleaner agent

) Cet6s assu%e the f&ll&0ing
Ahe perf&r%ance %easure a0ar$s &ne p&int f&r each clean s7uare at each ti%e step# &"er a lifeti%e &f 1000 ti%e steps Ahe ge&graphy &f the en"ir&n%ent is n&0n a pri&ri /ut the $irt $istri/uti&n an$ the initial l&cati&n &f the agent are n&tB Clean s7uares stay clean an$ the suc ing cleans the current s7uareB Ahe Ceft an$ *ight acti&ns %&"e the agent left an$ right e;cept 0hen this 0&ul$ ta e the agent &utsi$e the en"ir&n%ent# in 0hich case the agent re%ains 0here it is Ahe &nly a"aila/le acti&ns are Ceft# *ight# Suc an$ K&(p Ahe agent c&rrectly percei"es its l&cati&n an$ 0hether that l&cati&n c&ntains $irt

) !n$er these circu%stances the agent is rati&nal1its e;pecte$ perf&r%ance is at least as high as any &ther agent6s

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

10

?acuu% cleaner agent

) Sa%e agent 0&ul$ /e irrati&nal un$er $ifferent circu%stances
&nce all $irt is cleane$ up it 0ill &scillate nee$lessly /ac an$ f&rthB If the perf&r%ance %easure inclu$es a penalty &f &ne p&int f&r each %&"e%ent left &r right# the agent 0ill fare p&&rlyB A /etter agent f&r this case 0&ul$ $& n&thing &nce it is sure that all the s7uares are cleanB If the clean s7uares can /ec&%e $irty again# the agent sh&ul$ &ccasi&nally chec an$ clean the% if nee$e$B If the ge&graphy &f the en"ir&n%ent is un n&0n the agent 0ill nee$ t& e;pl&re it rather than stic t& s7uares A an$ B

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

11

*ati&nal agents
) *ati&nality is $istinct fr&% &%niscience ,all@ n&0ing 0ith infinite n&0le$ge) *ati&nality %a;i%i<es e;pecte$ perf&r%ance 0hile perfecti&n %a;i%i<es actual perf&r%ance ) Agents can perf&r% acti&ns in &r$er t& %&$ify future percepts s& as t& &/tain useful inf&r%ati&n ,inf&r%ati&n gathering# e;pl&rati&n) An agent is aut&n&%&us if its /eha"i&r is $eter%ine$ /y its &0n e;perience ,0ith a/ility t& learn an$ a$apt-

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

12

Specifying the tas en"ir&n%ent ,P+AS) P+AS1

Perf&r%ance %easure# En"ir&n%ent# Actuat&rs# Sens&rs

) In $esigning an agent# the first step %ust al0ays /e t& specify the tas en"ir&n%ent ,P+AS- as fully as p&ssi/le

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

1.

P+AS f&r an aut&%ate$ ta;i $ri"er

) Perf&r%ance %easure1 Safe# fast# legal# c&%f&rta/le trip# %a;i%i<e pr&fits ) +n"ir&n%ent1 *&a$s# &ther traffic# pe$estrians# cust&%ers ) Actuat&rs1 Steering 0heel# accelerat&r# /ra e# signal# h&rn ) Sens&rs1 Ca%eras# s&nar# spee$&%eter# LPS# &$&%eter# engine sens&rs# ey/&ar$

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

14

P+AS f&r a %e$ical $iagn&sis syste%

) Perf&r%ance %easure1 2ealthy patient# %ini%i<e c&sts# la0suits ) +n"ir&n%ent1 Patient# h&spital# staff ) Actuat&rs1 Screen $isplay ,7uesti&ns# tests# $iagn&ses# treat%ents# referrals) Sens&rs1 4ey/&ar$ ,entry &f sy%pt&%s# fin$ings# patientHs ans0ers-

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

15

P+AS f&r a satellite i%age analysis syste%

) Perf&r%ance %easure1 c&rrect i%age categ&ri<ati&n ) +n"ir&n%ent1 $&0nlin fr&% &r/iting satellite ) Actuat&rs1 $isplay categ&ri<ati&n &f scene ) Sens&rs1 c&l&r pi;el arrays

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

16

P+AS f&r a part@pic ing r&/&t

) Perf&r%ance %easure1 Percentage &f parts in c&rrect /ins ) +n"ir&n%ent1 C&n"ey&r /elt 0ith parts# /ins ) Actuat&rs1 M&inte$ ar% an$ han$ ) Sens&rs1 Ca%era# G&int angle sens&rs

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

1E

P+AS f&r a refinery c&ntr&ller

) Perf&r%ance %easure1 %a;i%i<e purity# yiel$# safety ) +n"ir&n%ent1 refinery# &perat&rs ) Actuat&rs1 "al"es# pu%ps# heaters# $isplays ) Sens&rs1 te%perature# pressure# che%ical sens&rs

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

18

P+AS f&r Interacti"e +nglish tut&r

) Perf&r%ance %easure1 'a;i%i<e stu$entHs sc&re &n test ) +n"ir&n%ent1 Set &f stu$ents ) Actuat&rs1 Screen $isplay ,e;ercises# suggesti&ns# c&rrecti&ns) Sens&rs1 4ey/&ar$

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

1J

+n"ir&n%ent types
) ) ) ) ) ) Iully &/ser"a/le "sB partially &/ser"a/le Deter%inistic "sB st&chastic +pis&$ic "sB se7uential Static "sB $yna%ic Discrete "sB c&ntinu&us Single agent "sB %ultiagent

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

20

+n"ir&n%ent types
Iully &/ser"a/le "sB partially &/ser"a/le1 ) An en"ir&n%ent is fully &/ser"a/le if an agentHs sens&rs gi"e it access t& the c&%plete state &f the en"ir&n%ent at each p&int in ti%eB ) Iully &/ser"a/le en"ir&n%ents are c&n"enient# /ecause the agent nee$ n&t %aintain any internal state t& eep trac &f the 0&rl$ ) An en"ir&n%ent %ight /e partially &/ser"a/le /ecause &f n&isy an$ inaccurate sens&rs &r /ecause parts &f the state are si%ply %issing fr&% the sens&r $ata ) +;a%ples1 "acuu% cleaner 0ith l&cal $irt sens&r# ta;i $ri"er

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

21

+n"ir&n%ent types
Deter%inistic "sB st&chastic1 ) Ahe en"ir&n%ent is $eter%inistic if the ne;t state &f the en"ir&n%ent is c&%pletely $eter%ine$ /y the current state an$ the acti&n e;ecute$ /y the agentB ) In principle# an agent nee$ n&t 0&rry a/&ut uncertainty in a fully &/ser"a/le# $eter%inistic en"ir&n%ent ) If the en"ir&n%ent is partially &/ser"a/le then it c&ul$ appear t& /e st&chastic ) +;a%ples1 ?acuu% 0&rl$ is $eter%inistic 0hile ta;i $ri"er is n&t ) If the en"ir&n%ent is $eter%inistic e;cept f&r the acti&ns &f &ther agents# then the en"ir&n%ent is strategic

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

22

+n"ir&n%ent types
+pis&$ic "sB se7uential1 ) In epis&$ic en"ir&n%ents# the agentHs e;perience is $i"i$e$ int& at&%ic Fepis&$esF ,each epis&$e c&nsists &f the agent percei"ing an$ then perf&r%ing a single acti&n-# an$ the ch&ice &f acti&n in each epis&$e $epen$s &nly &n the epis&$e itselfB ) +;a%ples1 classificati&n tas s ) In se7uential en"ir&n%ents# the current $ecisi&n c&ul$ affect all future $ecisi&ns ) +;a%ples1 chess an$ ta;i $ri"er

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

2.

+n"ir&n%ent types
Static "sB $yna%ic1 ) Ahe en"ir&n%ent is unchange$ 0hile an agent is $eli/eratingB ) Static en"ir&n%ents are easy t& $eal 0ith /ecause the agent nee$ n&t eep l&& ing at the 0&rl$ 0hile it is $eci$ing &n the acti&n &r nee$ it 0&rry a/&ut the passage &f ti%e ) Dyna%ic en"ir&n%ents c&ntinu&usly as the agent 0hat it 0ants t& $& ) Ahe en"ir&n%ent is se%i@$yna%ic if the en"ir&n%ent itself $&es n&t change 0ith the passage &f ti%e /ut the agentHs perf&r%ance sc&re $&es ) +;a%ples1 ta;i $ri"ing is $yna%ic# chess 0hen playe$ 0ith a cl&c is se%i@$yna%ic# cr&ss0&r$ pu<<les are static

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

24

+n"ir&n%ent types
Discrete "sB c&ntinu&us1 ) A li%ite$ nu%/er &f $istinct# clearly $efine$ states# percepts an$ acti&nsB ) +;a%ples1 Chess has finite nu%/er &f $iscrete states# an$ has $iscrete set &f percepts an$ acti&nsB Aa;i $ri"ing has c&ntinu&us states# an$ acti&ns

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

25

+n"ir&n%ent types
Single agent "sB %ultiagent1 ) An agent &perating /y itself in an en"ir&n%ent is single agent ) +;a%ples1 Cr&ss0&r$ is a single agent 0hile chess is t0&@agents ) Nuesti&n1 D&es an agent A ha"e t& treat an &/Gect B as an agent &r can it /e treate$ as a st&chastically /eha"ing &/Gect ) :hether BHs /eha"i&ur is /est $escri/e$ /y as %a;i%i<ing a perf&r%ance %easure 0h&se "alue $epen$s &n agentHs A /eha"i&ur ) +;a%ples1 chess is a c&%petiti"e %ultiagent en"ir&n%ent 0hile ta;i $ri"ing is a partially c&&perati"e %ultiagent en"ir&n%ent

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

26

+n"ir&n%ent types
Aas +n"ir&n%ent Cr&ss0&r$ pu<<le Chess 0ith a cl&c P& er Bac ga%%&n Aa;i $ri"ing 'e$ical Diagn&sis I%age Analysis Part@pic ing r&/&t *efinery c&ntr&ller Interacti"e +nglish Aut&r (/ser"a/le Deter%inistic +pis&$ic Static Discrete Agents

Iully Iully Partially Iully Partially Partially Iully Partially Partially Partially

Deter%inistic Strategic St&chastic St&chastic St&chastic St&chastic Deter%inistic St&chastic St&chastic St&chastic

Se7uential Se7uential Se7uential Se7uential Se7uential Se7uential +pis&$ic +pis&$ic Se7uential Se7uential

Static Se%i Static Static Dyna%ic Dyna%ic Se%i Dyna%ic Dyna%ic Dyna%ic

Discrete Discrete Discrete Discrete C&ntinu&us C&ntinu&us C&ntinu&us C&ntinu&us C&ntinu&us Discrete

Single 'ulti 'ulti 'ulti 'ulti Single Single Single Single 'ulti

) )

Ahe en"ir&n%ent type largely $eter%ines the agent $esign Ahe real 0&rl$ is ,&f c&urse- partially &/ser"a/le# st&chastic# se7uential# $yna%ic# c&ntinu&us# %ulti@agent
Spring 2008

CS461 Artificial Intelligence Pinar Duygulu

2E

Agent functi&ns an$ pr&gra%s

) An agent is c&%pletely specifie$ /y the agent functi&n %apping percept se7uences t& acti&ns ) (ne agent functi&n ,&r a s%all e7ui"alence classis rati&nal ) Ai%1 fin$ a 0ay t& i%ple%ent the rati&nal agent functi&n c&ncisely @O $esign an agent program
Agent = agent pr&gra% > architecture

) Architecture1 s&%e s&rt &f c&%puting $e"ice 0ith physical sens&rs an$ actuat&rs ,PC# r&/&tic car sh&ul$ /e appr&priate1 0al acti&n re7uires legs
CS461 Artificial Intelligence Pinar Duygulu Spring 2008

28

Agent functi&ns an$ pr&gra%s

) Agent pr&gra%1
Aa es the current percept as input fr&% the sens&rs *eturn an acti&n t& the actuat&rs

:hile agent functi&n ta es the 0h&le percept hist&ry# agent pr&gra% ta es Gust the current percept as input 0hich the &nly a"aila/le input fr&% the en"ir&n%ent Ahe agent nee$ t& re%e%/er the 0h&le percept se7uence# if it nee$s it

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

2J

Aa/le@l&& up agent
) A tri"ial agent pr&gra%1 eeps trac &f the percept se7uence an$ then uses it t& in$e; int& a ta/le &f acti&ns t& $eci$e 0hat t& $& ) Ahe $esigners %ust c&nstruct the ta/le that c&ntains the appr&priate acti&n f&r e"ery p&ssi/le percept se7uence
function AABC+@D*I?+K@AL+KA,percept- returns an acti&n static1 percepts# a se7uence# initially e%pty table# a ta/le &f acti&ns# in$e;e$ /y percept se7uences# initially fully specifie$ appen$ percept t& the en$ &f percepts action P@@C((4!P,percepts, tablereturn action

) Dra0/ac s1 2uge ta/le ,PQA # P1 set &f p&ssi/le percepts# A1 lifeti%e)Space t& st&re the ta/le )Aa e a l&ng ti%e t& /uil$ the ta/le )K& aut&n&%y )+"en 0ith learning# nee$ a l&ng ti%e t& learn the ta/le entries
CS461 Artificial Intelligence Pinar Duygulu Spring 2008

.0

Agent types
) *ather than a ta/le h&0 0e can pr&$uce rati&nal /eha"i&r fr&% a s%all a%&unt &f c&$e ) ) I&ur /asic types in &r$er &f increasing generality1
Si%ple refle; agents '&$el@/ase$ refle; agents L&al@/ase$ agents !tility@/ase$ agents

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

.1

Si%ple refle; agents

) Select acti&ns &n the /asis &f the current percept ign&ring the rest &f the percept hist&ry ) +;a%ple1 si%ple refle; "acuu% cleaner agent
function *+IC+R@?AC!!'@AL+KA,8location,status9- returns an acti&n if status = Dirty then return Suc else if location = A then return Right else if location = B then return Ceft

) Condition-action-rule ) +;a%ple1 if car-in-front-is-breaking then initiatebreaking

CS461 Artificial Intelligence Pinar Duygulu Spring 2008

.2

Si%ple refle; agents

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

..

Si%ple refle; agents

function SI'PC+@*+IC+R@AL+KA,percept- returns an acti&n static: rules, a set if c&n$iti&n@acti&n rules state P@@ IKA+*P*+ASIKP!A,perceptrule P@@ *!C+S'AAC2,state, rulesaction P@@ *!C+SACAI(K8rule9 return action

) Si%ple@refle; agents are si%ple# /ut they turn &ut t& /e &f "ery li%ite$ intelligence ) Ahe agent 0ill 0&r &nly if the c&rrect $ecisi&n can /e %a$e &n the /asis &f the current percept that is &nly if the en"ir&n%ent is fully &/ser"a/le ) Infinite l&&ps are &ften una"&i$a/le escape c&ul$ /e p&ssi/le /y ran$&%i<ing
CS461 Artificial Intelligence Pinar Duygulu Spring 2008

.4

'&$el@/ase$ refle; agents

) Ahe agent sh&ul$ eep trac &f the part &f the 0&rl$ it canHt see n&0 ) Ahe agent sh&ul$ %aintain s&%e s&rt &f internal state that $epen$s &n the percept hist&ry an$ reflects at least s&%e &f the un&/ser"e$ aspects &f the current state ) !p$ating the internal state inf&r%ati&n as ti%e g&es /y re7uires t0& in$s &f n&0le$ge t& /e enc&$e$ in the agent pr&gra% Inf&r%ati&n a/&ut h&0 the 0&rl$ e"&l"es in$epen$ently &f the agent Inf&r%ati&n a/&ut h&0 the agentHs &0n acti&ns affects the 0&rl$ ) '&$el &f the 0&rl$ %&$el /ase$ agents

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

.5

'&$el@/ase$ refle; agents

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

.6

'&$el@/ase$ refle; agents

function *+IC+R@AL+KA@:IA2@SAAA+,percept- returns an acti&n static: state, a $escripti&n &f the current 0&rl$ state rules, a set &f c&n$iti&n@acti&n rules action, the %&st recent acti&n# initially n&ne state P@@ !PDAA+SIKP!A,state, action, perceptrule P@@ *!C+S'AAC2,state, rulesaction P@@ *!C+SACAI(K8rule9 return action

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

.E

L&al@/ase$ agents
) 4n&0ing a/&ut the current state &f the en"ir&n%ent is n&t al0ays en&ugh t& $eci$e 0hat t& $& ,eBgB $ecisi&n at a r&a$ Guncti&n) Ahe agent nee$s s&%e s&rt &f g&al inf&r%ati&n that $escri/es situati&ns that are $esira/le ) Ahe agent pr&gra% can c&%/ine this 0ith inf&r%ati&n a/&ut the results &f p&ssi/le acti&ns in &r$er t& ch&&se acti&ns that achie"e the g&al ) !sually re7uires search an$ planning

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

.8

L&al@/ase$ agents

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

.J

L&al@/ase$ agents "s refle;@/ase$ agents

) Alth&ugh g&al@/ase$ agents appears less efficient# it is %&re fle;i/le /ecause the n&0le$ge that supp&rts its $ecisi&n is represente$ e;plicitly an$ can /e %&$ifie$ ) (n the &ther han$# f&r the refle;@agent# 0e 0&ul$ ha"e t& re0rite %any c&n$iti&n@acti&n rules ) Ahe g&al /ase$ agentHs /eha"i&r can easily /e change$ ) Ahe refle; agentHs rules %ust /e change$ f&r a ne0 situati&n

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

40

!tility@/ase$ agents
) L&als al&ne are n&t really en&ugh t& generate high 7uality /eha"i&r in %&st en"ir&n%ents they Gust pr&"i$e a /inary $istincti&n /et0een happy an$ unhappy states ) A %&re general perf&r%ance %easure sh&ul$ all&0 a c&%paris&n &f $ifferent 0&rl$ states acc&r$ing t& e;actly h&0 happy they 0&ul$ %a e the agent if they c&ul$ /e achie"e$ ) 2appy !tility ,the 7uality &f /eing useful) A utility functi&n %aps a state &nt& a real nu%/er 0hich $escri/es the ass&ciate$ $egree &f happiness

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

41

!tility@/ase$ agents

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

42

Cearning agents
) Auring instea$ &f actually pr&gra%%ing intelligent %achines /y han$# 0hich is t&& %uch 0&r # /uil$ learning %achines an$ then teach the% ) Cearning als& all&0s the agent t& &perate in initially un n&0n en"ir&n%ents an$ t& /ec&%e %&re c&%petent than its initial n&0le$ge al&ne %ight all&0

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

4.

Cearning agents

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008

44

Cearning agents
) Cearning ele%ent resp&nsi/le f&r %a ing i%pr&"e%ents ) Perf&r%ance ele%ent resp&nsi/le f&r selecting e;ternal acti&ns ,it is 0hat 0e ha$ $efine$ as the entire agent /ef&re) Cearning ele%ent uses fee$/ac fr&% the critic &n h&0 the agent is $&ing an$ $eter%ines h&0 the perf&r%ance ele%ent sh&ul$ /e %&$ifie$ t& $& /etter in the future ) Pr&/le% generat&r is resp&nsi/le f&r suggesting acti&ns that 0ill lea$ t& a ne0 an$ inf&r%ati"e e;periences

CS461 Artificial Intelligence Pinar Duygulu

Spring 2008