AgentsandEnvironments

Agents: An agent is embodied (i.e. situated) in an environment andmakesitowndecisions. It perceives the environment through sensors and acts on the environment through actuators(effectors).

Structureofagents
Asimpleagentprogramcanbedefinedmathematicallyasanagentfunctionwhichmaps everypossibleperceptssequencetoapossibleactiontheagentcanperformortoa coefficient,feedbackelement,functionorconstantthataffectseventualactions: Agentfunctionisanabstractconceptasitcouldincorporatevariousprinciplesofdecision makinglikecalculationofutilityofindividualoptions,deductionoverlogicrules,fuzzy logic,etc. Theprogramagent,instead,mapseverypossiblepercepttoanaction. Weusethetermpercepttorefertotheagent'sperceptionalinputsatanygiveninstant.Inthe followingfiguresanagentisanythingthatcanbeviewedasperceivingitsenvironment throughsensorsandactinguponthatenvironmentthroughactuators.

Classesofintelligentagents
Agentsaregroupedintofiveclassesbasedontheirdegreeofperceivedintelligenceand capability: 1. simplereflexagents 2. modelbasedreflexagents 3. goalbasedagents 4. utilitybasedagents 5. learningagents

Simplereflexagents Simplereflexagentsactonlyonthebasisofthecurrentpercept,ignoringtherestofthe percepthistory.Theagentfunctionisbasedontheconditionactionrule:ifconditionthen action. Thisagentfunctiononlysucceedswhentheenvironmentisfullyobservable.Somereflex agentscanalsocontaininformationontheircurrentstatewhichallowsthemtodisregard conditionswhoseactuatorsarealreadytriggered. Infiniteloopsareoftenunavoidableforsimplereflexagentsoperatinginpartiallyobservable environments.Note:Iftheagentcanrandomizeitsactions,itmaybepossibletoescape frominfiniteloops. However,wecansummarizeportionsofthetablebynotingcertaincommonlyoccurring input/outputassociations.Forexample,ifthecarinfrontbrakes,anditsbrakelightscome on, thenthedrivershouldnoticethisandinitiatebraking.Inotherwords,someprocessingisdone on thevisualinputtoestablishtheconditionwecallThecarinfrontisbrakingthenthis triggers someestablishedconnectionintheagentprogramtotheactioninitiatebraking.Wecall such aconnectionaconditionactionrulei.e.ifcarinfrontisbrakingtheninitiatebraking Modelbasedreflexagents Amodelbasedagentcanhandleapartiallyobservableenvironment.Itscurrentstateis storedinsidetheagentmaintainingsomekindofstructurewhichdescribesthepartofthe worldwhichcannotbeseen.Thisknowledgeabout"howtheworldworks"iscalledamodel oftheworld,hencethename"modelbasedagent". Amodelbasedreflexagentshouldmaintainsomesortofinternalmodelthatdependsonthe percepthistoryandtherebyreflectsatleastsomeoftheunobservedaspectsofthecurrent state.Itthenchoosesanactioninthesamewayasthereflexagent.

Goalbasedagents Goalbasedagentsfurtherexpandonthecapabilitiesofthemodelbasedagents,byusing "goal"information.Goalinformationdescribessituationsthataredesirable.Thisallowsthe agentawaytochooseamongmultiplepossibilities,selectingtheonewhichreachesagoal state.Searchandplanningarethesubfieldsofartificialintelligencedevotedtofindingaction sequencesthatachievetheagent'sgoals. Forexample,ataroadjunction,thetaxicanturnleft,right,orgostraighton.Theright decision dependsonwherethetaxiistryingtogetto.Inotherwords,aswellasacurrentstate description,theagentneedssomesortofgoalinformation,whichdescribessituationsthat aredesirabledescription,forexample,beingatthepassengersdestination.Theagent programcancombinethiswithinformationabouttheresultsofpossibleactions(thesame informationaswasusedtoupdateinternalstateinthereexagent)inordertochoose actionsthatachievethegoal. Insomeinstancesthegoalbasedagentappearstobeefficientitismoreflexiblebecause theknowledgethatsupportsitsdecisionsisrepresentedexplicitlyandcanbemodified.

Utilitybasedagents Goalbasedagentsonlydistinguishbetweengoalstatesandnongoalstates.Itispossibleto defineameasureofhowdesirableaparticularstateis.Thismeasurecanbeobtained throughtheuseofautilityfunctionwhichmapsastatetoameasureoftheutilityofthestate. Amoregeneralperformancemeasureshouldallowacomparisonofdifferentworldstates accordingtoexactlyhowhappytheywouldmaketheagent.Thetermutility,canbeusedto describehow"happy"theagentis. Arationalutilitybasedagentchoosestheactionthatmaximizestheexpectedutilityofthe actionoutcomesthatis,theagentexpectstoderive,onaverage,giventheprobabilitiesand utilitiesofeachoutcome.Autilitybasedagenthastomodelandkeeptrackofits environment,tasksthathaveinvolvedagreatdealofresearchonperception,representation, reasoning,andlearning. Goalsalonearenotreallyenoughtogeneratehighqualitybehavior.Forexample,thereare manyactionsequencesthatwillgetthetaxitoitsdestination,therebyachievingthegoal,but somearequicker,safer,morereliable,orcheaperthanothers.Goalsjustprovideacrude distinctionbetweenhappyandunhappystates,whereasamoregeneralperformance measureshouldallowacomparisonofdifferentworldstates(orsequencesofstates) accordingtoexactlyhowhappytheywouldmaketheagentiftheycouldbeachieved. Becausehappydoesnotsound.

Learningagents Learninghasanadvantagethatitallowstheagentstoinitiallyoperateinunknown environmentsandtobecomemorecompetentthanitsinitialknowledgealonemightallow. Themostimportantdistinctionisbetweenthe"learningelement",whichisresponsiblefor makingimprovements,andthe"performanceelement",whichisresponsibleforselecting externalactions. Thelearningelementusesfeedbackfromthe"critic"onhowtheagentisdoingand determineshowtheperformanceelementshouldbemodifiedtodobetterinthefuture.The performanceelementiswhatwehavepreviouslyconsideredtobetheentireagent:ittakesin perceptsanddecidesonactions. Thelastcomponentofthelearningagentisthe"problemgenerator".Itisresponsiblefor suggestingactionsthatwillleadtonewandinformativeexperiences.

Environments: Environmentscomeinseveralflavors.Theprincipaldistinctionstobemadeareasfollows: Accessiblevs.inaccessible:Ifanagentssensoryapparatusgivesitaccesstothecomplete stateoftheenvironment,thenwesaythattheenvironmentisaccessibletothatagent.An environmentiseffectivelyaccessibleifthesensorsdetectallaspectsthatarerelevanttothe choiceofaction.Anaccessibleenvironmentisconvenientbecausetheagentneednot maintainanyinternalstatetokeeptrackoftheworld. Deterministicvs.nondeterministic:Ifthenextstateoftheenvironmentiscompletely determinedbythecurrentstateandtheactionsselectedbytheagents,thenwesaythe environmentisdeterministic.Inprinciple,anagentneednotworryaboutuncertaintyinan accessible,deterministicenvironment.Iftheenvironmentisinaccessible,however,thenit mayappeartobenondeterministic.Thisisparticularlytrueiftheenvironmentiscomplex, makingithardtokeeptrackofalltheinaccessibleaspects.Thus,itisoftenbettertothinkof anenvironmentasdeterministicornondeterministicfromthepointofviewoftheagent. Episodicvs.nonepisodic:Inanepisodicenvironment,theagentsexperienceisdividedinto episodes.Eachepisodeconsistsoftheagentperceivingandthenacting.Thequalityofits actiondependsjustontheepisodeitself,becausesubsequentepisodesdonotdependon whatactionsoccurinpreviousepisodes.Episodicenvironmentsaremuchsimplerbecause theagentdoesnotneedtothinkahead. Staticvs.dynamic:Iftheenvironmentcanchangewhileanagentisdeliberating,thenwesay theenvironmentisdynamicforthatagentotherwiseitisstatic.Staticenvironmentsareeasy todealwithbecausetheagentneednotkeeplookingattheworldwhileitisdecidingonan action, norneeditworryaboutthepassageoftime.Iftheenvironmentdoesnotchangewiththe passageoftimebuttheagentsperformancescoredoes,thenwesaytheenvironmentis semidynamic. Discretevs.continuous:Iftherearealimitednumberofdistinct,clearlydefinedperceptsand actionswesaythattheenvironmentisdiscrete.Chessisdiscretethereareafixednumber ofpossiblemovesoneachturn.Taxidrivingiscontinuousthespeedandlocationofthetaxi andtheothervehiclessweepthrougharangeofcontinuousvalues. Examplesofenvironmentsandtheircharacteristics. EnvironmentAccessibleDeterministicEpisodicStaticDiscrete ChesswithaclockYesYesNoSemiYes ChesswithoutaclockYesYesNoYesYes PokerNoNoNoYesYes BackgammonYesNoNoYesYes TaxidrivingNoNoNoNoNo MedicaldiagnosissystemNoNoNoNoNo ImageanalysissystemYesYesYesSemiNo PartpickingrobotNoNoYesNoNo RefinerycontrollerNoNoNoNoNo InteractiveEnglishtutorNoNoNoNoYes