Chapter 2
What is an (Intelligent) Agent?
• An over-used, over-loaded, and misused term.
• Anything that can be viewed as perceiving its environment
through sensors and acting upon that environment through its
effectors to maximize progress towards its goals.
• PAGE (Percepts, Actions, Goals, Environment)
• Task-specific & specialized: well-defined goals and
environment
• The notion of an agent is meant to be a tool for analyzing
systems,
• It is not a different hardware or new programming
languages
Intelligent Agents and Artificial Intelligence
• Example: Human mind as network of thousands or millions
of agents working in parallel. To produce real artificial
intelligence, this school holds, we should build computer
systems that also contain many agents and systems for
arbitrating among the agents' competing results.
• Distributed decision-making
and control
• Challenges:
• Action selection: What next action to choose
• Conflict resolution
Agent Types
We can split agent research into two main strands:
• Distributed Artificial Intelligence (DAI) –
Multi-Agent Systems (MAS) (1980 – 1990)
• Much broader notion of "agent" (1990’s – present)
• interface, reactive, mobile, information
Rational Agents
A Windshield Wiper Agent
• Goals: Keep windshields clean & maintain visibility
• Percepts: Raining, Dirty
• Sensors: Camera (moist sensor)
• Effectors: Wipers (left, right, back)
• Actions: Off, Slow, Medium, Fast
• Environment: Inner city, freeways, highways, weather
Interacting Agents
Conflict Resolution by Action Selection Agents
• Override: CAA overrides LKA
• Arbitrate: if Obstacle is Close then CAA
else LKA
• Compromise: Choose action that satisfies both
agents
• Any combination of the above
• Challenges: Doing the right thing
The Right Thing = The Rational Action
• Rational Action: The action that maximizes the expected
value of the performance measure given the percept sequence
to date
• Rational = Best Yes, to the best of its knowledge
• Rational = Optimal Yes, to the best of its abilities (incl. its
constraints)
• Rational ≠ Omniscience
• Rational ≠ Clairvoyant
• Rational ≠ Successful
Behavior and performance of IAs
• Perception (sequence) to Action Mapping: f : P* → A
• Ideal mapping: specifies which actions an agent ought to
take at any point in time
• Description: Look-Up-Table, Closed Form, etc.
• Performance measure: a subjective measure to
characterize how successful an agent is (e.g., speed,
power usage, accuracy, money, etc.)
• (degree of) Autonomy: to what extent is the agent able to
make decisions and take actions on its own?
Look up table
Closed form
• Output (degree of rotation) = F(distance)
• E.g., F(d) = 10/d (distance cannot be less than 1/10)
How is an Agent different from other software?
• Agents are autonomous, that is, they act on behalf of the user
• Agents contain some level of intelligence, from fixed rules to
learning engines that allow them to adapt to changes in the
environment
• Agents don't only act reactively, but sometimes also
proactively
• Agents have social ability, that is, they communicate with the • Reflex agents with internal states
user, the system, and other agents as required • Goal-based agents
• Agents may also cooperate with other agents to carry out more • Utility-based agents
complex tasks than they themselves can handle • Reflex agents
• Agents may migrate from one system to another to access • Reactive: No memory
remote resources or even to meet other agents • Reflex agents with internal states
Environment Types • W/o previous state, may not be able to make decision
• Characteristics • E.g. brake lights at night.
• Accessible vs. inaccessible • Goal-based agents
• Sensors give access to complete state of the • Goal information needed to make decision
environment. • Utility-based agents
• Deterministic vs. nondeterministic • How well can the goal be achieved (degree of happiness)
• The next state can be determined based on the current • What to do if there are conflicting goals?
state and the action. • Speed and safety
• Episodic vs. nonepisodic (Sequential) • Which goal should be selected if several can be achieved?
• Episode: each perceive and action pairs Reflex agents
• The quality of action does not depend on the previous
episode.
• Hostile vs. friendly
• Static vs. dynamic
• Dynamic if the environment changes during deliberation
• Discrete vs. continuous
• Chess vs. driving

Structure of Intelligent Agents
• Agent = architecture + program
• Agent program: the implementation of f : P* → A, the
agent’s perception-action mapping
function Skeleton-Agent(Percept) returns Action
memory ← UpdateMemory(memory, Percept)
Action ← ChooseBestAction(memory)
memory ← UpdateMemory(memory, Action)
return Action
• Architecture: a device that can execute the agent program
(e.g., general-purpose computer, specialized device, beobot,
etc.)
Using a look-up-table to encode f : P* → A
Reactive agents
• Reactive agents do not have internal symbolic models.
• Act by stimulus-response to the current state of the
environment.
• Each reactive agent is simple and interacts with others in a
basic way.
• Complex patterns of behavior emerge from their interaction.
• Benefits: robustness, fast response time
• Challenges: scalability, how intelligent? and how do you
debug them?
Reflex agents w/ state

Goal-based agents

Agent types
• Reflex agents

Utility-based agents
Mobile agents
• Programs that can migrate from one machine to another.
• Execute in a platform-independent execution environment.
• Require agent execution environment (places).
• Mobility not necessary or sufficient condition for agenthood.
• Practical but non-functional advantages:
• Reduced communication cost (eg, from PDA)
• Asynchronous computing (when you are not connected)
• Two types:
• One-hop mobile agents (migrate to one other place)
• Multi-hop mobile agents (roam the network from place to
place)
• Applications:
• Distributed information retrieval.
• Telecommunication network routing.
• Programs that can migrate from one machine to another.
• Execute in a platform-independent execution environment.
• Require agent execution environment (places).
• Mobility not necessary or sufficient condition for agenthood.
• Practical but non-functional advantages:
• Reduced communication cost (e.g. from PDA)
• Asynchronous computing (when you are not connected)
• Two types:
• One-hop mobile agents (migrate to one other place)
• Multi-hop mobile agents (roam the network from place to
place)
• Applications:
• Distributed information retrieval.
• Telecommunication network routing.
Information agents
• Manage the explosive growth of information.
• Manipulate or collate information from many distributed
sources.
• Information agents can be mobile or static.
• Examples:
• BargainFinder comparison shops among Internet stores for
CDs
• FIDO the Shopping Doggie (out of service)
• Internet Softbot infers which internet facilities (finger, ftp,
gopher) to use and when from high-level search requests.
• Challenge: ontologies for annotating Web pages (eg, SHOE).
Summary
• Intelligent Agents:
• Anything that can be viewed as perceiving its
environment through sensors and acting upon that
environment through its effectors to maximize progress
towards its goals.
• PAGE (Percepts, Actions, Goals, Environment)
• Described as a Perception (sequence) to Action Mapping: f
: P* → A
• Using look-up-table, closed form, etc.
• Agent Types: Reflex, state-based, goal-based, utility-
based
• Rational Action: The action that maximizes the expected
value of the performance measure given the percept sequence
to date
Chapter 1
What is AI?
Acting Humanly: The Turing Test
• Alan Turing's 1950 article Computing Machinery and
Intelligence discussed conditions for considering a machine
to be intelligent
• “Can machines think?” ←→“Can machines behave
intelligently?”
•The Turing test (The Imitation Game): Operational definition
of intelligence.

• Computer needs to possess: Natural language processing,
Knowledge representation, Automated reasoning, and
Machine learning
• Are there any problems/limitations to the Turing Test?
What tasks require AI?
• “AI is the science and engineering of making intelligent
machines which can perform tasks that require intelligence
when performed by humans …”
• Tasks that require AI:
• Solving a differential equation
• Brain surgery
• Inventing stuff
• Playing Jeopardy
• Playing Wheel of Fortune
• What about walking?
• What about grabbing stuff?
• What about pulling your hand away from fire?
• What about watching TV?
• What about day dreaming?
Acting Humanly: The Full Turing Test
• Alan Turing's 1950 article Computing Machinery and
Intelligence discussed conditions for considering a machine
to be intelligent
• “Can machines think?” ←→“Can machines behave
intelligently?”
• The Turing test (The Imitation Game): Operational
definition of intelligence.
• Computer needs to posses:Natural language processing,
Knowledge representation, Automated reasoning, and
Machine learning
• Problem: 1) Turing test is not reproducible, constructive, and
amenable to mathematic analysis. 2) What about physical
interaction with interrogator and environment?
• Total Turing Test: Requires physical interaction and needs
perception and actuation.
What would a computer need to pass the Turing test?
• Natural language processing: to communicate with examiner.
• Knowledge representation: to store and retrieve information
provided before or during interrogation.
• Automated reasoning: to use the stored information to answer
questions and to draw new conclusions.
• Machine learning: to adapt to new circumstances and to detect
and extrapolate patterns.
• Vision (for Total Turing test): to recognize the
examiner’s actions and various objects presented by the
examiner.
• Motor control (total test): to act upon objects as
requested.
• Other senses (total test): such as audition, smell, touch,
etc.
Thinking Humanly: Cognitive Science
• 1960 “Cognitive Revolution”: information-processing
psychology replaced behaviorism
• Cognitive science brings together theories and experimental
evidence to model internal activities of the brain
• What level of abstraction? “Knowledge” or “Circuits”?
• How to validate models?
• Predicting and testing behavior of human subjects (top-
down)
• Direct identification from neurological data (bottom-up)
• Building computer/machine simulated models and
reproduce results (simulation)
Thinking Rationally: Laws of Thought
• Aristotle (~ 450 B.C.) attempted to codify “right thinking”
What are correct arguments/thought processes?
• E.g., “Socrates is a man, all men are mortal; therefore Socrates
is mortal”
• Several Greek schools developed various forms of logic:
notation plus rules of derivation for thoughts.
• Problems:
• Uncertainty: Not all facts are certain (e.g., the flight might
be delayed).
• Resource limitations:
• Not enough time to compute/process
• Insufficient memory/disk/etc
Acting Rationally: The Rational Agent
• Rational behavior: Doing the right thing!
• The right thing: That which is expected to maximize the
expected return
• Provides the most general view of AI because it includes:
• Correct inference (“Laws of thought”)
• Uncertainty handling
• Resource limitation considerations (e.g., reflex vs.
deliberation)
• Cognitive skills (NLP, AR, knowledge representation, ML,
etc.)
• Advantages:
• More general
• Its goal of rationality is well defined
How to achieve AI?
• How is AI research done?
• AI research has both theoretical and experimental sides. The
experimental side has both basic and applied aspects.
• There are two main lines of research:
• One is biological, based on the idea that since humans are
intelligent, AI should study humans and imitate their
psychology or physiology.
• The other is phenomenal, based on studying and
formalizing common sense facts about the world and the
problems that the world presents to the achievement of
goals.
• The two approaches interact to some extent, and both should
eventually succeed. It is a race, but both racers seem to be
walking. [John McCarthy]
Branches of AI
• Logical AI
• Search
• Natural language processing
• pattern recognition
• Knowledge representation
• Inference From some facts, others can be inferred.
• Automated reasoning
• Learning from experience
• Planning To generate a strategy for achieving some goal
 • Epistemology Study of the kinds of knowledge that are
required for solving problems in the world.
• Ontology Study of the kinds of things that exist. In AI,
the programs and sentences deal with various kinds of
objects, and we study what these kinds are and what their
basic properties are.
• Genetic programming
• Emotions???
AI Prehistory

Note: This is offline problem-solving. Online problem-solving

involves acting w/o complete knowledge of the problem and
environment
Example: Romania
On holiday in Romania; currently in Arad.
Flight leaves tomorrow from Bucharest
Formulate goal:
 be in Bucharest
Formulate problem:
 states: various cities
 actions: drive between cities
Find solution:
 sequence of cities, e.g., Arad, Sibiu, Fagaras,
Bucharest

AI State of the art

• Have the following been achieved by AI?
• World-class chess playing
• Playing table tennis
• Cross-country driving Problem types
• Solving mathematical problems • Single-state problem: deterministic, accessible
• Discover and prove mathematical theories • Agent knows everything about world (the exact state),
• Engage in a meaningful conversation • Can calculate optimal action sequence to reach goal state.
• Understand spoken language • E.g., playing chess. Any action will result in an exact state
• Observe and understand human emotions • Multiple-state problem: deterministic, inaccessible
• Express emotions • Agent does not know the exact state (could be in any of the
Chapter 3 possible states)
Problem-solving agents • May not have sensor at all
• Assume states while working towards goal state.
• E.g., walking in a dark room
• If you are at the door, going straight will lead you to
the kitchen
• If you are at the kitchen, turning left leads you to the
bedroom
• Contingency problem: nondeterministic, inaccessible
• Must use sensors during execution
• Solution is a tree or policy
• Often interleave search and execution
• E.g., a new skater in an arena
• Sliding problem.
• Many skaters around
• Exploration problem: unknown state space
Discover and learn about environment while taking actions.
• E.g., Maze

Example: vacuum world

 Single-state, start in #5. Solution? Right, Suck]
 Sensorless, start in
{1,2,3,4,5,6,7,8} e.g.,  states? integer dirt and robot location
Right goes to {2,4,6,8}  actions? Left, Right, Suck
Solution? [Right,Suck,Left,Suck]  goal test? no dirt at all locations
Contingency  path cost? 1 per action
 Nondeterministic: Suck may Example: The 8-puzzle
dirty a clean carpet
 Partially observable: location, dirt at current
location.
 Percept: [L, Clean], i.e., start in #5 or #7
Solution? [Right, if dirt then Suck]  states? locations of tiles
 actions? move blank left, right, up, down
 goal test? = goal state (given)
 path cost? 1 per move
[Note: optimal solution of n-Puzzle family is NP-hard]
Example: robotic assembly

Single-state problem formulation

A problem is defined by four items:
1. initial state e.g., "at Arad"
2. actions or successor function S(x) = set of action–state  states?: real-valued coordinates of robot joint angles parts
pairs of the object to be assembled
 e.g., S(Arad) = {<Arad à Zerind, Zerind>, … }  actions? : continuous motions of robot joints
3. goal test, can be  goal test?: complete assembly
 explicit, e.g., x = "at Bucharest"  path cost?: time to execute
 implicit, e.g., Checkmate(x) Tree search algorithms
4. path cost (additive) Basic idea:
 e.g., sum of distances, number of actions  offline, simulated exploration of state space by generating
executed, etc. successors of already-explored states (a.k.a.~expanding
 c(x,a,y) is the step cost, assumed to be ≥ 0 states)
A solution is a sequence of actions leading from the initial
state to a goal state
Selecting a state space
Real world is absurdly complex
à state space must be abstracted for problem solving
 (Abstract) state = set of real states
(Abstract) action = complex combination of real actions
 e.g., "Arad à Zerind" represents a complex set of
possible routes, detours, rest stops, etc.
For guaranteed realizability, any real state "in Arad“ must
get to some real state "in Zerind"
(Abstract) solution =
 set of real paths that are solutions in the real world
Each abstract action should be "easier" than the original
problem
Vacuum world state space graph
 Breadth-first search
Expand shallowest unexpanded node
Implementation:
 fringe is a FIFO queue, i.e., new successors go at end

Implementation: general tree search

Properties of breadth-first search

 Complete? Yes (if b is finite)
 Time? 1+b+b2+b3+… +bd + b(bd-1) = O(bd+1)
 Space? O(bd+1) (keeps every node in memory)
 Optimal? Yes (if cost = 1 per step)
Space is the bigger problem (more than time)
Search algorithms are commonly evaluated according to the
following four criteria:
 Completeness: does it always find a solution if one exists?
 Time complexity: how long does it take as function of
num. of nodes?
Implementation: states vs. nodes  Space complexity: how much memory does it require?
 Optimality: does it guarantee the least-cost solution?
 Time and space complexity are measured in terms of:
 b – max branching factor of the search tree
 d – depth of the least-cost solution
 m – max depth of the search tree (may be infinity)
Space complexity of breadth-first
• Largest number of nodes in QUEUE is reached on the level
d of the goal node.

Search strategies
A search strategy is defined by picking the order of node
expansion
Strategies are evaluated along the following dimensions:
 completeness: does it always find a solution if one exists? Time complexity of breadth-first search
 time complexity: number of nodes generated • If a goal node is found on depth d of the tree, all nodes up
till that depth are created.
 space complexity: maximum number of nodes in memory
 optimality: does it always find a least-cost solution?
Time and space complexity are measured in terms of
 b: maximum branching factor of the search tree
 d: depth of the least-cost solution • Thus: O(bd)
 m: maximum depth of the state space (may be ∞) Uniform-cost search
Uninformed search strategies Expand least-cost unexpanded node
Uninformed search strategies use only the information Implementation:
available in the problem definition  fringe = queue ordered by path cost
Breadth-first search Equivalent to breadth-first if step costs all equal
Uniform-cost search  Complete? Yes, if step cost ≥ ε
Depth-first search  Time? # of nodes with g ≤ cost of optimal solution, O(bceiling(C*/
Depth-limited search ε)
) where C* is the cost of the optimal solution
Iterative deepening search
 Space? # of nodes with g ≤ cost of optimal solution, O(bceiling(C*/ Depth-limited search
ε)
) = depth-first search with depth limit l,
 Optimal? Yes – nodes expanded in increasing order of g(n) i.e., nodes at depth l have no successors
Depth-first search Recursive implementation:
Expand deepest unexpanded node
Implementation:
 fringe = LIFO queue, i.e., put successors at front

Properties of depth-first search

 Complete? No: fails in infinite-depth spaces, spaces with
loops
 Modify to avoid repeated states along path Iterative deepening search
à complete in finite spaces
 Time? O(bm): terrible if m is much larger than d
 but if solutions are dense, may be much faster
than breadth-first
 Space? O(bm), i.e., linear space!
 Optimal? No
Time complexity of depth-first: details
• In the worst case:
Iterative deepening search l =3
• the (only) goal node may be on the right-most
branch,

Space complexity of depth-first

 Number of nodes generated in a depth-limited search to depth

Avoiding repeated states
In increasing order of effectiveness and computational overhead:
• do not return to state we come from, i.e., expand function will
skip possible successors that are in same state as node’s
parent.
• do not create paths with cycles, i.e., expand function will skip
possible successors that are in same state as any of node’s
ancestors.
• do not generate any state that was ever generated before, by
keeping track (in memory) of every state generated, unless
the cost of reaching that state is lower than last time we
reached it.
d with branching factor b:
NDLS = b0 + b1 + b2 + … + bd-2 + bd-1 + bd
 Number of nodes generated in an iterative deepening search to
depth d with branching factor b:
NIDS = (d+1)b0 + d b^1 + (d-1)b^2 + … + 3bd-2 +2bd-1 + 1bd
 For b = 10, d = 5,
 NDLS = 1 + 10 + 100 + 1,000 + 10,000 + 100,000 = 111,111
 NIDS = 6 + 50 + 400 + 3,000 + 20,000 + 100,000 = 123,456
Overhead = (123,456 - 111,111)/111,111 = 11%
Properties of iterative deepening search
 Complete? Yes
 Time? (d+1)b0 + d b1 + (d-1)b2 + … + bd = O(bd)
 Space? O(bd)
 Optimal? Yes, if step cost = 1
Summary of algorithms

Repeated states
Failure to detect repeated states can turn a linear problem
into an exponential one!

Graph search

Summary
Problem formulation usually requires abstracting away real-
world details to define a state space that can feasibly be
explored
Variety of uninformed search strategies
Iterative deepening search uses only linear space and not much
more time than other uninformed algorithms