AI All

Artificial Intelligence: introduction
Lecturer: Lars Mehnen

Technikum-Wien
04.02.2012
AI 1
Practical stuff
Course server:
http://
Mail:
Lars.mehnen@technikum-wien.at
Textbook:
S. Russell and P. Norvig Artificial Intelligence: A Modern Approach Prentice Hall, 2003, Second Edition
Exam: written at end of semester (50%)
04.02.2012
AI 1
Course overview
What is AI. Problem solving. Uncertain knowledge and reasoning. Planning.
04.02.2012
AI 1
Practical stuff
Excercices:
Agents
Wumpus
Problem solving Evolutionary algorithms Knowledge and reasoning Expert system Fuzzy logic Planning Agents
04.02.2012
AI 1
What is Artificial Intelligence

Creative extension of philosophy:
Understand and BUILD intelligent entities
Origin after WWII Highly interdisciplinary Currently consist of huge variety of subfields
04.02.2012
AI 1
What is Artificial Intelligence

Different definitions due to different criteria
Two dimensions:
Thought processes/reasoning vs. behavior/action Success according to human standards vs. success according to an ideal concept of intelligence: rationality.
Systems that think like humans
Systems that think rationally
Systems that act like humans
Systems that act rationally
04.02.2012
AI 1

When does a system behave intelligently?
Turing (1950) Computing Machinery and Intelligence Operational test of intelligence: imitation game
Test still relevant now, yet might be the wrong question. Requires the collaboration of major components of AI: knowledge, reasoning, language understanding, learning,
04.02.2012
AI 1
Alan Turing, the enigma
Problem with Turing test: not reproducible, constructive or amenable to mathematical analysis.
04.02.2012 AI 1 8
Systems that think like humans

How do humans think? Requires scientific theories of internal brain activities (cognitive model):
Level of abstraction? (knowledge or circuitry?) Validation?
Predicting and testing human behavior Identification from neurological data
Cognitive Science vs. Cognitive neuroscience. Both approaches are now distinct from AI Share that the available theories do not explain anything resembling human intelligence. Three fields share a principal direction.
04.02.2012
AI 1
Systems that think rationally

Capturing the laws of thought
Aristotle: What are correct argument and thought processes?
Correctness depends on irrefutability of reasoning processes.
This study initiated the field of logic.

The logicist tradition in AI hopes to create intelligent systems using logic programming.
Problems:
Not all intelligence is mediated by logic behavior What is the purpose of thinking? What thought should one have?
04.02.2012 AI 1 10
Rational behavior: doing the right thing

The Right thing is that what is expected to maximize goal achievement given the available information.
Can include thinking, yet in service of rational action.

Action without thinking: e.g. reflexes.
04.02.2012
AI 1
11

Two advantages over previous approaches:
More general than law of thoughts approach More amenable to scientific development.
Yet rationality is only applicable in ideal environments. Moreover rationality is not a very good model of reality.
04.02.2012
AI 1
12
Rational agents
An agent is an entity that perceives and acts This course is about designing rational agents
An agent is a function from percept histories to actions: For any given class of environments and task we seek the agent (or class of agents) with the best performance. Problem: computational limitations make perfect rationality unachievable.
04.02.2012
AI 1
13
Foundations of AI
Different fields have contributed to AI in the form of ideas,viewpoints and techniques.
Philosophy: Logic, reasoning, mind as a physical system, foundations of learning, language and rationality. Mathematics: Formal representation and proof algorithms, computation, (un)decidability, (in)tractability, probability. Psychology: adaptation, phenomena of perception and motor control. Economics: formal theory of rational decisions, game theory. Linguistics: knowledge represetatio, grammar. Neuroscience: physical substrate for mental activities. Control theory: homeostatic systems, stability, optimal agent design.
04.02.2012
AI 1
14
A brief history
What happened after WWII?
1943: Warren Mc Culloch and Walter Pitts: a model of artificial boolean neurons to perform computations.
First steps toward connectionist computation and learning (Hebbian learning). Marvin Minsky and Dann Edmonds (1951) constructed the first neural network computer
1950: Alan Turings Computing Machinery and Intelligence

First complete vision of AI.
04.02.2012 AI 1 15
A brief history (2)

The birth of AI (1956)
Darmouth Workshop bringing together top minds on automata theory, neural nets and the study of intelligence.
Allen Newell and Herbert Simon: The logic theorist (first nonnumerical thinking program used for theorem proving) For the next 20 years the field was dominated by these participants.
Great expectations (1952-1969)

Newell and Simon introduced the General Problem Solver.
Imitation of human problem-solving
Arthur Samuel (1952-)investigated game playing (checkers ) with great success. John McCarthy(1958-) :
Inventor of Lisp (second-oldest high-level language) Logic oriented, Advice Taker (separation between knowledge and reasoning)
04.02.2012
AI 1
16
A brief history (3)

The birth of AI (1956)
Great expectations continued ..
Marvin Minsky (1958 -)
Introduction of microworlds that appear to require intelligence to solve: e.g. blocks-world. Anti-logic orientation, society of the mind.
Collapse in AI research (1966 - 1973)

Progress was slower than expected.
Unrealistic predictions.
Some systems lacked scalability.

Combinatorial explosion in search.
Fundamental limitations on techniques and representations.

Minsky and Papert (1969) Perceptrons.
04.02.2012
AI 1
17
A brief history (4)

AI revival through knowledge-based systems (19691970)
General-purpose vs. domain specific
E.g. the DENDRAL project (Buchanan et al. 1969)
First successful knowledge intensive system.
Expert systems
MYCIN to diagnose blood infections (Feigenbaum et al.)
Introduction of uncertainty in reasoning.
Increase in knowledge representation research.

Logic, frames, semantic nets,
04.02.2012
AI 1
18
A brief history (5)

AI becomes an industry (1980 - present)
R1 at DEC (McDermott, 1982) Fifth generation project in Japan (1981) American response
Puts an end to the AI winter. Connectionist revival (1986 - present)

Parallel distributed processing (RumelHart and McClelland, 1986); backprop.
04.02.2012
AI 1
19
A brief history (6)

AI becomes a science (1987 - present)
Neats vs. scruffies.
In speech recognition: hidden markov models In neural networks In uncertain reasoning and expert systems: Bayesian network formalism
The emergence of intelligent agents (1995 - present)

The whole agent problem:
How does an agent act/behave embedded in real environments with continuous sensory inputs
04.02.2012
AI 1
20
Artificial Intelligence: Intelligent Agents

Lecturer: Lars Mehnen Technikum Wien
04/02/2012
AI 1
Outline
Agents and environments.
The vacuum-cleaner world
The concept of rational behavior. Environments. Agent structure.
04/02/2012
AI 1
Agents and environments

Agents include human, robots, softbots, thermostats, etc. The agent function maps percept sequence to actions
An agent can perceive its own actions, but not always it effects.
04/02/2012
AI 1
Agents and environments
The agent function will internally be represented by the agent program. The agent program runs on the physical architecture to produce f.
04/02/2012
AI 1
Environment: square A and B Percepts: [location and content] e.g. [A, Dirty] Actions: left, right, suck, and no-op
04/02/2012
AI 1
Percept sequence [A,Clean] [A, Dirty] [B, Clean] [B, Dirty] [A, Clean],[A, Clean] [A, Clean],[A, Dirty]
Action Right Suck Left Suck Right Suck
04/02/2012
AI 1
function REFLEX-VACUUM-AGENT ([location, status]) return an action if status == Dirty then return Suck else if location == A then return Right else if location == B then return Left
What is the right function? Can it be implemented in a small agent program?
04/02/2012
AI 1
The concept of rationality

A rational agent is one that does the right thing.
Every entry in the table is filled out correctly.
What is the right thing?

Approximation: the most successful agent. Measure of success?
Performance measure should be objective

E.g. the amount of dirt cleaned within a certain time. E.g. how clean the floor is.
Performance measure according to what is wanted in the environment instead of how the agents should behave.
04/02/2012
AI 1
Rationality
What is rational at a given time depends on four things:
Performance measure, Prior environment knowledge, Actions, Percept sequence to date (sensors).
DEF: A rational agent chooses whichever action maximizes the expected value of the performance measure given the percept sequence to date and prior environment knowledge.
04/02/2012
AI 1
Rationality
Rationality omniscience
An omniscient agent knows the actual outcome of its actions.
Rationality perfection
Rationality maximizes expected performance, while perfection maximizes actual performance.
04/02/2012
AI 1
10
Rationality
The proposed definition requires:
Information gathering/exploration
To maximize future rewards
Learn from percepts

Extending prior knowledge
Agent autonomy
Compensate for incorrect prior knowledge
04/02/2012
AI 1
11
Environments
To design a rational agent we must specify its task environment. PEAS description of the environment:
Performance Environment Actuators Sensors
04/02/2012
AI 1
12
Environments
E.g. Fully automated taxi:
PEAS description of the environment:
Performance Safety, destination, profits, legality, comfort Environment Streets/freeways, other traffic, pedestrians, weather,, Actuators Steering, accelerating, brake, horn, speaker/display, Sensors Video, sonar, speedometer, engine sensors, keyboard, GPS,
04/02/2012
AI 1
13
Environment types
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent??
Backgammon
Internet shopping
Taxi
04/02/2012
AI 1
14
Environment types
Fully vs. partially observable: an environment is full observable when the sensors can detect all aspects that are relevant to the choice of action.
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent??
Backgammon
Internet shopping
Taxi
04/02/2012
AI 1
15
Environment types
Fully vs. partially observable: an environment is full observable when the sensors can detect all aspects that are relevant to the choice of action.
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent?? FULL
Backgammon FULL
Internet shopping PARTIAL
Taxi PARTIAL
04/02/2012
AI 1
16
Environment types
Deterministic vs. stochastic: if the next environment state is completely determined by the current state the executed action then the environment is deterministic.
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent?? FULL Backgammon FULL Internet shopping PARTIAL Taxi PARTIAL
04/02/2012
AI 1
17
Environment types
Deterministic vs. stochastic: if the next environment state is completely determined by the current state the executed action then the environment is deterministic.
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent?? FULL YES Backgammon FULL NO Internet shopping PARTIAL YES Taxi PARTIAL NO
04/02/2012
AI 1
18
Environment types
Episodic vs. sequential: In an episodic environment the agents experience can be divided into atomic steps where the agents perceives and then performs A single action. The choice of action depends only on the episode itself
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent?? FULL YES Backgammon FULL NO Internet shopping PARTIAL YES Taxi PARTIAL NO
04/02/2012
AI 1
19
Environment types
Episodic vs. sequential: In an episodic environment the agents experience can be divided into atomic steps where the agents perceives and then performs A single action. The choice of action depends only on the episode itself
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent?? FULL YES NO Backgammon FULL NO NO Internet shopping PARTIAL YES NO Taxi PARTIAL NO NO
04/02/2012
AI 1
20
Environment types
Static vs. dynamic: If the environment can change while the agent is choosing an action, the environment is dynamic. Semi-dynamic if the agents performance changes even when the environment remains the same.
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent?? FULL YES NO Backgammon FULL NO NO Internet shopping PARTIAL YES NO Taxi PARTIAL NO NO
04/02/2012
AI 1
21
Environment types
Static vs. dynamic: If the environment can change while the agent is choosing an action, the environment is dynamic. Semi-dynamic if the agents performance changes even when the environment remains the same.
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent?? FULL YES NO YES Backgammon FULL NO NO YES Internet shopping PARTIAL YES NO SEMI Taxi PARTIAL NO NO NO
04/02/2012
AI 1
22
Environment types
Discrete vs. continuous: This distinction can be applied to the state of the environment, the way time is handled and to the percepts/actions of the agent.
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent?? FULL YES NO YES
Backgammon FULL NO NO YES
Internet shopping PARTIAL YES NO SEMI
Taxi PARTIAL NO NO NO
04/02/2012
AI 1
23
Environment types
Discrete vs. continuous: This distinction can be applied to the state of the environment, the way time is handled and to the percepts/actions of the agent.
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent?? FULL YES NO YES YES
Backgammon FULL NO NO YES YES
Internet shopping PARTIAL YES NO SEMI YES
Taxi PARTIAL NO NO NO NO
04/02/2012
AI 1
24
Environment types
Single vs. multi-agent: Does the environment contain other agents who are also maximizing some performance measure that depends on the current agents actions?
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent?? FULL YES NO YES YES Backgammon FULL NO NO YES YES Internet shopping PARTIAL YES NO SEMI YES Taxi PARTIAL NO NO NO NO
04/02/2012
AI 1
25
Environment types
Single vs. multi-agent: Does the environment contain other agents who are also maximizing some performance measure that depends on the current agents actions?
Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent?? FULL YES NO YES YES YES Backgammon FULL NO NO YES YES NO Internet shopping PARTIAL YES NO SEMI YES NO Taxi PARTIAL NO NO NO NO NO
04/02/2012
AI 1
26
Environment types
The simplest environment is

Fully observable, deterministic, episodic, static, discrete and single-agent.
Most real situations are:

Partially observable, stochastic, sequential, dynamic, continuous and multi-agent.
04/02/2012
AI 1
27
Agent types
How does the inside of the agent work?
Agent = architecture + program
All agents have the same skeleton:

Input = current percepts Output = action Program= manipulates input to produce output
Note difference with agent function (page 3-4)
04/02/2012
AI 1
28
Agent types
Function TABLE-DRIVEN_AGENT(percept) returns an action static: percepts, a sequence initially empty table, a table of actions, indexed by percept sequence append percept to the end of percepts action LOOKUP(percepts, table) return action
This approach is doomed to failure!
04/02/2012
AI 1
29
Agent types
Four basic kind of agent programs will be discussed:
Simple reflex agents Model-based reflex agents Goal-based agents Utility-based agents
All these can be turned into learning agents.
04/02/2012
AI 1
30
Agent types; simple reflex
Select action on the basis of only the current percept.

E.g. the vacuum-agent
Large reduction in possible percept/action situations(next page). Implemented through conditionaction rules
If dirty then suck
04/02/2012
AI 1
31
function REFLEX-VACUUM-AGENT ([location, status]) return an action if status == Dirty then return Suck else if location == A then return Right else if location == B then return Left
04/02/2012
AI 1
32
Agent types; simple reflex
function SIMPLE-REFLEX-AGENT(percept) returns an action static: rules, a set of condition-action rules state INTERPRET-INPUT(percept) rule RULE-MATCH(state, rule) action RULE-ACTION[rule] return action
Will only work if the environment is fully observable otherwise infinite loops may occur.
04/02/2012
AI 1
33
Agent types; reflex and state
To tackle partially observable environments.

Maintain internal state
Over time update state using world knowledge

How does the world change. How do actions affect world. Model of World
04/02/2012
AI 1
34
Agent types; reflex and state
function REFLEX-AGENT-WITH-STATE(percept) returns an action static: rules, a set of condition-action rules state, a description of the current world state action, the most recent action. state UPDATE-STATE(state, action, percept) rule RULE-MATCH(state, rule) action RULE-ACTION[rule] return action
04/02/2012
AI 1
35
Agent types; goal-based

The agent needs a goal to know which situations are desirable.
Things become difficult when long sequences of actions are required to find the goal.
Typically investigated in search and planning research. Major difference: future is taken into account Is more flexible since knowledge is represented explicitly and can be manipulated.
04/02/2012
AI 1
36
Agent types; utility-based

Certain goals can be reached in different ways.
Some are better, have a higher utility.
Utility function maps a (sequence of) state(s) onto a real number. Improves on goals:
Selecting between conflicting goals Select appropriately between several goals based on likelihood of success.
04/02/2012
AI 1
37
Agent types; learning
All previous agent-programs describe methods for selecting actions.

Yet it does not explain the origin of these programs. Learning mechanisms can be used to perform this task. Teach them instead of instructing them. Advantage is the robustness of the program toward initially unknown environments.
04/02/2012
AI 1
38
Agent types; learning
Learning element: introduce improvements in performance element.

Critic provides feedback on agents performance based on fixed performance standard.
Performance element: selecting actions based on percepts.

Corresponds to the previous agent programs
Problem generator: suggests actions that will lead to new and informative experiences.
Exploration vs. exploitation
04/02/2012
AI 1
39
Artificial Intelligence: problem solving and search

Outline
Problem-solving agents
A kind of goal-based agent
Problem types
Single state (fully observable) Search with partial information
Problem formulation
Example problems
Basic search algorithms

Uninformed
04/02/2012
AI 1
Example: Romania
04/02/2012
AI 1
Example: Romania
On holiday in Romania; currently in Arad
Flight leaves tomorrow from Bucharest
Formulate goal
Be in Bucharest
Formulate problem
States: various cities Actions: drive between cities
Find solution
Sequence of cities; e.g. Arad, Sibiu, Fagaras, Bucharest,
04/02/2012
AI 1
Problem-solving agent
Four general steps in problem solving:
Goal formulation
What are the successful world states
Problem formulation
What actions and states to consider given the goal
Search
Determine the possible sequence of actions that lead to the states of known values and then choosing the best sequence.
Execute
Give the solution perform the actions.
04/02/2012
AI 1
Problem-solving agent
function SIMPLE-PROBLEM-SOLVING-AGENT(percept) return an action static: seq, an action sequence state, some description of the current world state goal, a goal problem, a problem formulation state UPDATE-STATE(state, percept) if seq is empty then goal FORMULATE-GOAL(state) problem FORMULATE-PROBLEM(state,goal) seq SEARCH(problem) action FIRST(seq) seq REST(seq) return action
04/02/2012
AI 1
Problem types
Deterministic, fully observable single state problem

Agent knows exactly which state it will be in; solution is a sequence.
Partial knowledge of states and actions:

Non-observable sensorless or conformant problem
Agent may have no idea where it is; solution (if any) is a sequence.
Nondeterministic and/or partially observable contingency problem

Percepts provide new information about current state; solution is a tree or policy; often interleave search and execution.
Unknown state space exploration problem (online)

When states and actions of the environment are unknown.
04/02/2012
AI 1
Example: vacuum world

Single state, start in #5. Solution??
04/02/2012
AI 1

[Right, Suck]
04/02/2012
AI 1

[Right, Suck]
Sensorless: start in {1,2,3,4,5,6,7,8} e.g Right goes to {2,4,6,8}. Solution?? Contingency: start in {1,3}. (assume Murphys law, Suck can dirty a clean carpet and local sensing: [location,dirt] only. Solution??
04/02/2012
AI 1
10
Problem formulation
A problem is defined by:
An initial state, e.g. Arad Successor function S(X)= set of action-state pairs
e.g. S(Arad)={<Arad Zerind, Zerind>,}
intial state + successor function = state space Goal test, can be

Explicit, e.g. x=at bucharest Implicit, e.g. checkmate(x)
Path cost (additive)

e.g. sum of distances, number of actions executed, c(x,a,y) is the step cost, assumed to be >= 0
A solution is a sequence of actions from initial to goal state. Optimal solution has the lowest path cost.
04/02/2012
AI 1
11
Selecting a state space
Real world is absurdly complex.

State space must be abstracted for problem solving.
(Abstract) state = set of real states. (Abstract) action = complex combination of real actions.
e.g. Arad Zerind represents a complex set of possible routes, detours, rest stops, etc. The abstraction is valid if the path between two states is reflected in the real world.
(Abstract) solution = set of real paths that are solutions in the real world. Each abstract action should be easier than the real problem.
04/02/2012
AI 1
12
States?? Initial state?? Actions?? Goal test?? Path cost??
04/02/2012
AI 1
13
States?? two locations with or without dirt: 2 x 22=8 states. Initial state?? Any state can be initial Actions?? {Left, Right, Suck} Goal test?? Check whether squares are clean. Path cost?? Number of actions to reach goal.
04/02/2012
AI 1
14
Example: 8-puzzle
04/02/2012
AI 1
15
Example: 8-puzzle
States?? Integer location of each tile Initial state?? Any state can be initial Actions?? {Left, Right, Up, Down} Goal test?? Check whether goal configuration is reached Path cost?? Number of actions to reach goal
04/02/2012
AI 1
16
Example: 8-queens problem
04/02/2012
AI 1
17
Incremental formulation vs. complete-state formulation States?? Initial state?? Actions?? Goal test?? Path cost??
04/02/2012
AI 1
18
Incremental formulation States?? Any arrangement of 0 to 8 queens on the board Initial state?? No queens Actions?? Add queen in empty square Goal test?? 8 queens on board and none attacked Path cost?? None 3 x 1014 possible sequences to investigate
04/02/2012
AI 1
19
Incremental formulation (alternative) States?? n (0 n 8) queens on the board, one per column in the n leftmost columns with no queen attacking another. Actions?? Add queen in leftmost empty column such that is not attacking other queens 2057 possible sequences to investigate; Yet makes no difference when n=100
04/02/2012
AI 1
20
Example: robot assembly
04/02/2012
AI 1
21
Example: robot assembly
States?? Real-valued coordinates of robot joint angles; parts of the object to be assembled. Initial state?? Any arm position and object configuration. Actions?? Continuous motion of robot joints Goal test?? Complete assembly (without robot) Path cost?? Time to execute
04/02/2012
AI 1
22
Basic search algorithms

How do we find the solutions of previous problems?
Search the state space (remember complexity of space depends on state representation) Here: search through explicit tree generation
ROOT= initial state. Nodes and leafs generated through successor function.
In general search generates a graph (same state through multiple paths)
04/02/2012
AI 1
23
Simple tree search example
function TREE-SEARCH(problem, strategy) return a solution or failure Initialize search tree to the initial state of the problem do if no candidates for expansion then return failure choose leaf node for expansion according to strategy if node contains goal state then return solution else expand the node and add resulting nodes to the search tree enddo
04/02/2012
AI 1
24
04/02/2012
AI 1
25
Determines search process!!
04/02/2012
AI 1
26
State space vs. search tree
A state is a (representation of) a physical configuration A node is a data structure belong to a search tree
A node has a parent, children, and ncludes path cost, depth, Here node= <state, parent-node, action, path-cost, depth> FRINGE= contains generated nodes which are not yet expanded.
White nodes with black outline
04/02/2012
AI 1
27
Tree search algorithm
function TREE-SEARCH(problem,fringe) return a solution or failure fringe INSERT(MAKE-NODE(INITIAL-STATE[problem]), fringe) loop do if EMPTY?(fringe) then return failure node REMOVE-FIRST(fringe) if GOAL-TEST[problem] applied to STATE[node] succeeds then return SOLUTION(node) fringe INSERT-ALL(EXPAND(node, problem), fringe)
04/02/2012
AI 1
28
Tree search algorithm (2)

function EXPAND(node,problem) return a set of nodes successors the empty set for each <action, result> in SUCCESSOR-FN[problem](STATE[node]) do s a new NODE STATE[s] result PARENT-NODE[s] node ACTION[s] action PATH-COST[s] PATH-COST[node] + STEP-COST(node, action,s) DEPTH[s] DEPTH[node]+1 add s to successors return successors
04/02/2012
AI 1
29
Search strategies
A strategy is defined by picking the order of node expansion. Problem-solving performance is measured in four ways:
Completeness; Does it always find a solution if one exists? Optimality; Does it always find the least-cost solution? Time Complexity; Number of nodes generated/expanded? Space Complexity; Number of nodes stored in memory during search?
Time and space complexity are measured in terms of problem difficulty defined by:
b - maximum branching factor of the search tree d - depth of the least-cost solution m - maximum depth of the state space (may be )
04/02/2012
AI 1
30
Uninformed search strategies

(a.k.a. blind search) = use only information available in problem definition.
When strategies can determine whether one non-goal state is better than another informed search.
Categories defined by expansion algorithm:

Breadth-first search Uniform-cost search Depth-first search Depth-limited search Iterative deepening search. Bidirectional search
04/02/2012
AI 1
31
BF-search, an example
Expand shallowest unexpanded node Implementation: fringe is a FIFO queue
04/02/2012
AI 1
32
A B C
04/02/2012
AI 1
33
A B D E C
04/02/2012
AI 1
34
A B D E F C G
04/02/2012
AI 1
35
BF-search; evaluation
Completeness:
Does it always find a solution if one exists?
YES
If shallowest goal node is at some finite depth d Condition: If b is finite
(maximum num. Of succ. nodes is finite)
04/02/2012
AI 1
36
Completeness:
YES (if b is finite)
Time complexity:
Assume a state space where every state has b successors.
root has b successors, each node at the next level has again b successors (total b2), Assume solution is at depth d Worst case; expand all but the last node at depth d Total numb. of nodes generated:
b + b 2 + b 3 + ...+ b d + (b d +1 b) = O(b d +1 )
04/02/2012
AI 1
37
Completeness:
Time complexity:
Total numb. of nodes generated:
b + b 2 + b 3 + ...+ b d + (b d +1 b) = O(b d +1 )
Space complexity:
Idem if each node is retained in memory
04/02/2012
AI 1
38
Completeness:
Time complexity:
Total numb. of nodes generated:
b + b 2 + b 3 + ...+ b d + (b d +1 b) = O(b d +1 )
Space complexity:
Idem if each node is retained in memory
Optimality:
Does it always find the least-cost solution? In general YES
unless actions have different cost.
04/02/2012
AI 1
39
Two lessons:
Memory requirements are a bigger problem than its execution time. Exponential complexity search problems cannot be solved by uninformed search methods for any but the smallest instances.
DEPTH2 2 4 6 8 10 12 14
NODES 1100 111100 107 109 1011 1013 1015
TIME 0.11 seconds 11 seconds 19 minutes 31 hours 129 days 35 years 3523 years
MEMORY 1 megabyte 106 megabytes 10 gigabytes 1 terabyte 101 terabytes 10 petabytes 1 exabyte
04/02/2012
AI 1
40
Uniform-cost search
Extension of BF-search:
Expand node with lowest path cost
Implementation: fringe = queue ordered by path cost. UC-search is the same as BF-search when all step-costs are equal.
04/02/2012
AI 1
41
Uniform-cost search
Completeness:
YES, if step-cost > (smal positive constant)
Time complexity:
Assume C* the cost of the optimal solution. Assume that every action costs at least Worst-case: C */
O(b
Space complexity:
Idem to time complexity
Optimality:
nodes expanded in order of increasing path cost. YES, if complete.
04/02/2012
AI 1
42
DF-search, an example
Expand deepest unexpanded node Implementation: fringe is a LIFO queue (=stack)
04/02/2012
AI 1
43
A B C
04/02/2012
AI 1
44
A B D E C
04/02/2012
AI 1
45
A B D H I E C
04/02/2012
AI 1
46
A B D H I E C
04/02/2012
AI 1
47
A B D H I E C
04/02/2012
AI 1
48
A B D H I J E K C
04/02/2012
AI 1
49
A B D H I J E K C
04/02/2012
AI 1
50
A B D H I J E K C
04/02/2012
AI 1
51
A B D H
04/02/2012
C E F K
52
I J
AI 1
A B D H
04/02/2012
C E F K L
AI 1
G M
53
A B D H
04/02/2012
C E F K L M
54
I J
AI 1
DF-search; evaluation
Completeness;
Does it always find a solution if one exists?
NO
unless search space is finite and no loops are possible.
04/02/2012
AI 1
55
Completeness;
NO unless search space is finite.
Time complexity;
O(b m )
Terrible if m is much larger than d (depth of optimal solution) But if many solutions, then faster than BF-search
04/02/2012
AI 1
56
Completeness;
Time complexity; Space complexity;
O(b m ) O(bm + 1)
Backtracking search uses even less memory

One successor instead of all b.
04/02/2012
AI 1
57
Completeness;
Time complexity; Space complexity; Optimallity; No
O(b m ) O(bm + 1)
Same issues as completeness Assume node J and C contain goal states
04/02/2012
AI 1
58
Depth-limited search
Is DF-search with depth limit l.
i.e. nodes at depth l have no successors. Problem knowledge can be used
Solves the infinite-path problem. If l < d then incompleteness results. If l > d then not optimal. Time complexity: O(b l ) Space complexity: O(bl)
04/02/2012
AI 1
59
Depth-limited algorithm
function DEPTH-LIMITED-SEARCH(problem,limit) return a solution or failure/cutoff return RECURSIVE-DLS(MAKE-NODE(INITIAL-STATE[problem]),problem,limit) function RECURSIVE-DLS(node, problem, limit) return a solution or failure/cutoff cutoff_occurred? false if GOAL-TEST[problem](STATE[node]) then return SOLUTION(node) else if DEPTH[node] == limit then return cutoff else for each successor in EXPAND(node, problem) do result RECURSIVE-DLS(successor, problem, limit) if result == cutoff then cutoff_occurred? true else if result failure then return result if cutoff_occurred? then return cutoff else return failure
04/02/2012
AI 1
60
Iterative deepening search
What?
A general strategy to find best depth limit l.
Goals is found at depth d, the depth of the shallowest goal-node.
Often used in combination with DF-search
Combines benefits of DF- en BF-search
04/02/2012
AI 1
61
Iterative deepening search

function ITERATIVE_DEEPENING_SEARCH(problem) return a solution or failure inputs: problem for depth 0 to do result DEPTH-LIMITED_SEARCH(problem, depth) if result cuttoff then return result
04/02/2012
AI 1
62
ID-search, example
Limit=0
04/02/2012
AI 1
63
ID-search, example
Limit=1
04/02/2012
AI 1
64
ID-search, example
Limit=2
04/02/2012
AI 1
65
ID-search, example
Limit=3
04/02/2012
AI 1
66
ID search, evaluation
Completeness:
YES (no infinite paths)
04/02/2012
AI 1
67
Completeness:
YES (no infinite paths)
Time complexity:
Algorithm seems costly due to repeated generation of certain states. Node generation: 2 d
level d: once level d-1: 2 level d-2: 3 level 2: d-1 level 1: d
O(b d )
N(IDS) = (d)b + (d 1)b + ...+ (1)b
N(BFS) = b + b 2 + ...+ b d + (b d +1 b)
Num. Comparison for b=10 and d=5 solution at far right
N(IDS) = 50 + 400 + 3000 + 20000 + 100000 = 123450 N(BFS) = 10 + 100 + 1000 + 10000 + 100000 + 999990 = 1111100
04/02/2012
AI 1
68
Completeness:
YES (no infinite paths) Time complexity:
Space complexity:
Cfr. depth-first search
O(b d ) O(bd)
04/02/2012
AI 1
69
Completeness:
YES (no infinite paths) Time complexity:
Space complexity: Optimality:
O(b d ) O(bd)
YES if step cost is 1. Can be extended to iterative lengthening search

Same idea as uniform-cost search Increases overhead.
04/02/2012
AI 1
70
Bidirectional search
Two simultaneous searches from start an goal. d d /2 d /2 Motivation:
+b
Check whether the node belongs to the other fringe before expansion. Space complexity is the most significant weakness. Complete and optimal if both searches are BF.
04/02/2012
AI 1
71
How to search backwards?
The predecessor of each node should be efficiently computable.

When actions are easily reversible.
04/02/2012
AI 1
72
Summary of algorithms
Criterion
Breadth-First
Uniform-cost
Depth-First
Depth-limited
Iterative deepening
Bidirectional search
Complete?
YES*
YES*
NO
YES, if l d bl
YES
YES*
Time
bd+1
bC*/e
bm
bd
bd/2
Space
bd+1
bC*/e
bm
bl
bd
bd/2
Optimal?
YES*
YES*
NO
NO
YES
YES
04/02/2012
AI 1
73
Repeated states
Failure to detect repeated states can turn a solvable problems into unsolvable ones.
04/02/2012
AI 1
74
Graph search algorithm

Closed list stores all expanded nodes
function GRAPH-SEARCH(problem,fringe) return a solution or failure closed an empty set fringe INSERT(MAKE-NODE(INITIAL-STATE[problem]), fringe) loop do if EMPTY?(fringe) then return failure node REMOVE-FIRST(fringe) if GOAL-TEST[problem] applied to STATE[node] succeeds then return SOLUTION(node) if STATE[node] is not in closed then add STATE[node] to closed fringe INSERT-ALL(EXPAND(node, problem), fringe)
04/02/2012
AI 1
75
Graph search, evaluation

Optimality:
GRAPH-SEARCH discard newly discovered paths.
This may result in a sub-optimal solution YET: when uniform-cost search or BF-search with constant step cost
Time and space complexity,

proportional to the size of the state space
(may be much smaller than O(bd)).
DF- and ID-search with closed list no longer has linear space requirements since all nodes are stored in closed list!!
04/02/2012
AI 1
76
Search with partial information

Previous assumption:
Environment is fully observable Environment is deterministic Agent knows the effects of its actions
What if knowledge of states or actions is incomplete?
04/02/2012
AI 1
77
Search with partial information

Partial knowledge of states and actions:
sensorless or conformant problem
Agent may have no idea where it is; solution (if any) is a sequence.
contingency problem
Percepts provide new information about current state; solution is a tree or policy; often interleave search and execution. If uncertainty is caused by actions of another agent: adversarial problem
exploration problem
When states and actions of the environment are unknown.
04/02/2012
AI 1
78
Conformant problems
start in {1,2,3,4,5,6,7,8} e.g Right goes to {2,4,6,8}. Solution??
[Right, Suck, Left,Suck]
When the world is not fully observable: reason about a set of states that migth be reached
=belief state
04/02/2012
AI 1
79
Conformant problems
Search space of belief states Solution = belief state with all members goal states. If S states then 2S belief states. Murphys law:
Suck can dirty a clear square.
04/02/2012
AI 1
80
Belief state of vacuum-world
04/02/2012
AI 1
81
Contingency problems
Contingency, start in {1,3}. Murphys law, Suck can dirty a clean carpet. Local sensing: dirt, location only.
Percept = [L,Dirty] ={1,3} [Suck] = {5,7} [Right] ={6,8} [Suck] in {6}={8} (Success) BUT [Suck] in {8} = failure
Solution??
Belief-state: no fixed action sequence guarantees solution
Relax requirement:
[Suck, Right, if [R,dirty] then Suck] Select actions based on contingencies arising during execution.
04/02/2012
AI 1
82
Artificial intelligence: informed search

Outline
Informed = use problem-specific knowledge Which search strategies?
Best-first search and its variants
Heuristic functions?
How to invent them
Local search and optimization

Hill climbing, local beam search, genetic algorithms,
Local search in continuous spaces Online search agents
4 februari 2012
AI 1
Previously: tree-search
function TREE-SEARCH(problem,fringe) return a solution or failure fringe INSERT(MAKE-NODE(INITIAL-STATE[problem]), fringe) loop do if EMPTY?(fringe) then return failure node REMOVE-FIRST(fringe) if GOAL-TEST[problem] applied to STATE[node] succeeds then return SOLUTION(node) fringe INSERT-ALL(EXPAND(node, problem), fringe)
A strategy is defined by picking the order of node expansion
4 februari 2012
AI 1
Best-first search
General approach of informed search:
Best-first search: node is selected for expansion based on an evaluation function f(n)
Idea: evaluation function measures distance to the goal.

Choose node which appears best
Implementation: fringe is queue sorted in decreasing order of desirability.

Special cases: greedy search, A* search
4 februari 2012
AI 1
A heuristic function
[dictionary]A rule of thumb, simplification, or educated guess that reduces or limits the search for solutions in domains that are difficult and poorly understood.
h(n) = estimated cost of the cheapest path from node n to goal node. If n is goal then h(n)=0 More information later.
4 februari 2012
AI 1
Romania with step costs in km

hSLD=straight-line distance heuristic. hSLD can NOT be computed from the problem description itself In this example f(n)=h(n)
Expand node that is closest to goal = Greedy best-first search
4 februari 2012
AI 1
Greedy search example

Arad (366)
Assume that we want to use greedy search to solve the problem of travelling from Arad to Bucharest. The initial state=Arad
4 februari 2012
AI 1

Arad
Sibiu(253)
The first expansion step produces:
Sibiu, Timisoara and Zerind
Timisoara (329)
Zerind(374)
Greedy best-first will select Sibiu.
4 februari 2012
AI 1

Arad Sibiu
Arad (366)
Fagaras (176)
Oradea (380)
Rimnicu Vilcea (193)
If Sibiu is expanded we get:

Arad, Fagaras, Oradea and Rimnicu Vilcea
Greedy best-first search will select: Fagaras
4 februari 2012
AI 1

Arad Sibiu
Fagaras Sibiu (253) Bucharest (0)
If Fagaras is expanded we get:

Sibiu and Bucharest
Goal reached !!
Yet not optimal (see Arad, Sibiu, Rimnicu Vilcea, Pitesti)
4 februari 2012
AI 1
10
Greedy search, evaluation

Completeness: NO (cfr. DF-search)
Check on repeated states Minimizing h(n) can result in false starts, e.g. Iasi to Fagaras.
4 februari 2012
AI 1
11

Completeness: NO (cfr. DF-search) Time complexity?
O(b )
Cfr. Worst-case DF-search (with m is maximum depth of search space) Good heuristic can give dramatic improvement.
4 februari 2012
AI 1
12

Completeness: NO (cfr. DF-search) O(b m ) Time complexity: Space complexity: O(b m )
Keeps all nodes in memory
4 februari 2012
AI 1
13

Completeness: NO (cfr. DF-search) Time complexity: O(b m ) Space complexity: O(b m ) Optimality? NO
Same as DF-search
4 februari 2012
AI 1
14
A* search
Best-known form of best-first search. Idea: avoid expanding paths that are already expensive. Evaluation function f(n)=g(n) + h(n)
g(n) the cost (so far) to reach the node. h(n) estimated cost to get from the node to the goal. f(n) estimated total cost of path through n to goal.
4 februari 2012
AI 1
15
A* search
A* search uses an admissible heuristic
A heuristic is admissible if it never overestimates the cost to reach the goal Are optimistic
Formally: 1. h(n) <= h*(n) where h*(n) is the true cost from n 2. h(n) >= 0 so h(G)=0 for any goal G. e.g. hSLD(n) never overestimates the actual road distance
4 februari 2012
AI 1
16
Romania example
4 februari 2012
AI 1
17
A* search example
Find Bucharest starting at Arad

f(Arad) = c(??,Arad)+h(Arad)=0+366=366
4 februari 2012
AI 1
18
A* search example
Expand Arrad and determine f(n) for each node

f(Sibiu)=c(Arad,Sibiu)+h(Sibiu)=140+253=393 f(Timisoara)=c(Arad,Timisoara)+h(Timisoara)=118+329=447 f(Zerind)=c(Arad,Zerind)+h(Zerind)=75+374=449
Best choice is Sibiu
4 februari 2012
AI 1
19
A* search example
Expand Sibiu and determine f(n) for each node

f(Arad)=c(Sibiu,Arad)+h(Arad)=280+366=646 f(Fagaras)=c(Sibiu,Fagaras)+h(Fagaras)=239+179=415 f(Oradea)=c(Sibiu,Oradea)+h(Oradea)=291+380=671 f(Rimnicu Vilcea)=c(Sibiu,Rimnicu Vilcea)+ h(Rimnicu Vilcea)=220+192=413
Best choice is Rimnicu Vilcea

4 februari 2012 AI 1 20
A* search example
Expand Rimnicu Vilcea and determine f(n) for each node

f(Craiova)=c(Rimnicu Vilcea, Craiova)+h(Craiova)=360+160=526 f(Pitesti)=c(Rimnicu Vilcea, Pitesti)+h(Pitesti)=317+100=417 f(Sibiu)=c(Rimnicu Vilcea,Sibiu)+h(Sibiu)=300+253=553
Best choice is Fagaras
4 februari 2012
AI 1
21
A* search example
Expand Fagaras and determine f(n) for each node

f(Sibiu)=c(Fagaras, Sibiu)+h(Sibiu)=338+253=591 f(Bucharest)=c(Fagaras,Bucharest)+h(Bucharest)=450+0=450
Best choice is Pitesti !!!
4 februari 2012
AI 1
22
A* search example
Expand Pitesti and determine f(n) for each node

f(Bucharest)=c(Pitesti,Bucharest)+h(Bucharest)=418+0=418
Best choice is Bucharest !!!

Optimal solution (only if h(n) is admissable)
Note values along optimal path !!
4 februari 2012
AI 1
23
Optimality of A*(standard proof)
Suppose suboptimal goal G2 in the queue. Let n be an unexpanded node on a shortest to optimal goal G.
= g(G2 ) since h(G2 )=0 > g(G) since G2 is suboptimal >= f(n) since h is admissible Since f(G2) > f(n), A* will never select G2 for expansion f(G2 )
4 februari 2012
AI 1
24
BUT graph search

Discards new paths to repeated state.
Previous proof breaks down
Solution:
Add extra bookkeeping i.e. remove more expsive of two paths. Ensure that optimal path to any repeated state is always first followed.
Extra requirement on h(n): consistency (monotonicity)
4 februari 2012
AI 1
25
Consistency
A heuristic is consistent if If h is consistent, we have
h(n) c(n,a,n') + h(n')
f (n') = g(n') + h(n') = g(n) + c(n,a,n') + h(n') i.e. f(n) is nondecreasing along any path. g(n) + h(n) f (n)
4 februari 2012
AI 1
26
Optimality of A*(more usefull)

A* expands nodes in order of increasing f value Contours can be drawn in state space
Uniform-cost search adds circles. F-contours are gradually Added: 1) nodes with f(n)<C* 2) Some nodes on the goal Contour (f(n)=C*). Contour I has all Nodes with f=fi, where fi < fi+1.
4 februari 2012
AI 1
27
A* search, evaluation
Completeness: YES
Since bands of increasing f are added Unless there are infinitly many nodes with f<f(G)
4 februari 2012
AI 1
28
Completeness: YES Time complexity:
Number of nodes expanded is still exponential in the length of the solution.
4 februari 2012
AI 1
29
Completeness: YES Time complexity: (exponential with path length) Space complexity:
It keeps all generated nodes in memory Hence space is the major problem not time
4 februari 2012
AI 1
30
Completeness: YES Time complexity: (exponential with path length) Space complexity:(all nodes are stored) Optimality: YES
Cannot expand fi+1 until fi is finished. A* expands all nodes with f(n)< C* A* expands some nodes with f(n)=C* A* expands no nodes with f(n)>C*
Also optimally efficient (not including ties)
4 februari 2012
AI 1
31
Memory-bounded heuristic search

Some solutions to A* space problems (maintain completeness and optimality)
Iterative-deepening A* (IDA*)
Here cutoff information is the f-cost (g+h) instead of depth
Recursive best-first search(RBFS)

Recursive algorithm that attempts to mimic standard best-first search with linear space.
(simple) Memory-bounded A* ((S)MA*)

Drop the worst-leaf node when memory is full
4 februari 2012
AI 1
32
Recursive best-first search

function RECURSIVE-BEST-FIRST-SEARCH(problem) return a solution or failure return RFBS(problem,MAKE-NODE(INITIAL-STATE[problem]),) function RFBS( problem, node, f_limit) return a solution or failure and a new f-cost limit if GOAL-TEST[problem](STATE[node]) then return node successors EXPAND(node, problem) if successors is empty then return failure, for each s in successors do f [s] max(g(s) + h(s), f [node]) repeat best the lowest f-value node in successors if f [best] > f_limit then return failure, f [best] alternative the second lowest f-value among successors result, f [best] RBFS(problem, best, min(f_limit, alternative)) if result failure then return result
4 februari 2012
AI 1
33
Recursive best-first search

Keeps track of the f-value of the best-alternative path available.
If current f-values exceeds this alternative f-value than backtrack to alternative path. Upon backtracking change f-value to best f-value of its children. Re-expansion of this result is thus still possible.
4 februari 2012
AI 1
34
Recursive best-first search, ex.
Path until Rumnicu Vilcea is already expanded Above node; f-limit for every recursive call is shown on top. Below node: f(n) The path is followed until Pitesti which has a f-value worse than the f-limit.
4 februari 2012
AI 1
35
Unwind recursion and store best f-value for current best leaf Pitesti
result, f [best] RBFS(problem, best, min(f_limit, alternative))
best is now Fagaras. Call RBFS for new best

best value is now 450
4 februari 2012
AI 1
36
Unwind recursion and store best f-value for current best leaf Fagaras
result, f [best] RBFS(problem, best, min(f_limit, alternative))
best is now Rimnicu Viclea (again). Call RBFS for new best
Subtree is again expanded. Best alternative subtree is now through Timisoara.
Solution is found since because 447 > 417.
4 februari 2012
AI 1
37
RBFS evaluation
RBFS is a bit more efficient than IDA*
Still excessive node generation (mind changes)
Like A*, optimal if h(n) is admissible Space complexity is O(bd).

IDA* retains only one single number (the current f-cost limit)
Time complexity difficult to characterize

Depends on accuracy if h(n) and how often best path changes.
IDA* en RBFS suffer from too little memory.
4 februari 2012
AI 1
38
(simplified) memory-bounded A*
Use all available memory.
I.e. expand best leafs until available memory is full When full, SMA* drops worst leaf node (highest f-value) Like RFBS backup forgotten node to its parent
What if all leafs have the same f-value?

Same node could be selected for expansion and deletion. SMA* solves this by expanding newest best leaf and deleting oldest worst leaf.
SMA* is complete if solution is reachable, optimal if optimal solution is reachable.
4 februari 2012
AI 1
39
Learning to search better

All previous algorithms use fixed strategies. Agents can learn to improve their search by exploiting the meta-level state space.
Each meta-level state is a internal (computational) state of a program that is searching in the object-level state space. In A* such a state consists of the current search tree
A meta-level learning algorithm from experiences at the meta-level.
4 februari 2012
AI 1
40
Heuristic functions
E.g for the 8-puzzle

Avg. solution cost is about 22 steps (branching factor +/- 3) Exhaustive search to depth 22: 3.1 x 1010 states. A good heuristic function can reduce the search process.
4 februari 2012
AI 1
41
Heuristic functions
E.g for the 8-puzzle knows two commonly used heuristics h1 = the number of misplaced tiles
h1(s)=8
h2 = the sum of the distances of the tiles from their goal positions (manhattan distance).
h2(s)=3+1+2+2+2+3+3+2=18
4 februari 2012
AI 1
42
Heuristic quality
Effective branching factor b*
Is the branching factor that a uniform tree of depth d would have in order to contain N+1 nodes.
N + 1 = 1+ b * +(b*) 2 + ...+ (b*) d

Measure is fairly constant for sufficiently hard problems.
Can thus provide a good guide to the heuristics overall usefulness. A good value of b* is 1.
4 februari 2012
AI 1
43
Heuristic quality and dominance

1200 random problems with solution lengths from 2 to 24.
If h2(n) >= h1(n) for all n (both admissible) then h2 dominates h1 and is better for search
4 februari 2012
AI 1
44
Inventing admissible heuristics

Admissible heuristics can be derived from the exact solution cost of a relaxed version of the problem:
Relaxed 8-puzzle for h1 : a tile can move anywhere
As a result, h1(n) gives the shortest solution
Relaxed 8-puzzle for h2 : a tile can move to any adjacent square.

As a result, h2(n) gives the shortest solution.
The optimal solution cost of a relaxed problem is no greater than the optimal solution cost of the real problem. ABSolver found a usefull heuristic for the rubic cube.
4 februari 2012
AI 1
45

Admissible heuristics can also be derived from the solution cost of a subproblem of a given problem. This cost is a lower bound on the cost of the real problem. Pattern databases store the exact solution to for every possible subproblem instance.
The complete heuristic is constructed using the patterns in the DB
4 februari 2012
AI 1
46

Another way to find an admissible heuristic is through learning from experience:
Experience = solving lots of 8-puzzles An inductive learning algorithm can be used to predict costs for other states that arise during search.
4 februari 2012
AI 1
47

Previously: systematic exploration of search space.
Path to goal is solution to problem
YET, for some problems path is irrelevant.

E.g 8-queens
Different algorithms can be used

Local search
4 februari 2012
AI 1
48

Local search= use single current state and move to neighboring states. Advantages:
Use very little memory Find often reasonable solutions in large or infinite state spaces.
Are also useful for pure optimization problems.

Find best state according to some objective function. e.g. survival of the fittest as a metaphor for optimization.
4 februari 2012
AI 1
49
4 februari 2012
AI 1
50
Hill-climbing search
is a loop that continuously moves in the direction of increasing value
It terminates when a peak is reached.
Hill climbing does not look ahead of the immediate neighbors of the current state. Hill-climbing chooses randomly among the set of best successors, if there is more than one. Hill-climbing a.k.a. greedy local search
4 februari 2012
AI 1
51
Hill-climbing search
function HILL-CLIMBING( problem) return a state that is a local maximum input: problem, a problem local variables: current, a node. neighbor, a node. current MAKE-NODE(INITIAL-STATE[problem]) loop do neighbor a highest valued successor of current if VALUE [neighbor] VALUE[current] then return STATE[current] current neighbor
4 februari 2012
AI 1
52
Hill-climbing example
8-queens problem (complete-state formulation). Successor function: move a single queen to another square in the same column. Heuristic function h(n): the number of pairs of queens that are attacking each other (directly or indirectly).
4 februari 2012
AI 1
53
Hill-climbing example
a) b)
a) shows a state of h=17 and the h-value for each possible successor. b) A local minimum in the 8-queens state space (h=1).
4 februari 2012
AI 1
54
Drawbacks
Ridge = sequence of local maxima difficult for greedy algorithms to navigate Plateaux = an area of the state space where the evaluation function is flat. Gets stuck 86% of the time.
4 februari 2012
AI 1
55
Hill-climbing variations
Stochastic hill-climbing
Random selection among the uphill moves. The selection probability can vary with the steepness of the uphill move.
First-choice hill-climbing
cfr. stochastic hill climbing by generating successors randomly until a better one is found.
Random-restart hill-climbing
Tries to avoid getting stuck in local maxima.
4 februari 2012
AI 1
56
Simulated annealing
Escape local maxima by allowing bad moves.
Idea: but gradually decrease their size and frequency.
Origin; metallurgical annealing Bouncing ball analogy:

Shaking hard (= high temperature). Shaking less (= lower the temperature).
If T decreases slowly enough, best state is reached. Applied for VLSI layout, airline scheduling, etc.
4 februari 2012
AI 1
57
Simulated annealing
function SIMULATED-ANNEALING( problem, schedule) return a solution state input: problem, a problem schedule, a mapping from time to temperature local variables: current, a node. next, a node. T, a temperature controlling the probability of downward steps current MAKE-NODE(INITIAL-STATE[problem]) for t 1 to do T schedule[t] if T = 0 then return current next a randomly selected successor of current E VALUE[next] - VALUE[current] if E > 0 then current next else current next only with probability eE /T
4 februari 2012
AI 1
58
Local beam search

Keep track of k states instead of one
Initially: k random states Next: determine all successors of k states If any of successors is goal finished Else select k best from successors and repeat.
Major difference with random-restart search

Information is shared among k search threads.
Can suffer from lack of diversity.

Stochastic variant: choose k successors at proportionally to state success.
4 februari 2012
AI 1
59
Genetic algorithms
Variant of local beam search with sexual recombination.
4 februari 2012
AI 1
60
Genetic algorithms
Variant of local beam search with sexual recombination.
4 februari 2012
AI 1
61
Genetic algorithm
function GENETIC_ALGORITHM( population, FITNESS-FN) return an individual input: population, a set of individuals FITNESS-FN, a function which determines the quality of the individual repeat new_population empty set loop for i from 1 to SIZE(population) do x RANDOM_SELECTION(population, FITNESS_FN) y RANDOM_SELECTION(population, FITNESS_FN) child REPRODUCE(x,y) if (small random probability) then child MUTATE(child ) add child to new_population population new_population until some individual is fit enough or enough time has elapsed return the best individual
4 februari 2012
AI 1
62
Exploration problems
Until now all algorithms were offline.
Offline= solution is determined before executing it. Online = interleaving computation and action
Online search is necessary for dynamic and semi-dynamic environments

It is impossible to take into account all possible contingencies.
Used for exploration problems:

Unknown states and actions. e.g. any robot in a new environment, a newborn baby,
4 februari 2012
AI 1
63
Online search problems

Agent knowledge:
ACTION(s): list of allowed actions in state s C(s,a,s): step-cost function (! After s is determined) GOAL-TEST(s)
An agent can recognize previous states. Actions are deterministic. Access to admissible heuristic h(s) e.g. manhattan distance
4 februari 2012
AI 1
64
Online search problems

Objective: reach goal with minimal cost
Cost = total cost of travelled path Competitive ratio=comparison of cost with cost of the solution path if search space is known. Can be infinite in case of the agent accidentally reaches dead ends
4 februari 2012
AI 1
65
The adversary argument
Assume an adversary who can construct the state space while the agent explores it
Visited states S and A. What next?
Fails in one of the state spaces
No algorithm can avoid dead ends in all state spaces.
4 februari 2012
AI 1
66
Online search agents

The agent maintains a map of the environment.
Updated based on percept input. This map is used to decide next action. Note difference with e.g. A* An online version can only expand the node it is physically in (local order)
4 februari 2012
AI 1
67
Online DF-search
function ONLINE_DFS-AGENT(s) return an action input: s, a percept identifying current state static: result, a table indexed by action and state, initially empty unexplored, a table that lists for each visited state, the action not yet tried unbacktracked, a table that lists for each visited state, the backtrack not yet tried s,a, the previous state and action, initially null if GOAL-TEST(s) then return stop if s is a new state then unexplored[s] ACTIONS(s) if s is not null then do result[a,s] s add s to the front of unbackedtracked[s] if unexplored[s] is empty then if unbacktracked[s] is empty then return stop else a an action b such that result[b, s]=POP(unbacktracked[s]) else a POP(unexplored[s]) s s return a
4 februari 2012
AI 1
68
Online DF-search, example

Assume maze problem on 3x3 grid. s = (1,1) is initial state Result, unexplored (UX), unbacktracked (UB), are empty S,a are also empty
4 februari 2012
AI 1
69

GOAL-TEST((,1,1))?
S not = G thus false
S=(1,1)
(1,1) a new state?

True ACTION((1,1)) -> UX[(1,1)]
{RIGHT,UP}
s is null?
True (initially)
UX[(1,1)] empty?
False
POP(UX[(1,1)])->a
A=UP
s = (1,1) Return a
4 februari 2012
AI 1
70

GOAL-TEST((2,1))?
S=(2,1)
(2,1) a new state?

True ACTION((2,1)) -> UX[(2,1)]
{DOWN}
s is null?
false (s=(1,1)) result[UP,(1,1)] <- (2,1) UB[(2,1)]={(1,1)}
UX[(2,1)] empty?
False
A=DOWN, s=(2,1) return A
4 februari 2012
AI 1
71

GOAL-TEST((1,1))?
S=(1,1)
(1,1) a new state?

false
s is null?
false (s=(2,1)) result[DOWN,(2,1)] <- (1,1) UB[(1,1)]={(2,1)}
UX[(1,1)] empty?
False
A=RIGHT, s=(1,1) return A
4 februari 2012
AI 1
72

GOAL-TEST((1,2))?
S=(1,2)
(1,2) a new state?

True, UX[(1,2)]={RIGHT,UP,LEFT}
s is null?
false (s=(1,1)) result[RIGHT,(1,1)] <- (1,2) UB[(1,2)]={(1,1)}
UX[(1,2)] empty?
False
A=LEFT, s=(1,2) return A
4 februari 2012
AI 1
73

GOAL-TEST((1,1))?
S not = G thus false false
(1,1) a new state?
S=(1,1)
s is null?
false (s=(1,2)) result[LEFT,(1,2)] <- (1,1) UB[(1,1)]={(1,2),(2,1)}
UX[(1,1)] empty?
True UB[(1,1)] empty? False
A= b for b in result[b,(1,1)]=(1,2)
B=RIGHT
A=RIGHT, s=(1,1)
4 februari 2012
AI 1
74
Online DF-search
Worst case each node is visited twice. An agent can go on a long walk even when it is close to the solution. An online iterative deepening approach solves this problem. Online DF-search works only when actions are reversible.
4 februari 2012
AI 1
75
Online local search

Hill-climbing is already online
One state is stored.
Bad performancd due to local maxima

Random restarts impossible.
Solution: Random walk introduces exploration (can produce exponentially many steps)
4 februari 2012
AI 1
76
Online local search

Solution 2: Add memory to hill climber
Store current best estimate H(s) of cost to reach goal H(s) is initially the heuristic estimate h(s) Afterward updated with experience (see below)
Learning real-time A* (LRTA*)
4 februari 2012
AI 1
77
Learning real-time A*
function LRTA*-COST(s,a,s,H) return an cost estimate if s is undefined the return h(s) else return c(s,a,s) + H[s] function LRTA*-AGENT(s) return an action input: s, a percept identifying current state static: result, a table indexed by action and state, initially empty H, a table of cost estimates indexed by state, initially empty s,a, the previous state and action, initially null if GOAL-TEST(s) then return stop if s is a new state (not in H) then H[s] h(s) unless s is null result[a,s] s H[s] MIN LRTA*-COST(s,b,result[b,s],H)
b ACTIONS(s)
a an action b in ACTIONS(s) that minimizes LRTA*-COST(s,b,result[b,s],H) s s return a
4 februari 2012
AI 1
78
Artificial Intelligence: Constraint Satisfaction problems

Outline
CSP? Backtracking for CSP Local search for CSPs Problem structure and decomposition
4 februari 2012
AI 1
Constraint satisfaction problems

What is a CSP?
Finite set of variables V1, V2, , Vn Finite set of variables C1, C2, , Cm Non-emtpy domain of possible values for each variable DV1, DV2, DVn Each constraint Ci limits the values that variables can take, e.g., V1 V2
A state is defined as an assignment of values to some or all variables. Consistent assignment: assignment does not violate the constraints.
4 februari 2012
AI 1
Constraint satisfaction problems

An assignment is complete when every value is mentioned. A solution to a CSP is a complete assignment that satisfies all constraints. Some CSPs require a solution that maximizes an objective function. Applications: Scheduling the time of observations on the Hubble Space Telescope, Floor planning, Map coloring, Cryptography
4 februari 2012
AI 1
CSP example: map coloring
Variables: WA, NT, Q, NSW, V, SA, T Domains: Di={red,green,blue} Constraints:adjacent regions must have different colors.
E.g. WA NT (if the language allows this) E.g. (WA,NT) {(red,green),(red,blue),(green,red),}
4 februari 2012
AI 1
CSP example: map coloring
Solutions are assignments satisfying all constraints, e.g. {WA=red,NT=green,Q=red,NSW=green,V=red,SA=blue,T=green}
4 februari 2012
AI 1
Constraint graph
CSP benefits
Standard representation pattern Generic goal and successor functions Generic heuristics (no domain specific expertise).
Constraint graph = nodes are variables, edges show constraints.

Graph can be used to simplify search.
e.g. Tasmania is an independent sub-problem.
4 februari 2012
AI 1
Varieties of CSPs
Discrete variables
Finite domains; size d O(dn) complete assignments.
E.g. Boolean CSPs, include. Boolean satisfiability (NP-complete).
Infinite domains (integers, strings, etc.)

E.g. job scheduling, variables are start/end days for each job Need a constraint language e.g StartJob1 +5 StartJob3. Linear constraints solvable, nonlinear undecidable.
Continuous variables
e.g. start/end times for Hubble Telescope observations. Linear constraints solvable but time costy.
4 februari 2012
AI 1
Varieties of constraints
Unary constraints involve a single variable. e.g. SA green Binary constraints involve pairs of variables.
e.g. SA WA
Higher-order constraints involve 3 or more variables.

e.g. cryptharithmetic column constraints.
Preference (soft constraints) e.g. red is better than green often representable by a cost for each variable assignment constrained optimization problems.
4 februari 2012
AI 1
CSP as a standard search problem

A CSP can easily expressed as a standard search problem. Incremental formulation
Initial State: the empty assignment {}. Successor function: Assign value to unassigned variable provided that there is not conflict. Goal test: the current assignment is complete. Path cost: as constant cost for every step.
4 februari 2012
AI 1
10
CSP as a standard search problem

This is the same for all CSPs !!! Solution is found at depth n (if there are n variables).
Hence depth first search can be used.
Path is irrelevant, so complete state representation can also be used. Branching factor b at the top level is nd. b=(n-l)d at depth l, hence n!dn leaves (only dn complete assignments).
4 februari 2012
AI 1
11
Commutativity
CSPs are commutative.
The order of any given set of actions has no effect on the outcome. Example: choose colors for Australian territories one at a time
[WA=red then NT=green] same as [NT=green then WA=red] All CSP search algorithms consider a single variable assignment at a time there are dn leaves.
4 februari 2012
AI 1
12
Backtracking search
Cfr. Depth-first search Chooses values for one variable at a time and backtracks when a variable has no legal values left to assign. Uninformed algorithm
No good general performance
4 februari 2012
AI 1
13
Backtracking search
function BACKTRACKING-SEARCH(csp) return a solution or failure return RECURSIVE-BACKTRACKING({} , csp) function RECURSIVE-BACKTRACKING(assignment, csp) return a solution or failure if assignment is complete then return assignment var SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],assignment,csp) for each value in ORDER-DOMAIN-VALUES(var, assignment, csp) do if value is consistent with assignment according to CONSTRAINTS[csp] then add {var=value} to assignment result RRECURSIVE-BACTRACKING(assignment, csp) if result failure then return result remove {var=value} from assignment return failure
4 februari 2012
AI 1
14
Backtracking example
4 februari 2012
AI 1
15
4 februari 2012
AI 1
16
4 februari 2012
AI 1
17
4 februari 2012
AI 1
18
Improving backtracking efficiency

Previous improvements introduce heuristics General-purpose methods can give huge gains in speed:
Which variable should be assigned next? In what order should its values be tried? Can we detect inevitable failure early? Can we take advantage of problem structure?
4 februari 2012
AI 1
19
Minimum remaining values
var SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],assignment,csp)
A.k.a. most constrained variable heuristic Rule: choose variable with the fewest legal moves Which variable shall we try first?
4 februari 2012
AI 1
20
Degree heuristic
Use degree heuristic Rule: select variable that is involved in the largest number of constraints on other unassigned variables. Degree heuristic is very useful as a tie breaker. In what order should its values be tried?
4 februari 2012
AI 1
21
Least constraining value
Least constraining value heuristic Rule: given a variable choose the least constraing value i.e. the one that leaves the maximum flexibility for subsequent variable assignments.
4 februari 2012
AI 1
22
Forward checking
Can we detect inevitable failure early?

And avoid it later?
Forward checking idea: keep track of remaining legal values for unassigned variables. Terminate search when any variable has no legal values.
4 februari 2012
AI 1
23
Forward checking
Assign {WA=red} Effects on other variables connected by constraints with WA

NT can no longer be red SA can no longer be red
4 februari 2012
AI 1
24
Forward checking
Assign {Q=green} Effects on other variables connected by constraints with WA

NT can no longer be green NSW can no longer be green SA can no longer be green
MRV heuristic will automatically select NT and SA next, why?
4 februari 2012
AI 1
25
Forward checking
If V is assigned blue Effects on other variables connected by constraints with WA

SA is empty NSW can no longer be blue
FC has detected that partial assignment is inconsistent with the constraints and backtracking can occur.
4 februari 2012
AI 1
26
Example: 4-Queens Problem

X1 {1,2,3,4} X2 {1,2,3,4}
1 1 2 3 4
X3 {1,2,3,4}
X4 {1,2,3,4}
[4-Queens slides copied from B.J. Dorr CMSC 421 course on AI]

X1 {1,2,3,4} X2 {1,2,3,4}
1 1 2 3 4
X3 {1,2,3,4}
X4 {1,2,3,4}
4 februari 2012
AI 1
28

X1 {1,2,3,4} X2 { , ,3,4}
1 1 2 3 4
X3 { ,2, ,4}
X4 { ,2,3, }
4 februari 2012
AI 1
29

X1 {1,2,3,4} X2 { , ,3,4}
1 1 2 3 4
X3 { ,2, ,4}
X4 { ,2,3, }
4 februari 2012
AI 1
30

X1 {1,2,3,4} X2 { , ,3,4}
1 1 2 3 4
X3 { , , , }
X4 { ,2,3, }
4 februari 2012
AI 1
31

X1 { ,2,3,4} X2 {1,2,3,4}
1 1 2 3 4
X3 {1,2,3,4}
X4 {1,2,3,4}
4 februari 2012
AI 1
32

X1 { ,2,3,4} X2 { , , ,4}
1 1 2 3 4
X3 {1, ,3, }
X4 {1, ,3,4}
4 februari 2012
AI 1
33

X1 { ,2,3,4} X2 { , , ,4}
1 1 2 3 4
X3 {1, ,3, }
X4 {1, ,3,4}
4 februari 2012
AI 1
34

X1 { ,2,3,4} X2 { , , ,4}
1 1 2 3 4
X3 {1, , , }
X4 {1, ,3, }
4 februari 2012
AI 1
35

X1 { ,2,3,4} X2 { , , ,4}
1 1 2 3 4
X3 {1, , , }
X4 {1, ,3, }
4 februari 2012
AI 1
36

X1 { ,2,3,4} X2 { , , ,4}
1 1 2 3 4
X3 {1, , , }
X4 { , ,3, }
4 februari 2012
AI 1
37

X1 { ,2,3,4} X2 { , , ,4}
1 1 2 3 4
X3 {1, , , }
X4 { , ,3, }
4 februari 2012
AI 1
38
Constraint propagation
Solving CSPs with combination of heuristics plus forward checking is more efficient than either approach alone. FC checking propagates information from assigned to unassigned variables but does not provide detection for all failures.
NT and SA cannot be blue!
Constraint propagation repeatedly enforces constraints locally
4 februari 2012
AI 1
39
Arc consistency
X Y is consistent iff for every value x of X there is some allowed y SA NSW is consistent iff SA=blue and NSW=red
4 februari 2012
AI 1
40
Arc consistency
X Y is consistent iff for every value x of X there is some allowed y NSW SA is consistent iff NSW=red and SA=blue NSW=blue and SA=??? Arc can be made consistent by removing blue from NSW
4 februari 2012
AI 1
41
Arc consistency
Arc can be made consistent by removing blue from NSW RECHECK neighbours !!
Remove red from V
4 februari 2012
AI 1
42
Arc consistency
Arc can be made consistent by removing blue from NSW RECHECK neighbours !!
Remove red from V
Arc consistency detects failure earlier than FC Can be run as a preprocessor or after each assignment.
Repeated until no inconsistency remains
4 februari 2012
AI 1
43
Arc consistency algorithm

function AC-3(csp) return the CSP, possibly with reduced domains inputs: csp, a binary csp with variables {X1, X2, , Xn} local variables: queue, a queue of arcs initially the arcs in csp while queue is not empty do (Xi, Xj) REMOVE-FIRST(queue) if REMOVE-INCONSISTENT-VALUES(Xi, Xj) then for each Xk in NEIGHBORS[Xi ] do add (Xi, Xj) to queue function REMOVE-INCONSISTENT-VALUES(Xi, Xj) return true iff we remove a value removed false for each x in DOMAIN[Xi] do if no value y in DOMAIN[Xi] allows (x,y) to satisfy the constraints between Xi and Xj then delete x from DOMAIN[Xi]; removed true return removed
K-consistency
Arc consistency does not detect all inconsistencies:
Partial assignment {WA=red, NSW=red} is inconsistent.
Stronger forms of propagation can be defined using the notion of kconsistency. A CSP is k-consistent if for any set of k-1 variables and for any consistent assignment to those variables, a consistent value can always be assigned to any kth variable.
E.g. 1-consistency or node-consistency E.g. 2-consistency or arc-consistency E.g. 3-consistency or path-consistency
4 februari 2012
AI 1
45
K-consistency
A graph is strongly k-consistent if
It is k-consistent and Is also (k-1) consistent, (k-2) consistent, all the way down to 1consistent.
This is ideal since a solution can be found in time O(nd) instead of O(n2d3) YET we didnt win already: any algorithm for establishing n-consistency must take time exponential in n, in the worst case.
4 februari 2012
AI 1
46
Further improvements
Checking special constraints
Checking Alldif() constraint
E.g. {WA=red, NSW=red}
Checking Atmost() constraint

Bounds propagation for larger value domains
Intelligent backtracking
Standard form is chronological backtracking i.e. try different value for preceding variable. More intelligent, backtrack to conflict set.
Set of variables that caused the failure or set of previously assigned variables that are connected to X by constraints. Backjumping moves back to most recent element of the conflict set. Forward checking can be used to determine conflict set.
4 februari 2012
AI 1
47
Local search for CSP

Use complete-state representation For CSPs
allow states with unsatisfied constraints operators reassign variable values
Variable selection: randomly select any conflicted variable Value selection: min-conflicts heuristic
Select new value that results in a minimum number of conflicts with the other variables
4 februari 2012
AI 1
48
Local search for CSP

function MIN-CONFLICTS(csp, max_steps) return solution or failure inputs: csp, a constraint satisfaction problem max_steps, the number of steps allowed before giving up current an initial complete assignment for csp for i = 1 to max_steps do if current is a solution for csp then return current var a randomly chosen, conflicted variable from VARIABLES[csp] value the value v for var that minimize CONFLICTS(var,v,current,csp) set var = value in current return failure
4 februari 2012
AI 1
49
Min-conflicts example 1
h=5
h=3
h=1
Use of min-conflicts heuristic in hill-climbing.
4 februari 2012
AI 1
50
Min-conflicts example 2
A two-step solution for an 8-queens problem using min-conflicts heuristic. At each stage a queen is chosen for reassignment in its column. The algorithm moves the queen to the min-conflict square breaking ties randomly.
4 februari 2012
AI 1
51
Advantages of local search

The runtime of min-conflicts is roughly independent of problem size.
Solving the millions-queen problem in roughly 50 steps.
Local search can be used in an online setting.

Backtrack search requires more time
4 februari 2012
AI 1
52
Problem structure
How can the problem structure help to find a solution quickly? sub-problem identification is important:
Coloring Tasmania and mainland are independent sub-problems Identifiable as connected components of constrained graph.
Improves performance
4 februari 2012
AI 1
53
Problem structure
Suppose each problem has c variables out of a total of n. Worst case solution cost is O(n/c dc), i.e. linear in n
Instead of O(d n), exponential in n
E.g. n= 80, c= 20, d=2

280 = 4 billion years at 1 million nodes/sec. 4 * 220= .4 second at 1 million nodes/sec
Tree-structured CSPs
Theorem: if the constraint graph has no loops then CSP can be solved in O(nd 2) time Compare difference with general CSP, where worst case is O(d n)
4 februari 2012
AI 1
55
Tree-structured CSPs
In most cases sub-problems of a CSP are connected as a tree Any tree-structured CSP can be solved in time linear in the number of variables.
Choose a variable as root, order variables from root to leaves such that every nodes parent precedes it in the ordering. (label var from X1 to Xn) For j from n down to 2, apply REMOVE-INCONSISTENT-VALUES(Parent(Xj),Xj) For j from 1 to n assign Xj consistently with Parent(Xj )
4 februari 2012
AI 1
56
Nearly tree-structured CSPs
Can more general constraint graphs be reduced to trees? Two approaches:

Remove certain nodes Collapse certain nodes
4 februari 2012
AI 1
57
Idea: assign values to some variables so that the remaining variables form a tree. Assume that we assign {SA=x} cycle cutset
And remove any values from the other variables that are inconsistent. The selected value for SA could be the wrong one so we have to try all of them
This approach is worthwhile if cycle cutset is small. Finding the smallest cycle cutset is NP-hard
Approximation algorithms exist
This approach is called cutset conditioning.


Tree decomposition of the constraint graph in a set of connected subproblems. Each sub-problem is solved independently Resulting solutions are combined. Necessary requirements:
Every variable appears in ar least one of the sub-problems. If two variables are connected in the original problem, they must appear together in at least one sub-problem. If a variable appears in two sub-problems, it must appear in each node on the path.
4 februari 2012
AI 1
60
CSP and sudoku

Variabes?
81
Domains?
1-9
Constraints?
There may not be 2 numbers in the same row and column and in a 3x3 group box.
4 februari 2012
AI 1
61
Summary
CSPs are a special kind of problem: states defined by values of a fixed set of variables, goal test defined by constraints on variable values Backtracking=depth-first search with one variable assigned per node Variable ordering and value selection heuristics help significantly Forward checking prevents assignments that lead to failure. Constraint propagation does additional work to constrain values and detect inconsistencies. The CSP representation allows analysis of problem structure. Tree structured CSPs can be solved in linear time. Iterative min-conflicts is usually effective in practice.
4 februari 2012
AI 1
62
Artificial Intelligence: game playing

Outline
What are games? Optimal decisions in games
Which strategy leads to success?
- pruning Games of imperfect information Games that include an element of chance
4 februari 2012
AI 1
What are and why study games?

Games are a form of multi-agent environment
What do other agents do and how do they affect our success? Cooperative vs. competitive multi-agent environments. Competitive multi-agent environments give rise to adversarial search
Why study games?

Fun; historically entertaining Interesting subject of study because they are hard problems Easy to represent and agents restricted to small number of actions
4 februari 2012
AI 1
Relation of Games to Search

Search no adversary
Solution is (heuristic) method for finding goal Heuristics and CSP techniques can find optimal solution Evaluation function: estimate of cost from start to goal through given node Examples: path planning, scheduling activities
Games adversary
Solution is strategy (strategy specifies move for every possible opponent reply). Time limits force an approximate solution Evaluation function: evaluate goodness of game position Examples: chess, checkers, Othello, backgammon
4 februari 2012
AI 1
Types of Games
4 februari 2012
AI 1
Game setup
Two players: MAX and MIN MAX moves first and they take turns until the game is over. Winner gets award, looser gets penalty. Games as search:
Initial state: e.g. board configuration of chess (TICTACTOE) Successor function: list of (move,state) pairs specifying legal moves. Terminal test: Is the game finished? Utility function: Gives numerical value of terminal states. E.g. win (+1), loose (-1) and draw (0) in tic-tac-toe (next)
MAX uses search tree to determine next move.
4 februari 2012
AI 1
Partial Game Tree for Tic-Tac-Toe
4 februari 2012
AI 1
Optimal strategies
Find the contingent strategy for MAX assuming an infallible MIN opponent. Assumption: Both players play optimally !! Given a game tree, the optimal strategy can be determined by using the minimax value of each node: MINIMAX-VALUE(n)= UTILITY(n) If n is a terminal maxs successors(n) MINIMAX-VALUE(s) If n is a max node mins successors(n) MINIMAX-VALUE(s) If n is a min node
4 februari 2012
AI 1
Two-Ply Game Tree
4 februari 2012
AI 1
Two-Ply Game Tree
4 februari 2012
AI 1
10
Two-Ply Game Tree
4 februari 2012
AI 1
11
Two-Ply Game Tree
The minimax decision
Minimax maximizes the worst-case outcome for max.

What if MIN does not play optimally?
Definition of optimal play for MAX assumes MIN plays optimally: maximizes worst-case outcome for MAX. But if MIN does not play optimally, MAX will do even better. [proven.]
4 februari 2012
AI 1
13
Minimax Algorithm
function MINIMAX-DECISION(state) returns an action inputs: state, current state in game vMAX-VALUE(state) return the action in SUCCESSORS(state) with value v function MAX-VALUE(state) returns a utility value if TERMINAL-TEST(state) then return UTILITY(state) v for a,s in SUCCESSORS(state) do v MAX(v,MIN-VALUE(s)) return v function MIN-VALUE(state) returns a utility value if TERMINAL-TEST(state) then return UTILITY(state) v for a,s in SUCCESSORS(state) do v MIN(v,MAX-VALUE(s)) return v
Properties of Minimax
Criterion Minimax
Time
O(bm)
Space
O(bm)
4 februari 2012
AI 1
15
Multiplayer games
Games allow more than two players Single minimax values become vectors
4 februari 2012
AI 1
16
Problem of minimax search

Number of games states is exponential to the number of moves.
Solution: Do not examine every node ==> Alpha-beta pruning
Remove branches that do not influence final decision
Revisit example
4 februari 2012
AI 1
17
Alpha-Beta Example
Do DF-search until first leaf
Range of possible values
[-,+]
[-, +]
4 februari 2012
AI 1
18
Alpha-Beta Example (continued)
[-,+]
[-,3]
4 februari 2012
AI 1
19
[-,+]
[-,3]
4 februari 2012
AI 1
20
[3,+]
[3,3]
4 februari 2012
AI 1
21
[3,+]
This node is worse for MAX
[3,3]
[-,2]
4 februari 2012
AI 1
22
[3,14]
[3,3]
[-,2]
[-,14]
4 februari 2012
AI 1
23
[3,5]
[3,3]
[,2]
[-,5]
4 februari 2012
AI 1
24
[3,3]
[3,3]
[,2]
[2,2]
4 februari 2012
AI 1
25
[3,3]
[3,3]
[-,2]
[2,2]
4 februari 2012
AI 1
26
Alpha-Beta Algorithm
function ALPHA-BETA-SEARCH(state) returns an action inputs: state, current state in game vMAX-VALUE(state, - , +) return the action in SUCCESSORS(state) with value v function MAX-VALUE(state, , ) returns a utility value if TERMINAL-TEST(state) then return UTILITY(state) v- for a,s in SUCCESSORS(state) do v MAX(v,MIN-VALUE(s, , )) if v then return v MAX( ,v) return v
Alpha-Beta Algorithm
function MIN-VALUE(state, , ) returns a utility value if TERMINAL-TEST(state) then return UTILITY(state) v+ for a,s in SUCCESSORS(state) do v MIN(v,MAX-VALUE(s, , )) if v then return v MIN( ,v) return v
4 februari 2012
AI 1
28
General alpha-beta pruning

Consider a node n somewhere in the tree If player has a better choice at
Parent node of n Or any choice point further up
n will never be reached in actual play. Hence when enough is known about n, it can be pruned.
4 februari 2012
AI 1
29
Final Comments about Alpha-Beta Pruning

Pruning does not affect final results Entire subtrees can be pruned. Good move ordering improves effectiveness of pruning With perfect ordering, time complexity is O(bm/2)
Branching factor of sqrt(b) !! Alpha-beta pruning can look twice as far as minimax in the same amount of time
Repeated states are again possible.

Store them in memory = transposition table
4 februari 2012
AI 1
30
Games of imperfect information

Minimax and alpha-beta pruning require too much leaf-node evaluations. May be impractical within a reasonable amount of time. SHANNON (1950):
Cut off search earlier (replace TERMINAL-TEST by CUTOFFTEST) Apply heuristic evaluation function EVAL (replacing utility function of alpha-beta)
4 februari 2012
AI 1
31
Cutting off search

Change: if TERMINAL-TEST(state) then return UTILITY(state) into
if CUTOFF-TEST(state,depth) then return EVAL(state)
Introduces a fixed-depth limit depth

Is selected so that the amount of time will not exceed what the rules of the game allow.
When cuttoff occurs, the evaluation is performed.
4 februari 2012
AI 1
32
Heuristic EVAL
Idea: produce an estimate of the expected utility of the game from a given position. Performance depends on quality of EVAL. Requirements:
EVAL should order terminal-nodes in the same way as UTILITY. Computation may not take too long. For non-terminal states the EVAL should be strongly correlated with the actual chance of winning.
Only useful for quiescent (no wild swings in value in near future) states
4 februari 2012
AI 1
33
Heuristic EVAL example
Eval(s) = w1 f1(s) + w2 f2(s) + + wnfn(s)

Heuristic EVAL example
Addition assumes independence
Eval(s) = w1 f1(s) + w2 f2(s) + + wnfn(s)

Heuristic difficulties
Heuristic counts pieces won
4 februari 2012
AI 1
36
Horizon effect
Fixed depth search thinks it can avoid the queening move
4 februari 2012
AI 1
37
Games that include chance
Possible moves (5-10,5-11), (5-11,19-24),(5-10,10-16) and (5-11,1116)
4 februari 2012
AI 1
38

chance nodes
Possible moves (5-10,5-11), (5-11,19-24),(5-10,10-16) and (5-11,11-16) [1,1], [6,6] chance 1/36, all other chance 1/18
4 februari 2012
AI 1
39
[1,1], [6,6] chance 1/36, all other chance 1/18

Can not calculate definite minimax value, only expected value
4 februari 2012
AI 1
40
Expected minimax value

EXPECTED-MINIMAX-VALUE(n)= UTILITY(n) If n is a terminal maxs successors(n) MINIMAX-VALUE(s) If n is a max node mins successors(n) MINIMAX-VALUE(s) If n is a max node s successors(n) P(s) . EXPECTEDMINIMAX(s) If n is a chance node
These equations can be backed-up recursively all the way to the root of the game tree.
4 februari 2012
AI 1
41
Position evaluation with chance nodes
Left, A1 wins Right A2 wins Outcome of evaluation function may not change when values are scaled differently. Behavior is preserved only by a positive linear transformation of EVAL.
4 februari 2012
AI 1
42
Discussion
Examine section on state-of-the-art games yourself Minimax assumes right tree is better than left, yet
Return probability distribution over possible values Yet expensive calculation
4 februari 2012
AI 1
43
Discussion
Utility of node expansion
Only expand those nodes which lead to significanlty better moves
Both suggestions require meta-reasoning
4 februari 2012
AI 1
44
Summary
Games are fun (and dangerous) They illustrate several important points about AI
Perfection is unattainable -> approximation Good idea what to think about Uncertainty constrains the assignment of values to states
Games are to AI as grand prix racing is to automobile design.
4 februari 2012
AI 1
45
Artificial Intelligence: Logic agents

Thinking Rationally
Computational models of human thought processes Computational models of human behavior Computational systems that think rationally Computational systems that behave rationally
4 februari 2012
AI 1
Logical Agents
Reflex agents find their way from Arad to Bucharest by dumb luck Chess program calculates legal moves of its king, but doesnt know that no piece can be on 2 different squares at the same time Logic (Knowledge-Based) agents combine general knowledge with current percepts to infer hidden aspects of current state prior to selecting actions
Crucial in partially observable environments
4 februari 2012
AI 1
Outline
Knowledge-based agents Wumpus world Logic in general Propositional and first-order logic
Inference, validity, equivalence and satisfiability Reasoning patterns
Resolution Forward/backward chaining
4 februari 2012
AI 1
Knowledge Base
Knowledge Base : set of sentences represented in a knowledge representation language and represents assertions about the world.
tell
ask
Inference rule: when one ASKs questions of the KB, the answer should follow from what has been TELLed to the KB previously.
4 februari 2012
AI 1
Generic KB-Based Agent
4 februari 2012
AI 1
Abilities KB agent
Agent must be able to:
Represent states and actions, Incorporate new percepts Update internal representation of the world Deduce hidden properties of the world Deduce appropriate actions
4 februari 2012
AI 1
Description level
The KB agent is similar to agents with internal state Agents can be described at different levels
Knowledge level
What they know, regardless of the actual implementation. (Declarative description)
Implementation level
Data structures in KB and algorithms that manipulate them e.g propositional logic and resolution.
4 februari 2012
AI 1
A Typical Wumpus World
Wumpus
4 februari 2012
AI 1
Wumpus World PEAS Description
4 februari 2012
AI 1
10
Wumpus World Characterization

Observable? Deterministic? Episodic? Static? Discrete? Single-agent?
4 februari 2012
AI 1
11

Observable? No, only local perception Deterministic? Episodic? Static? Discrete? Single-agent?
4 februari 2012
AI 1
12

Observable? No, only local perception Deterministic? Yes, outcome exactly specified Episodic? Static? Discrete? Single-agent?
4 februari 2012
AI 1
13

Observable? No, only local perception Deterministic? Yes, outcome exactly specified Episodic? No, sequential at the level of actions Static? Discrete? Single-agent?
4 februari 2012
AI 1
14

Observable? No, only local perception Deterministic? Yes, outcome exactly specified Episodic? No, sequential at the level of actions Static? Yes, Wumpus and pits do not move Discrete? Single-agent?
4 februari 2012
AI 1
15

Observable? No, only local perception Deterministic? Yes, outcome exactly specified Episodic? No, sequential at the level of actions Static? Yes, Wumpus and pits do not move Discrete? Yes Single-agent?
4 februari 2012
AI 1
16

Observable? No, only local perception Deterministic? Yes, outcome exactly specified Episodic? No, sequential at the level of actions Static? Yes, Wumpus and pits do not move Discrete? Yes Single-agent? Yes, Wumpus is essentially a natural feature.
4 februari 2012
AI 1
17
Exploring the Wumpus World
[1,1] The KB initially contains the rules of the environment. The first percept is [none, none,none,none,none], move to safe cell e.g. 2,1 [2,1] breeze which indicates that there is a pit in [2,2] or [3,1], return to [1,1] to try next safe cell
4 februari 2012
AI 1
18
[1,2] Stench in cell which means that wumpus is in [1,3] or [2,2] YET not in [1,1] YET not in [2,2] or stench would have been detected in [2,1] THUS wumpus is in [1,3] THUS [2,2] is safe because of lack of breeze in [1,2] THUS pit in [1,3]
move to next safe cell [2,2]
4 februari 2012
AI 1
19
[2,2] move to [2,3] [2,3] detect glitter , smell, breeze THUS pick up gold THUS pit in [3,3] or [2,4]
4 februari 2012
AI 1
20
What is a logic?
A formal language
Syntax what expressions are legal (well-formed) Semantics what legal expressions mean
in logic the truth of each sentence with respect to each possible world.
E.g the language of arithmetic

X+2 >= y is a sentence, x2+y is not a sentence X+2 >= y is true in a world where x=7 and y =1 X+2 >= y is false in a world where x=0 and y =6
4 februari 2012
AI 1
21
Entailment
One thing follows from another KB |= KB entails sentence if and only if is true in worlds where KB is true. .g. x+y=4 entails 4=x+y Entailment is a relationship between sentences that is based on semantics.
4 februari 2012
AI 1
22
Models
Logicians typically think in terms of models, which are formally structured worlds with respect to which truth can be evaluated. m is a model of a sentence if is true in m M() is the set of all models of
4 februari 2012
AI 1
23
Wumpus world model
4 februari 2012
AI 1
24
Wumpus world model
4 februari 2012
AI 1
25
Wumpus world model
4 februari 2012
AI 1
26
Wumpus world model
4 februari 2012
AI 1
27
Wumpus world model
4 februari 2012
AI 1
28
Wumpus world model
4 februari 2012
AI 1
29
Logical inference
The notion of entailment can be used for logic inference.
Model checking (see wumpus example): enumerate all possible models and check whether is true.
If an algorithm only derives entailed sentences it is called sound or truth preserving.

Otherwise it just makes things up.
i is sound if whenever KB |-i it is also true that KB|=
Completeness : the algorithm can derive any sentence that is entailed.

i is complete if whenever KB |= it is also true that KB|-i
4 februari 2012
AI 1
30
Schematic perspective
If KB is true in the real world, then any sentence derived From KB by a sound inference procedure is also true in the real world.
4 februari 2012
AI 1
31
Artificial Intelli-gence: logic agents

Standard Logical Equivalences
February 4, 2012
Terminology
A sentence is valid if its truth value is t in all interpretations
Valid sentences: true, false, P P
A sentence is satisfiable if its truth value is t in at least one interpretation

Satisfiable sentences: P, true, P
A sentence is unsatisfiable iff its truth value is f in all interpretations

Unsatisfiable sentences: P P, false, true
February 4, 2012 3
Inference
KB i Soundness: Inference procedure i is sound if whenever KB i , it is also true that KB Completeness: Inference procedure i is complete if whenever KB , it is also true that KB i
February 4, 2012 4
Rules of Inference
Valid Rules of Inference:
Modus Ponens And-Elimination And-Introduction Or-Introduction Double Negation Unit Resolution Resolution
February 4, 2012
Searching for Proofs

Finding proofs is exactly like finding solutions to search problems. Can search forward (forward chaining) to derive goal or search backward (backward chaining) from the goal. Searching for proofs is not more efficient than enumerating models, but in many practical cases, its more efficient because we can ignore irrelevant propositions
February 4, 2012
Full Resolution Rule Revisited

Start with Unit Resolution Inference Rule: Full Resolution Rule is a generalization of this rule: For clauses of length two:
February 4, 2012
Resolution: Complete Inference Procedure

Any complete search algorithm, applying only the resolution rule, can derive any conclusion entailed by any knowledge base in propositional logic. Refutation completeness: Resolution can always be used to either confirm or refute a sentence, but it cannot be used to enumerate true sentences.
February 4, 2012 8
Horn Clauses
Real World KBs are often a conjunction of Horn clauses Horn clause =
proposition symbol; or (conjunction of symbols) symbol
February 4, 2012
Forward Chaining
Fire any rule whose premises are satisfied in the KB. Add its conclusion to the KB until query is found.
February 4, 2012
10
Backward Chaining
Motivation: Need goal-directed reasoning in order to keep from getting overwhelmed with irrelevant consequences Main idea:
Work backwards from query q To prove q:
Check if q is known already Prove by backward chaining all premises of some rule concluding q
February 4, 2012
11
Forward Chaining vs. Backward Chaining

FC is data-drivenit may do lots of work irrelevant to the goal BC is goal-drivenappropriate for problem-solving
February 4, 2012
12
Artificial Intelligence: Agents and Propositional Logic.

Propositional Logic
4 februari 2012
AI 1
Propositional Logic
4 februari 2012
AI 1
Propositional Logic
4 februari 2012
AI 1
Wumpus world logic
4 februari 2012
AI 1
Wumpus world logic
4 februari 2012
AI 1
Truth tables for inference

Enumerate the models and check that is true in every model In which KB is true.
4 februari 2012
AI 1
Inference by enumeration
Depth-first enumeration of all models is sound and complete
For n symbols, time complexity is O(2n), space complexity is O(n).
4 februari 2012
AI 1
Logical equivalence
Two sentences are logically equivalent iff true in same set of models or iff |= and |= .
4 februari 2012
AI 1
Validity and satisfiability

A sentence is valid if it is true in all models,
e.g., True, A A, A A, (A (A B)) B
Validity is connected to inference via the Deduction Theorem:

KB |= if and only if (KB ) is valid
A sentence is satisfiable if it is true in some model

e.g., A B, C e.g., AA
A sentence is unsatisfiable if it is true in no models Satisfiability is connected to inference via the following: KB |= if and only if (KB ) is unsatisfiable
Remember proof by contradiction.
4 februari 2012
AI 1
10
Inference rules in PL
Modens Ponens
And-elimination: from a conjuction any conjunction can be inferred:
All logical equivalences can be used as inference rules.
( ) ( )
4 februari 2012
AI 1
11
Example
Assume R1 through R5: How can we prove P1,2?
P1,1,B1,1 (P1,1 P2,1 ),B2,1 (P1,1 P2,2 P3,1 ),B1,1,B2,1

R6 : (B1,1 (P1,2 P2,1 )) ((P1,2 P2,1 ) B1,1 ) R7 : (P1,2 P2,1 ) B1,1 R8 :B1,1 (P1,2 P2,1 ) R9 :(P1,2 P2,1 ) R10 :P1,2 P2,1
4 februari 2012 AI 1
Biconditional elim. And elim. Contraposition Modens ponens Morgans rule

12
Searching for proofs

Finding proofs is exactly like finding solutions to search problems. Search can be done forward (forward chaining) to derive goal or backward (backward chaining) from the goal. Searching for proofs is not more efficient than enumerating models, but in many practical cases, it is more efficient because we can ignore irrelevant properties. Monotonicity: the set of entailed sentences can only increase as information is added to the knowledge base.
for any sentence and : if KB |= then KB |=
4 februari 2012
AI 1
13
Proof methods
Proof methods divide into (roughly) two kinds: Application of inference rules
Legitimate (sound) generation of new sentences from old Proof = a sequence of inference rule application can use inference rules as operators in a standard search algorithm Typically require transformation of sentences into a normal form
Model checking
truth table enumeration (always exponential in n) improved backtracking, e.g., Davis--Putnam-Logemann-Loveland (DPLL) heuristic search in model space (sound but incomplete) e.g., min-conflicts-like hill-climbing algorithms
4 februari 2012
AI 1
14
Resolution
Start with Unit Resolution Inference Rule: Full Resolution Rule is a generalization of this rule: For clauses of length two:
4 februari 2012
AI 1
15
Resolution in Wumpus world

At some point we can derive the absence of a pit in square 2,2:
P1,1,B1,1 (P1,1 P2,1 ),B2,1 (P1,1 P2,2 P3,1 ),B1,1,B2,1,P2,2 ,P3,1
Now after biconditional elimination of R3 followed by a modens ponens with R5: R15 : (P1,1 P2,2 P3,1) Resolution :
P1,1 P2,2 P3,1 P2,2 P1,1 P3,1
4 februari 2012
AI 1
16
Resolution
Uses CNF (Conjunctive normal form)
Conjunction of disjunctions of literals (clauses)
The resolution rule is sound:

Only entailed sentences are derived
Resolution is complete in the sense that it can always be used to either confirm or refute a sentence (it can not be used to enumerate true sentences.)
4 februari 2012
AI 1
17
Conversion to CNF
B1,1 (P1,2 P2,1) Eliminate , replacing with ( )( ).
(B1,1 (P1,2 P2,1)) ((P1,2 P2,1) B1,1) (B1,1 P1,2 P2,1) ((P1,2 P2,1) B1,1)
Eliminate , replacing with .

Move inwards using de Morgan's rules and double-negation:

(B1,1 P1,2 P2,1) ((P1,2 P2,1) B1,1)
Apply distributivity law ( over ) and flatten:
(B1,1 P1,2 P2,1) (P1,2 B1,1) (P2,1 B1,1)
4 februari 2012
AI 1
18
Resolution algorithm
Proof by contradiction, i.e., show KB unsatisfiable
4 februari 2012
AI 1
19
Resolution algorithm
First KB is converted into CNF Then apply resolution rule to resulting clauses. The process continues until:
There are no new clauses that can be added
Hence does not ential
Two clauses resolve to entail the empty clause.

Hence does ential
4 februari 2012
AI 1
20
Resolution example
KB = (B1,1 (P1,2 P2,1)) B1,1 = P1,2
4 februari 2012
AI 1
21
Forward and backward chaining

The completeness of resolution makes it a very important inference model. Real-world knowledge only requires a restricted form of clauses:
Horn clauses = disjunction of literals with at most one positive literal Three important properties
Can be written as an implication Inference through forward chaining and backward chaining. Deciding entailment can be done in a time linear size of the knowledge base.
(L1,1 Breeze B1,1) (L1,1 Breeze) B1,1
4 februari 2012
AI 1
22
Forward chaining
Idea: fire any rule whose premises are satisfied in the KB,
add its conclusion to the KB, until query is found
4 februari 2012
AI 1
23
Forward chaining algorithm

Forward chaining is sound and complete for Horn KB
4 februari 2012
AI 1
24
Forward chaining example
4 februari 2012
AI 1
25
4 februari 2012
AI 1
26
4 februari 2012
AI 1
27
4 februari 2012
AI 1
28
4 februari 2012
AI 1
29
4 februari 2012
AI 1
30
4 februari 2012
AI 1
31
4 februari 2012
AI 1
32
Proof of completeness
FC derives every atomic sentence that is entailed by KB
1. 2. 3. 4. 5. FC reaches a fixed point where no new atomic sentences are derived. Consider the final state as a model m, assigning true/false to symbols. Every clause in the original KB is true in m
a1 a k b
Hence m is a model of KB If KB |= q, q is true in every model of KB, including m
4 februari 2012
AI 1
33
Backward chaining
Idea: work backwards from the query q:
to prove q by BC,
check if q is known already, or prove by BC all premises of some rule concluding q
Avoid loops: check if new subgoal is already on the goal stack Avoid repeated work: check if new subgoal
1. 2.
has already been proved true, or has already failed
4 februari 2012
AI 1
34
Backward chaining example
4 februari 2012
AI 1
35
4 februari 2012
AI 1
36
4 februari 2012
AI 1
37
4 februari 2012
AI 1
38
4 februari 2012
AI 1
39
4 februari 2012
AI 1
40
4 februari 2012
AI 1
41
4 februari 2012
AI 1
42
4 februari 2012
AI 1
43
4 februari 2012
AI 1
44
Forward vs. backward chaining

FC is data-driven, automatic, unconscious processing,
e.g., object recognition, routine decisions
May do lots of work that is irrelevant to the goal BC is goal-driven, appropriate for problem-solving,
e.g., Where are my keys? How do I get into a PhD program?
Complexity of BC can be much less than linear in size of KB
4 februari 2012
AI 1
45
Effective propositional inference

Two families of efficient algorithms for propositional inference based on model checking: Are used for checking satisfiability Complete backtracking search algorithms
DPLL algorithm (Davis, Putnam, Logemann, Loveland)
Improves TT-Entails? Algorithm.
Incomplete local search algorithms

WalkSAT algorithm
4 februari 2012
AI 1
46
The DPLL algorithm

Determine if an input propositional logic sentence (in CNF) is satisfiable. Improvements over truth table enumeration:
1. Early termination
A clause is true if any literal is true. A sentence is false if any clause is false. E.g. (B1,1 P1,2 P2,1) (P1,2 B1,1) (P2,1 B1,1)
2. Pure symbol heuristic

Pure symbol: always appears with the same "sign" in all clauses. e.g., In the three clauses (A B), (B C), (C A), A and B are pure, C is impure. Assign a pure symbol so that their literals are true.
3. Unit clause heuristic

Unit clause: only one literal in the clause or only one literal which has not yet received a value. The only literal in a unit clause must be true. First do this assignments before continuing with the rest (unit propagation!).
4 februari 2012
AI 1
47
The DPLL algorithm
4 februari 2012
AI 1
48
The WalkSAT algorithm

Incomplete, local search algorithm. Evaluation function: The min-conflict heuristic of minimizing the number of unsatisfied clauses. Steps are taken in the space of complete assignments, flipping the truth value of one variable at a time. Balance between greediness and randomness.
To avoid local minima
4 februari 2012
AI 1
49
The WalkSAT algorithm
4 februari 2012
AI 1
50
Hard satisfiability problems

Underconstrained problems are easy: e.g n-queens in CSP. In SAT: e.g.,
(D B C) (B A C) (C B E) (E D B) (B E C)
Increase in complexity by keeping the number of symbols fixed and increasing the amount of clauses.
m = number of clauses n = number of symbols Hard problems seem to cluster near m/n = 4.3 (critical point)
4 februari 2012
AI 1
51
4 februari 2012
AI 1
52

Median runtime for 100 satisfiable random 3-CNF sentences, n = 50
4 februari 2012
AI 1
53
Inference-based agents in the wumpus world

A wumpus-world agent using propositional logic ( a knowledge base about the physics of the W-world):
P1,1 W1,1 Bx,y (Px,y+1 Px,y-1 Px+1,y Px-1,y) Sx,y (Wx,y+1 Wx,y-1 Wx+1,y Wx-1,y) W1,1 W1,2 W4,4 (at least one wumpus) W1,1 W1,2 (at most one wumpus) W1,1 W1,3
64 distinct proposition symbols, 155 sentences A fringe square is provably safe if the sentence is entailed by the knowledge base.
4 februari 2012 AI 1
(P
i, j
W i, j )
54
4 februari 2012
AI 1
55
Expressiveness limitation of propositional logic

KB contains "physics" sentences for every single square
With all consequences for large KB
Better would be to have just two sentences for breezes and stenches for all squares.
Impossible for propositional logic.
Simplification in agent: location info is not in KB!!

For every time t and every location [x,y], Ltx,y FacingRightt Forwardt Ltx+1,y PROBLEM: Rapid proliferation of clauses.
4 februari 2012
AI 1
56
Summary
Logical agents apply inference to a knowledge base to derive new information and make decisions. Basic concepts of logic:
syntax: formal structure of sentences semantics: truth of sentences wrt models entailment: necessary truth of one sentence given another inference: deriving sentences from other sentences soundness: derivations produce only entailed sentences completeness: derivations can produce all entailed sentences
Wumpus world requires the ability to represent partial and negated information, reason by cases, etc. Resolution is complete for propositional logic Forward, backward chaining are linear-time, complete for Horn clauses Propositional logic lacks expressive power
Artificial Intelligence: Agents and First-Order Logic

Outline
Why FOL? Syntax and semantics of FOL Using FOL Wumpus world in FOL Knowledge engineering in FOL
4 februari 2012
AI 1
Pros and cons of propositional logic

Propositional logic is declarative Propositional logic allows partial/disjunctive/negated information
(unlike most data structures and databases)
Propositional logic is compositional:
meaning of B1,1 P1,2 is derived from meaning of B1,1 and of P1,2 (unlike natural language, where meaning depends on context)
Meaning in propositional logic is context-independent Propositional logic has very limited expressive power
(unlike natural language) E.g., cannot say "pits cause breezes in adjacent squares
except by writing one sentence for each square
4 februari 2012
AI 1
First-order logic
Whereas propositional logic assumes the world contains facts, first-order logic (like natural language) assumes the world contains
Objects: people, houses, numbers, colors, baseball games, wars, Relations: red, round, prime, brother of, bigger than, part of, comes between, Functions: father of, best friend, one more than, plus,
4 februari 2012
AI 1
Logics in General
Ontological Commitment: What exists in the world TRUTH
PL : facts hold or do not hold. FL : objects with relatios between them that hold or do not hold
Epistemoligical Commitment: What an agent believes about facts BELIEF
4 februari 2012
AI 1
Syntax of FOL: Basic elements

Constants KingJohn, 2, NUS,... Predicates Brother, >,... Functions Sqrt, LeftLegOf,... Variables x, y, a, b,... Connectives , , , , Equality = Quantifiers ,
4 februari 2012
AI 1
Atomic sentences
Atomic sentence = predicate (term1,...,termn) or term1 = term2 Term = function (term1,...,termn) or constant or variable
E.g., Brother(KingJohn,RichardTheLionheart) > (Length(LeftLegOf(Richard)), Length(LeftLegOf(KingJohn)))
4 februari 2012
AI 1
Complex sentences
Complex sentences are made from atomic sentences using connectives
S, S1 S2, S1 S2, S1 S2, S1 S2,
E.g. Sibling(KingJohn,Richard) Sibling(Richard,KingJohn) >(1,2) (1,2) >(1,2) >(1,2)
4 februari 2012
AI 1
Truth in first-order logic

Sentences are true with respect to a model and an interpretation Model contains objects (domain elements) and relations among them Interpretation specifies referents for
constant symbols predicate symbols function symbols objects relations functional relations
An atomic sentence predicate(term1,...,termn) is true iff the objects referred to by term1,...,termn are in the relation referred to by predicate.
4 februari 2012
AI 1
Models for FOL: Example
4 februari 2012
AI 1
10
Models for FOL

We can enumerate the models for a given KB vocabulary:
Computing entailment by enumerating the models will not be easy !!
4 februari 2012
AI 1
11
Quantifiers
Allows us to express properties of collections of objects instead of enumerating objects by name Universal: for all Existential: there exists
4 februari 2012
AI 1
12
Universal quantification
<variables> <sentence> Everyone at VUB is smart: x At(x,VUB) Smart(x) x P is true in a model m iff P is true with x being each possible object in the model Roughly speaking, equivalent to the conjunction of instantiations of P
At(KingJohn,VUB) Smart(KingJohn) At(Richard,VUB) Smart(Richard) At(VUB,VUB) Smart(VUB) ...
A common mistake to avoid

Typically, is the main connective with
A universally quantifier is also equivalent to a set of implications over all objects
Common mistake: using as the main connective with :

x At(x, VUB) Smart(x) means Everyone is at VUB and everyone is smart
4 februari 2012
AI 1
14
Existential quantification
<variables> <sentence> Someone at VUB is smart: x At(x, VUB) Smart(x) x P is true in a model m iff P is true with x being some possible object in the model Roughly speaking, equivalent to the disjunction of instantiations of P
At(KingJohn,VUB) Smart(KingJohn) At(Richard,VUB) Smart(Richard) At(VUB, VUB) Smart(VUB) ...
AI 1
4 februari 2012
15
Another common mistake to avoid

Typically, is the main connective with Common mistake: using as the main connective with : x At(x, VUB) Smart(x) is true even if there is anyone who is not at VUB!
4 februari 2012
AI 1
16
Properties of quantifiers
x y is the same as y x x y is the same as y x x y is not the same as y x x y Loves(x,y) y x Loves(x,y)
There is a person who loves everyone in the world Everyone in the world is loved by at least one person
Quantifier duality: each can be expressed using the other x Likes(x,IceCream) x Likes(x,IceCream) x Likes(x,Broccoli) x Likes(x,Broccoli)
Equality
term1 = term2 is true under a given interpretation if and only if term1 and term2 refer to the same object E.g., definition of Sibling in terms of Parent:
x,y Sibling(x,y) [(x = y) m,f (m = f) Parent(m,x) Parent(f,x) Parent(m,y) Parent(f,y)]
4 februari 2012
AI 1
18
Interacting with FOL KBs

Suppose a wumpus-world agent is using an FOL KB and perceives a smell and a breeze (but no glitter) at t=5:
Tell(KB,Percept([Smell,Breeze,None],5)) (= assertion) Ask(KB,a BestAction(a,5)) (=queries)
I.e., does the KB entail some best action at t=5? Answer: Yes, {a/Shoot} substitution (binding list) Given a sentence S and a substitution , S denotes the result of plugging into S; e.g.,
S = Smarter(x,y) = {x/Hillary,y/Bill} S = Smarter(Hillary,Bill)
Ask(KB,S) returns some/all such that KB |= S.
4 februari 2012
AI 1
19
Using FOL
The kinship domain:
Brothers are siblings
x,y Brother(x,y) Sibling(x,y)
One's mother is one's female parent

m,c Mother(c) = m (Female(m) Parent(m,c))
Sibling is symmetric
x,y Sibling(x,y) Sibling(y,x)
A first cousin is a child of a parents sibling

x,y FirstCousin(x,y) p,ps Parent(p,x) Sibling(ps,p) Parent(ps,y)
4 februari 2012
AI 1
20
Using FOL
The set domain: s Set(s) (s = {} ) (x,s2 Set(s2) s = {x|s2}) x,s {x|s} = {} x,s x s s = {x|s} x,s x s [ y,s2} (s = {y|s2} (x = y x s2))] s1,s2 s1 s2 (x x s1 x s2) s1,s2 (s1 = s2) (s1 s2 s2 s1) x,s1,s2 x (s1 s2) (x s1 x s2) x,s1,s2 x (s1 s2) (x s1 x s2)
4 februari 2012
AI 1
21
FOL Version of Wumpus World

Typical percept sentence:
Percept([Stench,Breeze,Glitter,None,None],5)
Actions:
Turn(Right), Turn(Left), Forward, Shoot, Grab, Release, Climb
To determine best action, construct query: a BestAction(a,5) ASK solves this and returns {a/Grab}
And TELL about the action.
4 februari 2012
AI 1
22
Knowledge base for the wumpus world

Perception
b,g,t Percept([Smell,b,g],t) Smelt(t) s,b,t Percept([s,b,Glitter],t) Glitter(t)
Reflex
t Glitter(t) BestAction(Grab,t)
Reflex with internal state

t Glitter(t) Holding(Gold,t) BestAction(Grab,t) Holding(Gold,t) can not be observed: keep track of change. All synchronic sentences!
4 februari 2012
AI 1
23
Deducing hidden properties

Environment definition:
x,y,a,b Adjacent([x,y],[a,b]) [a,b] {[x+1,y], [x-1,y],[x,y+1],[x,y-1]} Properties of locations: s,t At(Agent,s,t) Smelt(t) Smelly(s) s,t At(Agent,s,t) Breeze(t) Breezy(s) Squares are breezy near a pit: Diagnostic rule---infer cause from effect
s Breezy(s) r Adjacent(r,s) Pit(r)
Causal rule---infer effect from cause (model based reasoning)

r Pit(r) [s Adjacent(r,s) Breezy(s)]
4 februari 2012
AI 1
24
Knowledge engineering in FOL

Identify the task (what will the KB be used for) Assemble the relevant knowledge
Knowledge acquisition Translate domain-level knowledge into logic-level names. define axioms
Decide on a vocabulary of predicates, functions, and constants Encode general knowledge about the domain Encode a description of the specific problem instance Pose queries to the inference procedure and get answers Debug the knowledge base
4 februari 2012
AI 1
25
The electronic circuits domain

One-bit full adder
4 februari 2012
AI 1
26
Identify the task

Does the circuit actually add properly? (circuit verification) Composed of wires and gates; Types of gates (AND, OR, XOR, NOT) Connections between terminals Irrelevant: size, shape, color, cost of gates Alternatives:
Type(X1) = XOR Type(X1, XOR) XOR(X1)
Assemble the relevant knowledge
Decide on a vocabulary
4 februari 2012
AI 1
27

4. Encode general knowledge of the domain
t1,t2 Connected(t1, t2) Signal(t1) = Signal(t2) t Signal(t) = 1 Signal(t) = 0 10 t1,t2 Connected(t1, t2) Connected(t2, t1) g Type(g) = OR Signal(Out(1,g)) = 1 n Signal(In(n,g)) = 1 g Type(g) = AND Signal(Out(1,g)) = 0 n Signal(In(n,g)) = 0 g Type(g) = XOR Signal(Out(1,g)) = 1 Signal(In(1,g)) Signal(In(2,g)) g Type(g) = NOT Signal(Out(1,g)) Signal(In(1,g))
4 februari 2012
AI 1
28

5. Encode the specific problem instance
Type(X1) = XOR Type(A1) = AND Type(O1) = OR Connected(Out(1,X1),In(1,X2)) Connected(Out(1,X1),In(2,A2)) Connected(Out(1,A2),In(1,O1)) Connected(Out(1,A1),In(2,O1)) Connected(Out(1,X2),Out(1,C1)) Connected(Out(1,O1),Out(2,C1)) Type(X2) = XOR Type(A2) = AND
Connected(In(1,C1),In(1,X1)) Connected(In(1,C1),In(1,A1)) Connected(In(2,C1),In(2,X1)) Connected(In(2,C1),In(2,A1)) Connected(In(3,C1),In(2,X2)) Connected(In(3,C1),In(1,A2))
4 februari 2012
AI 1
29

6. Pose queries to the inference procedure
What are the possible sets of values of all the terminals for the adder circuit?
i1,i2,i3,o1,o2 Signal(In(1,C_1)) = i1 Signal(In(2,C1)) = i2 Signal(In(3,C1)) = i3 Signal(Out(1,C1)) = o1 Signal(Out(2,C1)) = o2
7.
Debug the knowledge base

May have omitted assertions like 1 0
4 februari 2012
AI 1
30
Summary
First-order logic:
objects and relations are semantic primitives syntax: constants, functions, predicates, equality, quantifiers.
Increased expressive power: sufficient to define wumpus world
4 februari 2012
AI 1
31
Artificial intelligence: Inference in first-order logic

Outline
Reducing first-order inference to propositional inference Unification Generalized Modus Ponens Forward chaining Backward chaining Resolution
4 februari 2012
AI 11
FOL to PL
First order inference can be done by converting the knowledge base to PL and using propositional inference.
How to convert universal quantifiers?
Replace variable by ground term.
How to convert existential quantifiers?

Skolemization.
4 februari 2012
AI 11
Universal instantiation (UI)

Every instantiation of a universally quantified sentence is entailed by it:
v Subst({v/g}, )
for any variable v and ground term g (Subst(x,y) = substitution of y by x) E.g., x King(x) Greedy(x) Evil(x) yields:
King(John) Greedy(John) Evil(John) King(Richard) Greedy(Richard) Evil(Richard) King(Father(John)) Greedy(Father(John)) Evil(Father(John)) . . .
4 februari 2012
AI 11
Existential instantiation (EI)

For any sentence , variable v, and constant symbol k that does not appear elsewhere in the knowledge base:
v Subst({v/k}, )
E.g., x Crown(x) OnHead(x,John) yields: Crown(C1) OnHead(C1,John) provided C1 is a new constant symbol, called a Skolem constant
4 februari 2012
AI 11
EI versus UI
UI can be applied several times to add new sentences; the new KB is logically equivalent to the old. EI can be applied once to replace the existential sentence; the new KB is not equivalent to the old but is satisfiable if the old KB was satisfiable.
4 februari 2012
AI 11
Reduction to propositional inference

Suppose the KB contains just the following:
x King(x) Greedy(x) Evil(x) King(John) Greedy(John) Brother(Richard,John)
Instantiating the universal sentence in all possible ways, we have:

King(John) Greedy(John) Evil(John) King(Richard) Greedy(Richard) Evil(Richard) King(John) Greedy(John) Brother(Richard,John)
The new KB is propositionalized: proposition symbols are

John, Richard and also King(John), Greedy(John), Evil(John), King(Richard), etc.
4 februari 2012
AI 11
Reduction contd.
CLAIM: A ground sentence is entailed by a new KB iff entailed by the original KB. CLAIM: Every FOL KB can be propositionalized so as to preserve entailment IDEA: propositionalize KB and query, apply resolution, return result PROBLEM: with function symbols, there are infinitely many ground terms,
e.g., Father(Father(Father(John)))
4 februari 2012
AI 11
Reduction contd.
THEOREM: Herbrand (1930). If a sentence is entailed by an FOL KB, it is entailed by a finite subset of the propositionalized KB IDEA: For n = 0 to do
create a propositional KB by instantiating with depth-n terms see if is entailed by this KB
PROBLEM: works if is entailed, loops if is not entailed THEOREM: Turing (1936), Church (1936) Entailment for FOL is semi decidable
algorithms exist that say yes to every entailed sentence, but no algorithm exists that also says no to every non-entailed sentence.
4 februari 2012
AI 11
Problems with propositionalization

Propositionalization seems to generate lots of irrelevant sentences.
E.g., from:
x King(x) Greedy(x) Evil(x) King(John) y Greedy(y) Brother(Richard,John)
It seems obvious that Evil(John), but propositionalization produces lots of facts such as Greedy(Richard) that are irrelevant.
With p k-ary predicates and n constants, there are pnk instantiations!
4 februari 2012
AI 11
10
Lifting and Unification

Instead of translating the knowledge base to PL, we can redefine the inference rules into FOL.
Lifting; they only make those substitutions that are required to allow particular inferences to proceed. E.g. generalised Modus Ponens
To intriduce substitutions different logical expressions have to be look identical Unification
4 februari 2012
AI 11
11
Unification
We can get the inference immediately if we can find a substitution such that King(x) and Greedy(x) match King(John) and Greedy(y)
= {x/John,y/John} works
Unify( ,) = if = p q Knows(John,x) Knows(John,Jane) Knows(John,x) Knows(y,OJ) Knows(John,x) Knows(y,Mother(y)) Knows(John,x) Knows(x,OJ)
Standardizing apart eliminates overlap of variables, e.g., Knows(z17,OJ)
4 februari 2012
AI 11
12
Unification
= {x/John,y/John} works {x/Jane}
Unify( ,) = if = p q Knows(John,x) Knows(John,Jane) Knows(John,x) Knows(y,OJ) Knows(John,x) Knows(y,Mother(y)) Knows(John,x) Knows(x,OJ)
4 februari 2012
AI 11
13
Unification
Unify( ,) = if = p q Knows(John,x) Knows(John,Jane) Knows(John,x) Knows(John,x) Knows(John,x) Knows(y,OJ) Knows(y,Mother(y)) Knows(x,OJ)
{x/Jane} {x/OJ,y/John}
4 februari 2012
AI 11
14
Unification
{x/Jane} {x/OJ,y/John} {y/John,x/Mother(John)}}
4 februari 2012
AI 11
15
Unification
{x/Jane} {x/OJ,y/John} {y/John,x/Mother(John)}} {fail}
4 februari 2012
AI 11
16
Unification
To unify Knows(John,x) and Knows(y,z),
= {y/John, x/z } or = {y/John, x/John, z/John}
The first unifier is more general than the second. There is a single most general unifier (MGU) that is unique up to renaming of variables.
MGU = { y/John, x/z }
4 februari 2012
AI 11
17
The unification algorithm
4 februari 2012
AI 11
18
The unification algorithm
4 februari 2012
AI 11
19
Generalized Modus Ponens (GMP)

p1', p2', , pn', ( p1 p2 pn q) q p1' is King(John) p1 is King(x) p2' is Greedy(y) p2 is Greedy(x) is {x/John,y/John} q is Evil(x) q is Evil(John)
where pi' = pi for all i
GMP used with KB of definite clauses (exactly one positive literal). All variables assumed universally quantified.
4 februari 2012
AI 11
20
Soundness of GMP
Need to show that
p1', , pn', (p1 pn q) |= q
provided that pi' = pi for all I
LEMMA: For any sentence p, we have p |= p by UI

1. 2. 3. (p1 pn q) |= (p1 pn q) = (p1 pn q) p1', , pn' |= p1' pn' |= p1' pn' From 1 and 2, q follows by ordinary Modus Ponens.
4 februari 2012
AI 11
21
Example knowledge base

The law says that it is a crime for an American to sell weapons to hostile nations. The country Nono, an enemy of America, has some missiles, and all of its missiles were sold to it by Colonel West, who is American. Prove that Col. West is a criminal
4 februari 2012
AI 11
22
Example knowledge base contd.

... it is a crime for an American to sell weapons to hostile nations:
4 februari 2012
AI 11
23

American(x) Weapon(y) Sells(x,y,z) Hostile(z) Criminal(x)
4 februari 2012
AI 11
24

American(x) Weapon(y) Sells(x,y,z) Hostile(z) Criminal(x)
Nono has some missiles
4 februari 2012
AI 11
25

... it is a crime for an American to sell weapons to hostile nations: Nono has some missiles, i.e., x Owns(Nono,x) Missile(x):
Owns(Nono,M1) and Missile(M1) American(x) Weapon(y) Sells(x,y,z) Hostile(z) Criminal(x)
4 februari 2012
AI 11
26

all of its missiles were sold to it by Colonel West
4 februari 2012
AI 11
27


Missile(x) Owns(Nono,x) Sells(West,x,Nono)
4 februari 2012
AI 11
28


Missile(x) Owns(Nono,x) Sells(West,x,Nono)
Missiles are weapons:
4 februari 2012
AI 11
29


Missile(x) Owns(Nono,x) Sells(West,x,Nono) Missile(x) Weapon(x)
4 februari 2012
AI 11
30


Missile(x) Owns(Nono,x) Sells(West,x,Nono) Missile(x) Weapon(x)
An enemy of America counts as "hostile:
4 februari 2012
AI 11
31


Missile(x) Owns(Nono,x) Sells(West,x,Nono) Missile(x) Weapon(x) Enemy(x,America) Hostile(x)
An enemy of America counts as "hostile:
4 februari 2012
AI 11
32


Missile(x) Owns(Nono,x) Sells(West,x,Nono) Missile(x) Weapon(x) Enemy(x,America) Hostile(x)
An enemy of America counts as "hostile: West, who is American
4 februari 2012
AI 11
33


Missile(x) Owns(Nono,x) Sells(West,x,Nono) Missile(x) Weapon(x) Enemy(x,America) Hostile(x) American(West)
An enemy of America counts as "hostile: West, who is American
4 februari 2012
AI 11
34


An enemy of America counts as "hostile: West, who is American The country Nono, an enemy of America
4 februari 2012
AI 11
35


An enemy of America counts as "hostile: West, who is American The country Nono, an enemy of America
Enemy(Nono,America)
4 februari 2012
AI 11
36
Forward chaining algorithm
4 februari 2012
AI 11
37
4 februari 2012
AI 11
38
4 februari 2012
AI 11
39
4 februari 2012
AI 11
40
Properties of forward chaining

Sound and complete for first-order definite clauses.
Cfr. Propositional logic proof.
Datalog = first-order definite clauses + no functions (e.g. crime KB)

FC terminates for Datalog in finite number of iterations
May not terminate in general DF clauses with functions if is not entailed

This is unavoidable: entailment with definite clauses is semidecidable
4 februari 2012
AI 11
41
Efficiency of forward chaining

Incremental forward chaining: no need to match a rule on iteration k if a premise wasn't added on iteration k-1
match each rule whose premise contains a newly added positive literal.
Matching itself can be expensive: Database indexing allows O(1) retrieval of known facts
e.g., query Missile(x) retrieves Missile(M1)
Matching conjunctive premises against known facts is NP-hard. (Pattern matching) Forward chaining is widely used in deductive databases
4 februari 2012
AI 11
42
Hard matching example

Diff(wa,nt) Diff(wa,sa) Diff(nt,q) Diff(nt,sa) Diff(q,nsw) Diff(q,sa) Diff(nsw,v) Diff(nsw,sa) Diff(v,sa) Colorable() Diff(Red,Blue) Diff (Red,Green) Diff(Green,Red) Diff(Green,Blue) Diff(Blue,Red) Diff(Blue,Green)
Colorable() is inferred iff the CSP has a solution CSPs include 3SAT as a special case, hence matching is NP-hard
4 februari 2012
AI 11
43
Backward chaining algorithm
SUBST(COMPOSE(1, 2), p) = SUBST(2, SUBST(1, p))
4 februari 2012
AI 11
44
4 februari 2012
AI 11
45
4 februari 2012
AI 11
46
4 februari 2012
AI 11
47
4 februari 2012
AI 11
48
4 februari 2012
AI 11
49
4 februari 2012
AI 11
50
4 februari 2012
AI 11
51
4 februari 2012
AI 11
52
Properties of backward chaining

Depth-first recursive proof search: space is linear in size of proof. Incomplete due to infinite loops
fix by checking current goal against every goal on stack
Inefficient due to repeated subgoals (both success and failure)

fix using caching of previous results (extra space!!)
Widely used for logic programming
4 februari 2012
AI 11
53
Logic programming
Logic programming
Identify problem Assemble information <coffee break> Encode info in KB Encode problem instances as facts Ask queries Find false facts.
Procedural programming
Identify problem Assemble information Figure out solution Program solution Encode problem instance as data Apply program to data Debug procedural errors
Should be easier to debug Capital(NY, US) than x=x+2

Logic programming: Prolog

BASIS: backward chaining with Horn clauses + bells & whistles Widely used in Europe, Japan (basis of 5th Generation project) Compilation techniques 60 million LIPS Program = set of clauses = head :- literal1, literaln.
criminal(X) :- american(X), weapon(Y), sells(X,Y,Z), hostile(Z).
Efficient unification and retrieval of matching clauses. Depth-first, left-to-right backward chaining Built-in predicates for arithmetic etc., e.g., X is Y*Z+3 Built-in predicates that have side effects (e.g., input and output predicates, assert/retract predicates) Closed-world assumption ("negation as failure")
e.g., given alive(X) :- not dead(X). alive(joe) succeeds if dead(joe) fails
4 februari 2012
AI 11
55
Prolog
Appending two lists to produce a third: append([],Y,Y). append([X|L],Y,[X|Z]) :- append(L,Y,Z). query: answers: append(A,B,[1,2]) ? A=[] B=[1,2] A=[1] B=[2] A=[1,2] B=[]
4 februari 2012
AI 11
56
Resolution: brief summary

Full first-order version: l1 lk, m1 mn (l1 li-1 li+1 lk m1 mj-1 mj+1 mn) where Unify(li, mj) = . The two clauses are assumed to be standardized apart so that they share no variables. For example, Rich(x) Unhappy(x) Rich(Ken) Unhappy(Ken)
with = {x/Ken}
Apply resolution steps to CNF(KB ); complete for FOL
4 februari 2012
AI 11
57
Conversion to CNF
Everyone who loves all animals is loved by someone: x [y Animal(y) Loves(x,y)] [y Loves(y,x)] Eliminate biconditionals and implications x [y Animal(y) Loves(x,y)] [y Loves(y,x)] Move inwards: x p x p, x p x p x [y (Animal(y) Loves(x,y))] [y Loves(y,x)] x [y Animal(y) Loves(x,y)] [y Loves(y,x)] x [y Animal(y) Loves(x,y)] [y Loves(y,x)]
4 februari 2012
AI 11
58
Conversion to CNF contd.

Standardize variables: each quantifier should use a different one: x [y Animal(y) Loves(x,y)] [z Loves(z,x)] Skolemize: a more general form of existential instantiation.Each existential variable is replaced by a Skolem function of the enclosing universally quantified variables: x [Animal(F(x)) Loves(x,F(x))] Loves(G(x),x) Drop universal quantifiers: [Animal(F(x)) Loves(x,F(x))] Loves(G(x),x) Distribute over : [Animal(F(x)) Loves(G(x),x)] [Loves(x,F(x)) Loves(G(x),x)]
4 februari 2012
AI 11
59
Resolution proof: definite clauses
4 februari 2012
AI 11
60
Artificial Intelligence: knowledge representation

Outline
Ontological engineering Categories and objects Actions, situations and events Mental events and mental objects The internet shopping world Reasoning systems for categories Reasoning with default information Truth maintenance systems
February 4, 2012 Mehnen 2
Ontological engineering
How to create more general and flexible representations.
Concepts like actions, time, physical object and beliefs Operates on a bigger scale than K.E.
Define general framework of concepts

Upper ontology
Limitations of logic representation

Red, green and yellow tomatoes: exceptions and uncertainty
The upper ontology of the world
February 4, 2012
Mehnen
Difference with special-purpose ontologies

A general-purpose ontology should be applicable in more or less any special-purpose domain.
Add domain-specific axioms
In any sufficiently demanding domain different areas of knowledge need to be unified.

Reasoning and problem solving could involve several areas simultaneously
What do we need to express?

Categories, Measures, Composite objects, Time, Space, Change, Events, Processes, Physical Objects, Substances, Mental Objects, Beliefs
Categories and objects

KR requires the organisation of objects into categories
Interaction at the level of the object Reasoning at the level of categories
Categories play a role in predictions about objects

Based on perceived properties
Categories can be represented in two ways by FOL

Predicates: apple(x) Reification of categories into objects: apples
Category = set of its members
February 4, 2012
Mehnen
Category organization
Relation = inheritance:
All instance of food are edible, fruit is a subclass of food and apples is a subclass of fruit then an apple is edible.
Defines a taxonomy
February 4, 2012
Mehnen
FOL and categories

An object is a member of a category
MemberOf(BB12,Basketballs)
A category is a subclass of another category

SubsetOf(Basketballs,Balls)
All members of a category have some properties

x (MemberOf(x,Basketballs) Round(x))
All members of a category can be recognized by some properties

x (Orange(x) Round(x) Diameter(x)=9.5in MemberOf(x,Balls) MemberOf(x,BasketBalls))
A category as a whole has some properties

MemberOf(Dogs,DomesticatedSpecies)
Relations between categories

Two or more categories are disjoint if they have no members in common:
Disjoint(s)( c1,c2 c1 s c2 s c1 c2 Intersection(c1,c2)
={})
Example; Disjoint({animals, vegetables})
A set of categories s constitutes an exhaustive decomposition of a category c if all members of the set c are covered by categories in s:
E.D.(s,c) ( i i c c2 c2 s i c2) Example: ExhaustiveDecomposition({Americans, Canadian, Mexicans},NorthAmericans).
Relations between categories

A partition is a disjoint exhaustive decomposition: Partition(s,c) Disjoint(s) E.D.(s,c)
Example: Partition({Males,Females},Persons).
Is ({Americans,Canadian,
Mexicans},NorthAmericans) a partition?
Categories can be defined by providing necessary and sufficient conditions for membership
x Bachelor(x) Male(x) Adult(x) Unmarried(x)
Natural kinds
Many categories have no clear-cut definitions (chair, bush, book). Tomatoes: sometimes green, red, yellow, black. Mostly round. One solution: category Typical(Tomatoes). x, x Typical(Tomatoes) Red(x) Spherical(x).
We can write down useful facts about categories without providing exact definitions.
What about bachelor? Quine challenged the utility of the notion of strict definition. We might question a statement such as the Pope is a bachelor.
Physical composition
One object may be part of another:
PartOf(Bucharest,Romania) PartOf(Romania,EasternEurope) PartOf(EasternEurope,Europe)
The PartOf predicate is transitive (and irreflexive), so we can infer that PartOf(Bucharest,Europe) More generally:
x PartOf(x,x) x,y,z PartOf(x,y) PartOf(y,z) PartOf(x,z)
Often characterized by structural relations among parts.

E.g. Biped(a)
February 4, 2012
Mehnen
12
Measurements
Objects have height, mass, cost, .... Values that we assign to these are measures Combine Unit functions with a number: Length(L1) = Inches(1.5) = Centimeters(3.81). Conversion between units: i Centimeters(2.54 x i)=Inches(i). Some measures have no scale: Beauty, Difficulty, etc.
Most important aspect of measures: is that they are orderable. Don't care about the actual numbers. (An apple can have deliciousness .9 or .1.)
Actions, events and situations

Reasoning about outcome of actions is central to KBagent. How can we keep track of location in FOL?
Remember the multiple copies in PL.
Representing time by situations (states resulting from the execution of actions).

Situation calculus
February 4, 2012
Mehnen
14

Situation calculus:
Actions are logical terms Situations are logical terms consiting of
The initial situation I All situations resulting from the action on I (=Result(a,s))
Fluent are functions and predicates that vary from one situation to the next.
E.g. Holding(G1, S0)
Eternal predicates are also allowed

February 4, 2012 Mehnen
E.g. Gold(G1)
15

Results of action sequences are determined by the individual actions. Projection task: an SC agent should be able to deduce the outcome of a sequence of actions. Planning task: find a sequence that achieves a desirable effect
February 4, 2012
Mehnen
17
Describing change
Simples Situation calculus requires two axioms to describe change:
Possibility axiom: when is it possible to do the action
At(Agent,x,s) Adjacent(x,y) Poss(Go(x,y),s)
Effect axiom: describe changes due to action

Poss(Go(x,y),s) At(Agent,y,Result(Go(x,y),s))
What stays the same?

Frame problem: how to represent all things that stay the same? Frame axiom: describe non-changes due to actions
At(o,x,s) (o Agent) Holding(o,s) At(o,x,Result(Go(y,z),s))
Representational frame problem

If there are F fluents and A actions then we need AF frame axioms to describe other objects are stationary unless they are held.
We write down the effect of each actions
Solution; describe how each fluent changes over time

Successor-state axiom:
Pos(a,s) (At(Agent,y,Result(a,s)) (a = Go(x,y)) (At(Agent,y,s) a Go(y,z))
Note that next state is completely specified by current state. Each action effect is mentioned only once.
Other problems
How to deal with secondary (implicit) effects?
If the agent is carrying the gold and the agent moves then the gold moves too. Ramification problem
How to decide EFFICIENTLY whether fluents hold in the future?

Inferential frame problem.
Extensions:
Event calculus (when actions have a duration) Process categories
February 4, 2012
Mehnen
20
Mental events and objects

So far, KB agents can have beliefs and deduce new beliefs What about knowledge about beliefs? What about knowledge about the inference proces?
Requires a model of the mental objects in someones head and the processes that manipulate these objects.
Relationships between agents and mental objects: believes, knows, wants,

Believes(Lois,Flies(Superman)) with Flies(Superman) being a function a candidate for a mental object (reification). Agent can now reason about the beliefs of agents.
The internet shopping world

A Knowledge Engineering example An agent that helps a buyer to find product offers on the internet.
IN = product description (precise or precise) OUT = list of webpages that offer the product for sale.
Environment = WWW Percepts = web pages (character strings)

Extracting useful information required.
February 4, 2012
Mehnen
22
The internet shopping world

Find relevant product offers
RelevantOffer(page,url,query) Relevant(page, url, query) Offer(page)
Write axioms to define Offer(x) Find relevant pages: Relevant(x,y,z) ?

Start from an initial set of stores. What is a relevant category? What are relevant connected pages?
Require rich category vocabulary.

Synonymy and ambiguity
How to retrieve pages: GetPage(url)?

Procedural attachment
Compare offers (information extraction).

February 4, 2012 Mehnen
23
Reasoning systems for categories

How to organise and reason with categories?
Semantic networks
Visualize knowledge-base Efficient algorithms for category membership inference
Description logics
Formal language for constructing and combining category definitions Efficient algorithms to decide subset and superset relationships between categories.
Semantic Networks
Logic vs. semantic networks Many variations
All represent individual objects, categories of objects and relationships among objects.
Allows for inheritance reasoning

Female persons inherit all properties from person. Cfr. OO programming.
Inference of inverse links

SisterOf vs. HasSister
February 4, 2012
Mehnen
25
Semantic network example
February 4, 2012
Mehnen
26
Semantic networks
Drawbacks
Links can only assert binary relations Can be resolved by reification of the proposition as an event
Representation of default values

Enforced by the inheritance mechanism.
February 4, 2012
Mehnen
27
Description logics
Are designed to describe defintions and properties about categories
A formalization of semantic networks
Principal inference task is

Subsumption: checking if one category is the subset of another by comparing their definitions Classification: checking whether an object belongs to a category. Consistency: whether the category membership criteria are logically satisfiable.
Truth maintenance systems

Many of the inferences have default status rather than being absolutely certain
Inferred facts can be wrong and need to be retracted = BELIEF REVISION. Assume KB contains sentence P and we want to execute TELL(KB, P)
To avoid contradiction: RETRACT(KB,P) But what about sentences inferred from P?
Truth maintenance systems are designed to handle these complications.
February 4, 2012
Mehnen
29
Artificial Intelligence: Planning
Planning
The Planning problem Planning with State-space search Partial-order planning Planning graphs Planning with propositional logic Analysis of planning approaches
What is Planning
Generate sequences of actions to perform tasks and achieve objectives.
States, actions and goals
Search for solution over abstract space of plans. Classical planning environment: fully observable, deterministic, finite, static and discrete. Assists humans in practical applications
design and manufacturing military operations games space exploration
4 februari 2012
AI 1
Difficulty of real world problems

Assume a problem-solving agent using some search method
Which actions are relevant? Exhaustive search vs. backward search What is a good heuristic functions? Good estimate of the cost of the state? Problem-dependent vs, -independent How to decompose the problem? Most real-world problems are nearly decomposable.
Planning language
What is a good language?
Expressive enough to describe a wide variety of problems. Restrictive enough to allow efficient algorithms to operate on it. Planning algorithm should be able to take advantage of the logical structure of the problem.
STRIPS and ADL

General language features

Representation of states
Decompose the world in logical conditions and represent a state as a conjunction of positive literals.
Propositional literals: Poor Unknown FO-literals (grounded and function-free): At(Plane1, Melbourne) At(Plane2, Sydney)
Closed world assumption
Representation of goals
Partially specified state and represented as a conjunction of positive ground literals A goal is satisfied if the state contains all literals in goal.
General language features

Representations of actions
Action = PRECOND + EFFECT
Action(Fly(p,from, to), PRECOND: At(p,from) Plane(p) Airport(from) Airport(to) EFFECT: AT(p,from) At(p,to))
= action schema (p, from, to need to be instantiated)

Action name and parameter list Precondition (conj. of function-free literals) Effect (conj of function-free literals and P is True and not P is false)
Add-list vs delete-list in Effect

Language semantics?
How do actions affect states?
An action is applicable in any state that satisfies the precondition. For FO action schema applicability involves a substitution for the variables in the PRECOND.
At(P1,JFK) At(P2,SFO) Plane(P1) Plane(P2) Airport(JFK) Airport(SFO) Satisfies : At(p,from) Plane(p) Airport(from) Airport(to) With ={p/P1,from/JFK,to/SFO} Thus the action is applicable.
4 februari 2012
AI 1
Language semantics?
The result of executing action a in state s is the state s
s is same as s except
Any positive literal P in the effect of a is added to s Any negative literal P is removed from s
EFFECT: AT(p,from) At(p,to): At(P1,SFO) At(P2,SFO) Plane(P1) Plane(P2) Airport(JFK) Airport(SFO)
STRIPS assumption: (avoids representational frame problem)

every literal NOT in the effect remains unchanged
Expressiveness and extensions

STRIPS is simplified
Important limit: function-free literals
Allows for propositional representation Function symbols lead to infinitely many states and actions
Recent extension:Action Description language (ADL)

Action(Fly(p:Plane, from: Airport, to: Airport), PRECOND: At(p,from) (from to) EFFECT: At(p,from) At(p,to))
Standardization : Planning domain definition language (PDDL)
4 februari 2012
AI 1
10
Example: air cargo transport

Init(At(C1, SFO) At(C2,JFK) At(P1,SFO) At(P2,JFK) Cargo(C1) Cargo(C2) Plane(P1) Plane(P2) Airport(JFK) Airport(SFO)) Goal(At(C1,JFK) At(C2,SFO)) Action(Load(c,p,a) PRECOND: At(c,a) At(p,a) Cargo(c) Plane(p) Airport(a) EFFECT: At(c,a) In(c,p)) Action(Unload(c,p,a) PRECOND: In(c,p) At(p,a) Cargo(c) Plane(p) Airport(a) EFFECT: At(c,a) In(c,p)) Action(Fly(p,from,to) PRECOND: At(p,from) Plane(p) Airport(from) Airport(to) EFFECT: At(p,from) At(p,to)) [Load(C1,P1,SFO), Fly(P1,SFO,JFK), Load(C2,P2,JFK), Fly(P2,JFK,SFO)]
Example: Spare tire problem

Init(At(Flat, Axle) At(Spare,trunk)) Goal(At(Spare,Axle)) Action(Remove(Spare,Trunk) PRECOND: At(Spare,Trunk) EFFECT: At(Spare,Trunk) At(Spare,Ground)) Action(Remove(Flat,Axle) PRECOND: At(Flat,Axle) EFFECT: At(Flat,Axle) At(Flat,Ground)) Action(PutOn(Spare,Axle) PRECOND: At(Spare,Groundp) At(Flat,Axle) EFFECT: At(Spare,Axle) At(Spare,Ground)) Action(LeaveOvernight PRECOND: EFFECT: At(Spare,Ground) At(Spare,Axle) At(Spare,trunk) At(Flat,Ground) At(Flat,Axle) ) This example goes beyond STRIPS: negative literal in pre-condition (ADL description)
Example: Blocks world

Init(On(A, Table) On(B,Table) On(C,Table) Block(A) Block(B) Block(C) Clear(A) Clear(B) Clear(C)) Goal(On(A,B) On(B,C)) Action(Move(b,x,y) PRECOND: On(b,x) Clear(b) Clear(y) Block(b) (b x) (b y) (x y) EFFECT: On(b,y) Clear(x) On(b,x) Clear(y)) Action(MoveToTable(b,x) PRECOND: On(b,x) Clear(b) Block(b) (b x) EFFECT: On(b,Table) Clear(x) On(b,x))
Spurious actions are possible: Move(B,C,C)
4 februari 2012
AI 1
13
Planning with state-space search

Both forward and backward search possible Progression planners
forward state-space search Consider the effect of all possible actions in a given state
Regression planners
backward state-space search To achieve a goal, what must have been true in the previous state.
Progression and regression
4 februari 2012
AI 1
15
Progression algorithm
Formulation as state-space search problem:
Initial state = initial state of the planning problem
Literals not appearing are false
Actions = those whose preconditions are satisfied

Add positive effects, delete negative
Goal test = does the state satisfy the goal Step cost = each action costs 1
No functions any graph search that is complete is a complete planning algorithm.

E.g. A*
Inefficient:
(1) irrelevant action problem (2) good heuristic required for efficient search
Regression algorithm
How to determine predecessors?
What are the states from which applying a given action leads to the goal?
Goal state = At(C1, B) At(C2, B) At(C20, B) Relevant action for first conjunct: Unload(C1,p,B) Works only if pre-conditions are satisfied. Previous state= In(C1, p) At(p, B) At(C2, B) At(C20, B) Subgoal At(C1,B) should not be present in this state.
Actions must not undo desired literals (consistent) Main advantage: only relevant actions are considered.
Often much lower branching factor than forward search.
4 februari 2012
AI 1
17
Regression algorithm
General process for predecessor construction
Give a goal description G Let A be an action that is relevant and consistent The predecessors is as follows:
Any positive effects of A that appear in G are deleted. Each precondition literal of A is added , unless it already appears.
Any standard search algorithm can be added to perform the search. Termination when predecessor satisfied by initial state.
In FO case, satisfaction might require a substitution.
Heuristics for state-space search

Neither progression or regression are very efficient without a good heuristic.
How many actions are needed to achieve the goal? Exact solution is NP hard, find a good estimate
Two approaches to find admissible heuristic:

The optimal solution to the relaxed problem.
Remove all preconditions from actions
The subgoal independence assumption:

The cost of solving a conjunction of subgoals is approximated by the sum of the costs of solving the subproblems independently.
Partial-order planning
Progression and regression planning are totally ordered plan search forms.
They cannot take advantage of problem decomposition. Decisions must be made on how to sequence actions on all the subproblems
Least commitment strategy:

Delay choice during search
4 februari 2012
AI 1
20
Shoe example
Goal(RightShoeOn LeftShoeOn) Init() Action(RightShoe, PRECOND: RightSockOn EFFECT: RightShoeOn) Action(RightSock,PRECOND: EFFECT: RightSockOn) Action(LeftShoe, PRECOND: LeftSockOn EFFECT: LeftShoeOn) Action(LeftSock, PRECOND: EFFECT: LeftSockOn)
Planner: combine two action sequences (1)leftsock, leftshoe (2)rightsock, rightshoe

Partial-order planning(POP)
Any planning algorithm that can place two actions into a plan without which comes first is a PO plan.
4 februari 2012
AI 1
22
POP as a search problem

States are (mostly unfinished) plans.
The empty plan contains only start and finish actions.
Each plan has 4 components:

A set of actions (steps of the plan) A set of ordering constraints: A < B (A before B)
Cycles represent contradictions.
A set of causal links

The plan may not be extended by adding a new action C that conflicts with the causal link. (if the effect of C is p and if C could come after A and before B)
A set of open preconditions.

If precondition is not achieved by action in the plan. p
A B
AI 1
4 februari 2012
23
Example of final plan

Actions={Rightsock, Rightshoe, Leftsock, Leftshoe, Start, Finish} Orderings={Rightsock < Rightshoe; Leftsock < Leftshoe} Links={Rightsock->Rightsockon -> Rightshoe, Leftsock->Leftsockon-> Leftshoe, Rightshoe>Rightshoeon->Finish, } Open preconditions={}
4 februari 2012
AI 1
24
POP as a search problem

A plan is consistent iff there are no cycles in the ordering constraints and no conflicts with the causal links. A consistent plan with no open preconditions is a solution. A partial order plan is executed by repeatedly choosing any of the possible next actions.
This flexibility is a benefit in non-cooperative environments.
4 februari 2012
AI 1
25
Solving POP
Assume propositional planning problems:
The initial plan contains Start and Finish, the ordering constraint Start < Finish, no causal links, all the preconditions in Finish are open. Successor function :
picks one open precondition p on an action B and generates a successor plan for every possible consistent way of choosing action A that achieves p.
Test goal
Enforcing consistency
When generating successor plan:
The causal link A->p->B and the ordering constraint A < B is added to the plan.
If A is new also add start < A and A < B to the plan
Resolve conflicts between new causal link and all existing actions Resolve conflicts between action A (if new) and all existing causal links.
4 februari 2012
AI 1
27
Process summary
Operators on partial plans
Add link from existing plan to open precondition. Add a step to fulfill an open condition. Order one step w.r.t another to remove possible conflicts
Gradually move from incomplete/vague plans to complete/correct plans Backtrack if an open condition is unachievable or if a conflict is irresolvable.

Init(At(Flat, Axle) At(Spare,trunk)) Goal(At(Spare,Axle)) Action(Remove(Spare,Trunk) PRECOND: At(Spare,Trunk) EFFECT: At(Spare,Trunk) At(Spare,Ground)) Action(Remove(Flat,Axle) PRECOND: At(Flat,Axle) EFFECT: At(Flat,Axle) At(Flat,Ground)) Action(PutOn(Spare,Axle) PRECOND: At(Spare,Groundp) At(Flat,Axle) EFFECT: At(Spare,Axle) Ar(Spare,Ground)) Action(LeaveOvernight PRECOND: EFFECT: At(Spare,Ground) At(Spare,Axle) At(Spare,trunk) At(Flat,Ground) At(Flat,Axle) )
Solving the problem
Initial plan: Start with EFFECTS and Finish with PRECOND.
4 februari 2012
AI 1
30
Solving the problem
Initial plan: Start with EFFECTS and Finish with PRECOND. Pick an open precondition: At(Spare, Axle) Only PutOn(Spare, Axle) is applicable PutOn(Spare, Axle) At(Spare,Axle ) Finish Add causal link: Add constraint : PutOn(Spare, Axle) < Finish
4 februari 2012
AI 1
31
Solving the problem
Pick an open precondition: At(Spare, Ground) Only Remove(Spare, Trunk) is applicable Re move(Spare,Trunk) At(Spare,Ground ) PutOn(Spare, Axle) Add causal link: Add constraint : Remove(Spare, Trunk) < PutOn(Spare,Axle)
4 februari 2012
AI 1
32
Solving the problem
Pick an open precondition: At(Flat, Axle) LeaveOverNight is applicable conflict: LeaveOverNight also has the effect At(Spare,Ground)
Re move(Spare,Trunk) At(Spare,Ground ) PutOn(Spare, Axle)

To resolve, add constraint : LeaveOverNight < Remove(Spare, Trunk)
4 februari 2012
AI 1
33
Solving the problem
Pick an open precondition: At(Spare, Ground) LeaveOverNight is applicable conflict: At(Spare,Ground )
Re move(Spare,Trunk) PutOn(Spare, Axle)

LeaveOverNight At(Spare,Ground ) PutOn(Spare, Axle)
AI 1 34
To resolve, add constraint : LeaveOverNight < Remove(Spare, Trunk) Add causal link:
4 februari 2012
Solving the problem
Pick an open precondition: At(Spare, Trunk) Only Start is applicable Add causal link: Start At(Spare,Trunk ) Re move(Spare,Trunk) Conflict: of causal link with effect At(Spare,Trunk) in LeaveOverNight
No re-ordering solution possible.
backtrack
4 februari 2012
AI 1
35
Solving the problem
Remove LeaveOverNight, Remove(Spare, Trunk) and causal links Repeat step with Remove(Spare,Trunk) Add also RemoveFlatAxle and finish
4 februari 2012
AI 1
36
Some details
What happens when a first-order representation that includes variables is used?
Complicates the process of detecting and resolving conflicts. Can be resolved by introducing inequality constraint.
CSPs most-constrained-variable constraint can be used for planning algorithms to select a PRECOND.
4 februari 2012
AI 1
37
Planning graphs
Used to achieve better heuristic estimates.
A solution can also directly extracted using GRAPHPLAN.
Consists of a sequence of levels that correspond to time steps in the plan.

Level 0 is the initial state. Each level consists of a set of literals and a set of actions.
Literals = all those that could be true at that time step, depending upon the actions executed at the preceding time step. Actions = all those actions that could have their preconditions satisfied at that time step, depending on which of the literals actually hold.
4 februari 2012
AI 1
38
Planning graphs
Could?
Records only a restricted subset of possible negative interactions among actions.
They work only for propositional problems. Example:

Init(Have(Cake)) Goal(Have(Cake) Eaten(Cake)) Action(Eat(Cake), PRECOND: Have(Cake) EFFECT: Have(Cake) Eaten(Cake)) Action(Bake(Cake), PRECOND: Have(Cake) EFFECT: Have(Cake))
4 februari 2012
AI 1
39
Cake example
Start at level S0 and determine action level A0 and next level S1.
A0 >> all actions whose preconditions are satisfied in the previous level. Connect precond and effect of actions S0 --> S1 Inaction is represented by persistence actions.
Level A0 contains the actions that could occur

Conflicts between actions are represented by mutex links
4 februari 2012
AI 1
40
Cake example
Level S1 contains all literals that could result from picking any subset of actions in A0
Conflicts between literals that can not occur together (as a consequence of the selection action) are represented by mutex links. S1 defines multiple states and the mutex links are the constraints that define this set of states.
Continue until two consecutive levels are identical: leveled off

Or contain the same amount of literals (explanation follows later)
Cake example
A mutex relation holds between two actions when:

Inconsistent effects: one action negates the effect of another. Interference: one of the effects of one action is the negation of a precondition of the other. Competing needs: one of the preconditions of one action is mutually exclusive with the precondition of the other.
A mutex relation holds between two literals when (inconsistent support):

If one is the negation of the other OR if each possible action pair that could achieve the literals is mutex.
PG and heuristic estimation

PGs provide information about the problem
A literal that does not appear in the final level of the graph cannot be achieved by any plan.
Useful for backward search (cost = inf).
Level of appearance can be used as cost estimate of achieving any goal literals = level cost. Small problem: several actions can occur
Restrict to one action using serial PG (add mutex links between every pair of actions, except persistence actions).
Cost of a conjunction of goals? Max-level, sum-level and setlevel heuristics.
PG is a relaxed problem.
4 februari 2012
AI 1
43
The GRAPHPLAN Algorithm

How to extract a solution directly from the PG
function GRAPHPLAN(problem) return solution or failure graph INITIAL-PLANNING-GRAPH(problem) goals GOALS[problem] loop do if goals all non-mutex in last level of graph then do solution EXTRACT-SOLUTION(graph, goals, LENGTH(graph)) if solution failure then return solution else if NO-SOLUTION-POSSIBLE(graph) then return failure graph EXPAND-GRAPH(graph, problem)
4 februari 2012
AI 1
44

Init(At(Flat, Axle) At(Spare,trunk)) Goal(At(Spare,Axle)) Action(Remove(Spare,Trunk) PRECOND: At(Spare,Trunk) EFFECT: At(Spare,Trunk) At(Spare,Ground)) Action(Remove(Flat,Axle) PRECOND: At(Flat,Axle) EFFECT: At(Flat,Axle) At(Flat,Ground)) Action(PutOn(Spare,Axle) PRECOND: At(Spare,Groundp) At(Flat,Axle) EFFECT: At(Spare,Axle) At(Spare,Ground)) Action(LeaveOvernight PRECOND: EFFECT: At(Spare,Ground) At(Spare,Axle) At(Spare,trunk) At(Flat,Ground) At(Flat,Axle) ) This example goes beyond STRIPS: negative literal in pre-condition (ADL description)
4 februari 2012
AI 1
45
GRAPHPLAN example
Initially the plan consist of 5 literals from the initial state and the CWA literals (S0). Add actions whose preconditions are satisfied by EXPAND-GRAPH (A0) Also add persistence actions and mutex relations. Add the effects at level S1 Repeat until goal is in level Si
GRAPHPLAN example
EXPAND-GRAPH also looks for mutex relations

Inconsistent effects
E.g. Remove(Spare, Trunk) and LeaveOverNight due to At(Spare,Ground) and not At(Spare, Ground)
Interference
E.g. Remove(Flat, Axle) and LeaveOverNight At(Flat, Axle) as PRECOND and not At(Flat,Axle) as EFFECT
Competing needs
E.g. PutOn(Spare,Axle) and Remove(Flat, Axle) due to At(Flat.Axle) and not At(Flat, Axle)
Inconsistent support
E.g. in S2, At(Spare,Axle) and At(Flat,Axle)
GRAPHPLAN example
In S2, the goal literals exist and are not mutex with any other
Solution might exist and EXTRACT-SOLUTION will try to find it
EXTRACT-SOLUTION can use Boolean CSP to solve the problem or a search process:
Initial state = last level of PG and goal goals of planning problem Actions = select any set of non-conflicting actions that cover the goals in the state Goal = reach level S0 such that all goals are satisfied Cost = 1 for each action.
4 februari 2012
AI 1
48
GRAPHPLAN example
Termination? YES PG are monotonically increasing or decreasing:

Literals increase monotonically Actions increase monotonically Mutexes decrease monotonically
Because of these properties and because there is a finite number of actions and literals, every PG will eventually level off !
Planning with propositional logic

Planning can be done by proving theorem in situation calculus. Here: test the satisfiability of a logical sentence:
initial state all possible action descriptions goal

Sentence contains propositions for every action occurrence.
A model will assign true to the actions that are part of the correct plan and false to the others An assignment that corresponds to an incorrect plan will not be a model because of inconsistency with the assertion that the goal is true. If the planning is unsolvable the sentence will be unsatisfiable.
4 februari 2012
AI 1
50
SATPLAN algorithm
function SATPLAN(problem, Tmax) return solution or failure inputs: problem, a planning problem Tmax, an upper limit to the plan length for T= 0 to Tmax do cnf, mapping TRANSLATE-TO_SAT(problem, T) assignment SAT-SOLVER(cnf) if assignment is not null then return EXTRACT-SOLUTION(assignment, mapping) return failure
4 februari 2012
AI 1
51
cnf, mapping TRANSLATE-TO_SAT(problem, T)
Distinct propositions for assertions about each time step.

Superscripts denote the time step
At(P1,SFO)0 At(P2,JFK)0
No CWA thus specify which propositions are not true

At(P1,SFO)0 At(P2,JFK)0
Unknown propositions are left unspecified.
The goal is associated with a particular time-step

But which one?
4 februari 2012
AI 1
52
cnf, mapping TRANSLATE-TO_SAT(problem, T) How to determine the time step where the goal will be reached?
Start at T=0
Assert At(P1,SFO)0 At(P2,JFK)0
Failure .. Try T=1 Repeat this until some minimal path length is reached.
Termination is ensured by Tmax
Assert At(P1,SFO)1 At(P2,JFK)1
4 februari 2012
AI 1
53
cnf, mapping TRANSLATE-TO_SAT(problem, T)
How to encode actions into PL?

Propositional versions of successor-state axioms At(P1,JFK)1 (At(P1,JFK)0 (Fly(P1,JFK,SFO)0 At(P1,JFK)0)) (Fly(P1,SFO,JFK)0 At(P1,SFO)0) Such an axiom is required for each plane, airport and time step If more airports add another way to travel than additional disjuncts are required
Once all these axioms are in place, the satisfiability algorithm can start to find a plan.
assignment SAT-SOLVER(cnf)
Multiple models can be found They are NOT satisfactory: (for T=1)
Fly(P1,SFO,JFK)0 Fly(P1,JFK,SFO)0 Fly(P2,JFK.SFO)0
The second action is infeasible Yet the plan IS a model of the sentence
initial state all possible action descriptions goal1
Avoiding illegal actions: pre-condition axioms

Fly(P1,SFO,JFK)0 At(P1,JFK)
Exactly one model now satisfies all the axioms where the goal is achieved at T=1.
4 februari 2012
AI 1
55
assignment SAT-SOLVER(cnf) A plane can fly at two destinations at once

They are NOT satisfactory: (for T=1)
Fly(P1,SFO,JFK)0 Fly(P2,JFK,SFO)0 Fly(P2,JFK.LAX)0
The second action is infeasible Yet the plan allows spurious relations
Avoid spurious solutions: action-exclusion axioms

(Fly(P2,JFK,SFO)0 Fly(P2,JFK,LAX))
Prevents simultaneous actions
Lost of flexibility since plan becomes totally ordered : no actions are allowed to occur at the same time.
Restrict exclusion to preconditions
4 februari 2012
AI 1
56
Analysis of planning approach

Planning is an area of great interest within AI
Search for solution Constructively prove a existence of solution
Biggest problem is the combinatorial explosion in states. Efficient methods are under research
E.g. divide-and-conquer
4 februari 2012
AI 1
57
Artificial Intelligence: Planning in the real world

Outline
Time, schedules and resources Hierarchical task network planning Non-deterministic domains
Conditional planning Execution monitoring and replanning Continuous planning
Multi-agent planning
Time, schedules and resources

Until know:
what actions to do
Real-world:
+ actions occur at certain moments in time. + actions have a beginning and an end. + actions take a certain amount of time.
Job-shop scheduling:
Complete a set of jobs, each of which consists of a sequence of actions, Where each action has a given duration and might require resources. Determine a schedule that minimizes the total time required to complete all jobs (respecting resource constraints).
Car construction example

Init(Chassis(C1) Chassis(C2) Engine(E1,C1,30) Engine(E1,C2,60) Wheels(W1,C1,30) Wheels(W2,C2,15)) Goal(Done(C1) Done(C2)) Action(AddEngine(e,c,m) PRECOND: Engine(e,c,d) Chassis(c) EngineIn(c) EFFECT: EngineIn(c) Duration(d)) Action(AddWheels(w,c) PRECOND: Wheels(w,c,d) Chassis(c) EFFECT: WheelsOn(c) Duration(d)) Action(Inspect(c) PRECOND: EngineIn(c) WheelsOn(c) Chassis(c) EFFECT: Done(c) Duration(10))
4 februari 2012
AI 1
Solution found by POP

Slack of 15
critical path
4 februari 2012
AI 1
Planning vs. scheduling

How does the problem differ from a standard planning problem? When does an action start and when does it end?
So next to order (planning) duration is also considered
Duration(d)
Critical path method is used to determine start and end times:

Path = linear sequence from start to end Critical path = path with longest total duration
Determines the duration of the entire plan Critical path should be executed without delay
4 februari 2012
AI 1
ES and LS
Earliest possible (ES) and latest possible (LS) start times. LS-ES = slack of an action for all actions determines the schedule for the entire problem.
ES(Start) = 0 ES(B)=maxA<B ES(A) + Duration(A) LS(Finish)=ES(Finish) LS(A) = minA<B LS(B) -Duration(A)
Complexity is O(Nb) (given a PO)
4 februari 2012
AI 1
Scheduling with resources

Resource constraints = required material or objects to perform task
Reusable resources
A resource that is occupied during an action but becomes available when the action is finished. Require extension of action syntax: Resource:R(k)
k units of resource are required by the action. Is a pre-requisite before the action can be performed. Resource can not be used for k time units by other.
4 februari 2012
AI 1
Car example with resources

Init(Chassis(C1) Chassis(C2) Engine(E1,C1,30) Engine(E1,C2,60) Wheels(W1,C1,30) Wheels(W2,C2,15) EngineHoists(1) WheelStations(1) Inspectors(2)) Goal(Done(C1) Done(C2)) Action(AddEngine(e,c,m) PRECOND: Engine(e,c,d) Chassis(c) EngineIn(c) EFFECT: EngineIn(c) Duration(d), RESOURCE: EngineHoists(1)) Action(AddWheels(w,c) PRECOND: Wheels(w,c,d) Chassis(c) EFFECT: WheelsOn(c) Duration(d) RESOURCE: WheelStations(1)) Action(Inspect(c) PRECOND: EngineIn(c) WheelsOn(c) Chassis(c) EFFECT: Done(c) Duration(10) RESOURCE: Inspectors(1)) aggregation
Car example with resources
4 februari 2012
AI 1
10
Scheduling with resources

Aggregation = group individual objects into quantities when the objects are undistinguishable with respect to their purpose.
Reduces complexity
Resource constraints make scheduling problems more complicated.

Additional interactions among actions
Heuristic: minimum slack algorithm

Select an action with all pre-decessors scheduled and with the least slack for the earliest possible start.
Hierarchical task network planning

Reduce complexity hierarchical decomposition
At each level of the hierarchy a computational task is reduced to a small number of activities at the next lower level. The computational cost of arranging these activities is low.
Hierarchical task network (HTN) planning uses a refinement of actions through decomposition.
e.g. building a house = getting a permit + hiring a contractor + doing the construction + paying the contractor. Refined until only primitive actions remain.
Pure and hybrid HTN planning.
4 februari 2012
AI 1
12
Representation decomposition
General descriptions are stored in plan library.
Each method = Decompos(a,d); a= action and d= PO plan.
See buildhouse example Start action supplies all preconditions of actions not supplied by other actions.
=external preconditions
Finish action has all effects of actions not present in other actions
=external effects
Primary effects (used to achieve goal) vs. secondary effects
Buildhouse example
External precond
External effects
4 februari 2012
AI 1
14
Buildhouse example
Action(Buyland, PRECOND: Money, EFFECT: Land Money) Action(GetLoan, PRECOND: Goodcredit, EFFECT: Money Mortgage) Action(BuildHouse, PRECOND: Land, EFFECT: House) Action(GetPermit, PRECOND: LAnd, EFFECT: Permit) Action(HireBuilder, EFFECT: Contract) Action(Construction, PRECOND: Permit Contract, EFFECT: HouseBuilt Permit), Action(PayBuilder, PRECOND: Money HouseBuilt, EFFECT: Money House Contract), Decompose(BuildHouse, Plan ::STEPS{ S1: GetPermit, S2:HireBuilder, S3:Construction, S4 PayBuilder} ORDERINGS: {Start < S1 < S3< S4<Finish, Start<S2<S3}, LINKS
Start Land S1,Start Money S4,S1 Permit S3,S2 Contract S3, S3 HouseBuilt S4,S4 house Finish,S4 Money Finish
Properties of decomposition
Should be correct implementation of action a
Correct if plan d is complete and consistent PO plan for the problem of achieving the effects of a given the preconditions of a.
A decomposition is not necessarily unique. Performs information hiding:

STRIPS action description of higher-level action hides some preconditions and effects Ignore all internal effects of decomposition Does not specify the intervals inside the activity during which preconditions and effects must hold.
Information hiding is essential to HTN planning.

Recapitulation of POP (1)

Assume propositional planning problems:
The initial plan contains Start and Finish, the ordering constraint Start < Finish, no causal links, all the preconditions in Finish are open. Successor function : picks one open precondition p on an action B and generates a successor plan for every possible consistent way of choosing action A that achieves p. Test goal
Recapitulation of POP (2)

When generating successor plan:
The causal link A--p->B and the ordering constrain A < B is added to the plan. If A is new also add start < A and A < B to the plan Resolve conflicts between new causal link and all existing actions Resolve conflicts between action A (if new) and all existing causal links.
4 februari 2012
AI 1
18
Adapting POP to HTN planning

Remember POP?
Modify the successor function: apply decomposition to current plan
NEW Successor function:

Select non-primitive action a in P For any Decompose(a,d) method in library where a and a unify with substitution
Replace a with d = subst(,d)
4 februari 2012
AI 1
19
POP+HTN example
a
4 februari 2012
AI 1
20
POP+HTN example
a
4 februari 2012
AI 1
21
How to hook up d in a?
Remove action a from P and replace with d
For each step s in d select an action that will play the role of s (either new s or existing s from P) Possibility of subtask sharing
Connect ordering steps for a to the steps in d

Put all constraints so that constraints of the form B < a are maintained. Watch out for too strict orderings !
Connect the causal links

If B -p-> a is a causal link in P, replace it by a set of causal links from B to all steps in d with preconditions p that were supplied by the start step Idem for a -p-> C
4 februari 2012
AI 1
22
What about HTN?

Additional modification to POP are necessary BAD news: pure HTN planning is undecidable due to recursive decomposition actions.
Walk=make one step and walk
Resolve problems by
Rule out recursion. Bound the length of relevant solutions, Hybridize HTN with POP
Yet HTN can be efficient (see motivations in book)
4 februari 2012
AI 1
23
The Gift of magi
4 februari 2012
AI 1
24
Non-deterministic domains
So far: fully observable, static and deterministic domains.
Agent can plan first and then execute plan with eyes closed
Uncertain environment: incomplete (partially observable and/or nondeterministic) and incorrect (differences between world and model) information
Use percepts Adapt plan when necessary
Degree of uncertainty defined by indeterminacy

Bounded: actions can have unpredictable effects, yet can be listed in action description axioms. Unbounded: preconditions and effects unknown or to large to enumerate.
4 februari 2012
AI 1
25
Handling indeterminacy
Sensorless planning (conformant planning)
Find plan that achieves goal in all possible circumstances (regardless of initial state and action effects).
Conditional planning (Contingency planning)

Construct conditional plan with different branches for possible contingencies.
Execution monitoring and replanning

While constructing plan judge whether plan requires revision.
Continuous planning
Planning active for a life time: adapt to changed circumstances and reformulate goals if necessary.
4 februari 2012
AI 1
26
Sensorless planning
4 februari 2012
AI 1
27
Abstract example
Initial state = <chair,table, cans of paint, unknown colors>, goal state=<color(table) = color(chair)> Sensorless planning (conformant planning)
Open any can of paint and apply it to both chair and table.
Conditional planning (Contingency planning)

Sense color of table and chair, if they are the same then finish else sense labels paint if color(label) =color(Furniture) then apply color to othe piece else apply color to both
Execution monitoring and replanning

Same as conditional and can fix errors (missed spots)
Continuous planning
Can revise goal when we want to first eat before painting the table and the chair.
4 februari 2012
AI 1
28
Conditional planning
Deal with uncertainty by checking the environment to see what is really happening. Used in fully observable and nondeterministic environments:
The outcome of an action is unknown. Conditional steps will check the state of the environment. How to construct a conditional plan?
4 februari 2012
AI 1
29
Example, the vacuum-world
4 februari 2012
AI 1
30
Actions: left, right, suck Propositions to define states: AtL, AtR, CleanL, CleanR How to include indeterminism?
Actions can have more than one effect
E.g. moving left sometimes fails Action(Left, PRECOND: AtR, EFFECT: AtL) Becomes : Action(Left, PRECOND: AtR, EFFECT: AtLAtR)
Actions can have conditional effects

Action(Left, PRECOND:AtR, EFFECT: AtL(AtLwhen cleanL: cleanL) Both disjunctive and conditional
4 februari 2012
AI 1
31
Conditional plans require conditional steps:
If <test> then plan_A else plan_B
if AtLCleanL then Right else Suck
Plans become trees
Games against nature:

Find conditional plans that work regardless of which action outcomes actually occur. Assume vacuum-world
Initial state = AtR CleanL CleanR Double murphy: possibility of desposit dirt when moving to other square and possibility of despositing dirt when action is Suck.
4 februari 2012
AI 1
32
Game tree
State node chance node
4 februari 2012
AI 1
33
Solution of games against N.

Solution is a subtree that
Has a goal node at every leaf Specifies one action at each of its state nodes Includes every outcome branch at each of the chance nodes.
In previous example:
[Left, if AtL CleanL CleanR then [] else Suck]
For exact solutions: use minimax algorithm with 2 modifications:

Max and Min nodes become OR and AND nodes Algorithm returns conditional plan instead of single move
4 februari 2012
AI 1
34
And-Or-search algorithm
function AND-OR-GRAPH-SEARCH(problem) returns a conditional plan or failure return OR-SEARCH(INITIAL-STATE[problem], problem, []) function OR-SEARCH(state, problem, path) returns a conditional plan or failure if GOAL-TEST[problem](state) then return the empty plan if state is on path then return failure for action,state_set in SUCCESSORS[problem](state) do plan AND-SEARCH(state_set, problem, [state | plan] ) if plan failure then return [action | plan] return failure function AND-SEARCH(state_set, problem, path) returns a conditional plan or failure for each si in state_set do plani OR-SEARCH(si, problem,path ) if plan = failure then return failure return [ if s1 then plan1 else if s2 then plan2 else if sn-1 then plann-1 else plann]
How does it deal with cycles?
When a state that already is on the path appears, return failure
No non-cyclic solution
Ensures algorithm termination

The algorithm does not check whether some state is already on some other path from the root.
4 februari 2012
AI 1
36
Sometimes only a cyclic solution exists e.g. tripple murphy: sometimes the move is not performed
[Left, if CleanL then [] else Suck] is not a solution
Use label to repeat parts of plan (but infinite loops)

[L1: Left, if AtR then L1 else if CleanL then [] else Suck]
4 februari 2012
AI 1
37
CP and partially observable env.

Fully observable: conditional tests can ask any question and get an answer Partially observable???
The agent has limited information about the environment. Modeled by a state-set = belief states E.g. assume vacuum agent which can not sense presence or absence of dirt in other squares than the one it is on.
+ alternative murphy: dirt can be left behind when moving to other square. Solution in fully observable world: keep moving left and right, sucking dirt whenever it appears until both squares are clean and Im in square left.
4 februari 2012
AI 1
38
PO: alternate double murphy
4 februari 2012
AI 1
39
Belief states
Representation?
Sets of full state descriptions
{(AtRCleanRCleanL) (AtRCleanRCleanL)}
Logical sentences that capture the set of possible worlds in the belief state (OWA)
AtR CleanR
Knowledge propositions describing the agents knowledge (CWA)

K(AtR) K(CleanR)
4 februari 2012
AI 1
40
Belief states
Choice 2 and 3 are equivalent (lets continue with 3) Symbols can appear in three ways in three ways: positive, negative or unknown: 3n possible belief states for n proposition symbols.
YET, set of belief sets is a power set of the phyiscal states which is much larger than 3n Hence 3 is restricted as representation
Any scheme capable of representing every possible belief state will require O(2n) bit to represent each one in the worst case. The current scheme only requires O(n)
4 februari 2012
AI 1
41
Sensing in Cond. Planning

How does it work?
Automatic sensing
At every time step the agent gets all available percepts
Active sensing
Percepts are obtained through the execution of specific sensory actions. checkDirt and checkLocation
Given the representation and the sensing, action descriptions can now be formulated.
4 februari 2012
AI 1
42
Monitoring and replanning

Execution monitoring: check whether everything is going as planned.
Unbounded indeterminancy: some unanticipated circumstances will arise. A necessity in realistic environments.
Kinds of monitoring:
Action monitoring: verify whether the next action will work. Plan monitoring: verify the entire remaining plan.
4 februari 2012
AI 1
43
Monitoring and replanning

When something unexpected happens: replan
To avoid too much time on planning try to repair the old plan.
Can be applied in both fully and partially observable environments, and to a variety of planning representations.
4 februari 2012
AI 1
44
Replanning-agent
function REPLANNING-AGENT(percept) returns an action static: KB, a knowledge base (+ action descriptions) plan, a plan initially [] whole_plan, a plan initially [] goal, a goal TELL(KB, MAKE-PERCEPT-SENTENCE(percept,t)) current STATE-DESCRIPTION(KB,t) if plan = [] then return the empty plan whole_plan plan PLANNER(current, goal, KB) if PRECONDITIONS(FIRST(plan)) not currently true in KB then candidates SORT(whole_plan,ordered by distance to current) find state s in candidates such that failure repair PLANNER(current, s, KB) continuation the tail of whole_plan starting at s whole_plan plan APPEND(repair, continuation) return POP(plan)
4 februari 2012
AI 1
45
Repair example
4 februari 2012
AI 1
46
Repair example: painting

Init(Color(Chair, Blue) Color(Table,Green) ContainsColor(BC,Blue) PaintCan(BC) ContainsColor(RC,Red) PaintCan(RC)) Goal(Color(Chair,x) Color(Table,x)) Action(Paint(object, color) PRECOND: HavePaint(color) EFFECT: Color(object, color)) Action(Open(can) PRECOND: PaintCan(can) ContainsColor(can,color) EFFECT: HavePaint(color)) [Start; Open(BC); Paint(Table,Blue), Finish]
4 februari 2012
AI 1
47
Repair example: painting

Suppose that the agent now perceives that the colors of table and chair are different
Figure out point in whole_plan to aim for
Current state is identical as the precondition before Paint
Repair action sequence to get there.

Repair =[] and plan=[Paint, Finish]
Continue performing this new plan

Will loop until table and chair are perceived as the same.
Action monitoring can lead to less intelligent behavior

Assume the red is selected and there is not enough paint to apply to both chair and table. Improved by doing plan monitoring
4 februari 2012
AI 1
48
Plan monitoring
Check the preconditions for success of the entire plan.
Except those which are achieved by another step in the plan. Execution of doomed plan is cut of earlier.
Limitation of replanning agent:

It can not formulate new goals or accept new goals in addition to the current one
4 februari 2012
AI 1
49
Continuous planning.
Agent persists indefinitely in an environment
Phases of goal formulation, planning and acting
Execution monitoring + planner as one continuous process Example:Blocks world

Assume a fully observable environment Assume partially ordered plan
4 februari 2012
AI 1
50
Block world example
Initial state (a) Action(Move(x,y), PRECOND: Clear(x) Clear(y) On(x,z) EFFECT: On(x,y) Clear(z) On(x,z) Clear(y) The agent first need to formulate a goal: On(C,D) On(D,B) Plan is created incrementally, return NoOp and check percepts
4 februari 2012
AI 1
51
Block world example
Assume that percepts dont change and this plan is constructed Ordering constraint between Move(D,B) and Move(C,D) Start is label of current state during planning. Before the agent can execute the plan, nature intervenes:
D is moved onto B
4 februari 2012
AI 1
52
Block world example
Start contains now On(D,B) Agent perceives: Clear(B) and On(D,G) are no longer true
Update model of current state (start)
Causal links from Start to Move(D,B) (Clear(B) and On(D,G)) no longer valid.
Remove causal relations and two PRECOND of Move(D,B) are open Replace action and causal links to Finish by connecting Start to Finish.
4 februari 2012
AI 1
53
Block world example
Extending causal link
Extending: whenever a causal link can be supplied by a previous step All redundant steps (Move(D,B) and its causal links) are removed from the plan Execute new plan, perform action Move(C,D)
This removes the step from the plan
4 februari 2012
AI 1
54
Block world example
Execute new plan, perform action Move(C,D)

Assume agent is clumsy and drops C on A
No plan but still an open PRECOND Determine new plan for open condition Again Move(C,D)
4 februari 2012
AI 1
55
Block world example
Similar to POP On each iteration find plan-flaw and fix it Possible flaws: Missing goal, Open precondition, Causal conflict, Unsupported link, Redundant action, Unexecuted action, unnecessary historical goal
4 februari 2012
AI 1
56
Multi-agent planning
So far we only discussed single-agent environments. Other agents can simply be added to the model of the world:
Poor performance since agents are not indifferent ot other agents intentions
In general two types of multi-agent environments:

Cooperative Competitive
4 februari 2012
AI 1
57
Cooperation: Joint goals and plans

Multi-planning problem: assume double tennis example where agents want to return ball.
Agents(A,B) Init(At(A,[Left,Baseline]) At(B,[Right, Net]) Approaching(Ball,[Right, Baseline]) PArtner(A,B) Partner(B,A)) Goal(Returned(Ball) At(agent,[x,Net])) Action(Hit(agent, Ball) PRECOND: Approaching(Ball,[x,y]) At(agent,[x,y]) Partner(agent, partner) At(partner,[x,y]) EFFECT: Returned(Ball)) Action(Go(agent,[x,y]) PRECOND: At(agent,[a,b]) EFFECT: At(agent,[x,y]) At(agent,[a,b]))
4 februari 2012
AI 1
58
Cooperation: Joint goals and plans

A solution is a joint-plan consisting of actions for both agents. Example:
A: [Go(A,[Right, Baseline]), Hit(A,Ball)] B: [NoOp(B), NoOp(B)]
Or
A: [Go(A,[Left, net), NoOp(A)] B: [Go(B,[Right, Baseline]), Hit(B, Ball)]
Coordination is required to reach same joint plan
4 februari 2012
AI 1
59
Multi-body planning
Planning problem faced by a single centralized agent that can dictate action to each of several physical entities. Hence not truly multi-agent Important: synchronization of actions
Assume for simplicity that every action takes one time step and at each point in the joint plan the actions are performed simultaneously
[<Go(A,[Left,Net]), Go(B,[Right,Baseline]>; <NoOp(A), Hit(B, Ball)>]
Planning can be performed using POP applied to the set of all possible joint actions.
Size of this set???
4 februari 2012
AI 1
60
Multi-body planning
Alternative to set of all joint actions: add extra concurrency lines to action description
Concurrent action
Action(Hit(A, Ball) CONCURRENT: Hit(B,Ball) PRECOND: Approaching(Ball,[x,y]) At(A,[x,y]) EFFECT: Returned(Ball))
Required actions (carrying object by two agents)

Action(Carry(A, cooler, here, there) CONCURRENT: Carry(B,cooler, here there) PRECOND: )
Planner similar to POP with some small changes in possible ordering relations
4 februari 2012
AI 1
61
Coordination mechanisms
To ensure agreement on joint plan: use convention.
Convention = a constraint on the selection of joint plans (beyond the constraint that the joint plan must work if the agents adopt it).
e.g. stick to your court or one player stays at the net.
Conventions which are widely adopted= social laws e.g. language. Can be domain-specific or independent. Could arise through evolutionary process (flocking behavior).
4 februari 2012
AI 1
62
Flocking example
Three rules:
Separation:
Steer away from neighbors when you get too close
Cohesion
Steer toward the average position of neighbors
Alignment
Steer toward average orientation (heading) of neighbors
Flock exhibits emergent behavior of flying as a pseudo-rigid body.
4 februari 2012
AI 1
63
Coordination mechanisms
In the absence of conventions: Communication
e.g. Mine! Or Yours! in tennis example
The burden of arriving at a succesfull joint plan can be placed on

Agent designer (agents are reactive, no explicit models of other agents) Agent (agents are deliberative, model of other agents required)
4 februari 2012
AI 1
64
Competitive environments
Agents can have conflicting utilities
e.g. zero-sum games like chess
The agent must:

Recognize that there are other agents Compute some of the other agents plans Compute how the other agents interact with its own plan Decide on the best action in view of these interactions.
Model of other agent is required YET, no commitment to joint action plan.
4 februari 2012
AI 1
65
Artificial Intelligence: Neural Networks

Technikum-Wien
17.05.11
AI NN
Neural Networks
McCulloch & Pitts (1943) are generally recognised as the designers of the first neural network Many of their ideas still used today (e.g. many simple units combine to give increased computational power and the idea of a threshold)
AI NN 2
17.05.11
Neural Networks
Hebb (1949) developed the first learning rule (on the premise that if two neurons were active at the same time the strength between them should be increased)
AI NN 3
17.05.11
Neural Networks
During the 50s and 60s many researchers worked on the perceptron amidst great excitement. 1969 saw the death of neural network research for about 15 years Minsky & Papert Only in the mid 80s (Parker and LeCun) was interest revived (in fact Werbos discovered algorithm in 1974)
AI NN 4
17.05.11
Neural Networks
17.05.11
AI NN
Neural Networks
We are born with about 100 billion neurons A neuron may connect to as many as 100,000 other neurons
17.05.11
AI NN
Neural Networks
Signals move via electrochemical signals The synapses release a chemical transmitter the sum of which can cause a threshold to be reached causing the neuron to fire Synapses can be inhibitory or excitatory
17.05.11
AI NN
The First Neural Neural Networks

McCulloch and Pitts produced the first neural network in 1943
Many of the principles can still be seen in neural networks of today
17.05.11
AI NN

X1 2
X2
Y -1
X3
The activation of a neuron is binary. That is, the neuron either fires (activation of one) or does not fire (activation of zero).
17.05.11
AI NN

X1 2
X2
Y -1
X3
For the network shown here the activation function for unit Y is f(y_in) = 1, if y_in >= else 0 where y_in is the total input signal received is the threshold for Y
17.05.11
AI NN
10

X1 2
X2
Y -1
X3
Neurons in a McCulloch-Pitts network are connected by directed, weighted paths
17.05.11
AI NN
11

X1 2
X2
Y -1
X3
If the weight on a path is positive the path is excitatory, otherwise it is inhibitory
17.05.11
AI NN
12

X1 2
X2
Y -1
X3
All excitatory connections into a particular neuron have the same weight, although different weighted connections can be input to different neurons
17.05.11
AI NN
13

X1 2
X2
Y -1
X3
Each neuron has a fixed threshold. If the net input into the neuron is greater than or equal to the threshold, the neuron fires
17.05.11
AI NN
14

X1 2
X2
Y -1
X3
The threshold is set such that any non-zero inhibitory input will prevent the neuron from firing
17.05.11
AI NN
15

X1 2
X2
Y -1
X3
It takes one time step for a signal to pass over one connection.
17.05.11
AI NN
16

X1 1 Y X2
AND Function
AND X1 1 1 0 0
X2 1 0 1 0
Y 1 0 0 0
Threshold(Y) = 2
17.05.11
AI NN
17

X1 2 Y X2 2 ANDFunction OR Function
OR X1 1 1 0 0
X2 1 0 1 0
Y 1 1 1 0
Threshold(Y) = 2
17.05.11
AI NN
18

X1 2 Y X2
-1 AND NOT Function
AND NOT X1 1 1 0 0
X2 1 0 1 0
Y 0 1 0 0
Threshold(Y) = 2
17.05.11
AI NN
19

2 2 X1 -1 Z1 Y X2 -1 2 XOR Function Z2
XOR X1 1 1 0 0
X2 1 0 1 0
Y 0 1 1 0
X1 XOR X2 = (X1 AND NOT X2) OR (X2 AND NOT X1)
17.05.11
AI NN
20

If we touch something cold we perceive heat
If we keep touching something cold we will perceive cold
If we touch something hot we will perceive heat

17.05.11 AI NN 21

To model this we will assume that time is discrete If cold is applied for one time step then heat will be perceived If a cold stimulus is applied for two time steps then cold will be perceived If heat is applied then we should perceive heat
17.05.11 AI NN 22

2 Y1 Heat X1 Z1 2 Hot
-1
Cold
X2
Z2
1 1
Y2
Cold
17.05.11
AI NN
23

2 Y1 X1 Z1 2 2 Hot Hea t
-1
Cold
X2
Z2
1 1
Y2
Cold
It takes time for the stimulus (applied at X1 and X2) to make its way to Y1 and Y2 where we perceive either heat or cold
At t(0), we apply a stimulus to X1 and X2 At t(1) we can update Z1, Z2 and Y1 At t(2) we can perceive a stimulus at Y2 At t(2+n) the network is fully functional AI NN 24
17.05.11

We want the system to perceive cold if a cold stimulus is applied for two time steps Y2(t) = X2(t 2) AND X2(t 1)
X2(t 2) X2( t 1) 1 1 1 0 0 1 0 0
17.05.11 AI NN
Y2(t) 1 0 0 0
25

We want the system to perceive heat if either a hot stimulus is applied or a cold stimulus is applied (for one time step) and then removed Y1(t) = [ X1(t 1) ] OR [ X2(t 3) AND NOT X2(t 2) ]
X2(t 3) 1 1 0 0 1 1 0 0 X2(t 2) 1 0 1 0 1 0 1 0 AND NOT 0 1 0 0 0 1 0 0 X1(t 1) 1 1 1 1 0 0 0 0 OR 1 1 1 1 0 1 0 0
17.05.11
AI NN
26

The network shows Y1(t) = X1(t 1) OR Z1(t 1) Z1(t 1) = Z2( t 2) AND NOT X2(t 2) Z2(t 2) = X2(t 3) Substituting, we get Y1(t) = [ X1(t 1) ] OR [ X2(t 3) AND NOT X2(t 2) ] which is the same as our original requirements 17.05.11 AI NN 27
Modelling a Neuron
in i= j W j , i a j
aj wj,I inI aI g
:Activation value of unit j :Weight on the link from unit j to unit i :Weighted sum of inputs to unit i :Activation value of unit i :Activation function AI NN 28
17.05.11
Activation Functions
17.05.11
Stept(x)= 0 Sign(x) = else 1
1 if x >= t, else +1 if x >= 0,

-x
AI NN
29
Simple Networks
AND
Input 1 Input 2 Output
OR
NOT
0 0 0
0 1 1 1 0 1 0 0 1
0 0 0
0 1 1 1 0 1 1 1 1
0 1
1 0
17.05.11
AI NN
30
Simple Networks
-1 W = 1.5 x W=1 y
t = 0.0
17.05.11
AI NN
31
Perceptron
Synonym for SingleLayer, Feed-Forward Network First Studied in the 50s Other networks were known about but the perceptron was the only one capable of learning and thus all research was concentrated in this area 17.05.11 AI NN 32
Perceptron
A single weight only affects one output so we can restrict our investigations to a model as shown on the right Notation can be simpler, i.e.
O= Step 0 j WjIj
17.05.11
AI NN
33
What can perceptrons represent?

AND 0 0 0 XOR 0 0 0
0 1 0
1 0 0
1 1 1
0 1 1
1 0 1
1 1 0
17.05.11
AI NN
34

1,1 0,1 0,1 1,1
0,0
1,0
AND
0,0
1,0
XOR
Functions which can be separated in this way are called Linearly Separable Only linearly Separable functions can be represented by a perceptron AI NN 35
17.05.11
Linear Separability is also possible in more than 3 dimensions but it is harder to visualise 17.05.11 AI NN 36
Training a perceptron
Aim
AND 0 0 0
0 1 0
1 0 0
1 1 1
17.05.11
AI NN
37
Training a perceptrons
-1 W = 0.3 x t = 0.0
W = 0.5
W = -0.4 y
I1 -1 -1 -1 -1
I2 0 0 1 1
I3 0 1 0 1
Summation (-1*0.3) + (0*0.5) + (0*-0.4) = -0.3 (-1*0.3) + (0*0.5) + (1*-0.4) = -0.7 (-1*0.3) + (1*0.5) + (0*-0.4) = 0.2 (-1*0.3) + (1*0.5) + (1*-0.4) = -0.2
AI NN 38
Output 0 0 1 0
17.05.11
Learning
While epoch produces an error
Present network with next inputs from epoch Err = T O If Err <> 0 then Wj = Wj + LR * Ij * Err End If
End While
17.05.11 AI NN 39
Learning
End While
Epoch : Presentation of the entire training set to the neural network. In the case of the AND function an epoch consists of four sets of inputs being presented to the network (i.e. [0,0], [0,1], [1,0], [1,1]) 17.05.11 AI NN 40
Learning
End While
Training Value, T : When we are training a network we not only present it with the input but also with a value that we require the network to produce. For example, if we present the network with [1,1] for the AND function the training value will be 1 17.05.11 AI NN 41
Learning
End While
Error, Err : The error value is the amount by which the value output by the network differs from the training value. For example, if we required the network to output 0 and it output a 1, then Err = -1 17.05.11 AI NN 42
Learning
Present network with next inputs from epoch Err = T O If Err <> 0 then Wj = Wj + LR * Ij * Err End OutputIffrom Neuron, O : The output
End While
value from the neuron
Ij : Inputs being presented to the neuron Wj : Weight from input neuron (Ij) to the output neuron LR : The learning rate. This dictates how quickly the network converges. It is set by a matter of experimentation. It is typically 0.1 17.05.11 AI NN 43
Learning
After First Epoch Note I1 point = W0/W1 I2 point = W0/W2
0,1
I1 1,1
0,0
1,0
I2
I1 1,1 0,1
At Convergence
0,0 1,0 I2
17.05.11
AI NN
44
Biological Neural Nets
Pigeons as art experts (Watanabe et al.

1995)
Experiment:
Pigeon in Skinner box Present paintings of two different artists (e.g. Chagall / Van Gogh) Reward for pecking when presented a particular artist (e.g. Van Gogh)
17.05.11
AI NN
45
17.05.11
AI NN
46
17.05.11
AI NN
47
17.05.11
AI NN
48
Pigeons were able to discriminate between Van Gogh and Chagall with 95% accuracy (when presented with pictures they had been trained on) Discrimination still 85% successful for previously unseen paintings of the artists Pigeons do not simply memorise the pictures They can extract and recognise patterns (the style) They generalise from the already seen to make predictions This is what neural networks (biological and artificial) are good at (unlike conventional computer) AI NN 49
17.05.11
Feeding
data through the net:
(1 0.25) + (0.5 (-1.5)) = 0.25 + (-0.75) = - 0.5
Squashing: 17.05.11
1 = 0 . 3775 1+ e 0 . 5
AI NN 50
Data is presented to the network in the form of activations in the input layer Examples

Pixel intensity (for pictures) Molecule concentrations (for artificial nose) Share prices (for stock market prediction)
Data usually requires preprocessing
Analogous to senses in biology
How to represent more abstract data, e.g. a name?
Choose a pattern, e.g.

17.05.11
0-0-1 for Chris 0-1-0 for Becky
AI NN
51
Weight settings determine the behaviour of a network How can we find the right weights?
17.05.11
AI NN
52
Training the Network - Learning
Backpropagation
Requires training set (input / output pairs) Starts with small random weights Error is used to adjust weights (supervised learning) Gradient descent on error landscape
17.05.11
AI NN
53
17.05.11
AI NN
54
Advantages

It works! Relatively fast
Downsides

Requires a training set Can be slow Probably not biologically realistic
Alternatives to Backpropagation
Hebbian learning
Not successful in feed-forward nets Only limited success More general, but can be even slower than backprop
Reinforcement learning
Artificial evolution
17.05.11
AI NN
55
Example: Voice Recognition
Task: Learn to discriminate between two different voices saying Hello Data
Sources

Steve Simpson David Raubenheimer Frequency distribution (60 bins) Analogy: cochlea
Format

17.05.11
AI NN
56
Network architecture
Feed forward network

60 input (one for each frequency bin) 6 hidden 2 output (0-1 for Steve, 1-0 for David)
17.05.11
AI NN
57
Presenting the data
Steve
David
17.05.11
AI NN
58
Presenting the data (untrained network)

0.43 0.26
Steve
David
0.73 0.55 17.05.11 AI NN 59
Calculate error
0.43 0 0.26 1 = 0.43 = 0.74
Steve
David
0.73 1 0.55 0 17.05.11 AI NN 60
= 0.27 = 0.55
Backprop error and adjust weights

0.43 0 0.26 1 = 0.43 = 0.74 1.17
Steve
David
0.73 1 0.55 0 17.05.11 AI NN 61
= 0.27 = 0.55 0.82
Repeat process (sweep) for all training pairs

Present data Calculate error Backpropagate error Adjust weights
Repeat process multiple times
17.05.11
AI NN
62
Presenting the data (trained network)

0.01 0.99
Steve
David
0.99 0.01 17.05.11 AI NN 63
Results Voice Recognition
Performance of trained network
Discrimination accuracy between known Hellos

100%
Discrimination accuracy between new Hellos

100%
17.05.11
AI NN
64
Results Voice Recognition (ctnd.)
Network has learnt to generalise from original data Networks with different weight settings can have same functionality Trained networks concentrate on lower frequencies
Network is robust against non-functioning 17.05.11 AI NN 65 nodes
Applications of Feed-forward nets
Pattern recognition

Character recognition Face Recognition
Sonar mine/rock recognition (Gorman & Sejnowksi, 1988) Navigation of a car (Pomerleau, 1989) Stock-market prediction Pronunciation (NETtalk)
(Sejnowksi & Rosenberg, 1987)
17.05.11
AI NN
66
Cluster analysis of hidden layer
17.05.11
AI NN
67
FFNs as Biological Modelling Tools
Signalling / Sexual Selection
Enquist & Arak (1994)
Preference for symmetry not selection for good genes, but instead arises through the need to recognise objects irrespective of their orientation Exaggerated, symmetric ornaments facilitate mate recognition
Johnstone (1994)
(but see Dawkins & Guilford, 1995)

17.05.11 AI NN 68
Recurrent Networks
Feed forward networks:

Information only flows one way One input pattern produces one output No sense of time (or memory of previous state)
Recurrency

Nodes connect back to other nodes or themselves Information flow is multidirectional Sense of time and memory of previous state(s)
Biological nervous systems show high levels of recurrency (but feed-forward structures exists too) 17.05.11 AI NN 69
Elman Nets
Elman nets are feed forward networks with partial recurrency
Unlike feed forward nets, Elman nets have a memory or sense of time 17.05.11 AI NN 70
Classic experiment on language acquisition and processing (Elman, 1990)
Task
Elman net to predict successive words in sentences.
Data
Suite of sentences, e.g.

The boy catches the ball. The girl eats an apple.
Words are input one at a time
Representation
Binary representation for each word, e.g.
0-1-0-0-0 for girl
Training method
Backpropagation
17.05.11
AI NN
71
Internal representation of words
17.05.11
AI NN
72
Hopfield Networks
Sub-type of recurrent neural nets

Fully recurrent Weights are symmetric Nodes can only be on or off Random updating
Learning: Hebb rule (cells that fire together wire

together)
Biological equivalent to LTP and LTD
Can recall a memory, if presented with a corrupt or incomplete version
auto-associative or content-addressable memory
17.05.11
AI NN
73
Task: store images with resolution of 20x20 pixels Hopfield net with 400 nodes
Memorise:
1. 2.
Present image Apply Hebb rule (cells that fire together, wire together)
Increase weight between two nodes if both have same activity, otherwise decrease
3.
Go to 1
Recall:
1. 2. 3.
Present incomplete pattern Pick random node, update Go to 2 until settled
17.05.11
AI NN
74
Memories are attractors in state space
17.05.11
AI NN
75
Catastrophic forgetting
Problem: memorising new patterns corrupts the memory of older ones
Old memories cannot be recalled, or spurious memories arise
Solution: allow Hopfield net to sleep
17.05.11
AI NN
76
Two approaches (both using randomness): Unlearning (Hopfield, 1986)

Recall old memories by random stimulation, but use an inverse Hebb rule Makes room for new memories (basins of attraction shrink)
Pseudorehearsal (Robins, 1995)

While learning new memories, recall old memories by random stimulation Use standard Hebb rule on new and old memories Restructure memory Needs short-term + long term memory Mammals: hippocampus plays back new memories to neo-cortex, which is randomly stimulated at the same time
17.05.11
AI NN
77
RNNs as Central Pattern Generators
CPGs: group of neurones creating rhythmic muscle activity for locomotion, heart-beat etc. Identified in several invertebrates and vertebrates Hard to study
Computer modelling
E.g. lamprey swimming (Ijspeert et al., 1998)
17.05.11
AI NN
78
Evolution of Bipedal Walking (Reil & Husbands, 2001)
17.05.11
AI NN
79
CPG cycles are cyclic attractors in state space
17.05.11
AI NN
80
Recap Neural Networks
Components biological plausibility

Neurone / node Synapse / weight
Feed forward networks

Unidirectional flow of information Good at extracting patterns, generalisation and prediction Distributed representation of data Parallel processing of data Training: Backpropagation Not exact models, but good at demonstrating principles
Recurrent networks

17.05.11
Multidirectional flow of information Memory / sense of time Complex temporal dynamics (e.g. CPGs) Various training methods (Hebbian, evolution) Often better biological models than FFNs
AI NN
81
Supervised Vs. Unsupervised
Previously discussed networks are supervised
Need to be trained ahead of time with lots of data
Unsupervised networks adapt to the input

Applications in Clustering and reducing dimensionality Learning may be very slow
17.05.11
AI NN
82
Current Applications
Investment Analysis

Predicting movement of stocks Replacing earlier linear models Bank Checks, VISA, etc. Chemistry related Sensor networks may gather more data than can be processed by operators Inputs: Cues from camera data, vibration levels, sound, radar, lydar, etc. Output: Number of people at a terminal, engine warning light, control for light switch
Signature Analysis
Process Control
Monitoring

17.05.11
AI NN
83
Current Work

Work of S. Cho, Chow and C. Leung Training Neural Network to approximate crowd size based on image cues Network approximates a non linear mapping between input and output space Their input cues: Black Pixels, White Pixels, Edge Pixels Our proposed inputs: Edge Histogram, Blob Size histogram (based on background model)
AI NN 84
17.05.11
References
[1] L. Smith, ed. (1996, 2001), "An Introduction to Neural Networks", URL: http://www.cs.stir.ac.uk/~lss/NNIntro/InvSlides.html [2] Sarle, W.S., ed. (1997), Neural Network FAQ, URL: ftp://ftp.sas.com/pub/neural/FAQ.html [3] StatSoft, "Neural Networks", URL: http://www.statsoftinc.com/textbook/stneunet.html [4] S. Cho, T. Chow, and C. Leung, "A Neural-Based Crowd Estimation by Hybrid Global Learning Algorithm", IEEE Transactions on Systems, Man and Cybernetics, Part B, No. 4. 1999. 17.05.11 AI NN 85
What is an Evolutionary Algorithm?

Lars Mehnen
Contents
Recap of Evolutionary Metaphor Basic scheme of an EA Basic Components:
Representation / Evaluation / Population / Parent Selection / Recombination / Mutation / Survivor Selection / Termination
Examples : eight queens / knapsack Typical behaviours of EAs EC in context of global optimisation
Recap of EC metaphor
A population of individuals exists in an environment with limited resources Competition for those resources causes selection of those fitter individuals that are better adapted to the environment These individuals act as seeds for the generation of new individuals through recombination and mutation The new individuals have their fitness evaluated and compete (possibly also with parents) for survival. Over time Natural selection causes a rise in the fitness of the population
Recap 2:
EAs fall into the category of generate and test algorithms They are stochastic, population-based algorithms Variation operators (recombination and mutation) create the necessary diversity and thereby facilitate novelty Selection reduces diversity and acts as a force pushing quality
General Scheme of EAs
Pseudo-code for typical EA
What are the different types of EAs

Historically different flavours of EAs have been associated with different representations
Binary strings : Genetic Algorithms Real-valued vectors : Evolution Strategies Finite state Machines: Evolutionary Programming LISP trees: Genetic Programming
These differences are largely irrelevant, best strategy

choose representation to suit problem choose variation operators to suit representation
Selection operators only use fitness and so are independent of representation
Representations
Candidate solutions (individuals) exist in phenotype space They are encoded in chromosomes, which exist in genotype space
Encoding : phenotype=> genotype (not necessarily one to one) Decoding : genotype=> phenotype (must be one to one)
Chromosomes contain genes, which are in (usually fixed) positions called loci (sing. locus) and have a value (allele) In order to find the global optimum, every feasible solution must be represented in genotype space
Evaluation (Fitness) Function

Represents the requirements that the population should adapt to a.k.a. quality function or objective function Assigns a single real-valued fitness to each phenotype which forms the basis for selection
So the more discrimination (different values) the better

Typically we talk about fitness being maximised
Some problems may be best posed as minimisation problems, but conversion is trivial
Population
Holds (representations of) possible solutions Usually has a fixed size and is a multiset of genotypes Some sophisticated EAs also assert a spatial structure on the population e.g., a grid. Selection operators usually take whole population into account i.e., reproductive probabilities are relative to current generation Diversity of a population refers to the number of different fitnesses / phenotypes / genotypes present (note not the same thing)
Parent Selection Mechanism

Assigns variable probabilities of individuals acting as parents depending on their fitnesses Usually probabilistic
high quality solutions more likely to become parents than low quality but not guaranteed even worst in current population usually has non-zero probability of becoming a parent
This stochastic nature can aid escape from local optima
Variation Operators
Role is to generate new candidate solutions Usually divided into two types according to their arity (number of inputs):
Arity 1 : mutation operators Arity >1 : Recombination operators Arity = 2 typically called crossover
There has been much debate about relative importance of recombination and mutation
Nowadays most EAs use both Choice of particular variation operators is representation dependant
Mutation
Acts on one genotype and delivers another Element of randomness is essential and differentiates it from other unary heuristic operators Importance ascribed depends on representation and dialect:
Binary GAs background operator responsible for preserving and introducing diversity EP for FSMs/ continuous variables only search operator GP hardly used
May guarantee connectedness of search space and hence convergence proofs
Recombination
Merges information from parents into offspring Choice of what information to merge is stochastic Most offspring may be worse, or the same as the parents Hope is that some are better by combining elements of genotypes that lead to good traits Principle has been used for millennia by breeders of plants and livestock
Survivor Selection
a.k.a. replacement Most EAs use fixed population size so need a way of going from (parents + offspring) to next generation Often deterministic
Fitness based : e.g., rank parents+offspring and take best Age based: make as many offspring as parents and delete all parents
Sometimes do combination (elitism)
Initialisation / Termination
Initialisation usually done at random,
Need to ensure even spread and mixture of possible allele values Can include existing solutions, or use problem-specific heuristics, to seed the population
Termination condition checked every generation

Reaching some (known/hoped for) fitness Reaching some maximum allowed number of generations Reaching some minimum level of diversity Reaching some specified number of generations without fitness improvement
Example: the 8 queens problem
Place 8 queens on an 8x8 chessboard in such a way that they cannot check each other
The 8 queens problem: representation
Phenotype: a board configuration
Genotype: a permutation of the numbers 1 - 8
Obvious mapping
1 3 5 2 6 4 7 8
8 Queens Problem: Fitness evaluation

Penalty of one queen: the number of queens she can check. Penalty of a configuration: the sum of the penalties of all queens. Note: penalty is to be minimized Fitness of a configuration: inverse penalty to be maximized
The 8 queens problem: Mutation

Small variation in one permutation, e.g.:
swapping values of two randomly chosen positions,
1 3 5 2 6 4 7 8
1 3 7 2 6 4 5 8
The 8 queens problem: Recombination

Combining two permutations into two new permutations: choose random crossover point copy first parts into children create second part by inserting values from other parent: in the order they appear there beginning after crossover point skipping values already in child
1 3 526 4 7 8 8 7 654 3 2 1
1 3 542 8 7 6 8 7 624 1 3 5
The 8 queens problem: Selection

Parent selection:
Pick 5 parents and take best two to undergo crossover
Survivor selection (replacement)

When inserting a new child into the population, choose an existing member to replace by: sorting the whole population by decreasing fitness enumerating this list from high to low replacing the first with a fitness lower than the given child
8 Queens Problem: summary
Note that is is only one possible set of choices of operators and parameters
Typical behaviour of an EA
Phases in optimising on a 1-dimensional fitness landscape
Early phase: quasi-random population distribution
Mid-phase: population arranged around/on hills
Late phase: population concentrated on high hills
Typical run: progression of fitness

Best fitness in population
Time (number of generations)
Typical run of an EA shows so-called anytime behavior
Are long runs beneficial?

Progress in 2nd half
Progress in 1st half
Answer: - it depends how much you want the last bit of progress - it may be better to do more shorter runs
Is it worth expending effort on smart initialisation?

F: fitness after smart initialisation T: time needed to reach level F after random initialisation
T

Answer : it depends:
- possibly, if good solutions/methods exist. - care is needed, see chapter on hybridisation
Evolutionary Algorithms in Context

There are many views on the use of EAs as robust problem solving tools For most problems a problem-specific tool may:
perform better than a generic search algorithm on most instances, have limited utility, not do well on all instances
Goal is to provide robust tools that provide:
evenly good performance over a range of problems and instances
EAs as problem solvers: Goldbergs 1989 view

Performance of methods on problems
Special, problem tailored method Evolutionary algorithm
Random search
Scale of all problems
EAs and domain knowledge

Trend in the 90s: adding problem specific knowledge to EAs (special variation operators, repair, etc) Result: EA performance curve deformation:
better on problems of the given type worse on problems different from given type amount of added knowledge is variable
Recent theory suggests the search for an all-purpose algorithm may be fruitless
Michalewicz 1996 view

EA 4 Performance of methods on problems
EA 2
EA 3
EA 1
Scale of all problems
EC and Global Optimisation

Global Optimisation: search for finding best solution x* out of some fixed set S Deterministic approaches
e.g. box decomposition (branch and bound etc) Guarantee to find x* , but may run in superpolynomial time
Heuristic Approaches (generate and test)
rules for deciding which x S to generate next no guarantees that best solutions found are globally optimal
EC and Neighbourhood Search

Many heuristics impose a neighbourhood structure on S Such heuristics may guarantee that best point found is locally optimal e.g. Hill-Climbers:
But problems often exhibit many local optima Often very quick to identify good solutions
EAs are distinguished by:
Use of population, Use of multiple, stochastic search operators Especially variation operators with arity >1 Stochastic selection
GA Quick Overview
Developed: USA in the 1970s Early names: J. Holland, K. DeJong, D. Goldberg Typically applied to:
discrete optimization
Attributed features:
not too fast good heuristic for combinatorial problems
Special Features:
Traditionally emphasizes combining information from good parents (crossover) many variants, e.g., reproduction models, operators
Genetic algorithms
Hollands original GA is now known as the simple genetic algorithm (SGA) Other GAs use different:
Representations Mutations Crossovers Selection mechanisms
SGA technical summary tableau

Representation Recombination Mutation Parent selection Survivor selection Speciality Binary strings N-point or uniform Bitwise bit-flipping with fixed probability Fitness-Proportionate All children replace parents Emphasis on crossover
Representation
Phenotype space Genotype space = {0,1}L 10010001 10010010 010001001 011101001 Decoding (inverse representation)
Encoding (representation)
SGA reproduction cycle
1. Select parents for the mating pool (size of mating pool = population size) 2. Shuffle the mating pool 3. For each consecutive pair apply crossover with probability pc , otherwise copy parents 4. For each offspring apply mutation (bit-flip with probability pm independently for each bit) 5. Replace the whole population with the resulting offspring
SGA operators: 1-point crossover

Choose a random point on the two parents Split parents at this crossover point Create children by exchanging tails Pc typically in range (0.6, 0.9)
SGA operators: mutation

Alter each gene independently with a probability pm pm is called the mutation rate
Typically between 1/pop_size and 1/ chromosome_length
SGA operators: Selection

Main idea: better individuals get higher chance
Chances proportional to fitness Implementation: roulette wheel technique

Assign to each individual a part of the roulette wheel Spin the wheel n times to select n individuals fitness(A) = 3 fitness(B) = 1 fitness(C) = 2
1/6 = 17%
A
3/6 = 50%
2/6 = 33%
An example after Goldberg 89 (1) Simple problem: max x2 over {0,1,,31} GA approach:
Representation: binary code, e.g. 01101 13 Population size: 4 1-point xover, bitwise mutation Roulette wheel selection Random initialisation
We show one generational cycle done by hand
x2 example: selection
X2 example: crossover
X2 example: mutation
The simple GA
Has been subject of many (early) studies
still often used as benchmark for novel GAs
Shows many shortcomings, e.g.

Representation is too restrictive Mutation & crossovers only applicable for bit-string & integer representations Selection mechanism sensitive for converging populations with close fitness values Generational population model (step 5 in SGA repr. cycle) can be improved with explicit survivor selection
Alternative Crossover Operators

Performance with 1 Point Crossover depends on the order that variables occur in the representation
more likely to keep together genes that are near each other Can never keep together genes from opposite ends of string This is known as Positional Bias Can be exploited if we know about the structure of our problem, but this is not usually the case
n-point crossover
Choose n random crossover points Split along those points Glue parts, alternating between parents Generalisation of 1 point (still some positional bias)
Uniform crossover
Assign 'heads' to one parent, 'tails' to the other Flip a coin for each gene of the first child Make an inverse copy of the gene for the second child Inheritance is independent of position
Crossover OR mutation?
Decade long debate: which one is better / necessary / main-background Answer (at least, rather wide agreement):
it depends on the problem, but in general, it is good to have both both have another role mutation-only-EA is possible, xover-only-EA would not work
Crossover OR mutation? (contd)

Exploration: Discovering promising areas in the search space, i.e. gaining information on the problem Exploitation: Optimising within a promising area, i.e. using information There is co-operation AND competition between them Crossover is explorative, it makes a big jump to an area somewhere in between two (parent) areas Mutation is exploitative, it creates random small diversions, thereby staying near (in the area of ) the parent
Crossover OR mutation? (contd)

Only crossover can combine information from two parents Only mutation can introduce new information (alleles) Crossover does not change the allele frequencies of the population (thought experiment: 50% 0s on first bit in the population, ?% after performing n crossovers) To hit the optimum you often need a lucky mutation
Other representations
Gray coding of integers (still binary chromosomes)
Gray coding is a mapping that means that small changes in the genotype cause small changes in the phenotype (unlike binary coding). Smoother genotype-phenotype mapping makes life easier for the GA
Nowadays it is generally accepted that it is better to encode numerical variables directly as Integers Floating point variables
Integer representations
Some problems naturally have integer variables, e.g. image processing parameters Others take categorical values from a fixed set e.g. {blue, green, yellow, pink} N-point / uniform crossover operators work Extend bit-flipping mutation to make
creep i.e. more likely to move to similar value Random choice (esp. categorical variables) For ordinal problems, it is hard to know correct range for creep, so often use two mutation operators in tandem
Real valued problems

Many problems occur as real valued problems, e.g. continuous parameter optimisation f : n Illustration: Ackleys function (often used in EC)
Mapping real values on bit strings

z [x,y] represented by {a1,,aL} {0,1}L [x,y] {0,1}L must be invertible (one phenotype per genotype) : {0,1}L [x,y] defines the representation
y x L1 ( a1 ,..., aL ) = x + L ( aL j 2 j ) [ x, y ] 2 1 j =0
Only 2L values out of infinite are represented L determines possible maximum precision of solution High precision long chromosomes (slow evolution)
Floating point mutations 1

General scheme of floating point mutations
x = x1 , ..., xl x = x1 , ..., xl xi , xi [LBi , UBi ]
Uniform mutation:
xi drawn randomly (uniform) from [LBi ,UBi ]
Analogous to bit-flipping (binary) or random resetting (integers)
Floating point mutations 2

Non-uniform mutations:
Many methods proposed,such as time-varying range of change etc. Most schemes are probabilistic but usually only make a small change to value Most common method is to add random deviate to each variable separately, taken from N(0, ) Gaussian distribution and then curtail to range Standard deviation controls amount of change (2/3 of deviations will lie in range (- to + )
Crossover operators for real valued GAs

Discrete:
each allele value in offspring z comes from one of its parents (x,y) with equal probability: zi = xi or yi Could use n-point or uniform
Intermediate
exploits idea of creating children between parents (hence a.k.a. arithmetic recombination) zi = xi + (1 - ) yi where : 0 1. The parameter can be:
constant: uniform arithmetical crossover variable (e.g. depend on the age of the population) picked at random every time
Single arithmetic crossover

Parents: x1,,xn and y1,,yn Pick a single gene (k) at random, child1 is:
x1 , ..., xk , yk + (1 ) xk , ..., xn
reverse for other child. e.g. with = 0.5
Simple arithmetic crossover

Parents: x1,,xn and y1,,yn Pick random gene (k) after this point mix values child1 is:
x , ..., x , y + (1 ) x , ..., y + (1 ) x 1 k k +1 k +1 n n
Whole arithmetic crossover

Most commonly used Parents: x1,,xn and y1,,yn child1 is:
a x + (1 a ) y
Permutation Representations
Ordering/sequencing problems form a special type Task is (or can be solved by) arranging some objects in a certain order
Example: sort algorithm: important thing is which elements occur before others (order) Example: Travelling Salesman Problem (TSP) : important thing is which elements occur next to each other (adjacency)
These problems are generally expressed as a permutation:

if there are n variables then the representation is as a list of n integers, each of which occurs exactly once
Permutation representation: TSP example

Problem: Given n cities Find a complete tour with minimal length Encoding: Label the cities 1, 2, , n One complete tour is one permutation (e.g. for n =4 [1,2,3,4], [3,4,2,1] are OK) Search space is BIG: for 30 cities there are 30! 1032 possible tours
Mutation operators for permutations

Normal mutation operators lead to inadmissible solutions
e.g. bit-wise mutation : let gene i have value j changing to some other value k would mean that k occurred twice and j no longer occurred
Therefore must change at least two values Mutation parameter now reflects the probability that some operator is applied once to the whole string, rather than individually in each position
Insert Mutation for permutations

Pick two allele values at random Move the second to follow the first, shifting the rest along to accommodate Note that this preserves most of the order and the adjacency information
Swap mutation for permutations

Pick two alleles at random and swap their positions Preserves most of adjacency information (4 links broken), disrupts order more
Inversion mutation for permutations

Pick two alleles at random and then invert the substring between them. Preserves most adjacency information (only breaks two links) but disruptive of order information
Scramble mutation for permutations

Pick a subset of genes at random Randomly rearrange the alleles in those positions
(note subset does not have to be contiguous)
Crossover operators for permutations

Normal
crossover operators will often lead to inadmissible solutions

12345 54321 12321 54345
Many specialised operators have been devised which focus on combining order or adjacency information from the two parents
Order 1 crossover
Idea is to preserve relative order that elements occur Informal procedure:
1. Choose an arbitrary part from the first parent 2. Copy this part to the first child 3. Copy the numbers that are not in the first part, to the first child:
starting right from cut point of the copied part, using the order of the second parent and wrapping around at the end
4. Analogous for the second child, with parent roles reversed
Order 1 crossover example

Copy randomly selected set from first parent
Copy rest from second parent in order 1,9,3,8,2
Partially Mapped Crossover (PMX)

Informal procedure for parents P1 and P2: 1. Choose random segment and copy it from P1 2. Starting from the first crossover point look for elements in that segment of P2 that have not been copied 3. For each of these i look in the offspring to see what element j has been copied in its place from P1 4. Place i into the position occupied j in P2, since we know that we will not be putting j there (as is already in offspring) 5. If the place occupied by j in P2 has already been filled in the offspring k, put i in the position occupied by k in P2 6. Having dealt with the elements from the crossover segment, the rest of the offspring can be filled from P2. Second child is created analogously
PMX example
Step 1
Step 2
Step 3
Cycle crossover
Basic idea: Each allele comes from one parent together with its position.
Informal procedure:
1. Make a cycle of alleles from P1 in the following way.
(a) Start with the first allele of P1. (b) Look at the allele at the same position in P2. (c) Go to the position with the same allele in P1. (d) Add this allele to the cycle. (e) Repeat step b through d until you arrive at the first allele of P1.
2. Put the alleles of the cycle in the first child on the positions they have in the first parent. 3. Take next cycle from second parent
Cycle crossover example

Step 1: identify cycles
Step 2: copy alternate cycles into offspring
Edge Recombination
Works by constructing a table listing which edges are present in the two parents, if an edge is common to both, mark with a + e.g. [1 2 3 4 5 6 7 8 9] and [9 3 7 8 2 6 5 1 4]
Edge Recombination 2
Informal procedure once edge table is constructed
1. Pick an initial element at random and put it in the offspring 2. Set the variable current element = entry 3. Remove all references to current element from the table 4. Examine list for current element:
If there is a common edge, pick that to be next element Otherwise pick the entry in the list which itself has the shortest list Ties are split at random
5. In the case of reaching an empty list:

Examine the other end of the offspring is for extension Otherwise a new element is chosen at random
Edge Recombination example
Multiparent recombination
Recall that we are not constricted by the practicalities of nature Noting that mutation uses 1 parent, and traditional crossover 2, the extension to a>2 is natural to examine Been around since 1960s, still rare but studies indicate useful Three main types:
Based on allele frequencies, e.g., p-sexual voting generalising uniform crossover Based on segmentation and recombination of the parents, e.g., diagonal crossover generalising n-point crossover Based on numerical operations on real-valued alleles, e.g., center of mass crossover, generalising arithmetic recombination operators
Population Models
SGA uses a Generational model:
each individual survives for exactly one generation the entire set of parents is replaced by the offspring
At the other end of the scale are Steady-State models:

one offspring is generated per generation, one member of population replaced,
Generation Gap
the proportion of the population replaced 1.0 for GGA, 1/pop_size for SSGA
Fitness Based Competition

Selection can occur in two places:
Selection from current generation to take part in mating (parent selection) Selection from parents + offspring to go into next generation (survivor selection)
Selection operators work on whole individual

i.e. they are representation-independent
Distinction between selection

operators: define selection probabilities algorithms: define how probabilities are implemented
Implementation example: SGA

Expected number of copies of an individual i E( ni ) = f(i)/ f
( = pop.size, f(i) = fitness of i, f avg. fitness in pop.)
Roulette wheel algorithm:

Given a probability distribution, spin a 1-armed wheel n times to make n selections No guarantees on actual value of ni
Bakers SUS algorithm:

n evenly spaced arms on wheel and spin once Guarantees floor(E( ni ) ) ni ceil(E( ni ) )
Fitness-Proportionate Selection
Problems include
One highly fit member can rapidly take over if rest of population is much less fit: Premature Convergence At end of runs when fitnesses are similar, lose selection pressure Highly susceptible to function transposition
Scaling can fix last two problems

Windowing: f(i) = f(i) - t
where is worst fitness in this (last n) generations
Sigma Scaling: f(i) = max( f(i) ( f - c f ), 0.0)

where c is a constant, usually 2.0
Function transposition for FPS
Rank Based Selection

Attempt to remove problems of FPS by basing selection probabilities on relative rather than absolute fitness Rank population according to fitness and then base selection probabilities on rank where fittest has rank and worst rank 1 This imposes a sorting overhead on the algorithm, but this is usually negligible compared to the fitness evaluation time
Linear Ranking
Parameterised by factor s: 1.0 < s 2.0
measures advantage of best individual in GGA this is the number of children allotted to it
Simple 3 member example
Exponential Ranking
Linear Ranking is limited to selection pressure Exponential Ranking can allocate more than 2 copies to fittest individual Normalise constant factor c according to population size
Tournament Selection
All methods above rely on global population statistics
Could be a bottleneck esp. on parallel machines Relies on presence of external fitness function which might not exist: e.g. evolving game players
Informal Procedure:
Pick k members at random then select the best of these Repeat to select more individuals
Tournament Selection 2
Probability of selecting i will depend on:
Rank of i Size of sample k
higher k increases selection pressure
Whether contestants are picked with replacement

Picking without replacement increases selection pressure
Whether fittest contestant always wins (deterministic) or this happens with probability p
For k = 2, time for fittest individual to take over population is the same as linear ranking with s = 2 p
Survivor Selection
Most of methods above used for parent selection Survivor selection can be divided into two approaches:
Age-Based Selection
e.g. SGA In SSGA can implement as delete-random (not recommended) or as first-in-first-out (a.k.a. deleteoldest)
Fitness-Based Selection
Using one of the methods above or
Two Special Cases

Elitism
Widely used in both population models (GGA, SSGA) Always keep at least one copy of the fittest solution so far
GENITOR: a.k.a. delete-worst

From Whitleys original Steady-State algorithm (he also used linear ranking for parent selection) Rapid takeover : use with large populations or no duplicates policy
Example application of order based GAs: JSSP

Precedence constrained job shop scheduling problem
J is a set of jobs. O is a set of operations M is a set of machines Able O M defines which machines can perform which operations Pre O O defines which operation should precede which Dur : O M IR defines the duration of o O on m M The goal is now to find a schedule that is: Complete: all jobs are scheduled Correct: all conditions defined by Able and Pre are satisfied Optimal: the total duration of the schedule is minimal
Precedence constrained job shop scheduling GA

Representation: individuals are permutations of operations Permutations are decoded to schedules by a decoding procedure
take the first (next) operation from the individual look up its machine (here we assume there is only one) assign the earliest possible starting time on this machine, subject to
machine occupation precedence relations holding for this operation in the schedule created so far
fitness of a permutation is the duration of the corresponding schedule (to be minimized) use any suitable mutation and crossover use roulette wheel parent selection on inverse fitness Generational GA model for survivor selection use random initialisation
JSSP example: operator comparison
Hodgkin-Huxley Model and FitzHugh-Nagumo Model

Technikum-Wien
04.02.2012
AI HH
Andrew Fielding Huxley
Alan Lloyd Hodgkin
A quantitative description of membrane current and its application to conduction and excitation in nerve. The Journal of Physiology 117:500-544 (1952)
Nobelpreice for medicine, 1963
04.02.2012
The squid (Loligo)
AI HH
Loligo giant axones
04.02.2012
AI HH
Nervous System
Signals are propagated from nerve cell to nerve cell (neuron) via electro-chemical mechanisms ~100 billion neurons in a person Hodgkin and Huxley experimented on squid giant axons and discovered how the signal is produced and propagate within the neuron H.-H. model was published in Jour. of Physiology (1952) H.-H. were awarded 1963 Nobel Prize FitzHugh-Nagumo model is a simplification
04.02.2012 AI HH 4
Neurons
Basic morphology
Presynaptic terminal (axon or dendrite) Postsynaptic process (dendrite or cell body)
Cerebral cortex
108 - 109 synapses/mm3 0.5 - 1.0 m diameter of synaptic contact
04.02.2012
AI HH
Neuron
04.02.2012
AI C. GeorgeHH Boeree: www.ship.edu/~cgboeree/
Electron Micrograph of a Real Neuron
04.02.2012
AI HH
Remember? Transmenbrane Ionic Transport

Ion channels act as gates that allow or block the flow of specific ions into and out of the cell.
04.02.2012
AI HH
04.02.2012
AI HH
Ein Aktionspotential besteht aus verschiedenen Phasen
Depolarisation
Repolarisation
Restingpotential
Posthyperpolarisation
Restingpotential
04.02.2012
AI HH
10
Das Modell von Hodgkin and Huxley

The Aktionspotential: During the resting-potential the Na+-channels are closed (not inactivated!, not activated) After depolarization the Na+-channels open fast, GNa is increased The membrane depolarizes further more Slowly the K+-channels, GK increases Na+-channels close (inactivate) K+-flow (current) dominates den allover current The membrane-potential repolarizes The K+-channels remain still in an open state with decreased GNa After hyper-polarization (Vm near EK) As long as Na+-channels are inactivated: absolute refractory period
11
04.02.2012
AI HH
Synaptic transmission
Depolarization due to action potential Influx of calcium Exocystosis of synaptic vesicles Diffusion of neurotransmitter Binding to postsynaptic receptors
04.02.2012
AI HH
12
The basic Hodgkin - Huxley model

The standard model of an exicitatory neuron consists of the equation for the total membrane current IM
dV I M Cm = + I K + I Na + I L dt
04.02.2012
AI HH
13
Currents
V denotes the membrane voltage IK is the potassium current INa is the sodium current IL is a leakage current
04.02.2012
AI HH
14
Circuit
04.02.2012
AI HH
15
The Cable Equation

See http://diwww.epfl.ch/~gerstner/SPNM/SPNM.html for excellent additional material Just a piece of passive dendrite can yield complicated differential equations which have been extensively studied by electro-technicians in the context of the study of coaxial cables (TV antenna cable):
04.02.2012
AI HH
16
Action Potential
mV _ 30
_0
10 msec -70
04.02.2012 AI HH
Axon membrane potential difference V = Vi Ve When the axon is excited, V spikes because sodium Na+ and potassium K+ ions flow through the membrane.
17
Charge across a capacitor is q = CE, C is capacitance, E is the voltage across the capacitor, differentiating:
Ohms law:
dq dE =C dt dt
V = IR I = V / R = gV
Hodkin Huxley equations K and Na nonlinear functions of membrane voltage and time.
I c= g c (V , t ) (V (t ) E (t ))
04.02.2012
V-Vc is the voltage difference across the membrane and the equilibrium value for the ion is Vc, which may change in time.
AI HH
18
Circuit Model for Axon Membrane

Since the membrane separates charge, it is modeled as a capacitor with capacitance C. Ion channels are resistors. 1/R = g = conductance iC = C dV/dt iNa = gNa (V VNa) iK= gK (V VK) iL = gL (V V:)
04.02.2012 AI HH 19
Nernst Potential VNa , VK and Vr
Ion flow due to electrical signal
Traveling wave
04.02.2012
AI HH C. George Boeree: www.ship.edu/~cgboeree/
20
Circuit Equations
Since the sum of the currents is 0, it follows that
dV C = Na (V V Na ) g K (V V K ) gr (V Vr ) + Iap g dt
where Iap is applied current. If ion conductances are constants then group constants to obtain 1st order, linear eq
dV C = g (V V *) + Iap dt
Solving gives
04.02.2012
V (t ) V * + Iap / g
AI HH
21
Variable Conductance
g
Experiments showed that gNa and gK varied with time and V. After stimulus, Na responds much more rapidly than K . 04.02.2012 AI HH 22
Hodgkin-Huxley System
Four state variables are used: v(t)=V(t)-Veq is membrane potential, m(t) is Na activation, n(t) is K activation and h(t) is Na inactivation.
In terms of these variables gK=gKn4 and gNa=gNam3h. The resting potential Veq-70mV. Voltage clamp experiments determined gK and n as functions of t and hence the parameter dependences on v in the differential eq. for n(t). Likewise for m(t) and h(t).
04.02.2012 AI HH 23
Hodgkin-Huxley System
dv 3 4 C = g Na m h(v VNa ) g K n (v VK ) gr (v Vr ) + I ap dt
dm = m( v )(1 m) m( v )m dt dn = n ( v )(1 n ) n ( v )n dt
04.02.2012
dh = h ( v )(1 h ) h ( v )h dt
AI HH
24
110 mV
Iap =8, v(t)

1.2 m(t)
n(t) 40msec
h(t) 10msec
04.02.2012
IapHH v(t) AI =7,
25
Fast-Slow Dynamics
m(t)
m(v) dm/dt = m(v) m. m(v) is much smaller than

n(t)
h(t)
n(v) and h(v). An increase in v results in an increase in m(v) and a large dm/dt. Hence Na activates more rapidly than K in response to a change in v.
10msec
v, m are on a fast time scale and n, h are slow.

04.02.2012 AI HH 26
FitzHugh-Nagumo System
dv = f (v ) w + I dt
and
dw = v 0.5w dt
dw ( v 0.5w) = dv f (v ) w + I
Observe that in the (v,w) phase plane
I represents applied current, is small and f(v) is a cubic nonlinearity.
which is small unless the solution is near f(v)-w+I=0. Thus the slow manifold is the cubic w=f(v)+I which is the nullcline of the fast variable v. And w is the slow variable with nullcline w=2v.
04.02.2012 AI HH 27
Take f(v)=v(1-v)(v-a) . Stable rest state w I=0 w Stable oscillation I=0.2
04.02.2012
AI HH
28
FitzHugh-Nagumo Orbits
04.02.2012
AI HH
29
How do the channels really work: Patch Clamp
04.02.2012
AI HH
30
The Patch Clamp Technique single channel measurements
Bert Sakmann, Erwin Neher Nobelpreice for medicin, 1991
04.02.2012
AI HH
31
04.02.2012
AI HH
32
Major Functional Areas

Primary motor: voluntary movement Primary somatosensory: tactile, pain, pressure, position, temp., mvt. Motor association: coordination of complex movements Sensory association: processing of multisensorial information Prefrontal: planning, emotion, judgement Speech center (Brocas area): speech production and articulation Wernickes area: comprehension of speech Auditory: hearing Auditory association: complex auditory processing Visual: low-level vision Visual association: higher-level vision
AI HH 33
04.02.2012
Interconnect
04.02.2012 Felleman & Van Essen, 1991
AI HH
34
Detailed Neural Modeling

A simulator, called Neuron has been developed at Yale to simulate the Hodgkin-Huxley equations, as well as other membranes/channels/etc. See http://www.neuron.yale.edu/
04.02.2012
AI HH
35
Basic neurobiology: Combine inhibitory and excitatory signals at different times to fire a neuron.
04.02.2012
AI HH
36
Computer science: Programing a simple early computer demonstrates basic computer science concepts.
04.02.2012
AI HH
37
Neural networks: Use a learning algorithm to understand how the brain stabilizes the eyes during head movement.
04.02.2012
AI HH
38
Active neural membranes: Explore the HodgkinHuxley equations, the classical explanation of neural membrane biophysics.
04.02.2012
AI HH
39
04.02.2012
AI HH
40
References
1. C.G. Boeree, The Neuron, www.ship.edu/~cgboeree/. 2. R. FitzHugh, Mathematical models of excitation and propagation in nerve, In: Biological Engineering, Ed: H.P. Schwan, McGraw-Hill, New York, 1969. 3. L. Edelstein-Kesket, Mathematical Models in Biology, Random House, New York, 1988. 4. A.L. Hodgkin, A.F. Huxley and B. Katz, J. Physiology 116, 424448,1952. 5. A.L. Hodgkin and A.F. Huxley, J. Physiol. 116, 449-566, 1952. 6. F.C. Hoppensteadt and C.S. Peskin, Modeling and Simulation in Medicine and the Life Sciences, 2nd ed, Springer-Verlag, New York, 2002. 7. J. Keener and J. Sneyd, Mathematical Physiology, SpringerVerlag, New York, 1998. 8. J. Rinzel, Bull. Math. Biology 52, 5-23, 1990. 9. E.K. Yeargers, R.W. Shonkwiler and J.V. Herod, An Introduction to the Mathematics of Biology: with Computer 41 04.02.2012 AI HH Algebra Models, Birkhauser, Boston, 1996.

AI All

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

AI All

Uploaded by

Copyright:

Available Formats

Artificial Intelligence: introduction

Lecturer: Lars Mehnen

Exam: written at end of semester (50%)

What is AI. Problem solving. Uncertain knowledge and reasoning. Planning.

What is Artificial Intelligence

What is Artificial Intelligence

Systems that think like humans

Systems that think rationally

Systems that act like humans

Systems that act rationally

Systems that act like humans

Systems that act like humans

Alan Turing, the enigma

Systems that think like humans

Systems that think rationally

This study initiated the field of logic.

Systems that act rationally

Rational behavior: doing the right thing

Can include thinking, yet in service of rational action.

Systems that act rationally

1950: Alan Turings Computing Machinery and Intelligence

A brief history (2)

Great expectations (1952-1969)

A brief history (3)

Collapse in AI research (1966 - 1973)

Some systems lacked scalability.

Fundamental limitations on techniques and representations.

A brief history (4)

Increase in knowledge representation research.

A brief history (5)

Puts an end to the AI winter. Connectionist revival (1986 - present)

A brief history (6)

The emergence of intelligent agents (1995 - present)

Artificial Intelligence: Intelligent Agents

The concept of rational behavior. Environments. Agent structure.

Agents and environments

Agents and environments

The vacuum-cleaner world

The vacuum-cleaner world

Action Right Suck Left Suck Right Suck

The vacuum-cleaner world

What is the right function? Can it be implemented in a small agent program?

The concept of rationality

What is the right thing?

Performance measure should be objective

Learn from percepts

Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent??

Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent??

Solitaire Observable?? Deterministic?? Episodic?? Static?? Discrete?? Single-agent?? FULL

Internet shopping PARTIAL

Backgammon FULL NO NO YES

Internet shopping PARTIAL YES NO SEMI

Backgammon FULL NO NO YES YES

Internet shopping PARTIAL YES NO SEMI YES

The simplest environment is

Most real situations are:

All agents have the same skeleton:

Note difference with agent function (page 3-4)

This approach is doomed to failure!

All these can be turned into learning agents.

Agent types; simple reflex

Select action on the basis of only the current percept.