4.

Design Space Exploration of

Embedded Systems

Lothar Thiele
ETH Zurich, Switzerland

Swiss Federal Computer Engineering

Institute of Technology 4-1 and Networks Laboratory
Contents of Lectures (Lothar Thiele)

1. Introduction to Embedded System Design

2. Software for Embedded Systems

3. Real-Time Scheduling

4. Design Space Exploration

5. Performance Analysis

Swiss Federal Computer Engineering

Institute of Technology 4-2 and Networks Laboratory
Topics
Multi-Objective Optimization
! Introduction
! Multi-Objective Optimization
! Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
! Implementation
Design Space Exploration
! System Design
! Problem Specification
! Example

Swiss Federal Computer Engineering

Institute of Technology 4-3 and Networks Laboratory
Evolutionary Multiobjective Optimization Algorithms

What are Evolutionary Algorithms?

randomized, problem-independent search heuristics

applicable to black-box optimization problems

How do they work?

by iteratively improving a population of solutions by

variation and selection
can find many different optimal solution in a single run

Swiss Federal Computer Engineering

Institute of Technology 4-4 and Networks Laboratory
 Black-Box Optimization
objective function
Stretch-Module Decision-Module Handling-Module
DX lane
100

DV

lane a require
v, a
Br
v Dr
v=f(t)

?
point of
gear
change Dr

decision lane
t
n
gear R
4
2

5
3
1
gear
objective
x, v, a

vector x Vehicle-Module gear

vector f(x)
s clutch

lane, x, v
a,gear, n

(e.g. simulation model)

Optimization Algorithm:
only allowed to evaluate f
(direct search)

Swiss Federal Computer Engineering

Institute of Technology 4-5 and Networks Laboratory
 The Knapsack Problem
weight = 750g weight = weight = 300g weight =
profit = 5 1500g profit = 7 1000g
profit = 8 profit = 3

?
Goal: choose subset that
maximizes overall profit
minimizes total weight

Swiss Federal Computer Engineering

Institute of Technology 4-6 and Networks Laboratory
 The Solution Space

profit

20

15

10

weight
500g 1000g 1500g 2000g 2500g 3000g 3500g

Swiss Federal Computer Engineering

Institute of Technology 4-7 and Networks Laboratory
 The Trade-off Front
Observations: " there is no single optimal solution, but
# some solutions ( ) are better than others ( )
profit

selecting a
20 solution

15
finding the good
10 solutions

weight
500g 1000g 1500g 2000g 2500g 3000g 3500g
Swiss Federal Computer Engineering
Institute of Technology 4-8 and Networks Laboratory
 Decision Making: Selecting a Solution
Approaches: profit more important than cost (ranking)
weight must not exceed 2400g (constraint)
profit

20

15
too heavy
10

weight
500g 1000g 1500g 2000g 2500g 3000g 3500g
Swiss Federal Computer Engineering
Institute of Technology 4-9 and Networks Laboratory
 Optimization Alternatives

Use of classical single objective optimization methods

simulated annealing, tabu search
integer linear program
other constructive or iterative heuristic methods
Decision making (weighting the different objectives) is
done before the optimization.

Population based optimization methods

evolutionary algorithms
genetic algorithms
Decision making is done after the optimization.

Swiss Federal Computer Engineering

Institute of Technology 4 - 10 and Networks Laboratory
 Optimization Alternatives
scalarization population-based
weighted sum SPEA2

y2 y2

y1 y1

parameter-oriented set-oriented
scaling-dependent scaling-independent
Swiss Federal Computer Engineering
Institute of Technology 4 - 11 and Networks Laboratory
 Scalarization Approach

parameters
multiple single
objectives objective
(y1, y2, , yk) transformation y

example: weighting approach y2 maximization problem

(w1, w2, , wk)

y = w1y1 + + wkyk

y1
Swiss Federal Computer Engineering
Institute of Technology 4 - 12 and Networks Laboratory
 A Generic Multiobjective EA
population archive

evaluate
update
sample
truncate
vary

new population new archive

Swiss Federal Computer Engineering
Institute of Technology 4 - 13 and Networks Laboratory
 An Evolutionary Algorithm in Action
max. y2
hypothetical trade-off front

min. y1
Swiss Federal Computer Engineering
Institute of Technology 4 - 14 and Networks Laboratory
 Design Space Exploration

Specification
Specification Optimization
Optimization Evaluation
Evaluation Implementation
Implementation

power
consumption latency

cost

Swiss Federal Computer Engineering

Institute of Technology 4 - 15 and Networks Laboratory
 Packet Processing in Networks

UCB Rabaey
Embedded
Embedded
Internet
Internet
Devices
Devices method (a)(fsd)
for I=1 to n
do nothing
call comm(a,dsf,*e);
Wearable
WearableComputing
Computing
end for

Access Core

Mobile
MobileInternet
Internet

Swiss Federal Computer Engineering

Institute of Technology 4 - 16 and Networks Laboratory
 Network Processors
Network processor = high-performance, programmable device
designed to efficiently execute communication
workloads [Crowley et al.: 2003]

incoming flows routing / forwarding outgoing flows

(packet streams) transcoding (processed packets)
encryption / decryption
real-time flows
e.g., voice network
processor
(NP)
e.g., sftp
non-real-time flows

Swiss Federal
Institute of Technology
? 4 - 17
Computer Engineering
and Networks Laboratory
 Optimization Scenario: Overview

Given: " specification of the task structure (task model) =

for each flow the corresponding tasks to be executed
# different usage scenarios (flow model) =
sets of flows with different characteristics
Sought: network processor implementation (resource model)
= architecture + task mapping + scheduling
Objectives: " maximize performance
# minimize cost
(performance model)
Subject to: " memory constraint
# delay constraints

Swiss Federal Computer Engineering

Institute of Technology 4 - 18 and Networks Laboratory
 Topics

Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal Computer Engineering
Institute of Technology 4 - 19 and Networks Laboratory
 Dominance, Pareto Points

A (design) point Jk is dominated by Ji, if Ji is

better or equal than Jk in all criteria and
better in at least one criterion.

A point is Pareto-optimal or a Pareto-point, if it is not

dominated.

The domination relation imposes a partial order on all

design points
We are faced with a set of optimal solutions.
Divergence of solutions vs. convergence.

Swiss Federal Computer Engineering

Institute of Technology 4 - 20 and Networks Laboratory
 Multi-objective Optimization

Swiss Federal Computer Engineering

Institute of Technology 4 - 21 and Networks Laboratory
 Multi-objective Optimization
Maximize (y1, y2, , yk) = (x1, x2, , xn)

y2 y2

Pareto optimal = not dominated

better

incomparable dominated

worse incomparable

y1 y1

Pareto set = set of all Pareto-optimal solutions

Swiss Federal Computer Engineering
Institute of Technology 4 - 22 and Networks Laboratory
 Randomized (Black Box) Search Algorithms

Idea: find good solutions without investigating all solutions

Assumptions:better solutions can be found in the neighborhood
of good solutions
information available only by function evaluations

Randomized
search algorithm

t = 1: t t+1:
(randomly) choose a (randomly) choose a
solution x1 to start with solution xt+1 using solutions
x1, , xt

Swiss Federal Computer Engineering

Institute of Technology 4 - 23 and Networks Laboratory
 Types of Randomized Search Algorithms
memory selection variation

mating
selection

environmental
selection

EA 1 both
evolutionary algorithm

TS 1 no mating selection
tabu search

SA 1 no mating selection
simulated annealing

Swiss Federal Computer Engineering

Institute of Technology 4 - 24 and Networks Laboratory
 Limitations of Randomized Search Algorithms

The No-Free-Lunch Theorem

All search algorithms provide in average the same

performance on a all possible functions
with finite search and objective spaces.

[Wolpert, McReady: 1997]

Remarks:
Not all functions equally likely and realistic
We cannot expect to design the algorithm beating all others
Ongoing research: which algorithm suited for which class of
problem?
Swiss Federal Computer Engineering
Institute of Technology 4 - 25 and Networks Laboratory
 Topics

Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal Computer Engineering
Institute of Technology 4 - 26 and Networks Laboratory
 Design Choices

representation fitness assignment mating selection

0011 0111

parameters
0011 0100
0000
1011 0011

environmental selection variation operators

Swiss Federal Computer Engineering

Institute of Technology 4 - 27 and Networks Laboratory
 Issues in Multi-Objective Optimization
How to maintain a diverse
y2
Diversity
Pareto set approximation?
# density estimation

How to prevent nondominated

solutions from being lost?
$ environmental selection

y1
How to guide the population
towards the Pareto set?
Convergence
" fitness assignment
Swiss Federal Computer Engineering
Institute of Technology 4 - 28 and Networks Laboratory
 Comparison of Three Implementations
2-objective knapsack problem
SPEA2

VEGA

extended
VEGA

Trade-off between
distance and diversity?

Swiss Federal Computer Engineering

Institute of Technology 4 - 29 and Networks Laboratory
 Topics

Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal Computer Engineering
Institute of Technology 4 - 30 and Networks Laboratory
 Representation
search space decoder solution space objectives objective space

1 0 1 1 1 0

1 0 1 1 1 0
m
1 0 1 1 1 0

solutions encoded by vectors, matrices, trees, lists, ...

fixed length variable length
Issues:
completeness (each solution has an encoding)
uniformity (all solutions are represented equally)
redundancy (cardinality of search space vs. solution space)
feasibility (each encoding maps to a feasible solution)
Swiss Federal Computer Engineering
Institute of Technology 4 - 31 and Networks Laboratory
 Example: Binary Vector Encoding
Given: graph
Goal: find minimum subset of nodes such that each edge
is connected to at least one node of this subset
(minimum vertex cover)
A B C D E

nodes A B C D E F
selected? 1 0 1 1 1 0

Swiss Federal Computer Engineering

Institute of Technology 4 - 32 and Networks Laboratory
 Example: Integer Vector Encoding
Given: graph, k colors
Goal: assign each node one of the k colors such that the
number of connected nodes with the same color is
minimized (graph coloring problem)
A B C D E

nodes A B C D E F
colors 1 2 1 3 1 2

Swiss Federal Computer Engineering

Institute of Technology 4 - 33 and Networks Laboratory
 Example: Real Vector Encoding

parameters x1 x2 x3 x4 xn
values 0.33 0.53 1.03 3.25 9.83
[Michalewicz, Fogel: How to Solve it. Springer 2000]
Swiss Federal Computer Engineering
Institute of Technology 4 - 34 and Networks Laboratory
 Tree Example: Parking a Truck

steering u
angle

cab
d
dock
trailer

t
position (x,y)

Goal:
find function c with
constant speed u = c(x, y, d, t)
Swiss Federal Computer Engineering
Institute of Technology 4 - 35 and Networks Laboratory
 Search Space for the Truck Problem

Operators:

Arguments: X position x
Y position y
DIFF cab angle d
TANG trailer angle t
Search space : set of symbolic expression using the above
operators and arguments
Swiss Federal Computer Engineering
Institute of Technology 4 - 36 and Networks Laboratory
 Example Solution: Tree Representation

MULT

MINUS PLUS

X DIFF Y TANG

encodes the function (symbolic expression): u = (x d) * (y + t)

Swiss Federal Computer Engineering
Institute of Technology 4 - 37 and Networks Laboratory
 A Solution Found by an EA
truck simulation encoded tree

Swiss Federal Computer Engineering

Institute of Technology 4 - 38 and Networks Laboratory
 Topics

Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal Computer Engineering
Institute of Technology 4 - 39 and Networks Laboratory
 Fitness Assignment

Fitness = scalar value representing quality of an individual (usually).

Used for mating and environmental selection

Fi = f ( m (i) )

Difficulties:
multiple objectives have to be considered (Pareto set is sought)
multiple optima need to be approximated (how to consider
diversity?)
constraints are involved which have to be met

Swiss Federal Computer Engineering

Institute of Technology 4 - 40 and Networks Laboratory
 Example Pareto Ranking

Fitness function:

execution time

F (1) = 3
F ( 2) = 1
2 6
F ( 3) = 1
1 3
F ( 4) = 2
4 F (5) = 1
5 F (6) = 0
cost
Swiss Federal Computer Engineering
Institute of Technology 4 - 41 and Networks Laboratory
 Constraint Handling
Constraint = g(x1, x2, , xn) 0 feasible g0

infeasible
g<0
Approaches:
construct initialization and variation such that infeasible
solutions are not generated (resp. not inserted)
representation is such that decoding always yields a feasible
solution
calculate constraint violation (- g(x1, x2, , xn) ) and incorporate it
into fitness, e.g., Fi = f ( m (i) ) - g(x1, x2, , xn) (fitness to be
maximized)
code constraint as a new objective
Swiss Federal Computer Engineering
Institute of Technology 4 - 42 and Networks Laboratory
 Selection

Two conflicting goals:

trade-off
exploitation exploration
(converge fast) (avoid getting stuck)

Two types of selection:

mating selection = select for variation

environmental selection = select for survival

Swiss Federal Computer Engineering

Institute of Technology 4 - 43 and Networks Laboratory
 Example: Tournament Selection

= integrated sampling rate assignment and sampling

population mating pool

" #
uniformly choose compare fitness
T individuals at and copy best
random independently individual
of fitness in mating pool

T = tournament size (binary tournament selection means T=2)

Swiss Federal Computer Engineering
Institute of Technology 4 - 44 and Networks Laboratory
 Topics

Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal Computer Engineering
Institute of Technology 4 - 45 and Networks Laboratory
 Example: Vector Mutation

Bit vectors: 1 0 1 1 1 0
each bit is flipped with probability 1/6

1 0 0 1 1 0

Permutations: 1 2 3 4 5 6 1 4 3 2 5 6

swap
1 2 3 4 5 6 1 3 4 2 5 6

rearrange
Swiss Federal Computer Engineering
Institute of Technology 4 - 46 and Networks Laboratory
 Example: Vector Recombination
Bit vectors:
1 1 0 0 1 0
1 1 0 0 0 1
1 0 1 0 0 1

Permutations:

1 2 3 4 5 6 child

parents 1 2 3 6 4 5
6 2 3 4 1 5

Swiss Federal Computer Engineering

Institute of Technology 4 - 47 and Networks Laboratory
 Example: Recombination of Trees
MULT PLUS

MINUS PLUS TANG MINUS

MINUS DIFF Y TANG

PLUS DIFF

X Y
MULT DIFF

exchange
Y TANG

Swiss Federal Computer Engineering

Institute of Technology 4 - 48 and Networks Laboratory
 Example: SPEA2 Algorithm
Step 1: Generate initial population P0 and empty archive
(external set) A0. Set t = 0.
Step 2: Calculate fitness values of individuals in Pt and At.
Step 3: At+1 = nondominated individuals in Pt and At.
If size of At+1 > N then reduce At+1, else if
size of At+1 < N then fill At+1 with dominated
individuals in Pt and At.
Step 4: If t > T then output the nondominated set of At+1.
Stop.
Step 5: Fill mating pool by binary tournament selection.
Step 6: Apply recombination and mutation operators to
the mating pool and set Pt+1 to the resulting
population. Set t = t + 1 and go to Step 2.

Swiss Federal Computer Engineering

Institute of Technology 4 - 49 and Networks Laboratory
 SPEA2 Fitness Assignment

Idea (Step 2): calculate dominance rank weighted by

dominance count
y2

non-dominated solutions:
2
F = #dominated solutions
4+2
dominated solutions
2
3 F = # of non-Pareto solutions +
4+2+3 strengths of dominators
4+3
4+2+2+3

y1

Note: lower raw fitness = better quality

Swiss Federal Computer Engineering
Institute of Technology 4 - 50 and Networks Laboratory
 Topics

Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal Computer Engineering
Institute of Technology 4 - 51 and Networks Laboratory
 What Is Needed
A framework that
Provides ready-to-use modules (algorithms / applications)
Is simple to use
Is independent of programming language and OS
Comes with minimum overhead

Idea: separate problem-dependent from problem-

independent part cut

Representation
Selection
Recombination Objective functions
Fitness assignment
Archiving Mutation

Swiss Federal Computer Engineering

Institute of Technology 4 - 52 and Networks Laboratory
 PISA: Implementation
shared
shared
file
file
system
system

selector text variator

process files process

application independent: handshake protocol: application dependent:

mating / environmental state / action variation operators
selection individual IDs stores and manages
individuals are objective vectors individuals
described parameters
by IDs and objective
vectors
Swiss Federal Computer Engineering
Institute of Technology 4 - 53 and Networks Laboratory
 PISA Installation

Swiss Federal Computer Engineering

Institute of Technology 4 - 54 and Networks Laboratory
 Topics

Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal Computer Engineering
Institute of Technology 4 - 55 and Networks Laboratory
 Design Space Exploration

Application Architecture

Mapping

Estimation

Swiss Federal Computer Engineering

Institute of Technology 4 - 56 and Networks Laboratory
 Embedded System Design

architecture task binding

template graph restrictions
evaluation
construct map estimate
allocation performance / architecture application performance
bindings cost vector

multiobjective
optimization

architecture performance

Swiss Federal Computer Engineering

Institute of Technology 4 - 57 and Networks Laboratory
 Example: System Synthesis
Given:
algorithm mappings architectures

f 1, f 2, K

Goal:
schedule mapping architecture
1 3

5
2 4

Objectives: cost, latency, power consumption

Swiss Federal Computer Engineering

Institute of Technology 4 - 58 and Networks Laboratory
 Evolutionary Algorithms for DSE
individual

EA allocation decode allocation

" selection binding

# recombination decode binding
$ mutation
scheduling

design point
chromosome = encoded
allocation + binding (implementation)

fitness evaluation
fitness

user constraints

Swiss Federal Computer Engineering

Institute of Technology 4 - 59 and Networks Laboratory
 Topics

Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal Computer Engineering
Institute of Technology 4 - 60 and Networks Laboratory
 Basic Model Problem Graph

Problem graph GP(VP,EP):

1 2
Interpretation:
VP consists of functional
5 6 nodes VPf (task, proce-
dure) and communication
3
nodes VPc .
7 EP represent data depend-
encies
4

Swiss Federal Computer Engineering

Institute of Technology 4 - 61 and Networks Laboratory
 Basic model architecture graph

Architecture graph GA(VA,EA):

RISC HWM1
RISC HWM1

PTP bus shared bus

shared PTP bus
bus
HWM2 HWM2
Architecture Architecture graph
VA consists of functional resources VAf (RISC, ASIC) and
bus resources VAc. These components are potentially allo-
catable.
EA model directed communication.

Swiss Federal Computer Engineering

Institute of Technology 4 - 62 and Networks Laboratory
 Basic model specification graph

1
Definition: A specifica-
tion graph is a graph 5 RISC
GS=(VS,ES) consisting
of a problem graph GP, 3 SB

an architecture graph 7 HWM1

GA, and edges EM. In
particular, VS=VPVA, 2 PTP

ES=EPEAEM 6 HWM2

4
GP EM GA

Swiss Federal Computer Engineering

Institute of Technology 4 - 63 and Networks Laboratory
 Basic model - synthesis

Three main tasks of synthesis:

Allocation is a subset of VA.
Binding is a subset of EM, i.e., a mapping of functional
nodes of VP onto resource nodes of VA.
Schedule is a function that assigns a number (start time) to
each functional node.

Swiss Federal Computer Engineering

Institute of Technology 4 - 64 and Networks Laboratory
 Basic model - implementation

1
0 1
0
Definition: Given a 1 5 RISC
specification graph GS 8
an implementation is a 21 3 SB
triple (,,), where 29 7
1
HWM1
is a feasible allocation, 20

is a feasible binding, 1 2
and is a schedule. 1
21 6 RISC HWM1
2 PTP bus
30 4 shared
bus
HWM2

Swiss Federal Computer Engineering

Institute of Technology 4 - 65 and Networks Laboratory
 Example

Swiss Federal Computer Engineering

Institute of Technology 4 - 66 and Networks Laboratory
 Challenges

Encoding of (allocation+binding)
simple encoding
eg. one bit per resource, one variable per binding
easy to implement
many infeasible partitionings
encoding + repair
eg. simple encoding and modify such that for each vp VP there
exists at least one va VA with a (vp) = va
reduces number of infeasible partitionings
Generation of the initial population, mutation
Recombination

Swiss Federal Computer Engineering

Institute of Technology 4 - 67 and Networks Laboratory
 Topics

Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal Computer Engineering
Institute of Technology 4 - 68 and Networks Laboratory
 Exploration - Case Study (1)

Swiss Federal Computer Engineering

Institute of Technology 4 - 69 and Networks Laboratory
 Exploration - Case Study (2)

Swiss Federal Computer Engineering

Institute of Technology 4 - 70 and Networks Laboratory
 Exploration - Case Study (3)
frame memory dual ported frame memory block matching module
input module

output module
subtract/add module
DCT/IDCT module Huffman encoder

Swiss Federal Computer Engineering

Institute of Technology 4 - 71 and Networks Laboratory
 EA Case Study - Design Space

350
300
250
power consumption 200
150
1000
200
50
300
100
latency 400 cost
150
500
200

Swiss Federal Computer Engineering

Institute of Technology 4 - 72 and Networks Laboratory
 Exploration Case Study - Solution 1

INM OUTM FM RISC2

SBS

Swiss Federal Computer Engineering

Institute of Technology 4 - 73 and Networks Laboratory
 Exploration Case Study - Solution 2

INM OUTM DPFM HC DCTM BMM SAM

SBF

Swiss Federal Computer Engineering

Institute of Technology 4 - 74 and Networks Laboratory