Sensor fusion

What is it?
Web Economy Bullshit Generator
http://www.dack.com/web/bullshit.html
•synthesize holistic deliverables
•architect robust interfaces
•facilitate web-enabled interfaces

Combining data from different

sensor, in order to get a optimal
result.

1
 Sensors
• Radar
• Sonar (under water) and Ultrasonic (above water)
• Infra-red / thermal imaging camera
• Cameras
• Tilt sensors
• Gyro
• GNSS(Global NavigationSatelliteSystem)
• Laserscanners
• Odometri encoders
• Kinect(IR)

Bjoern Griesbach griesbac@in.tum.de

2
 Sensors
Can we use the result or data from one sensor alone?

That depends on the sensor.

☺
NO!
And the application it is used in.

3
 Sensor fusion, why?
• Each sensor has its strengths and weaknesses
• One sensor is never sufficient for reliable
tracking/measuring
• Optimal tracking/measuring = use multiple
sensors
Precise Estimation

Data Fusion Component

Noisy Noisy
Data Noisy
Data
Data

Sensor Sensor Sensor

Bjoern Griesbach griesbac@in.tum.de

4
 Sensor fusion/confusion
Said the eye one day, "I see beyond these valleys a mountain veiled with
blue mist. Is it not beautiful?“

The ear listened, and after listening intently awhile, said, "But where is
any
mountain? I do not hear "

Then the hand spoke and said, "I am trying in vain to feel it or touch it,
and I
can find no mountain.“

And the nose said, "There is no mountain, I cannot smell it.“

Then the eye turned the other way and they all began to talk together
way, about the eye's strange delusion. And they said, "Something
must be
the matter with the eye.“

K.Gibran: „The Madman“ (1946)

5
Mathmatical sensorfusion methods
Simple:
• Use normal math to correct a signal
• Use simple algorithms to select the most
accurate signal.

Complex:
• Kalman filter
• Dempster–Shafer
• Bayesian network

6
Simple Sensor fusion

Line drawed without sensor

fusion

7
Simple sensor fusion

8
Simple Sensor fusion

Line drawed with sensor

fusion

9
 Simple Sensor fusion
Gyrodometri

10
 Complex sensor fusion
Wanted: Fusion of odemetri and
GNSS data to detemine the most
accurate possition.

 Estimate pose of position

How to realize?

Kalmanfilter

Bjoern Griesbach griesbac@in.tum.de

11
 Complex Sensor fusion -
Kalmanfilter
Optimal data processing algorithm
• Filter out noise of measurement data
• Can fuse different measurements to find the optimal
result
• Optimal: incorporates all information (i.e. measurement
data) that can be provided to it
• Does not need to keep all previous measurement data in
storage!

Bjoern Griesbach griesbac@in.tum.de

12
 Complex Sensor fusion -
Kalmanfilter
• Pose of an object is represented as a vector
=(x,y,θ);
• By transforming poses of two different sensors
into the same coordinate system, one will get
two (noisy) measurements and for the same
object
• Each measurement is weighted differently 
depends on the variance of the measurement
• Each measurement has its own variance σi2
represented by a matrix Pi

Bjoern Griesbach griesbac@in.tum.de

13
 Complex Sensor fusion -
Kalmanfilter
• Given: pose measurements: z1 z2 with
covariance matrices P1 P2 from two different
sensors
• Wanted: optimal weights to get optimal estimate
• Solution: Optimal estimate x with minimal
combined covariance matrix P:

r P2 r P1 r
x= z1 + z2 ;
P1 + P2 P1 + P2
P2
P= P1 ;
P1 + P2

Bjoern Griesbach griesbac@in.tum.de

14
 Basic Kalman Filter
r r r r
• Process Model: x (t k ) = A ⋅ x (t k −1 ) + Bu (t k ) + w(t k −1 ) w(t ) → N (0, Q (t ))

r r r
• Measurement Model: z (t k ) = H ⋅ x (t k ) + v (t k ) v(t ) → N (0, R(t ))

• Algorithm: (vector arrows omitted!)

xˆ − (t k ) = Ax(t k −1 ) + Bu (t k ) [
xˆ(tk ) = xˆ− (tk ) + K ⋅ z(tk ) − H ⋅ xˆ− (tk ) ]
P − (t k ) = AP(t k −1 ) AT + Q (t k ) P(tk ) = [I − KH]P− (tk )

1. Predict 2. Correct

P − (t k ) H T
Kalman Gain K= −
HP (t k ) H T + R (t k )

Bjoern Griesbach griesbac@in.tum.de

15
Combine simple and complex
sensor fusion

16
 Sensor Fusion

What can it be used for?

Your imagination is the limit!

17