Visual Perception and Digital Image Representation: Klara Nahrstedt Spring 2014

CS 414 Multimedia Systems Design
Lecture 4 Visual Perception and Digital Image Representation

Klara Nahrstedt Spring 2014
CS 414 - Spring 2014
Administrative
Groups are formed and names have been sent to Engineering IT and Barb Leisner We will inform you about group directories as soon as we have information from Engineering IT
Administrative
Leasing Process from Barb Leisner
Lease one Logitech camera - two cameras within one group to start MP1, and then for MP2/MP3. Leasing process starts on January 31 Pick up the camera from Barb Leisner office, 2312 SC Bring your student ID to sign for the camera Each cs414 group is responsible for his/her own camera
if you loose it (or badly damage) and you dont have police report, you pay for it (charged to your student account at the end of the semester)
Hours to pick up camera: Monday Friday 9am-5pm No camera pickup on Saturday and Sunday
Today Introduced Concepts
Important Metric for Digital Audio

Signal-to-Noise
Ratio (dB)
Human Visual System Digital Images

Sampling Quantization Spatial
Resolution
Signal-to-Noise Ratio
(metric to quantify quality of digital audio)
Signal To Noise (SNR) Ratio
Measures strength of signal to noise

Signal energy SNR (in DB)= 10 log 10( Noise energy )
Given sound form with amplitude in [-A, A]

Signal energy =
A 2
0.75 0.5 0.25
-0.25 -0.5 -0.75
-A
-1
Modeling of Noise Quantization Error
Difference between actual and sampled value

amplitude
between [-A, A] quantization levels = N
2A N
1 0.75 0.5
e.g., if A = 1, N = 8, = 1/4
0.25 0 -0.25 -0.5
Compute Signal to Noise Ratio
2 2A A ; Noise energy = Signal energy = ; 12 N 22
A Noise energy = 3 N 2
Signal-to-Noise =
3N 2 10 log( SignalEnergy / NoiseEnergy ) 10 log 2
SNR depends on number of bits (number of quantization levels) assigned to signal Every bit increases SNR by ~ 6 decibels
Integrating Aspects of Multimedia

Image/Video Capture
Audio/Video Perception/ Playback Audio/Video Presentation Playback
Image/Video Information Representation
Transmission
Audio Capture Compression Processing
Transmission
Audio Information Representation
Media Server Storage
A/V Playback
Human Visual System
Eyes, optic nerve, parts of the brain Transforms electromagnetic energy
Human Visual System
Image Formation
cornea,
sclera, pupil, iris, lens, retina, fovea
Transduction
retina,
rods, and cones Retina has photosensitive receptors at back of eye
Processing
optic
nerve, brain
Rods vs Cones (Responsible for us seeing brightness and color)

Cones

Rods

Contain photo-pigment Respond to high energy Enhance perception Concentrated in fovea, exist sparsely in retina Three types, sensitive to different wavelengths
Contain photo-pigment Respond to low energy Enhance sensitivity Concentrated in retina, but outside of fovea One type, sensitive to grayscale changes
Tri-stimulus Theory
3 types of cones (6/7 Mil. of them)

Red = L cones, Green = M cones, Blue = S cones Ratio differentiates for each person E.g., Red (64%), Green (32%), rest S cones E.g., L(50.6%), M(44.2%), rest S cones
Each type most responsive to a narrow band electro-magnetic waves
red and green absorb most energy, blue the least
Light stimulates each set of cones differently, and the ratios produce CS 414 - Spring 2014 sensation of color
Color and Visual System
Color refers to how we perceive a narrow band of electromagnetic energy source, object, observer
Visual system transforms light energy into sensory experience of sight
Color Perception (Color Theory)
Hue
Refers
Hue Scale
to pure colors dominant wavelength of the light

Original
Saturation
Perceived
intensity of a specific color how far color is from a gray of equal intensity
Saturation
Brightness (lightness)
perceived
lightness
intensity
Source: Wikipedia
Digitalization of Images Capturing and Processing
Capturing Real-World Images
Picture two dimensional image captured from a real-world scene that represents a momentary event from the 3D spatial W2 world W1
r F s
W3 r= function of (W1/W3); s=function of (W2/W3)
Image Concepts - Sampling

An image is a function of intensity values over a 2D plane I(r,s) Sample function at discrete intervals to represent an image in digital form
matrix
of intensity values for each color plane intensity typically represented with 8 bits
Sample points are called pixels

Digital Image Sampling
Sample = pixel
Image Size (in pixels) Image Size = Height x Width (in pixels)
320x240 pixels 640x480 pixels 1920x1080pixels
Digital Images - Quantization

Quantization = number of bits per pixel Example: if we would sample and quantize standard TV picture (525 lines) by using VGA (Video Graphics Array),
video
controller creates matrix 640x480pixels,
and each pixel is represented by 8 bit integer (256 discrete gray levels)
Image Representations
Black and white image

single
color plane with 2
bits
Grey scale image

single
color plane with 8
bits
Color image
three
color planes each with 8 bits RGB, CMY, YIQ, etc.
Indexed color image

single
plane that indexes a color table
4 gray levels
2gray levels
Compressed images
TIFF,
JPEG, BMP, etc.
Digital Image Representation (3 Bit Quantization)
Color Quantization Example of 24 bit RGB Image
24-bit Color Monitor

Image Representation Example

24 bit RGB Representation (uncompressed)
128 129 229
135 255 213
166 105 134
138 189 111
190 167 138
132 190 187
128 129 229
138 189 111
135 255 213
190 167 138
166 105 134
132 190 187
Color Planes
Graphical Representation
Image Properties (Color)
Color Histogram
Spatial and Frequency Domains
Spatial domain
refers
to planar region of intensity values at time t
Frequency domain
think
of each color plane as a sinusoidal function of changing intensity values refers to organizing pixels according to their changing intensity (frequency)
Spatial Resolution and Brightness
Spatial Resolution (depends on: )

Image size Viewing distance Perception of brightness is higher than perception of color Different perception of primary colors Relative brightness: green:red:blue=
59%:30%:11%
Brightness
Source: wikipedia
Image Size (in Bits)
Image Size = Height x Width X Bits/pixel

Example:
Consider
image 320x240 pixels with 8 bits per
pixel Image takes storage 7680 x 8 bits = 61440 bits or 7680 bytes
Summary
Important Image Processing Functions (see Computer Vision/Image Processing classes)

Filtering Edge
detection Image segmentation Image recognition

Formatting Conditioning Marking Grouping Extraction Matching

Image synthesis

Visual Perception and Digital Image Representation: Klara Nahrstedt Spring 2014

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Visual Perception and Digital Image Representation: Klara Nahrstedt Spring 2014

Uploaded by

Copyright:

Available Formats

CS 414 Multimedia Systems Design

Lecture 4 Visual Perception and Digital Image Representation

CS 414 - Spring 2014

Leasing Process from Barb Leisner

CS 414 - Spring 2014

Today Introduced Concepts

Important Metric for Digital Audio

Human Visual System Digital Images

CS 414 - Spring 2014

Signal To Noise (SNR) Ratio

Measures strength of signal to noise

Given sound form with amplitude in [-A, A]

0.75 0.5 0.25

-0.25 -0.5 -0.75

CS 414 - Spring 2014

Modeling of Noise Quantization Error

Difference between actual and sampled value

between [-A, A] quantization levels = N

0.25 0 -0.25 -0.5

CS 414 - Spring 2014

Compute Signal to Noise Ratio

2 2A A ; Noise energy = Signal energy = ; 12 N 22

3N 2 10 log( SignalEnergy / NoiseEnergy ) 10 log 2

Integrating Aspects of Multimedia

Image/Video Information Representation

Audio Information Representation

Media Server Storage

CS 414 - Spring 2014

Human Visual System

Eyes, optic nerve, parts of the brain Transforms electromagnetic energy

Human Visual System

sclera, pupil, iris, lens, retina, fovea

rods, and cones Retina has photosensitive receptors at back of eye

Rods vs Cones (Responsible for us seeing brightness and color)

CS 414 - Spring 2014

3 types of cones (6/7 Mil. of them)

Each type most responsive to a narrow band electro-magnetic waves

red and green absorb most energy, blue the least

Color and Visual System

Visual system transforms light energy into sensory experience of sight

Color Perception (Color Theory)

to pure colors dominant wavelength of the light

Digitalization of Images Capturing and Processing

CS 414 - Spring 2014

Capturing Real-World Images

W3 r= function of (W1/W3); s=function of (W2/W3)

Image Concepts - Sampling

Sample points are called pixels

Digital Image Sampling

CS 414 - Spring 2014

Digital Images - Quantization

controller creates matrix 640x480pixels,

Black and white image

color plane with 2

Grey scale image

color plane with 8

color planes each with 8 bits RGB, CMY, YIQ, etc.

Indexed color image

plane that indexes a color table

JPEG, BMP, etc.

Digital Image Representation (3 Bit Quantization)

CS 414 - Spring 2014