AUDIO, VIDEO & ANIMATION IN MULTIMEDIA

DIGITAL AUDIO IN MULTIMEDIA

Objective


What is sound? Waveforms and attributes of sound Capturing digital audio Sampling MIDI (Musical Instrument Digital Interface)

Sound
Sound comprises the spoken word, voices, music and even noise. It is a complex relationship involving a vibrating object (sound source), a transmission medium (usually air), a receiver (ear) and a perceptor (brain).
Example banging drum.

As the sound vibrates it bumps into molecules of the surrounding medium causing pressure waves to travel away from the source in all directions

So, Sound are rapid vibrations that are transmitted as variations in air pressure.
4

Waveforms
Sound waves are manifest as waveforms A waveform that repeats itself at regular intervals is called a periodic waveform Waveforms that do not exhibit regularity are called noise The unit of regularity is called a cycle This is known as Hertz (or Hz) after Heinrich Hertz One cycle = 1 Hz Sometimes written as kHz or kiloHertz (1 kHz = 1000 Hz)
5

Waveforms
Time for one cycle distance along wave Cycle

The Characteristics of Sound Waves

Sound is described in terms of two characteristics: Frequency Amplitude (or loudness) Frequency The rate at which sound is measured Number of cycles per second or Hertz (Hz) Determines the pitch of the sound as heard by our ears The higher frequency, the clearer and sharper the sound the higher pitch of sound
7

The Characteristics of Sound Waves

Amplitude Sounds intensity or loudness The louder the sound, the larger amplitude. In addition, all sounds have a duration and successive musical sounds is called rhythm

The Characteristics of Sound Waves

Amplitude pitch Cycle Time for one cycle distance along wave

Example Waveforms

Piano

Snare drum

Pan flute
10

Capture & Playback of Digital Audio

Air pressure variations

Captured via microphone Signal is converted into binary (discrete form) 0101001101 0110101111

Converts back into voltage

Digital to Analogue Converter

DAC

Analogue to Digital Converter

ADC

Air pressure variations 11

Sampling
Two parameters:
1. Sampling Rate
Frequency of sampling Measure in Hertz The higher sampling rate, higher quality sound but size storage is big. Standard Sampling rate:

- 44.1 KHz for CD Audio

- 22.05 KHz - 11.025 KHz for spoken - 5.1025 KHz for audio effect

12

Sampling
2. Size sample The resolution of a sample is the number of bits it uses to store a given amplitude value, e.g.
8 bits (256 different values) 16 bits (65536 different values) A higher resolution will give higher quality but will require more memory (or disk storage)

13

Calculating the Size of Digital Audio

The formula is as follows:

rate v duration v resolution v number of channels 8

The answer will be in bytes Where: sampling rate is in Hz Duration/time is in seconds resolution is in bits (1 for 8 bits, 2 for 16 bits) number of channels = 1 for mono, 2 for stereo, etc.
14

Calculating the Size of Digital Audio

Example: Calculate the file size for 1 minute, 44.1 KHz, 16 bits, stereo sound

rate v duration v resolution v number of channels 8

Where: sampling rate is 44,100 Hz Duration/time is 60 seconds resolution is 16 bits number of channels for stereo is 2
15

Calculating the Size of Digital Audio

rate v duration v resolution v number of channels 8
44100 * 60 * 16 *2 8 10584000 bytes

16

Digital Audio Editing Software

One of the most powerful and professional PC-based packages is a tool called Sound Forge

17

Editing Digital Audio

Trimming Splicing and assembly Volume adjustments Format conversion Resampling or downsampling Fade ins and fade outs Equalization Time Stretching Digital Signal processing Reversing Sounds
18

Editing Digital Audio

Simple audio editing software allows:
Recording of digital audio segments Trimming Splicing and assembly Volume adjustments of the entire segment Reversing Sounds Copy, cut, paste and delete segments of digital audio

Others audio editing software:

COOL Edit Pro Gold Wave PROSONIQ SonicWORX Samplitude Studio
19

Audio Formats
Depend on O/S. For examples: AIFF (Audio Interchange File Format) SOU For Macintosh .WAV Waveform file format. For Windows/Microsoft .VOC Sound Blaster Card
20

MIDI (Musical Instrument Digital Interface)

MIDI is a standard for specifying a musical performance Rather than send raw digital audio, it sends instructions to musical instruments telling them what note to play, at what volume, using what sound, etc.(sound sequencer) The synthesiser that receives the MIDI events is responsible for generating the actual sounds.
Example: Keyboard Piano

21

MIDI Sequencers


A MIDI sequencer allows musicians to edit and create musical compositions like a word processor Cut and paste Insert / delete

22

MIDI Versus Wav

Quality recording, MIDI depend to the tools Audio .wav easier to create compare than MIDI MIDI Advantages
File Size small Size Storage also small

MIDI Disadvantages
Playback Cost and Skill

23

How Audio can be used effectively

Examples of uses of audio: Cautions and warnings It is a good medium for alerting users to critical information. Some uses include: Sounding an alarm when a limit is reached Alerting users when data is entered incorrectly

24

How Audio can be used effectively

Music and Sound Effects These make multimedia interaction more real. Some uses include:
Musical background for a video segment Birds Songs accompanying photographs in biological field training.

Sound-related data. Some uses include: Helping mechanics diagnose engine trouble Training medical students to recognize different breathing sounds
25

How Audio can be used effectively

Direct voice communication. Some uses include: Leaving a voice message for other users of an application Consulting with an expert during a troubleshooting procedure.

26

Advantages & Disadvantages of using Audio

Sound adds life to any multimedia application and plays important role in effective marketing presentations. Advantages Ensure important information is noticed Add interest Can communicate more directly than other media
27

Advantages & Disadvantages of using Audio

Disadvantages Easily overused Requires special equipment for quality production Not as memorable as visual media

--28

Lecture 3: Video DIGITAL VIDEO IN MULTIMEDIA

29

Objective
Analogue video What is digital video? Calculating the size of digital video Compression techniques Digital video formats Video capture hardware Digital video editing Consumer desktop video

30

Analog video
Video information that is stored using television video signals, film, videotape or other non-computer media. Each frame is represented by a fluctuating voltage signal known as an analogue wave form or composite video. Composite analogue video has all the video components: brightness, colour and synchronization Then, combined into one signal for delivery
31

Analog video
Usage : television Problems: colour blending, low clarity, high generation lost, difficult to edit

32

Type of Analogue Video Signal

There are two main analogue video formats PAL and NTSC PAL (Phase Alternate Line) standard television format used in the European, UK, Australia and South Africa Consist 625 scan lines drawn every 1/25 second

33

Type of Analogue Video Signal

NTSC, National Television System Committee, is the American and Japanese standard Consist of 525 scan lines drawn every 1/30 second Television usually has a 4:3 aspect ratio For every 1 pixel down there are 1.333 pixels across Aspect ratio is the comparison of width to height for a viewing area Digital TV has an aspect ratio of 16:9 (wide screen)

34

What is digital video?

Digital video is the digitisation of analogue video signals into numerical format It creates the illusion of full motion by displaying a rapid sequence of changing images on a display device.

35

File Size Considerations

Several elements determine the file size, in additional to the length of the video. These including: Frame rate Number of images displayed within a specified amount of time to convey a sense of motion Usually measured in frames per second Standard video movie Video digital 30 fps, movie film 24 fps

at least 15 fps
36

File Size Considerations

Frame or Image size width and height of each individual frame or image Determines the quality of the image displayed and the processing involved to display that image Standard full screen resolution is 640x480 pixels Video screen display is one fourth the size of the full window screen (320x240)

37

File Size Considerations

Color depth/Resolution Number of colors displayed on the screen at one time Ranging from 1 bit to 8 bit, 16 or 24 bit per pixel Quality directly related to frame size, image size and color depth. Qualitiy also depends on content. Motion picture needs higher frame rate

38

Calculating the Size of Digital Video

File size = frame size * frame rate * Color Depth * time Where: Frame size = image size ( width * height in pixels) Frame rate = frames per second Color depth = measured in bytes time = time in seconds

This does not include any sound data !

39

Compression Techniques
Since the size of raw digital video is so prohibitively large we need some means to compress the information Video compression and decompression program, known as Codecs Lossy compression techniques cause some information to be lost from the original image Redundant information
Example: Image and video

JPEG and MPEG Intraframe and Interframe

40

Compression Techniques
Lossless compression techniques do not lose information throughout the compression and decompression process Example use in text images Exactly same before and after compression Technique is to identify repeating words and assign them a code. Decompression, the code would be changed back to the actual word.

41

Digital Video Formats

MPEG / MJPEG Files with a .mpg extension Apple QuickTime Files with a .mov or .qt extension Microsoft AVI Files with a .avi extension

42

MPEG Video
Named after the Moving Picture Experts Group who devised the compression and file formats There are a number of MPEG formats: MPEG-1 is a format used for low quality video (generally displayed on computers) MPEG-1 Layer 3 is the popular encoding mechanism for MP3 audio MPEG-2 is used for digital TV broadcasts and DVDs MPEG-4 is a new format for multimedia presentations Can require separate hardware to decode higher quality MPEG video data
43

MPEG Compression Example

A simple scene showing a car moving across a desert landscape

Only the difference between the current and next frame needs to be stored This is called intraframe coding
44

QuickTime
Developed by Apple, Inc. Primarily for playback without any hardware assistance Can achieve compression ratios of 25 to 200:1 The QuickTime format can also store audio, graphics, 3D and text making it more much versatile for multimedia applications
45

Microsoft AVI
Audio Video Interleave format Interleaving is a technique used to embed two or more things into the same stream of information In every chunk of information you will find some video data and some audio data

8, 16 or 24 bits

001001010010010101010011110101... 001001010010010101010011110101... ... 001001010010010101010011110101... 001001010010010101010011110101...

Video information Audio information

46

Digital Video Editing

Analogue tape editing is a linear process To find the section you want, you may have to forward or rewind the video tape To move a section to another place in the sequence you have to either re-record the section onto another tape or physically cut and splice the video tape

47

Digital Video Editing

Digital video editing can be non-linear process You can move sections around inside the computer and play those sections back in any order

48

How Video can be used effectively

Example of uses for video Showing physical procedures. Some uses are: Installing a board in a PC Adjusting engine timing Attracting and holding attention. Some uses are: Advertising products and services Teaching new skills to busy employees

49

How Video can be used effectively

Presenting scenarios. Some uses are: Training technicians to respond to equipment malfunctions Demonstrating possible uses for a product Analyzing motion . Some uses are: Body motion to improve athletic performance Traffic patterns for transportation planning

50

Advantages and Disadvantages of using video

Video adds visual impact to multimedia applications: Advantages Captures interest Increase retention Clarifies complex physical actions and relationships Can incorporate other media
51

Advantages and Disadvantages of using video

Disadvantages Is expensive to produce Requires extensive memory and storage Requires special equipment Does not effectively illustrate abstract concepts and static situations --52

DIGITAL ANIMATION IN MULTIMEDIA

53

Objective
What is an animation? Animation categories Animation Special Effects

54

Animation
Such animation is made from a series of stills and relies on something called persistence of vision. Persistence of vision is the phenomenon were an object on the eyes retina remains for a brief time after viewing. This means that a series of still images which vary slightly, if shown rapidly will give the illusion of movement.
55

Animation
If each of the eight pictures below were shown at the same point in rapid succession, the result would be a rotating arrow.

The pictures shown below are stills of the rotating Ford logo from the Ford Motor Company site.

56

Animation
TV gives the illusion of continuous movement by showing the stills at the rate of 30 frames per second. Feature films are filmed at a rate of 24 frames per second but shown at a rate of 48 frames per second. Today, most animation is performed by computer.
Examples are Bugs, Toy Story, Jurassic Park etc.

The computer will produce a wire frame of the scene, then apply textures and light effects before moving onto the production of the next frame, each of which may take hours or even days to produce.

57

Animation Categories
Two categories : Traditional Animation All frame in an animation had to be drawn by hand Computer Graphics Animation The use of computers to create animations.

58

Traditional Animation
Different techniques for creating animation by hand: Key frames Cell Animation

Rotoscoping: An animation technique for tracing images of live action, often a part of motion capture
59

Computer Graphics Animation

Two types of Computer Graphics: Linear animation or 2D animation Page flipping Cell Animation Object Animation or Path Animation 3D animation Modelling Rendering
60

Animation Special Effects : Morphing

Morphing is the effect in which one image transforms into another through a series of intermediate frames. This is done by first creating or scanning the first and last images. Then key points are specified, i.e. points on the original image which should become points on the final image. The computer then produces the frames in between, with each successive frame becoming progressively more morphed into the last image.
61

Morphing
The creator can usually specify the number of intermediate frames to be produced as well the the number of key points. Both the number of the key points and number of frames will impact on the time taken to produce the sequence of frames.

62

Morphing
The diagrams show start and end frames and the intermediate frames produced by morphing software

Morphing software:
Photo Morph, Hijact Morph Elastic Reality
63

Animation Special Effects : Warping

Special effect that allows you to manipulate a single image. Example: You could stretch a facial feature to change a frown into a smile. Warping software : Kai Super Goo

64

Advantages and Disadvantages of using Animation

Animation captures the imagination like no other tool, portraying actions and spatial relationships that are not readily visible in reality or that may not exist at all. Advantages
Attracts and holds attention Show otherwise invisible actions or physical processes Increases retention Allows visualization of imagined concepts, objects and relationships.

65

Advantages and Disadvantages of using Animation

Disadvantages
Requires extensive memory and storage space Requires special equipment for a quality presentation Cannot depict actuality like video or photography

66