Professional Documents
Culture Documents
Up Conversion Example
Design
Application Note 427
Introduction
Altera Corporation
AN-427-4.0
1
Preliminary
Refer to the Interfaces chapter in the Video and Image Processing Suite User
Guide for a full description of how these interfaces are implemented.
For more information on DSP Builder, refer to the DSP Builder User
Guide.
SOPC Builder is a system development tool, allowing the user to create
hardware and software system modules with a customized set of system
peripherals. SOPC Builder automatically creates the bus arbitration logic
connecting the individual components together to create an overall
system.
2
Preliminary
For more information on SOPC Builder, refer to the SOPC Builder User
Guide.
Altera Corporation
Installing the
Example Design
The example design files are included on the Video Development Kit,
Cyclone II Edition CD-ROM or can be downloaded as a zip file from the
Altera website.
Figure 1 shows the directory structure for the example design files when
they have been extracted from the zip file.
Figure 1. Example Design Directory Structure
Video_IP_Example_Design_<version>
Contains top level block design file (Video_IP_Example_Design.bdf), Quartus II settings file
(Video_IP_Example_Design.qsf), Quartus II project file (Video_IP_Example_Design.qpf),
DSP Builder model file (example_design_data_path.mdl), SOPC Builder project file
(video_system_SOPC.sopc), and PLL files for the DDR2 Controller (ddr_pll_cycloneii*.*).
altera_avalon_i2c
Contains SOPC Builder components (<module name >_hw.tcl and VHDL files) for the
2
I C controller that communicates with the digital composite input card.
example_design_controller
Contains SOPC Builder components (<module name >_hw.tcl and VHDL files) for the
2
Avalon master used by the I C controller and NTSC composite input module.
frame_buffer_beta
Contains a beta version of the Frame Buffer MegaCore function. The frame buffer block
can perform a double or triple buffering function and is inserted between the other Video
and Image Processing Suite MegaCore functions and the VGA output.
ntsc_composite_input
Contains SOPC Builder components (<module name >_hw.tcl and VHDL files) which
decode the video signal from the digital composite input card.
vga_output
Contains SOPC Builder components (<module name >_hw.tcl and VHDL files) for the VGA
output driver.
docs
Contains this document (AN-427.pdf).
Notes to Figure 1:
(1)
(2)
Altera Corporation
3
Preliminary
Possible Video
System
Configurations
4
Preliminary
Altera Corporation
Deinterlacing
NTSC
Interface
Chroma
Resampling
Gamma
Correction
Color Space
Conversion
Scaling
(YCbCr->RGB)
2D 5x5 FIR
Filter
(Sharpening)
Nios II
Processor
Static
Image
Picture-inPicture
Mixing
Triple
Buffer
VGA
Controller
Video
Output
(VGA)
External Memory
The system uses a Nios II processor for control processes such as writing
gamma values to the gamma corrector look up table and writing the
relative location of a static image in on-chip memory to the picture-inpicture mixing function.
The standardization on Avalon-MM control interface ports makes Nios II
a convenient choice for run-time control.
Altera Corporation
5
Preliminary
NTSC
Interface
Deinterlacing
Chroma
Resampling
Gamma
Correction
Color Space
Conversion
Scaling
(YCbCr->RGB)
Nios II
Processor
Picture-inPicture
Mixing
Triple
Buffer
VGA
Controller
Video
Output
(VGA)
External
Memory
Triple
Buffer
Composite
Video Input
(Interlaced)
NTSC
Interface
Deinterlacing
Chroma
Resampling
2D 5x5 FIR
Filter
(Sharpening)
Color Space
Conversion
Gamma
Correction
(YCbCr->RGB)
Scaling
2D 5x5 FIR
Filter
(Sharpening)
Note that the two triple buffer blocks perform frame synchronization for
the two video streams.
6
Preliminary
Altera Corporation
SDI Video
Source
Deinterlacing
SDI
Chroma
Resampling
Color Space
Conversion
Scaling
Triple
Buffer
SDI
SDI
Video
(YCbCr->RGB)
External Memory
Altera Corporation
7
Preliminary
Functional
Description
Figure 5 shows a simple block diagram for the video and image
processing up conversion example design.
DSP Builder
Video Up
Conversion
Frame Buffer
HD Video Output
- VGA Controller
DDR2
SDRAM
SOPC Builder
8
Preliminary
Altera Corporation
Functional Description
Example
Design
Controller
I2 C
Controller
Y/C
Syncs
Video Data
Capture
Ready
Valid
Data
Dual-Clock FIFO
8
The NTSC input into the system requires three SOPC Builder components
to work together: the I2C Controller, the Example Design Controller and
the NTSC Composite Input block. These are shown in Figure 6, with a
dotted line delineating each SOPC Builder component.
The NTSC Composite Input block operates similarly to its counterpart the
VGA Output block. It is written in VHDL which can be found in the
<install_dir>\ntsc_composite_input folder.
Video data in YCbCr 4:2:2 format and associated synchronization signals
are input into the Video Data Capture block from the Video Input
Daughtercard. This block identifies active video parts of the picture and
inserts just this data into the Dual-Clock FIFO. A standard flow controlled
interface on the output of the FIFO allows this data to be read by the next
part of the system (the video processing data path in the case of the
example design) at the system clock rate of 130 MHz.
Altera Corporation
9
Preliminary
If the FIFO becomes empty, then the interface will not assert valid and the
rest of the system will wait. The FIFO should never become full - if this
were to happen then data from the video data capture block would be lost
and unpleasant visual artefacts would result.
The TVP5146 video decoder chip on the Video Input Daughtercard
performs analog-to-digital conversion of the video input signals and
needs to be configured before it can be used. An I2C interface is provided
for this purpose. The I2C Controller is an Avalon-MM slave which, when
controlled by an Avalon-MM master, performs I2C reads and writes on an
external bus as requested by that master.
The Example Design Controller is an Avalon-MM master which directs
the OpenCores I2C Controller to perform the necessary start-up sequence
for the TI5146 chip, and then sends a signal to the NTSC Composite Input
block telling it to start inputting data. The NTSC Composite Input block
provides an Avalon-MM slave port for this purpose. The slave port has
one register, one bit of which is significant: The least significant bit is a GO
bit - when this is logic '1' the NTSC Composite Input block tries to input
data, when it is logic '0' the NTSC Composite Input block does nothing.
The I2C master is written in VHDL which can be found in the directory
<install_dir>\altera_avalon_i2c. The example design controller VHDL
can be found in <install_dir>\example_design_controller.
Video Up Conversion
Figure 7 shows a block diagram of the video up conversion data path
subsystem.
Figure 7. Video Up Conversion Data Path Block Diagram
640 x 480
Interlaced 60 Hz
YCbCr 4:2:2
640 x 480
Progressive 30 Hz
YCbCr 4:2:2
Deinterlacer
MegaCore
8
640 x 480
Progressive 30 Hz
YCbCr 4:4:4
Chroma
Resampler
MegaCore
640 x 480
Progressive 30 Hz
RGB
Color
Space
Converter
MegaCore
1024 x 768
Progressive 30 Hz
RGB
Ready
Scaler
MegaCore
Valid
Data
8
10
Preliminary
Altera Corporation
Functional Description
The video up conversion data path performs all of the video processing
required to convert from an NTSC format input to a 1024768 VGA
output. It is composed entirely of Altera Video and Image Processing
MegaCore functions and is assembled in DSP Builder, where it can be
simulated independently of the rest of the system (see Simulate the Data
Path Component in DSP Builder on page 30). The entire data path is
exported from DSP Builder as a single SOPC Builder component, with
standard Avalon-ST input and output interfaces and two Avalon-MM
master ports.
The data path makes use of four MegaCore functions connected in
sequence to perform conversion. First a Deinterlacer converts from 60 Hz
interlaced to 30 Hz progressive. Next a Chroma Resampler interpolates to
convert from 4:2:2 subsampled color data to full 4:4:4 color data. This is
followed by a Color Space Converter which transforms between the
YCbCr and RGB color spaces. Finally a Scaler scales the 640480 input
image up to 1024768 using bicubic interpolation.
Ready
Valid
Data
8
ready-latency 1
rules
Altera Corporation
Avalon-ST
Timing
Adapter
Ready
Valid
Data
8
Avalon-ST
Data Format
Adapter
ready-latency 0
rules
Ready
Valid
Data
24
ready-latency 0
rules
Avalon-ST
Timing
Adapter
Ready
Valid
Data
24
ready-latency 1
rules
11
Preliminary
Ready
Ready
Memory
Writer
Valid
Data
Memory
Reader
Valid
Data
24
24
DDR2
The Frame Buffer block is a beta version of a new Video and Image
Processing Suite MegaCore function which can be found in the directory
<install_dir>\frame_buffer_beta.
The purpose of this block is to provide a triple buffering function which
allows the input and output sides to run asynchronously and at different
frame rates. In the example design, this is necessary because the output of
the video data path is 1024768 progressive video @ 30 fps, but the VGA
output cannot run slower than 60 fps.
The Frame Buffer block inputs and outputs flow-controlled streams of
video data over standard interfaces of the type described in Data and
Flow Control Signals on page 18.
The input and output data interfaces have 24-bit wide data ports and
output all three color planes in parallel, with blue on the least significant
bits and red on the most significant bits. This makes the output interface
compatible with the input interface on the VGA Output block.
The block uses three equal-sized frame buffers stored in external memory.
The base address of this memory in the Avalon-MM address space is set
at 0x10000000. There must be 8MByte of free memory at this address.
12
Preliminary
Altera Corporation
Functional Description
The external memory is a double data rate (DDR) RAM in the example
design, and is accessed via SOPC Builder arbitration logic as shown in
Figure 9. SOPC Builder is responsible for sharing access time to the DDR
between the frame buffer block and other blocks in the design. The block
includes a writer, which writes the input stream into one of the buffers,
and a reader, which reads the output stream from another (never the
same) buffer. There is always one buffer which is neither being written to
nor read from. This "spare" buffer is required to allow the input and
output to run at differing frame rates.
The frame buffer operates by swapping frames according to the following
algorithms.
Each time the input finishes writing a frame of data into a buffer:
Wait until the spare buffer has data that has already been displayed,
then start writing the next input frame into the spare buffer. The
buffer just written into then becomes the new spare buffer. No frame
is dropped in this configuration.
Each time the output finishes reading a frame of data from a buffer:
If the spare buffer has data which has not yet been displayed then
start reading the next output frame from the spare buffer. The frame
buffer just read from then becomes the new spare buffer. Otherwise,
start reading the next output frame from the same buffer (a frame is
repeated).
VGA Output
Figure 10 shows a simple block diagram of the VGA Output component.
Figure 10. VGA Output Component
SOPC System Clock
(130 MHz)
VGA Clock
(65 MHz)
Ready
Dual-Clock FIFO
Valid
Data
24
VGA Syncs
Generator
R
G
B
Syncs
Altera Corporation
13
Preliminary
A stream of video data is input into the block over a standard flow
controlled interface. This interface includes a 24-bit wide data port
because red, green, and blue data is input into the block in parallel, so that
all of the data for a pixel can be input in a single clock cycle. The least
significant eight bits form the blue channel, the green is in the middle and
the most significant eight bits carry the red channel. The interface also
includes ready and valid lines for flow control.
Refer to Data and Flow Control Signals on page 18 for details of the
flow controlled interface.
The video data is input via a flow controlled interface, so data is not
transferred on every clock cycle, only those clock cycles where valid = '1',
and the clock therefore need not be the same rate as the VGA output
clock. The clock actually used at this end is therefore the SOPC System
clock, which runs at 130MHz in the example design.
Data is input into a dual-clock FIFO which provides clock domain
crossing to the 65MHz VGA clock and also provides a queue where pixels
can wait when the VGA output is in blanking and does not need pixel
data.
If this FIFO ever becomes full, then the flow controlled interface will
indicate that the VGA output is not ready for data and earlier parts of the
pipe will stop.
The FIFO should never become empty while the system is running. If it
does, then the situation could arise that there is no pixel data available
when the VGA Syncs Generator needs it. In this case, the VGA output
module will generate red pixels on its outputs (R:255, G:0, B:0) as an
indication of data starvation.
Synchronization and blanking signals for VGA running the standard
video electronics standards association (VESA) 1024768 resolution are
calculated and output by the VGA Syncs Generator block. This block also
pulls data out of the dual-clock FIFO and presents it on R, G, and B
outputs when the syncs indicate the VGA should output active picture
data. Signals from the VGA Syncs Generator form the output of the VGA
Output block, and are wired directly to pins connected to the 2C35
board's VGA digital-to-analog converter (DAC) in the example design.
System
Requirements
14
Preliminary
This section describes the hardware and software requirements to run the
video up conversion example design.
Altera Corporation
System Requirements
Hardware Requirements
The video and image processing up conversion example design requires
the following hardware components:
Software Requirements
The up conversion example design is supported on Windows XP only.
Ensure that the software provided with the development kit is installed
onto your PC.
Altera Corporation
15
Preliminary
Review and
Simulate the
Example Design
This section reviews the Data Path component in DSP Builder and
describes how to simulate the Data Path component in DSP Builder.
2.
3.
4.
Review the top level DSP Builder model, as shown in Figure 11.
16
Preliminary
Altera Corporation
Altera Corporation
17
Preliminary
Refer to the Interfaces chapter in the Video and Image Processing Suite User
Guide for a full description of these data and flow control signals.
Two MegaCore functions can therefore be connected together by
connecting the common signal types valid to valid, ready to ready
and data to data.
The model based view in DSP Builder of the connection between the
Color Space Converter and the Scaler block is show in Figure 13.
Figure 13. Connections Between the Color Space Converter and Scaler
18
Preliminary
Altera Corporation
The Video Source block extracts video frames from a multimedia file
(vip_car.avi) and transmits them using the image streaming protocol
described in the Video and Image Processing Suite User Guide.
For more information on the Video Source block, refer to the DSP Builder
Reference Manual.
The Video Source block has one input signal and two output signals:
Input: ready
Data output: data
Valid output: valid
The block produces valid output data one clock cycle after the ready
input signal is driven high. The valid output signal is driven high when
the clock is valid.
In this example, the Video Source block is configured to output 8-bit wide
data in sequence, one color plane every clock cycle, over the data output
port as shown in Figure 15.
Figure 15. YCbCr Format Pixel Data at a 4:2:2 Sampling Rate
Y Cb
1
Altera Corporation
Y Cr Y Cb Y Cr
The file name and location of the input file is a parameter of the
Video Source block. Before running the simulation, you should
check that the input file vip_car.avi is in the current MATLAB
directory. Alternatively, you can change the Input file name
parameter of the Video Source block to the absolute path of the
multimedia file of your choice. The Video Source block can
extract video frames from a wide range of multimedia file types,
including still bitmap pictures, provided that the proper codecs
are installed on your system.
19
Preliminary
The Video Sink block builds video frames from the incoming stream of
data and uses an user-specified encoder to store them in an AVI file. The
Video Sink block is configured to output the file vip_car_out.avi in the
current MATLAB directory by default.
1
For more information on the Video Sink block, refer to the DSP
Builder Reference Manual.
When running a simulation, the Input Frames and Output Frames blocks
display the number of frames processed in decimal mode.
5.
20
Preliminary
Altera Corporation
To count frames correctly, the frame width and frame height are set to the
resolution of the video at the point in the data path where the frame
counter is connected. The number of channels in sequence is set to the
number of color planes transported in sequence at that point on the data
path.
For the frame counter placed after the Scaler block, the frame width,
frame height, and number of channels in sequence are set to the format
output by the Scaler block.
1
6.
Altera Corporation
21
Preliminary
7.
22
Preliminary
Altera Corporation
Note that the external RAM model can be used for simulation
only and will not generate HDL if the design is compiled with
Signal Compiler.
Altera Corporation
23
Preliminary
9.
10. Click Cancel to close the External RAM Function Block Parameters
dialog box.
Chroma Resampler
The example design resamples the subsampled video input using the
Chroma Resampler MegaCore function provided with the Altera Video
and Image Processing Suite. Figure 23 on page 25 shows the data and
control ports on the Chroma Resampler block.
24
Preliminary
Altera Corporation
Altera Corporation
25
Preliminary
26
Preliminary
Altera Corporation
14. Select the Operands tab to display the coefficients used to convert
between the YCbCr and RGB color spaces. See Figure 27.
Figure 27. Operands for the Color Space Converter MegaCore Function
15. Click Cancel to close the Color Space Converter MegaWizard page.
Scaler
The example design scales the input video from standard definition (SD)
NTSC resolution of 640480 pixels to the high definition (HD) resolution
1024768 pixels using the Scaler MegaCore function provided with the
Altera Video and Image Processing Suite.
Altera Corporation
27
Preliminary
Figure 28 shows the data and control ports on the scaler block.
Figure 28. Scaler Block
28
Preliminary
Altera Corporation
The scaler uses the Bicubic scaling algorithm with 16 vertical and
horizontal phases.
1
The Coefficients tab is only used when you select the Polyphase
scaling algorithm and is not used in the example design.
Altera Corporation
29
Preliminary
For more information about fast functional simulation, refer to the Using
the Simulation Accelerator chapter in the DSP Builder User Guide.
algebraic_loop_cut_dil Block
This block is used to prevent Simulink from detecting an algebraic loop
between the Deinterlacer and the two Avalon-MM master interfaces.
30
Preliminary
Altera Corporation
This issue is discussed in the DSP Builder Release Notes and Errata as an
errata: Deinterlacer Ports Cannot Be Simulated with Avalon-MM Master
Block. The design can be synthesized but it cannot be simulated correctly
if the External RAM Wait States Per Write parameter is set to anything
other than 0.
1.
1
2.
To simulate the design, select Start from the Simulation menu in the
example_design_data_path model window.
Any existing vip_car_out.avi file is overwritten when you run a
new simulation. You may want to archive the previous video
files before you run a simulation. Simulation may fail to start if
the previous video output file is open in your video player.
Allow the simulation to run for at least one output frame. The frame
counter blocks provides run time feedback on simulation progress.
If you stop the simulation, you will not be able to resume from
the same point and the simulation must be restarted from the
beginning.
You can then simulate the design in exactly the same way as for fast
functional simulation.
Altera Corporation
31
Preliminary
Review the
System
Integration
Using SOPC
Builder
32
Preliminary
Altera Corporation
The Read and Write Avalon-MM masters bound the synthesizable region
in a similar way to the Avalon-ST data interface bus limiters.
Altera Corporation
33
Preliminary
The External RAM block is for simulation only, and is not synthesizable.
The connection of the Deinterlacer to the external DDR2 memory is
performed in SOPC Builder.
The data and control interface's port blocks define the synthesizable
boundary of the data path design that will interface to other system
components in SOPC Builder.
A description of your model is written out as a .mdlxml file when you run
Signal Compiler.
Refer to these DSP Builder User Guide for information about compiling
your design.
A full compilation is not strictly necessary for integration of the design
within SOPC Builder. You can alternatively use the DSPBuilder command
alt_dspbuilder_mdl2xml to write the .mdlxml file by performing the
following steps:
1.
2.
34
Preliminary
Altera Corporation
2.
Choose Open Project from the File menu. Browse to the directory
where you installed the Video_IP_Example_Design and select the
Video_IP_Example_Design.qpf file. (This file contains project
definitions for the video up conversion example design.)
3.
Choose Open from the File menu in the Quartus II software and
select the file Video_IP_Example_Design.bdf. Click on Open to
display this top-level file as shown in Figure 36.
4.
5.
Altera Corporation
35
Preliminary
SOPC Builder automatically detects the DSP Builder generated data path
module because the .mdlxml file (example_design_data_path.mdlxml)
is located in the same directory as the SOPC Builder component file
(video_system_SOPC.sopc).
1
36
Preliminary
Altera Corporation
Altera Corporation
37
Preliminary
38
Preliminary
Altera Corporation
Click Cancel to close the Frame Buffer BETA block dialog box.
9.
Altera Corporation
39
Preliminary
Notice that the sizes of the data ports for the Pipeline Bridge components
are different. The data width is 128 for the Pipeline Bridge connected to
the Frame Buffer and 64 for the Pipeline Bridge connected to the up
conversion subsystem built with DSP Builder. This is consistent with the
parameterization of the Deinterlacer and Frame Buffer MegaCore
functions.
11. Click Cancel to close the second Avalon-MM Pipeline Bridge dialog
box.
12. Click Exit to close SOPC Builder after you review the components
within the system but leave the Quartus II project open for the next
section.
40
Preliminary
Set Up the
Hardware and
Configure the
FPGA
2.
Connect one end of the USB cable to the USB port on your PC.
3.
Connect the other end to the 10-pin header labeled (J21) on the
Cyclone II DSP Development Board.
4.
5.
6.
7.
For details of installing the USB Blaster software driver on the host PC
(located at <quartus_install_dir>\drivers\usb-blaster), refer to the
USB-Blaster Download Cable User Guide.
For details of the Cyclone II DSP Development Board, refer to the DSP
Development kit, Cyclone II Edition Getting Started User Guide.
Altera Corporation
41
Preliminary
2.
3.
4.
In the Save As type list, make sure you select Chain Description
File.
5.
Click Save.
6.
7.
8.
9.
If you are not using a licensed version of the Video and Image
Processing Suite, a message appears indicating that you are
running a time-limited configuration file on your target
hardware.
12. Press the USER RESET button on the board and confirm that a
scaled video stream is displayed on the monitor.
You have now successfully completed the Video and Image Processing
Up Conversion design example walkthrough.
42
Preliminary
Altera Corporation
Conclusion
Conclusion
Troubleshooting
Altera Corporation
1.
Open Video_IP_Example_Design.bdf.
2.
Double-click on Video_IP_Example_Design_SOPC_Component to
open the SOPC System in SOPC Builder.
3.
Make sure that the Quartus II user and global library paths are
correct for the project (see Review the Final System using SOPC
Builder on page 34).
4.
5.
43
Preliminary
44
Preliminary
Altera Corporation
Revision History
Revision History
Table 1 shows the revision history for the AN-427: Video and Image
Processing Up Conversion Example Design application note.
Date
Errata Summary
4.0
October 2007
Updated for Quartus version 7.2. The design now uses DSP Builder Video Source
and Video Sink blocks and the triple buffer block has been replaced by a Frame
Buffer MegaCore function. Removed obsolete troubleshooting issue Compilation
Fails in Analysis and Synthesis.
3.0
May 2007
2.0
December 2006
1.2
July 2006
1.1
July 2006
Updated algorithm used by the triple buffer block, effects of VGA starvation,
description of the image stream frame counter block, and other minor edits.
1.0
June 2006
Altera Corporation
Copyright 2007 Altera Corporation. All rights reserved. Altera, The Programmable Solutions Company,
the stylized Altera logo, specific device designations, and all other words and logos that are identified as
trademarks and/or service marks are, unless noted otherwise, the trademarks and service marks of Altera
Corporation in the U.S. and other countries. All other product or service names are the property of their respective holders. Altera products are protected under numerous U.S. and foreign patents and pending
applications, maskwork rights, and copyrights. Altera warrants performance of its semiconductor products
to current specifications in accordance with Altera's standard warranty, but reserves the right to make changes to any products and services at any time without notice. Altera assumes no responsibility or liability
arising out of the application or use of any information, product, or service described
herein except as expressly agreed to in writing by Altera Corporation. Altera customers
are advised to obtain the latest version of device specifications before relying on any published information and before placing orders for products or services.
45
Preliminary