Professional Documents
Culture Documents
(Chin-Feng Lai)
Assistant Professor, institute of CSIE, National Ilan University Oct 6th 2012
2012 MMN Lab. All Rights Reserved
2012 1
Outline
Video Device Driver Introduction Video Codec Format Raw Data Format Framebuffer Driver Video for Linux Two, V4L2 Driver OMAP3 Display SubSystem (DSS) LAB
Video Device Driver Introduction Video Codec Format Raw Data Format Framebuffer Driver Video for Linux Two, V4L2 Driver OMAP3 Display SubSystem (DSS) LAB
Media Player
File System
MediaPlayer
iSurface
MediaRecoder
LIB
OpenCore ./external/
stagefright ./frameworks/base/media/
HAL
/dev/graphics/fb0 /dev/video0 FB Driver /dev/eac Other Audio Driver /dev/snd ALSA Driver
Kernel
5
Kernel
V4L2 Driver
FB Driver
Video Driver
Hardware
Linux Embedded system
Hardware
Video Device Driver Introduction Video Codec Format Raw Data Format Framebuffer Driver Video for Linux Two, V4L2 Driver OMAP3 Display SubSystem (DSS) LAB
10
From Wiki
11
13
14
15
Video Device Driver Introduction Video Codec Format Raw Data Format Framebuffer Driver Video for Linux Two, V4L2 Driver OMAP3 Display SubSystem (DSS) LAB
16
17
18
MSB
LSB
R7 ~ R0
G7 ~ G0 Pixel
B7 ~ B0
19
MSB R4 ~ R0 G5 ~ G0 Pixel B5 ~ B0
LSB
20
YUV and Y'UV were used for a specific analog encoding of color information in television systems
=
Original Luminance
+
Chrominance
+
Chroma
21
22
Y
Y
U V U V
U V U V
Y
Y
Per Subsample
23
Y Y
U V U V
Per Subsample
Y Y
24
Y
Y
U V
Per Subsample
Y
Y
25
Video Device Driver Introduction Video Codec Format Raw Data Format Framebuffer Driver Video for Linux Two, V4L2 Driver OMAP3 Display SubSystem (DSS) LAB
26
Framebuffer Driver
What is Framebuffer
A framebuffer is a video output device that drives a video display from a memory buffer containing a complete frame of data Use mmap to allocate a block of memory space in kernel, and create virtual memory in user space, user space directly access framebuffer memory The information in the buffer typically consists of color values for every pixel (point that can be displayed) on the screen Typically use /dev/fbx (x=any number) as framebuffer device Support various video input signal
27
Framebuffer Driver
What is Framebuffer
The total amount of the memory required to drive the framebuffer depends on
Resolution Color depth Palette size
Framebuffers are commonly accessed via a memory mapping directly to memory space Double/Tripple buffering technology makes display of frames more smooth
28
Framebuffer Driver
Framebuffer Architecture
User Application
User Space Memory
User Mode
Kernel Space Memory
Memory Mapping
Kernel Mode
Hardware
29
LCD
Framebuffer Driver
Define
linux/include/linux/fb.h linux/drivers/video/fbmem.c linux/drivers/video/omap2/ (only OMAP platform) Major number29
30
Framebuffer Driver
Framebuffer Driver programming arcticture
User Application
file_operation
user space
fb_read
fb_write
fb_mmap
fb_ioctl
fbmen.c
register_fb
Fb_info
unregister_fb
fb_ops
xxxfb_pan_display
kernel space
LCD Controller
xxxfb.c
hardware
31
Framebuffer Driver
fb_info framebufer information structure
struct fb_info { int node; int flags; struct fb_var_screeninfo var; struct fb_fix_screeninfo fix; struct fb_monspecs monspecs; struct work_struct queue; struct fb_pixmap pixmap; struct fb_pixmap sprite; struct fb_cmap cmap; struct list_head modelist; struct fb_videomode *mode;
/* Current var /* Current fix /* Current Monitor specs /* Framebuffer event queue /* Image hardware mapper /* Cursor hardware mapper /* Current cmap /* mode list /* current mode
*/ */ */ */ */ */ */ */ */ */ */
struct backlight_device *bl_dev; /* assigned backlight device struct mutex bl_curve_mutex; /* Backlight level curve }
32
Framebuffer Driver
fb_var_screeninfo - used to describe the features of a video card or LCD Controller that are user defined
struct fb_var_screeninfo { __u32 xres; __u32 yres; __u32 xres_virtual; __u32 yres_virtual; __u32 xoffset; __u32 yoffset;
__u32 bits_per_pixel; __u32 grayscale; struct fb_bitfield red; struct fb_bitfield green; struct fb_bitfield blue; struct fb_bitfield transp; . . /* visible resolution /* virtual resolution /* offset from virtual to visible /* resolution /* guess what /* != 0 Graylevels instead of colors /* bitfield in fb mem if true color, /* else only length is significant /* transparency */ */ */ */ */ */ */ */ */
33
Framebuffer Driver
fb_var_screeninfo(cont.) - Timing: All values in pixclocks, except pixclock
.. .. __u32 pixclock; __u32 left_margin; __u32 right_margin; __u32 upper_margin; __u32 lower_margin; __u32 hsync_len; __u32 vsync_len; __u32 sync; __u32 vmode; __u32 rotate; __u32 reserved[5]; } /* pixel clock in ps (pico seconds) /* time from sync to picture /* time from picture to sync /* time from sync to picture */ */ */ */
/* length of horizontal sync */ /* length of vertical sync */ /* see FB_SYNC_* */ /* see FB_VMODE_* */ /* angle we rotate counter clockwise */ /* Reserved for future compatibility */
34
Framebuffer Driver
fb_fix_screeninfo - Fixed information about the video hardware, such as the start address and size of frame buffer struct fb_fix_screeninfo { memory
char id[16]; unsigned long smem_start; __u32 smem_len; __u32 type; __u32 type_aux; __u32 visual; __u16 xpanstep; __u16 ypanstep; __u16 ywrapstep; __u32 line_length; unsigned long mmio_start; __u32 mmio_len; __u32 accel; __u16 reserved[3]; /* identification string eg "TT Builtin" /* Start of frame buffer mem /* (physical address) /* Length of frame buffer mem /* see FB_TYPE_* /* Interleave for interleaved Planes /* see FB_VISUAL_* /* zero if no hardware panning /* zero if no hardware panning /* zero if no hardware ywrap /* length of a line in bytes /* Start of Memory Mapped I/O /* (physical address) /* Length of Memory Mapped I/O /* Indicate to driver which /* specific chip/card we have /* Reserved for future compatibility */ */ */ */ */ */ */ */ */ */ */ */ */ */ */ */ */
};
35
Framebuffer Driver
framebuffer Programming on linux
#include <sys/mman.h> #include <sys/ioctl.h> #include <linux/fb.h> int main(void){ int fb; unsigned char* fb_mem; fb= open("/dev/fb0", O_RDWR); fb_mem = mmap (NULL, 320*240*2, PROT_READ|PROT_WRITE, MAP_SHARED, fb, 0); memset(fb_mem, 0, 320*240*2); memcpy(fb_mem, (unsigned char *)ScreenBitmap, 320*240*2); close(fb); }
36
Memory mapping
clear framebuffer
Framebuffer Driver
Get the desktop picture
dd if=/dev/fb0 of=fbdata bs=1024 count=768 (if the resolution is 1024x768)
37
From esmertec
38
39
40
Framework
android.view.Surface ./frameworks/base/core/
android.view.Surface ./frameworks/base/core/
SurfaceFlinger Client
.framework/base/libs/surfaceflinger_client
SurfaceFlinger Client
.framework/base/libs/surfaceflinger_client
Binder LayerBaseClient
SurfaceFlinger Server
BClient
LIB
HAL
/dev/graphics/fb0
/dev/video0
FB Driver
Kernel
41
Binder
BClient
LIB HAL
42
43
swapBuffer
Screen
primary buffer
1. Flip
Graphics
primary buffer
Screen
Back buffer Flip 44
Video Device Driver Introduction Video Codec Format Raw Data Format Framebuffer Driver Video for Linux Two, V4L2 Driver OMAP3 Display SubSystem (DSS) LAB
45
V4L2
V4L2 is the second version of V4L ,and proposed by Bill Dirks V4L2 is a new design video capture APIs , support more capture device , but incompatible with V4L
46
V4L2 Program
User Mode
Hardware
47
48
49
50
51
52
2.Application use VIDIOC_DQBUF , driver get a device buffer from outgoing queue
53
VIDIOC_QUERYBUF
Application requests each allocated buffer characteristics
VIDIOC_QBUF
application requests the driver to enqueue each frame
VIDIOC_STREAMON
application starts the acquisition chain
VIDIOC_DQBUF
application waits and dequeues a frame
54
55
56
57
58
59
60
61
62
63
Both input are connect to TVP5146 decoder are using standard V4L2 interface These module support standard V4L2 device(/dev/video) Support buffer access mechanism through memory mapping and user pointers Supports standard V4L2 IOCTLs to get/set various control parameters like brightness, contrast and saturation
64
65
66
Video Device Driver Introduction Video Codec Format Raw Data Format Framebuffer Driver Video for Linux Two, V4L2 Driver OMAP3 Display SubSystem (DSS) LAB
67
68
70
/dev/video1
Video2 pipeline
71
72
73
74
arch/arm/plat-omap/dss/venc.c drivers/media/video/omap/omap_vout.c
core V4L2 driver file, It also implements the necessary ioctls supported by the V4L2
library file for the V4L2 driver, DSS provides some of the functionalities like cropping, window size and window position, This library file helps in setting all the parameters
drivers/media/video/omap/omap_voutlib.c
75
Video Device Driver Introduction Video Codec Format Raw Data Format Framebuffer Driver Video for Linux Two, V4L2 Driver OMAP3 Display SubSystem (DSS) LAB
76
LAB
Purpose
Use V4L2 driver to implement our lab Lab Programming Flow
Read Raw data picture via v4l2 driver Use Streaming mechanism streams to local buffer Push Raw data to display interface via v4l2 driver Display the picture on LCD panel
77
LAB
RAW Data
Read data
Kernel
Hardware
78
LAB
Audio Driver LAB program tip
Revise vidoe/saVideoARGB.c Revise Rule.make
KERNEL_DIR (kernel source path) LIB_DIR (External Library Dir) LIB_INC (External Headers Dir)
Fill the blank with right code make copy bin/* files to target file system
cp bin/* $(NFS_SYSTEM)
79
VIDIOC_DQBUF
Main Loop
VIDIOC_QUERYCAP
VIDIOC_S_INPUT VIDIOC_REQBUF
VIDIOC_STREAMON
VIDIOC_S_QUERYBUF VIDIOC_QBUF
LAB
Programming Stage
1. Open Stage Open the vl42 device 2. Set Parameter Stage Use IOCTL function to set parameter EX : ioctl ( fd , VIDIOC_G_FMT , &fmt) 3. Request & Query Buffer Stage Use IOCTL function to set parameter EX : ioctl ( fd , REQBUFS , &req) 4. Stream On Stage Start the video stream EX : ioctl ( fd , STREAMON , &a) 5. Queue & Dequeue Stage 6. Stream Off Stage
81
LAB
V4L2 API - Open device
82
LAB
V4L2 API - Set Image size and pixel format
83
LAB
V4L2 API - Get Window Parameter
84
LAB
V4L2 API - Set Window Size
85
LAB
V4L2 API - Request the buffer
86
LAB
V4L2 API - Pass the physical address to driver
87
LAB
V4L2 API - Enqueue the buffers
88
LAB
V4L2 API - Start Stream on
89
LAB
V4L2 API - Dqueue
90
LAB
V4L2 API - Stream Off the video loop
91
Appendix
mmap
mmap represent memory mapping Mapping the process address to user mode virtual address Let user space application directly access process memory via virtual address Commonly use in store a large number of data instead of read/write
92
Appendix
#include <sys/mman.h> void *mmap(void *addr, size_t len, int prot, int flag, int filedes, off_t off); int munmap(void *addr, size_t len);
Prot
PROT_EXEC PROT_READ PROT_WRITE PROT_NONE MAP_SHARED MAP_PRIVATE
Flag
93
Appendix
mmap
mmap (add , len ,port ,flags , fd ,offset ) Progress Space Address
94