V4L2 mini-summit Sep 2009
From OMAPpedia
This wikipage provides details about the Video4Linux mini-summit held at Linux Plumbers Conference 2009 in Portland, Oregon, USA on 23-25 Sep, 2009.
Contents |
[edit] Atendees
- Moderator: Hans Verkuil.
- V4L-dvb core devs:
- Mike Krufky
- Steven Toth
- Andy Walls
- Brandon Philips.
- Nokia:
- Sakari Ailus
- Laurent Pinchart
- Texas Instruments:
- Vaibhav Hiremath
- Murali Karicheri
- Sergio Aguirre
- Lajos Molnar
- Samsung:
- Marek Szyprowsky
- Tomasz Fujak
- Jin-Sung Yang
- Intel:
- Guru Raj
- Xiaolin Zhang.
[edit] Problems to Solve
This were the objects for discussion during the 3 days the summit was held:
[edit] SoC complex video devices handling
Recent System on Chip solutions have several internal video processing blocks, like:
- Scalers
- Colorspace Converters
- Composers
- Statistic collection modules
- Etc
Currently, V4L2 doesn’t offer a good API for this, and the functionality of current implementations is very limited to a set of fixed combinations.
Obviously, the need for an extension of v4l2 API for this arises.
[edit] Optimal buffer handling between video devices
Usually, a video buffer is shared between:
- Camera driver.
- Display driver.
- DSP for image processing.
Two choices are currently available:
- User pointers
- Pro: Zero copy
- Con: No real way to keep cache coherency
- Kernel mmaped buffers
- Pro: Cache coherency not a problem
- Con: Needs memcpys! (more memory footprint)
Therefore, the need for an optimal handling arises to have the best of both options above in a clean way.
[edit] Standard device to application events communication
Complex multimedia devices require an smart way to notify events to applications in order for application to know when they take place, and being able to control them optimally.
There wasn't a way to get this working meanwhile sticking to the v4l2 spec, therefore hacks were done by SoC drivers to overcome this.
[edit] Standard timings setting for video interfaces
There are video HW, like HDMI, that requires to negotiate timing settings between Transmitter and Receiver.
This is lacking on the v4l2 specification, and doing this negotiation is an actual need for many of this video interfaces.
[edit] Proposals
[edit] Media Controller
[edit] What is it?
It's a new Video4Linux device node that can be used to discover and modify the topology of the board and to give access to the low-level nodes (such as previewers, resizers, color space converters, etc.) that are part of the topology.
[edit] Why is needed?
- Discovering various device nodes created by a board.
- Being able to configure interconnections between subcomponents dynamically.
- Increasing demand to control sensors and/or video encoders/decoders more precisely.
[edit] Restrictions
- Shouldn’t affect existing applications.
- Don’t try to be too smart, just give full control to the application.
- Provide automatic default support for drivers that don’t create MC, for easing apps transition.
- All APIs specific to sub-devices _MUST_ be documented!
- Don’t abuse them to use direct register read/writes!
[edit] Concepts
- Entity: Subcomponent which could have multiple input and output pads. Has a unique ID per MC.
- Pad: Connecting point which could have several mutually exclusive links to other entities.
- Link: Connection between 2 pads.
[edit] Expected operation
- Links should be created by Media Controller, from user-space library, which should be helping abstract the wanted usecases.
- Library should be aware of the elements, and provide policy for interconnections.
[edit] RFC
http://www.spinics.net/lists/linux-media/msg09971.html
[edit] Global video buffers pool
- Allocation on kernel space.
- Solves cache coherency problem.
- Zero copy solution
- Globally available to all devices.
- Probably one per system.
- And probably v4l2 independent (Con: will have to deal with core kernel devs!)
- Conclusion: More investigation needs to be done. (Laurent)
[edit] RFC
http://www.spinics.net/lists/linux-media/msg10145.html
Also see Laurent notes about it:
http://www.spinics.net/lists/linux-media/msg10450.html
And Samsung solution:
http://www.spinics.net/lists/linux-media/msg10566.html
[edit] Video events API
- Standard V4L2 communication from device to application
- Can be used with select() since it arrives as an exception.
- Per filehandle event queue and event subscription
Example: Notify App when statistics are ready for the current frame.
[edit] RFC
http://www.spinics.net/lists/linux-media/msg10217.html
[edit] Video timings API
- Presets for timing standards
- Can Get/Set/query standards
- If HW allows, set custom timings (front & back porch, sync widths, pixelclock)
[edit] RFC
http://www.spinics.net/lists/linux-media/msg09979.html http://www.spinics.net/lists/linux-media/msg10025.html
[edit] Daily notes
These were written and shared by Hans Verkuil (moderator):
Day 1: http://www.spinics.net/lists/linux-media/msg10423.html
Day 2: http://www.spinics.net/lists/linux-media/msg10447.html
Day 3: http://www.spinics.net/lists/linux-media/msg10457.html
