From: Mauro Carvalho Chehab Date: Wed, 20 Jul 2016 17:14:37 +0000 (-0300) Subject: [media] doc-rst: Split v4l-core into one file per kAPI X-Git-Url: https://git.stricted.de/?a=commitdiff_plain;h=58759874002a71cbd48a01b615a210bb474e1f2b;p=GitHub%2Fmoto-9609%2Fandroid_kernel_motorola_exynos9610.git [media] doc-rst: Split v4l-core into one file per kAPI Sphinx produce a 1:1 mapping between a rst file and an html file. So, we need to split the kernel-doc tags on multiple documents. A side effect is that we're now having a better name for each section of the kAPI documentation. Signed-off-by: Mauro Carvalho Chehab --- diff --git a/Documentation/media/kapi/v4l2-async.rst b/Documentation/media/kapi/v4l2-async.rst new file mode 100644 index 000000000000..372aa29fbf29 --- /dev/null +++ b/Documentation/media/kapi/v4l2-async.rst @@ -0,0 +1,4 @@ +V4L2 Async kAPI +^^^^^^^^^^^^^^^ + +.. kernel-doc:: include/media/v4l2-async.h diff --git a/Documentation/media/kapi/v4l2-controls.rst b/Documentation/media/kapi/v4l2-controls.rst index 916fd4366cdf..2d433305fd4e 100644 --- a/Documentation/media/kapi/v4l2-controls.rst +++ b/Documentation/media/kapi/v4l2-controls.rst @@ -807,3 +807,8 @@ notify function is called. There can be only one notify function per control handler. Any attempt to set another notify function will cause a WARN_ON. + +V4L2 control kAPI +----------------- + +.. kernel-doc:: include/media/v4l2-ctrls.h diff --git a/Documentation/media/kapi/v4l2-core.rst b/Documentation/media/kapi/v4l2-core.rst index db571a4f498a..8c127ccdb0ae 100644 --- a/Documentation/media/kapi/v4l2-core.rst +++ b/Documentation/media/kapi/v4l2-core.rst @@ -1,38 +1,23 @@ Video2Linux devices ------------------- -.. kernel-doc:: include/media/tuner.h - -.. kernel-doc:: include/media/tuner-types.h - -.. kernel-doc:: include/media/tveeprom.h - -.. kernel-doc:: include/media/v4l2-async.h - -.. kernel-doc:: include/media/v4l2-ctrls.h - -.. kernel-doc:: include/media/v4l2-device.h - -.. kernel-doc:: include/media/v4l2-dv-timings.h - -.. kernel-doc:: include/media/v4l2-event.h - -.. kernel-doc:: include/media/v4l2-flash-led-class.h - -.. kernel-doc:: include/media/v4l2-mc.h - -.. kernel-doc:: include/media/v4l2-mediabus.h - -.. kernel-doc:: include/media/v4l2-mem2mem.h - -.. kernel-doc:: include/media/v4l2-of.h - -.. kernel-doc:: include/media/v4l2-rect.h - -.. kernel-doc:: include/media/v4l2-subdev.h - -.. kernel-doc:: include/media/videobuf2-core.h - -.. kernel-doc:: include/media/videobuf2-v4l2.h - -.. kernel-doc:: include/media/videobuf2-memops.h +.. toctree:: + :maxdepth: 1 + + v4l2-framework + v4l2-async + v4l2-controls + v4l2-device + v4l2-dv-timings + v4l2-event + v4l2-flash-led-class + v4l2-mc + v4l2-mediabus + v4l2-mem2mem + v4l2-of + v4l2-rect + v4l2-subdev + v4l2-tuner + v4l2-tveeprom + v4l2-videobuf2 + v4l2-videobuf diff --git a/Documentation/media/kapi/v4l2-device.rst b/Documentation/media/kapi/v4l2-device.rst new file mode 100644 index 000000000000..e324fbcb0353 --- /dev/null +++ b/Documentation/media/kapi/v4l2-device.rst @@ -0,0 +1,4 @@ +V4L2 Device kAPI +^^^^^^^^^^^^^^^^ + +.. kernel-doc:: include/media/v4l2-device.h diff --git a/Documentation/media/kapi/v4l2-dv-timings.rst b/Documentation/media/kapi/v4l2-dv-timings.rst new file mode 100644 index 000000000000..4b08a49c54a4 --- /dev/null +++ b/Documentation/media/kapi/v4l2-dv-timings.rst @@ -0,0 +1,4 @@ +V4L2 DV Timings kAPI +^^^^^^^^^^^^^^^^^^^^ + +.. kernel-doc:: include/media/v4l2-dv-timings.h diff --git a/Documentation/media/kapi/v4l2-event.rst b/Documentation/media/kapi/v4l2-event.rst new file mode 100644 index 000000000000..b54bd915fff6 --- /dev/null +++ b/Documentation/media/kapi/v4l2-event.rst @@ -0,0 +1,4 @@ +V4L2 event kAPI +^^^^^^^^^^^^^^^ + +.. kernel-doc:: include/media/v4l2-event.h diff --git a/Documentation/media/kapi/v4l2-flash-led-class.rst b/Documentation/media/kapi/v4l2-flash-led-class.rst new file mode 100644 index 000000000000..251ed6b3aab3 --- /dev/null +++ b/Documentation/media/kapi/v4l2-flash-led-class.rst @@ -0,0 +1,4 @@ +V4L2 Flash and LED class kAPI +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. kernel-doc:: include/media/v4l2-flash-led-class.h diff --git a/Documentation/media/kapi/v4l2-mc.rst b/Documentation/media/kapi/v4l2-mc.rst new file mode 100644 index 000000000000..c94ce0fa3839 --- /dev/null +++ b/Documentation/media/kapi/v4l2-mc.rst @@ -0,0 +1,4 @@ +V4L2 Media Controller kAPI +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. kernel-doc:: include/media/v4l2-mc.h diff --git a/Documentation/media/kapi/v4l2-mediabus.rst b/Documentation/media/kapi/v4l2-mediabus.rst new file mode 100644 index 000000000000..b3c246b51c2c --- /dev/null +++ b/Documentation/media/kapi/v4l2-mediabus.rst @@ -0,0 +1,4 @@ +V4L2 Media Bus kAPI +^^^^^^^^^^^^^^^^^^^ + +.. kernel-doc:: include/media/v4l2-mediabus.h diff --git a/Documentation/media/kapi/v4l2-mem2mem.rst b/Documentation/media/kapi/v4l2-mem2mem.rst new file mode 100644 index 000000000000..61f9923286c7 --- /dev/null +++ b/Documentation/media/kapi/v4l2-mem2mem.rst @@ -0,0 +1,3 @@ +V4L2 Memory to Memory kAPI +^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. kernel-doc:: include/media/v4l2-mem2mem.h diff --git a/Documentation/media/kapi/v4l2-of.rst b/Documentation/media/kapi/v4l2-of.rst new file mode 100644 index 000000000000..1ddf76b00944 --- /dev/null +++ b/Documentation/media/kapi/v4l2-of.rst @@ -0,0 +1,3 @@ +V4L2 Open Firmware kAPI +^^^^^^^^^^^^^^^^^^^^^^^ +.. kernel-doc:: include/media/v4l2-of.h diff --git a/Documentation/media/kapi/v4l2-rect.rst b/Documentation/media/kapi/v4l2-rect.rst new file mode 100644 index 000000000000..bb86dcbc5a3c --- /dev/null +++ b/Documentation/media/kapi/v4l2-rect.rst @@ -0,0 +1,4 @@ +V4L2 rect kAPI +^^^^^^^^^^^^^^ + +.. kernel-doc:: include/media/v4l2-rect.h diff --git a/Documentation/media/kapi/v4l2-subdev.rst b/Documentation/media/kapi/v4l2-subdev.rst new file mode 100644 index 000000000000..1b262aa7e250 --- /dev/null +++ b/Documentation/media/kapi/v4l2-subdev.rst @@ -0,0 +1,4 @@ +V4L2 subdev kAPI +^^^^^^^^^^^^^^^^ + +.. kernel-doc:: include/media/v4l2-subdev.h diff --git a/Documentation/media/kapi/v4l2-tuner.rst b/Documentation/media/kapi/v4l2-tuner.rst new file mode 100644 index 000000000000..37b0ef310a62 --- /dev/null +++ b/Documentation/media/kapi/v4l2-tuner.rst @@ -0,0 +1,6 @@ +Tuner kAPI +^^^^^^^^^^ + +.. kernel-doc:: include/media/tuner.h + +.. kernel-doc:: include/media/tuner-types.h diff --git a/Documentation/media/kapi/v4l2-tveeprom.rst b/Documentation/media/kapi/v4l2-tveeprom.rst new file mode 100644 index 000000000000..f7ef71742e93 --- /dev/null +++ b/Documentation/media/kapi/v4l2-tveeprom.rst @@ -0,0 +1,4 @@ +Hauppauge TV EEPROM kAPI +^^^^^^^^^^^^^^^^^^^^^^^^ + +.. kernel-doc:: include/media/tveeprom.h diff --git a/Documentation/media/kapi/v4l2-videobuf.rst b/Documentation/media/kapi/v4l2-videobuf.rst new file mode 100644 index 000000000000..01156728203c --- /dev/null +++ b/Documentation/media/kapi/v4l2-videobuf.rst @@ -0,0 +1,402 @@ +Videobuf Framework +================== + +Author: Jonathan Corbet + +Current as of 2.6.33 + +.. note:: + + The videobuf framework was deprecated in favor of videobuf2. Shouldn't + be used on new drivers. + +Introduction +------------ + +The videobuf layer functions as a sort of glue layer between a V4L2 driver +and user space. It handles the allocation and management of buffers for +the storage of video frames. There is a set of functions which can be used +to implement many of the standard POSIX I/O system calls, including read(), +poll(), and, happily, mmap(). Another set of functions can be used to +implement the bulk of the V4L2 ioctl() calls related to streaming I/O, +including buffer allocation, queueing and dequeueing, and streaming +control. Using videobuf imposes a few design decisions on the driver +author, but the payback comes in the form of reduced code in the driver and +a consistent implementation of the V4L2 user-space API. + +Buffer types +------------ + +Not all video devices use the same kind of buffers. In fact, there are (at +least) three common variations: + + - Buffers which are scattered in both the physical and (kernel) virtual + address spaces. (Almost) all user-space buffers are like this, but it + makes great sense to allocate kernel-space buffers this way as well when + it is possible. Unfortunately, it is not always possible; working with + this kind of buffer normally requires hardware which can do + scatter/gather DMA operations. + + - Buffers which are physically scattered, but which are virtually + contiguous; buffers allocated with vmalloc(), in other words. These + buffers are just as hard to use for DMA operations, but they can be + useful in situations where DMA is not available but virtually-contiguous + buffers are convenient. + + - Buffers which are physically contiguous. Allocation of this kind of + buffer can be unreliable on fragmented systems, but simpler DMA + controllers cannot deal with anything else. + +Videobuf can work with all three types of buffers, but the driver author +must pick one at the outset and design the driver around that decision. + +[It's worth noting that there's a fourth kind of buffer: "overlay" buffers +which are located within the system's video memory. The overlay +functionality is considered to be deprecated for most use, but it still +shows up occasionally in system-on-chip drivers where the performance +benefits merit the use of this technique. Overlay buffers can be handled +as a form of scattered buffer, but there are very few implementations in +the kernel and a description of this technique is currently beyond the +scope of this document.] + +Data structures, callbacks, and initialization +---------------------------------------------- + +Depending on which type of buffers are being used, the driver should +include one of the following files: + +.. code-block:: none + + /* Physically scattered */ + /* vmalloc() buffers */ + /* Physically contiguous */ + +The driver's data structure describing a V4L2 device should include a +struct videobuf_queue instance for the management of the buffer queue, +along with a list_head for the queue of available buffers. There will also +need to be an interrupt-safe spinlock which is used to protect (at least) +the queue. + +The next step is to write four simple callbacks to help videobuf deal with +the management of buffers: + +.. code-block:: none + + struct videobuf_queue_ops { + int (*buf_setup)(struct videobuf_queue *q, + unsigned int *count, unsigned int *size); + int (*buf_prepare)(struct videobuf_queue *q, + struct videobuf_buffer *vb, + enum v4l2_field field); + void (*buf_queue)(struct videobuf_queue *q, + struct videobuf_buffer *vb); + void (*buf_release)(struct videobuf_queue *q, + struct videobuf_buffer *vb); + }; + +buf_setup() is called early in the I/O process, when streaming is being +initiated; its purpose is to tell videobuf about the I/O stream. The count +parameter will be a suggested number of buffers to use; the driver should +check it for rationality and adjust it if need be. As a practical rule, a +minimum of two buffers are needed for proper streaming, and there is +usually a maximum (which cannot exceed 32) which makes sense for each +device. The size parameter should be set to the expected (maximum) size +for each frame of data. + +Each buffer (in the form of a struct videobuf_buffer pointer) will be +passed to buf_prepare(), which should set the buffer's size, width, height, +and field fields properly. If the buffer's state field is +VIDEOBUF_NEEDS_INIT, the driver should pass it to: + +.. code-block:: none + + int videobuf_iolock(struct videobuf_queue* q, struct videobuf_buffer *vb, + struct v4l2_framebuffer *fbuf); + +Among other things, this call will usually allocate memory for the buffer. +Finally, the buf_prepare() function should set the buffer's state to +VIDEOBUF_PREPARED. + +When a buffer is queued for I/O, it is passed to buf_queue(), which should +put it onto the driver's list of available buffers and set its state to +VIDEOBUF_QUEUED. Note that this function is called with the queue spinlock +held; if it tries to acquire it as well things will come to a screeching +halt. Yes, this is the voice of experience. Note also that videobuf may +wait on the first buffer in the queue; placing other buffers in front of it +could again gum up the works. So use list_add_tail() to enqueue buffers. + +Finally, buf_release() is called when a buffer is no longer intended to be +used. The driver should ensure that there is no I/O active on the buffer, +then pass it to the appropriate free routine(s): + +.. code-block:: none + + /* Scatter/gather drivers */ + int videobuf_dma_unmap(struct videobuf_queue *q, + struct videobuf_dmabuf *dma); + int videobuf_dma_free(struct videobuf_dmabuf *dma); + + /* vmalloc drivers */ + void videobuf_vmalloc_free (struct videobuf_buffer *buf); + + /* Contiguous drivers */ + void videobuf_dma_contig_free(struct videobuf_queue *q, + struct videobuf_buffer *buf); + +One way to ensure that a buffer is no longer under I/O is to pass it to: + +.. code-block:: none + + int videobuf_waiton(struct videobuf_buffer *vb, int non_blocking, int intr); + +Here, vb is the buffer, non_blocking indicates whether non-blocking I/O +should be used (it should be zero in the buf_release() case), and intr +controls whether an interruptible wait is used. + +File operations +--------------- + +At this point, much of the work is done; much of the rest is slipping +videobuf calls into the implementation of the other driver callbacks. The +first step is in the open() function, which must initialize the +videobuf queue. The function to use depends on the type of buffer used: + +.. code-block:: none + + void videobuf_queue_sg_init(struct videobuf_queue *q, + struct videobuf_queue_ops *ops, + struct device *dev, + spinlock_t *irqlock, + enum v4l2_buf_type type, + enum v4l2_field field, + unsigned int msize, + void *priv); + + void videobuf_queue_vmalloc_init(struct videobuf_queue *q, + struct videobuf_queue_ops *ops, + struct device *dev, + spinlock_t *irqlock, + enum v4l2_buf_type type, + enum v4l2_field field, + unsigned int msize, + void *priv); + + void videobuf_queue_dma_contig_init(struct videobuf_queue *q, + struct videobuf_queue_ops *ops, + struct device *dev, + spinlock_t *irqlock, + enum v4l2_buf_type type, + enum v4l2_field field, + unsigned int msize, + void *priv); + +In each case, the parameters are the same: q is the queue structure for the +device, ops is the set of callbacks as described above, dev is the device +structure for this video device, irqlock is an interrupt-safe spinlock to +protect access to the data structures, type is the buffer type used by the +device (cameras will use V4L2_BUF_TYPE_VIDEO_CAPTURE, for example), field +describes which field is being captured (often V4L2_FIELD_NONE for +progressive devices), msize is the size of any containing structure used +around struct videobuf_buffer, and priv is a private data pointer which +shows up in the priv_data field of struct videobuf_queue. Note that these +are void functions which, evidently, are immune to failure. + +V4L2 capture drivers can be written to support either of two APIs: the +read() system call and the rather more complicated streaming mechanism. As +a general rule, it is necessary to support both to ensure that all +applications have a chance of working with the device. Videobuf makes it +easy to do that with the same code. To implement read(), the driver need +only make a call to one of: + +.. code-block:: none + + ssize_t videobuf_read_one(struct videobuf_queue *q, + char __user *data, size_t count, + loff_t *ppos, int nonblocking); + + ssize_t videobuf_read_stream(struct videobuf_queue *q, + char __user *data, size_t count, + loff_t *ppos, int vbihack, int nonblocking); + +Either one of these functions will read frame data into data, returning the +amount actually read; the difference is that videobuf_read_one() will only +read a single frame, while videobuf_read_stream() will read multiple frames +if they are needed to satisfy the count requested by the application. A +typical driver read() implementation will start the capture engine, call +one of the above functions, then stop the engine before returning (though a +smarter implementation might leave the engine running for a little while in +anticipation of another read() call happening in the near future). + +The poll() function can usually be implemented with a direct call to: + +.. code-block:: none + + unsigned int videobuf_poll_stream(struct file *file, + struct videobuf_queue *q, + poll_table *wait); + +Note that the actual wait queue eventually used will be the one associated +with the first available buffer. + +When streaming I/O is done to kernel-space buffers, the driver must support +the mmap() system call to enable user space to access the data. In many +V4L2 drivers, the often-complex mmap() implementation simplifies to a +single call to: + +.. code-block:: none + + int videobuf_mmap_mapper(struct videobuf_queue *q, + struct vm_area_struct *vma); + +Everything else is handled by the videobuf code. + +The release() function requires two separate videobuf calls: + +.. code-block:: none + + void videobuf_stop(struct videobuf_queue *q); + int videobuf_mmap_free(struct videobuf_queue *q); + +The call to videobuf_stop() terminates any I/O in progress - though it is +still up to the driver to stop the capture engine. The call to +videobuf_mmap_free() will ensure that all buffers have been unmapped; if +so, they will all be passed to the buf_release() callback. If buffers +remain mapped, videobuf_mmap_free() returns an error code instead. The +purpose is clearly to cause the closing of the file descriptor to fail if +buffers are still mapped, but every driver in the 2.6.32 kernel cheerfully +ignores its return value. + +ioctl() operations +------------------ + +The V4L2 API includes a very long list of driver callbacks to respond to +the many ioctl() commands made available to user space. A number of these +- those associated with streaming I/O - turn almost directly into videobuf +calls. The relevant helper functions are: + +.. code-block:: none + + int videobuf_reqbufs(struct videobuf_queue *q, + struct v4l2_requestbuffers *req); + int videobuf_querybuf(struct videobuf_queue *q, struct v4l2_buffer *b); + int videobuf_qbuf(struct videobuf_queue *q, struct v4l2_buffer *b); + int videobuf_dqbuf(struct videobuf_queue *q, struct v4l2_buffer *b, + int nonblocking); + int videobuf_streamon(struct videobuf_queue *q); + int videobuf_streamoff(struct videobuf_queue *q); + +So, for example, a VIDIOC_REQBUFS call turns into a call to the driver's +vidioc_reqbufs() callback which, in turn, usually only needs to locate the +proper struct videobuf_queue pointer and pass it to videobuf_reqbufs(). +These support functions can replace a great deal of buffer management +boilerplate in a lot of V4L2 drivers. + +The vidioc_streamon() and vidioc_streamoff() functions will be a bit more +complex, of course, since they will also need to deal with starting and +stopping the capture engine. + +Buffer allocation +----------------- + +Thus far, we have talked about buffers, but have not looked at how they are +allocated. The scatter/gather case is the most complex on this front. For +allocation, the driver can leave buffer allocation entirely up to the +videobuf layer; in this case, buffers will be allocated as anonymous +user-space pages and will be very scattered indeed. If the application is +using user-space buffers, no allocation is needed; the videobuf layer will +take care of calling get_user_pages() and filling in the scatterlist array. + +If the driver needs to do its own memory allocation, it should be done in +the vidioc_reqbufs() function, *after* calling videobuf_reqbufs(). The +first step is a call to: + +.. code-block:: none + + struct videobuf_dmabuf *videobuf_to_dma(struct videobuf_buffer *buf); + +The returned videobuf_dmabuf structure (defined in +) includes a couple of relevant fields: + +.. code-block:: none + + struct scatterlist *sglist; + int sglen; + +The driver must allocate an appropriately-sized scatterlist array and +populate it with pointers to the pieces of the allocated buffer; sglen +should be set to the length of the array. + +Drivers using the vmalloc() method need not (and cannot) concern themselves +with buffer allocation at all; videobuf will handle those details. The +same is normally true of contiguous-DMA drivers as well; videobuf will +allocate the buffers (with dma_alloc_coherent()) when it sees fit. That +means that these drivers may be trying to do high-order allocations at any +time, an operation which is not always guaranteed to work. Some drivers +play tricks by allocating DMA space at system boot time; videobuf does not +currently play well with those drivers. + +As of 2.6.31, contiguous-DMA drivers can work with a user-supplied buffer, +as long as that buffer is physically contiguous. Normal user-space +allocations will not meet that criterion, but buffers obtained from other +kernel drivers, or those contained within huge pages, will work with these +drivers. + +Filling the buffers +------------------- + +The final part of a videobuf implementation has no direct callback - it's +the portion of the code which actually puts frame data into the buffers, +usually in response to interrupts from the device. For all types of +drivers, this process works approximately as follows: + + - Obtain the next available buffer and make sure that somebody is actually + waiting for it. + + - Get a pointer to the memory and put video data there. + + - Mark the buffer as done and wake up the process waiting for it. + +Step (1) above is done by looking at the driver-managed list_head structure +- the one which is filled in the buf_queue() callback. Because starting +the engine and enqueueing buffers are done in separate steps, it's possible +for the engine to be running without any buffers available - in the +vmalloc() case especially. So the driver should be prepared for the list +to be empty. It is equally possible that nobody is yet interested in the +buffer; the driver should not remove it from the list or fill it until a +process is waiting on it. That test can be done by examining the buffer's +done field (a wait_queue_head_t structure) with waitqueue_active(). + +A buffer's state should be set to VIDEOBUF_ACTIVE before being mapped for +DMA; that ensures that the videobuf layer will not try to do anything with +it while the device is transferring data. + +For scatter/gather drivers, the needed memory pointers will be found in the +scatterlist structure described above. Drivers using the vmalloc() method +can get a memory pointer with: + +.. code-block:: none + + void *videobuf_to_vmalloc(struct videobuf_buffer *buf); + +For contiguous DMA drivers, the function to use is: + +.. code-block:: none + + dma_addr_t videobuf_to_dma_contig(struct videobuf_buffer *buf); + +The contiguous DMA API goes out of its way to hide the kernel-space address +of the DMA buffer from drivers. + +The final step is to set the size field of the relevant videobuf_buffer +structure to the actual size of the captured image, set state to +VIDEOBUF_DONE, then call wake_up() on the done queue. At this point, the +buffer is owned by the videobuf layer and the driver should not touch it +again. + +Developers who are interested in more information can go into the relevant +header files; there are a few low-level functions declared there which have +not been talked about here. Also worthwhile is the vivi driver +(drivers/media/platform/vivi.c), which is maintained as an example of how V4L2 +drivers should be written. Vivi only uses the vmalloc() API, but it's good +enough to get started with. Note also that all of these calls are exported +GPL-only, so they will not be available to non-GPL kernel modules. diff --git a/Documentation/media/kapi/v4l2-videobuf2.rst b/Documentation/media/kapi/v4l2-videobuf2.rst new file mode 100644 index 000000000000..b4f2d6983ef3 --- /dev/null +++ b/Documentation/media/kapi/v4l2-videobuf2.rst @@ -0,0 +1,8 @@ +V4L2 videobuf2 kAPI +^^^^^^^^^^^^^^^^^^^ + +.. kernel-doc:: include/media/videobuf2-core.h + +.. kernel-doc:: include/media/videobuf2-v4l2.h + +.. kernel-doc:: include/media/videobuf2-memops.h diff --git a/Documentation/media/kapi/videobuf.rst b/Documentation/media/kapi/videobuf.rst deleted file mode 100644 index 01156728203c..000000000000 --- a/Documentation/media/kapi/videobuf.rst +++ /dev/null @@ -1,402 +0,0 @@ -Videobuf Framework -================== - -Author: Jonathan Corbet - -Current as of 2.6.33 - -.. note:: - - The videobuf framework was deprecated in favor of videobuf2. Shouldn't - be used on new drivers. - -Introduction ------------- - -The videobuf layer functions as a sort of glue layer between a V4L2 driver -and user space. It handles the allocation and management of buffers for -the storage of video frames. There is a set of functions which can be used -to implement many of the standard POSIX I/O system calls, including read(), -poll(), and, happily, mmap(). Another set of functions can be used to -implement the bulk of the V4L2 ioctl() calls related to streaming I/O, -including buffer allocation, queueing and dequeueing, and streaming -control. Using videobuf imposes a few design decisions on the driver -author, but the payback comes in the form of reduced code in the driver and -a consistent implementation of the V4L2 user-space API. - -Buffer types ------------- - -Not all video devices use the same kind of buffers. In fact, there are (at -least) three common variations: - - - Buffers which are scattered in both the physical and (kernel) virtual - address spaces. (Almost) all user-space buffers are like this, but it - makes great sense to allocate kernel-space buffers this way as well when - it is possible. Unfortunately, it is not always possible; working with - this kind of buffer normally requires hardware which can do - scatter/gather DMA operations. - - - Buffers which are physically scattered, but which are virtually - contiguous; buffers allocated with vmalloc(), in other words. These - buffers are just as hard to use for DMA operations, but they can be - useful in situations where DMA is not available but virtually-contiguous - buffers are convenient. - - - Buffers which are physically contiguous. Allocation of this kind of - buffer can be unreliable on fragmented systems, but simpler DMA - controllers cannot deal with anything else. - -Videobuf can work with all three types of buffers, but the driver author -must pick one at the outset and design the driver around that decision. - -[It's worth noting that there's a fourth kind of buffer: "overlay" buffers -which are located within the system's video memory. The overlay -functionality is considered to be deprecated for most use, but it still -shows up occasionally in system-on-chip drivers where the performance -benefits merit the use of this technique. Overlay buffers can be handled -as a form of scattered buffer, but there are very few implementations in -the kernel and a description of this technique is currently beyond the -scope of this document.] - -Data structures, callbacks, and initialization ----------------------------------------------- - -Depending on which type of buffers are being used, the driver should -include one of the following files: - -.. code-block:: none - - /* Physically scattered */ - /* vmalloc() buffers */ - /* Physically contiguous */ - -The driver's data structure describing a V4L2 device should include a -struct videobuf_queue instance for the management of the buffer queue, -along with a list_head for the queue of available buffers. There will also -need to be an interrupt-safe spinlock which is used to protect (at least) -the queue. - -The next step is to write four simple callbacks to help videobuf deal with -the management of buffers: - -.. code-block:: none - - struct videobuf_queue_ops { - int (*buf_setup)(struct videobuf_queue *q, - unsigned int *count, unsigned int *size); - int (*buf_prepare)(struct videobuf_queue *q, - struct videobuf_buffer *vb, - enum v4l2_field field); - void (*buf_queue)(struct videobuf_queue *q, - struct videobuf_buffer *vb); - void (*buf_release)(struct videobuf_queue *q, - struct videobuf_buffer *vb); - }; - -buf_setup() is called early in the I/O process, when streaming is being -initiated; its purpose is to tell videobuf about the I/O stream. The count -parameter will be a suggested number of buffers to use; the driver should -check it for rationality and adjust it if need be. As a practical rule, a -minimum of two buffers are needed for proper streaming, and there is -usually a maximum (which cannot exceed 32) which makes sense for each -device. The size parameter should be set to the expected (maximum) size -for each frame of data. - -Each buffer (in the form of a struct videobuf_buffer pointer) will be -passed to buf_prepare(), which should set the buffer's size, width, height, -and field fields properly. If the buffer's state field is -VIDEOBUF_NEEDS_INIT, the driver should pass it to: - -.. code-block:: none - - int videobuf_iolock(struct videobuf_queue* q, struct videobuf_buffer *vb, - struct v4l2_framebuffer *fbuf); - -Among other things, this call will usually allocate memory for the buffer. -Finally, the buf_prepare() function should set the buffer's state to -VIDEOBUF_PREPARED. - -When a buffer is queued for I/O, it is passed to buf_queue(), which should -put it onto the driver's list of available buffers and set its state to -VIDEOBUF_QUEUED. Note that this function is called with the queue spinlock -held; if it tries to acquire it as well things will come to a screeching -halt. Yes, this is the voice of experience. Note also that videobuf may -wait on the first buffer in the queue; placing other buffers in front of it -could again gum up the works. So use list_add_tail() to enqueue buffers. - -Finally, buf_release() is called when a buffer is no longer intended to be -used. The driver should ensure that there is no I/O active on the buffer, -then pass it to the appropriate free routine(s): - -.. code-block:: none - - /* Scatter/gather drivers */ - int videobuf_dma_unmap(struct videobuf_queue *q, - struct videobuf_dmabuf *dma); - int videobuf_dma_free(struct videobuf_dmabuf *dma); - - /* vmalloc drivers */ - void videobuf_vmalloc_free (struct videobuf_buffer *buf); - - /* Contiguous drivers */ - void videobuf_dma_contig_free(struct videobuf_queue *q, - struct videobuf_buffer *buf); - -One way to ensure that a buffer is no longer under I/O is to pass it to: - -.. code-block:: none - - int videobuf_waiton(struct videobuf_buffer *vb, int non_blocking, int intr); - -Here, vb is the buffer, non_blocking indicates whether non-blocking I/O -should be used (it should be zero in the buf_release() case), and intr -controls whether an interruptible wait is used. - -File operations ---------------- - -At this point, much of the work is done; much of the rest is slipping -videobuf calls into the implementation of the other driver callbacks. The -first step is in the open() function, which must initialize the -videobuf queue. The function to use depends on the type of buffer used: - -.. code-block:: none - - void videobuf_queue_sg_init(struct videobuf_queue *q, - struct videobuf_queue_ops *ops, - struct device *dev, - spinlock_t *irqlock, - enum v4l2_buf_type type, - enum v4l2_field field, - unsigned int msize, - void *priv); - - void videobuf_queue_vmalloc_init(struct videobuf_queue *q, - struct videobuf_queue_ops *ops, - struct device *dev, - spinlock_t *irqlock, - enum v4l2_buf_type type, - enum v4l2_field field, - unsigned int msize, - void *priv); - - void videobuf_queue_dma_contig_init(struct videobuf_queue *q, - struct videobuf_queue_ops *ops, - struct device *dev, - spinlock_t *irqlock, - enum v4l2_buf_type type, - enum v4l2_field field, - unsigned int msize, - void *priv); - -In each case, the parameters are the same: q is the queue structure for the -device, ops is the set of callbacks as described above, dev is the device -structure for this video device, irqlock is an interrupt-safe spinlock to -protect access to the data structures, type is the buffer type used by the -device (cameras will use V4L2_BUF_TYPE_VIDEO_CAPTURE, for example), field -describes which field is being captured (often V4L2_FIELD_NONE for -progressive devices), msize is the size of any containing structure used -around struct videobuf_buffer, and priv is a private data pointer which -shows up in the priv_data field of struct videobuf_queue. Note that these -are void functions which, evidently, are immune to failure. - -V4L2 capture drivers can be written to support either of two APIs: the -read() system call and the rather more complicated streaming mechanism. As -a general rule, it is necessary to support both to ensure that all -applications have a chance of working with the device. Videobuf makes it -easy to do that with the same code. To implement read(), the driver need -only make a call to one of: - -.. code-block:: none - - ssize_t videobuf_read_one(struct videobuf_queue *q, - char __user *data, size_t count, - loff_t *ppos, int nonblocking); - - ssize_t videobuf_read_stream(struct videobuf_queue *q, - char __user *data, size_t count, - loff_t *ppos, int vbihack, int nonblocking); - -Either one of these functions will read frame data into data, returning the -amount actually read; the difference is that videobuf_read_one() will only -read a single frame, while videobuf_read_stream() will read multiple frames -if they are needed to satisfy the count requested by the application. A -typical driver read() implementation will start the capture engine, call -one of the above functions, then stop the engine before returning (though a -smarter implementation might leave the engine running for a little while in -anticipation of another read() call happening in the near future). - -The poll() function can usually be implemented with a direct call to: - -.. code-block:: none - - unsigned int videobuf_poll_stream(struct file *file, - struct videobuf_queue *q, - poll_table *wait); - -Note that the actual wait queue eventually used will be the one associated -with the first available buffer. - -When streaming I/O is done to kernel-space buffers, the driver must support -the mmap() system call to enable user space to access the data. In many -V4L2 drivers, the often-complex mmap() implementation simplifies to a -single call to: - -.. code-block:: none - - int videobuf_mmap_mapper(struct videobuf_queue *q, - struct vm_area_struct *vma); - -Everything else is handled by the videobuf code. - -The release() function requires two separate videobuf calls: - -.. code-block:: none - - void videobuf_stop(struct videobuf_queue *q); - int videobuf_mmap_free(struct videobuf_queue *q); - -The call to videobuf_stop() terminates any I/O in progress - though it is -still up to the driver to stop the capture engine. The call to -videobuf_mmap_free() will ensure that all buffers have been unmapped; if -so, they will all be passed to the buf_release() callback. If buffers -remain mapped, videobuf_mmap_free() returns an error code instead. The -purpose is clearly to cause the closing of the file descriptor to fail if -buffers are still mapped, but every driver in the 2.6.32 kernel cheerfully -ignores its return value. - -ioctl() operations ------------------- - -The V4L2 API includes a very long list of driver callbacks to respond to -the many ioctl() commands made available to user space. A number of these -- those associated with streaming I/O - turn almost directly into videobuf -calls. The relevant helper functions are: - -.. code-block:: none - - int videobuf_reqbufs(struct videobuf_queue *q, - struct v4l2_requestbuffers *req); - int videobuf_querybuf(struct videobuf_queue *q, struct v4l2_buffer *b); - int videobuf_qbuf(struct videobuf_queue *q, struct v4l2_buffer *b); - int videobuf_dqbuf(struct videobuf_queue *q, struct v4l2_buffer *b, - int nonblocking); - int videobuf_streamon(struct videobuf_queue *q); - int videobuf_streamoff(struct videobuf_queue *q); - -So, for example, a VIDIOC_REQBUFS call turns into a call to the driver's -vidioc_reqbufs() callback which, in turn, usually only needs to locate the -proper struct videobuf_queue pointer and pass it to videobuf_reqbufs(). -These support functions can replace a great deal of buffer management -boilerplate in a lot of V4L2 drivers. - -The vidioc_streamon() and vidioc_streamoff() functions will be a bit more -complex, of course, since they will also need to deal with starting and -stopping the capture engine. - -Buffer allocation ------------------ - -Thus far, we have talked about buffers, but have not looked at how they are -allocated. The scatter/gather case is the most complex on this front. For -allocation, the driver can leave buffer allocation entirely up to the -videobuf layer; in this case, buffers will be allocated as anonymous -user-space pages and will be very scattered indeed. If the application is -using user-space buffers, no allocation is needed; the videobuf layer will -take care of calling get_user_pages() and filling in the scatterlist array. - -If the driver needs to do its own memory allocation, it should be done in -the vidioc_reqbufs() function, *after* calling videobuf_reqbufs(). The -first step is a call to: - -.. code-block:: none - - struct videobuf_dmabuf *videobuf_to_dma(struct videobuf_buffer *buf); - -The returned videobuf_dmabuf structure (defined in -) includes a couple of relevant fields: - -.. code-block:: none - - struct scatterlist *sglist; - int sglen; - -The driver must allocate an appropriately-sized scatterlist array and -populate it with pointers to the pieces of the allocated buffer; sglen -should be set to the length of the array. - -Drivers using the vmalloc() method need not (and cannot) concern themselves -with buffer allocation at all; videobuf will handle those details. The -same is normally true of contiguous-DMA drivers as well; videobuf will -allocate the buffers (with dma_alloc_coherent()) when it sees fit. That -means that these drivers may be trying to do high-order allocations at any -time, an operation which is not always guaranteed to work. Some drivers -play tricks by allocating DMA space at system boot time; videobuf does not -currently play well with those drivers. - -As of 2.6.31, contiguous-DMA drivers can work with a user-supplied buffer, -as long as that buffer is physically contiguous. Normal user-space -allocations will not meet that criterion, but buffers obtained from other -kernel drivers, or those contained within huge pages, will work with these -drivers. - -Filling the buffers -------------------- - -The final part of a videobuf implementation has no direct callback - it's -the portion of the code which actually puts frame data into the buffers, -usually in response to interrupts from the device. For all types of -drivers, this process works approximately as follows: - - - Obtain the next available buffer and make sure that somebody is actually - waiting for it. - - - Get a pointer to the memory and put video data there. - - - Mark the buffer as done and wake up the process waiting for it. - -Step (1) above is done by looking at the driver-managed list_head structure -- the one which is filled in the buf_queue() callback. Because starting -the engine and enqueueing buffers are done in separate steps, it's possible -for the engine to be running without any buffers available - in the -vmalloc() case especially. So the driver should be prepared for the list -to be empty. It is equally possible that nobody is yet interested in the -buffer; the driver should not remove it from the list or fill it until a -process is waiting on it. That test can be done by examining the buffer's -done field (a wait_queue_head_t structure) with waitqueue_active(). - -A buffer's state should be set to VIDEOBUF_ACTIVE before being mapped for -DMA; that ensures that the videobuf layer will not try to do anything with -it while the device is transferring data. - -For scatter/gather drivers, the needed memory pointers will be found in the -scatterlist structure described above. Drivers using the vmalloc() method -can get a memory pointer with: - -.. code-block:: none - - void *videobuf_to_vmalloc(struct videobuf_buffer *buf); - -For contiguous DMA drivers, the function to use is: - -.. code-block:: none - - dma_addr_t videobuf_to_dma_contig(struct videobuf_buffer *buf); - -The contiguous DMA API goes out of its way to hide the kernel-space address -of the DMA buffer from drivers. - -The final step is to set the size field of the relevant videobuf_buffer -structure to the actual size of the captured image, set state to -VIDEOBUF_DONE, then call wake_up() on the done queue. At this point, the -buffer is owned by the videobuf layer and the driver should not touch it -again. - -Developers who are interested in more information can go into the relevant -header files; there are a few low-level functions declared there which have -not been talked about here. Also worthwhile is the vivi driver -(drivers/media/platform/vivi.c), which is maintained as an example of how V4L2 -drivers should be written. Vivi only uses the vmalloc() API, but it's good -enough to get started with. Note also that all of these calls are exported -GPL-only, so they will not be available to non-GPL kernel modules. diff --git a/Documentation/media/media_kapi.rst b/Documentation/media/media_kapi.rst index 431fc3e43d6a..b71e8e8048ca 100644 --- a/Documentation/media/media_kapi.rst +++ b/Documentation/media/media_kapi.rst @@ -28,9 +28,6 @@ For more details see the file COPYING in the source distribution of Linux. :maxdepth: 5 :numbered: - kapi/v4l2-framework - kapi/v4l2-controls - kapi/videobuf kapi/v4l2-core kapi/dtv-core kapi/rc-core