Updated by Alexander Shishkov about 13 years ago
I am attaching three patches which decreases libv4l capture latency by few 10 ms for large (2592x1944) captures and even more if processing of captures takes more than one frame time.
The point is in reducing memory copying around (012 and 013 patch). Probably this (as read from comments) decreases stability for some cameras but it worked well for me Logitech HD Pro Webcam C910 and Logitech QuickCam [[QuickCam]] Pro 9000. So those two patches remove any copying and push the buffer directly to user.
In the case that processing of captures takes more than one frame time latency is decreased because when requesting new capture queue of buffers is emptied in FIFO fashion. This helps decreasing frame loss if processing was just temporarily longer than one frame time but it introduces additional latency of @length of queue length - 1 * one frame time@. The point is that if processing is always longer than frame time buffer queue is always full and just last (the one filled most in the past) is dequeued and used.
One solution would be to check how many frames are already in captured queue and pop all them and return only the last when requesting new capture. But much easier way is simply to decrease number of buffers to 2. So if processing takes longer old frames are dropped and just new ones are available.
In this way latency between event in real world and finished processing and capturing that event in computer is minimized. On the expense of more frame drops. But it is questionable how much does too much delayed frames help you in real-time application anyway. And if processing takes to long frames will be dropped anyway. With decreasing number of buffers in queue we just select which one would those be. But yes, we also drop somewhat more frames to keep really just the most recent ones available.
The point is in reducing memory copying around (012 and 013 patch). Probably this (as read from comments) decreases stability for some cameras but it worked well for me Logitech HD Pro Webcam C910 and Logitech QuickCam [[QuickCam]] Pro 9000. So those two patches remove any copying and push the buffer directly to user.
In the case that processing of captures takes more than one frame time latency is decreased because when requesting new capture queue of buffers is emptied in FIFO fashion. This helps decreasing frame loss if processing was just temporarily longer than one frame time but it introduces additional latency of @length of queue length - 1 * one frame time@. The point is that if processing is always longer than frame time buffer queue is always full and just last (the one filled most in the past) is dequeued and used.
One solution would be to check how many frames are already in captured queue and pop all them and return only the last when requesting new capture. But much easier way is simply to decrease number of buffers to 2. So if processing takes longer old frames are dropped and just new ones are available.
In this way latency between event in real world and finished processing and capturing that event in computer is minimized. On the expense of more frame drops. But it is questionable how much does too much delayed frames help you in real-time application anyway. And if processing takes to long frames will be dropped anyway. With decreasing number of buffers in queue we just select which one would those be. But yes, we also drop somewhat more frames to keep really just the most recent ones available.