Decrease libv4l capture latency (Patch #632)


Added by Mitar M over 6 years ago. Updated almost 4 years ago.


Status:Open Start date:
Priority:Low Due date:
Assignee:Evgeny Talanin % Done:

0%

Category:highgui-camera
Target version:Next Hackathon
Affected version:branch 'master' (2.4.9) Operating System:
Difficulty: HW Platform:
Pull request:

Description

I am attaching three patches which decreases libv4l capture latency by few 10 ms for large (2592x1944) captures and even more if processing of captures takes more than one frame time.

The point is in reducing memory copying around (012 and 013 patch). Probably this (as read from comments) decreases stability for some cameras but it worked well for me Logitech HD Pro Webcam C910 and Logitech QuickCam Pro 9000. So those two patches remove any copying and push the buffer directly to user.

In the case that processing of captures takes more than one frame time latency is decreased because when requesting new capture queue of buffers is emptied in FIFO fashion. This helps decreasing frame loss if processing was just temporarily longer than one frame time but it introduces additional latency of length of queue length - 1 * one frame time. The point is that if processing is always longer than frame time buffer queue is always full and just last (the one filled most in the past) is dequeued and used.

One solution would be to check how many frames are already in captured queue and pop all them and return only the last when requesting new capture. But much easier way is simply to decrease number of buffers to 2. So if processing takes longer old frames are dropped and just new ones are available.

In this way latency between event in real world and finished processing and capturing that event in computer is minimized. On the expense of more frame drops. But it is questionable how much does too much delayed frames help you in real-time application anyway. And if processing takes to long frames will be dropped anyway. With decreasing number of buffers in queue we just select which one would those be. But yes, we also drop somewhat more frames to keep really just the most recent ones available.


012-no-frame-copy.patch (405 Bytes) Mitar M, 2010-10-27 01:28 am

013-more-no-frame-copy.patch (3.4 kB) Mitar M, 2010-10-27 01:29 am

014-realtime.patch (490 Bytes) Mitar M, 2010-10-27 01:29 am

latency.patch (1.8 kB) Pieter Eendebak, 2012-05-08 11:14 pm

latency-patch.txt (1.6 kB) Pieter Eendebak, 2012-06-10 12:18 pm

patch.txt (4.3 kB) Pieter Eendebak, 2012-11-12 02:44 pm


History

Updated by Mitar M over 6 years ago

I also made a patch for improving libv4l MJPEG decoding to decrease its delay.

Updated by Alexander Shishkov about 5 years ago

  • Description changed from I am attaching three patches which decreases libv4l capture latency by few 10... to I am attaching three patches which decreases libv4l capture latency by few 10... More

Updated by Alexander Shishkov almost 5 years ago

  • Tracker changed from Feature to Patch
  • Priority changed from High to Normal
  • Target version deleted ()

Updated by Alexander Shishkov almost 5 years ago

  • Priority changed from Normal to Low

Updated by Alexander Shishkov almost 5 years ago

  • Target version deleted ()

Updated by Pieter Eendebak almost 5 years ago

Attached is the same patch, updated to OpenCV-2.4.0. Is there a possibility to include (part of) this patch by default?

  • File latency.patch added

Updated by Pieter Eendebak over 4 years ago

Update from the same patch

  • File latency-patch.txt added

Updated by Andrey Kamaev over 4 years ago

  • Category changed from highgui-images to highgui-camera

Updated by Vadim Pisarevsky over 4 years ago

  • Target version deleted ()

Updated by Evgeny Talanin over 4 years ago

Pieter, can you provide a new patch generated with git diff command?
It's unclear which revision should this patch be applied to

  • Assignee set to Evgeny Talanin

Updated by Kirill Kornyakov over 4 years ago

  • Target version set to Next Hackathon

Updated by Pieter Eendebak over 4 years ago

Dear Evgeny,

Attached you find the patch generated with 'git diff'. The patch is against the current git branch:

commit 4d059e9e5bc4aa4d7532a95037bb38ab3bdbef43

Updated by Kirill Kornyakov about 4 years ago

Pieter, it could last forever... I suggest you to make a pull request at the GitHub, this will allow automatic testing and simplify the reviewing process.

  • Affected version set to branch 'master' (2.4.9)

Updated by Pieter Eendebak almost 4 years ago

Ok, pull request was made:

https://github.com/eendebakpt/opencv/commit/f8f5b823a1c655642be80b7f7589d0cea3229683

Perhaps it can also be pushed to the 2.4 branch

Updated by Alexander Smorkalov almost 4 years ago

You need to create pull request to main OpenCV repository. Select your branch on Github project page and press "pull request" button on the top of the page. It will be automatically tested by Buildbot and proposed on code review. Also I recommend you to rebase it on 2.4 branch and make pull request to branch 2.4. All releases in 2.4.x series are prepared from it and your changes will be published in the next release. There is how-to page, I think it will be useful for you.

Also available in: Atom PDF