Connecting to Point Grey chameleon camera (Bug #2102)
Description
Hello,
we use Point Grey Chameleon camera connected via USB 2.0. This camera supports its own API, however, for compatibility, it supports DirectShow too. We would like to use this DirectShow interface to be independent on used camera. Problem is that OpenCV is not able to operate with it and crashes when cvCreateCameraCapture is called.
I'm providing the basic info which I found by debugging our application and OpenCV code.
- The crash is caused by negative image height reported in VIDEOINFOHEADER->bmiHeader (cap_dshow.cpp, int videoInput::start(int deviceID, videoDevice *VD))
The MSDN documentation (http://msdn.microsoft.com/en-us/library/windows/desktop/dd318229(v=vs.85).aspx) says that negative biHeight is allowed but OpenCV is not able to deal with it and tries to allocate memory block with negative size. This results in the crash.
- If I modify the code to use absolute value of biHeight than the connection to camera is created correctly.
- The camera should always return 8bit image only (with Bayer encoding), but OpenCV returns 24bit image which is totally distorted and flipped vertically. Normally, when using PointGrey API, it is enough to call cvCvtColor(image, result, CV_BayerGB2RGB) on captured image and it gives the correct image. But it does not work (and crashes) with OpenCV as Bayer->RGB conversion is 8bit->24bit. This does not necessarily be a bug, maybe only some other conversion is required (simple CV_BGR2GRAY does not work). But I am not sure that's why I'm reporting it too.
Could somebody help me with this problem? If it is needed, I will try to provide more information. Just ask.
Thanks in advance.
Associated revisions
Merge pull request #2102 from Nerei:compilatin_for_windows
History
Updated by Andrey Kamaev over 12 years ago
- Category set to highgui-images
Updated by Joel Mckay over 12 years ago
It is usually helpful to include a code snippet for the devs to reproduce the problem. However, that company provides its own "free" software framework for it's very expensive proprietary hardware.
You may wish to consider writing your own driver wrapper in the commercial framework, and link its interface into your opencv code with a conditional check for the camera hardware.
Good luck,
J
Updated by Tomas Kohlschutter over 12 years ago
Yes, they provide their own SDK to work with the camera. However, our requirement is to use DirectShow interface only and not to include any other DLL/SDK with our application. And since PointGrey Chameleon camera supports DirectShow, it should work.
I don't think it is necessary to provide any code snipet to this problem, because it can be easily tested on any code using OpenCV. Just call cvCreateCameraCapture and it crashes immedietaly.
Updated by Andrey Kamaev over 12 years ago
- Category changed from highgui-images to highgui-camera
Updated by Ben Buckner over 12 years ago
I've experienced this same problem with using the Point Grey DirectShow driver. Since the negative is a valid and meaningful value with uncompressed RGB bitmaps, I think it has to be supported.
This (admittedly trivial) code is below is sufficient to duplicate the error (console mode VC++ 2008, tested with OCV 2.3.1) with a Grasshopper 2.0 mono GigE camera using the DirectShow driver under Win 7 64 bit but running all 32 bit apps. Point Grey support seems to be unaware of the issue, but I just explained it to them. They usually test the driver with MS GraphEdit, which apparently has no problem with the negative height, but the latest version of vlc shows what looks like the same problem.
#include "stdafx.h" #include "opencv/cv.h" #include "opencv/highgui.h" #include "stdlib.h" int _tmain(int argc, _TCHAR* argv[]) { CvCapture* capture = cvCaptureFromCAM(0); IplImage* image = 0; cvNamedWindow("Window"); int count = 1; while (true) { image = cvQueryFrame(capture); cvShowImage("Window", image); int ch = cvWaitKey(1); if(ch == 27) break; } cvDestroyWindow("Window"); cvReleaseCapture(&capture); return 0; }
Thomas: To handle it correctly with uncompressed RGB, you have to invert the interpretation of the height coordinate. With YUV, you can just do the abs() though apparently. Given that OpenCV generally seems to leave it up to the user to use CV_CVTIMG_FLIP when necessary though, I suppose it makes sense to just abs() it in all cases and leave the interpretation up to the user.
Thanks for pointing it out - I've been beating on this for a day or two now.
Updated by Ben Buckner over 12 years ago
Oh, regarding the color conversion problem, the inverted vertical coordinate may be part of the issue there as well. The Bayer pattern is not usually symmetric relative to a flip (for the usual case of even valued heights), so if you use the inverted CV_BayerGB2RGB->CV_BayerRG2RGB that might help. The Point Grey DS driver also returns an RGB32 (according to GraphEdit), so if something's interpreting that as RGB24, that would explain your warping. It looks like OpenCV should recognize the RGB32 subtype though.
Updated by Ben Buckner over 12 years ago
I think I found what causes the "warping".
(current 2.5 code from the repository)
//MEDIA CONVERSION //Get video properties from the stream's mediatype and apply to the grabber (otherwise we don't get an RGB image) //zero the media type - lets try this :) - maybe this works? AM_MEDIA_TYPE mt; ZeroMemory(&mt,sizeof(AM_MEDIA_TYPE)); mt.majortype = MEDIATYPE_Video; mt.subtype = MEDIASUBTYPE_RGB24; mt.formattype = FORMAT_VideoInfo; //VD->pAmMediaType->subtype = VD->videoType; hr = VD->pGrabber->SetMediaType(&mt);
As I understand it, what this does is to create a RGB24 grabber and (later) attempt to connect it to the random-typed source. DirectShow is supposed to fail if the two pins can't agree on a data type, so at some point one of the filters in the chain is apparently agreeing on a data type but not following through with the conversion. It's possible that the Point Grey driver is doing that, which might be why it fails on vlc too, which presumably uses different code. I can understand why this would have trouble with OpenCV, in which the video capture support isn't so much core functionality, but it's definitely core in vlc. We might be seeing two different issues - the first is that OpenCV doesn't handle the negative height correctly, as Thomas found, but then after you fix that the PG filter might be agreeing to a data type and then not following through with it. Based on the docs for SampleGrabber (which incidentally has been deprecated), it looks to me like SampleGrabber is set up to only accept RGB24.
Updated by Vadim Pisarevsky almost 10 years ago
let's cancel very old bug in a hope that they are not relevant anymore; please, reopen if they are still relevant
- Status changed from Open to Cancelled
- HW Platform set to Any
- Operating System set to Any
- Affected version set to 2.4.0 - 2.4.8