Using SIFT with Mask (Bugfix #2892)
Description
I'd like to report a bug in the SIFT detector implementation of version 2.4.4 (possibly others too). When using a mask, I get SegFaults. The reason is, that the check, if keypoints are inside the mask is done on the scaled keypoint variants, given by the ScaleSpace and the original mask. This check has to be done after "backscaling" the keypoints. The problem is given in the operator():
if( !mask.empty() ) KeyPointsFilter::runByPixelsMask( keypoints, mask ); if( nfeatures > 0 ) KeyPointsFilter::retainBest(keypoints, nfeatures); //t = (double)getTickCount() - t; //printf("keypoint detection time: %g\n", t*1000./tf); if( firstOctave < 0 ) for( size_t i = 0; i < keypoints.size(); i++ ) { KeyPoint& kpt = keypoints[i]; float scale = 1.f/(float)(1 << -firstOctave); kpt.octave = (kpt.octave & ~255) | ((kpt.octave + firstOctave) & 255); kpt.pt *= scale; kpt.size *= scale; }
Simply doing the "is Mask empty" check after rescaling works for me.
if( nfeatures > 0 ) KeyPointsFilter::retainBest(keypoints, nfeatures); //t = (double)getTickCount() - t; //printf("keypoint detection time: %g\n", t*1000./tf); if( firstOctave < 0 ) for( size_t i = 0; i < keypoints.size(); i++ ) { KeyPoint& kpt = keypoints[i]; float scale = 1.f/(float)(1 << -firstOctave); kpt.octave = (kpt.octave & ~255) | ((kpt.octave + firstOctave) & 255); kpt.pt *= scale; kpt.size *= scale; } if( !mask.empty() ) KeyPointsFilter::runByPixelsMask( keypoints, mask );
It would be more efficent to just search in the given mask for regions of interest, though. Maybe this problem can be fixed in the next version? I'm new to the issue tracker and hopefully this post helps.
Associated revisions
Fixed a bug #2892
Added regression tests for SURF/SIFT (related to #2892)
Merge pull request #2892 from PhilLab:patch-1
History
Updated by Alexander Smorkalov almost 12 years ago
What is your OS and compiler? If you use Visual Studio, write run time library type.
Updated by Philipp Paier almost 12 years ago
Win 7, VisStudio 2008 Express.
But this should happen independent of the system, as this seems to be an error in the algorithm itself in sift::operator(). Code like the following should fail almost anytime (except if keypoints have small characteristic scale, which then would result in incorrect use of the mask) with any mask, because at some point the algorithm tries to access a pixel in the mask but uses the scaled keypoint coordinates. Thats what I found out during debugging.
#include <vector> #include <opencv2/highgui/highgui.hpp> #include <opencv2/nonfree/features2d.hpp> #include <opencv2/nonfree/nonfree.hpp> ... int main(int argc, char *argv[]) { cv::Mat img = imread("someImage.jpg", -1); cv::Mat mask(img.size(),CV_8U, cv::Scalar(255)); cv::initModule_nonfree(); cv::Ptr<cv::FeatureDetector> detector = cv::FeatureDetector::create("SIFT"); std::vector<cv::KeyPoint> keypoints; detector->detect(img,keypoints,mask); // this leads to a segfault }
Updated by Andrey Kamaev almost 12 years ago
- Target version changed from 2.4.5 to Next Hackathon
Updated by Ivan Korolev over 11 years ago
- Assignee changed from Vadim Pisarevsky to Ivan Korolev
- Affected version changed from 2.4.4 to 2.4.0 - 2.4.4
Updated by Ivan Korolev over 11 years ago
- Pull request set to https://github.com/Itseez/opencv/pull/1000
Updated by Andrey Pavlenko over 11 years ago
- Target version changed from Next Hackathon to 2.4.6
- HW Platform set to Other
- Operating System set to Any
- Start date deleted (
2013-03-16)