Using SIFT with Mask (Bugfix #2892)


Added by Philipp Paier almost 12 years ago. Updated over 11 years ago.


Status:Done Start date:
Priority:Normal Due date:
Assignee:Ivan Korolev % Done:

0%

Category:nonfree
Target version:2.4.6
Affected version:2.4.0 - 2.4.4 Operating System:Any
Difficulty: HW Platform:Other
Pull request:https://github.com/Itseez/opencv/pull/1000

Description

I'd like to report a bug in the SIFT detector implementation of version 2.4.4 (possibly others too). When using a mask, I get SegFaults. The reason is, that the check, if keypoints are inside the mask is done on the scaled keypoint variants, given by the ScaleSpace and the original mask. This check has to be done after "backscaling" the keypoints. The problem is given in the operator():

    if( !mask.empty() )
        KeyPointsFilter::runByPixelsMask( keypoints, mask );

    if( nfeatures > 0 )
        KeyPointsFilter::retainBest(keypoints, nfeatures);
    //t = (double)getTickCount() - t;
    //printf("keypoint detection time: %g\n", t*1000./tf);

    if( firstOctave < 0 )
        for( size_t i = 0; i < keypoints.size(); i++ )
        {
            KeyPoint& kpt = keypoints[i];
            float scale = 1.f/(float)(1 << -firstOctave);
            kpt.octave = (kpt.octave & ~255) | ((kpt.octave + firstOctave) & 255);
            kpt.pt *= scale;
            kpt.size *= scale;
        }


Simply doing the "is Mask empty" check after rescaling works for me.
    if( nfeatures > 0 )
        KeyPointsFilter::retainBest(keypoints, nfeatures);
    //t = (double)getTickCount() - t;
    //printf("keypoint detection time: %g\n", t*1000./tf);

    if( firstOctave < 0 )
        for( size_t i = 0; i < keypoints.size(); i++ )
        {
            KeyPoint& kpt = keypoints[i];
            float scale = 1.f/(float)(1 << -firstOctave);
            kpt.octave = (kpt.octave & ~255) | ((kpt.octave + firstOctave) & 255);
            kpt.pt *= scale;
            kpt.size *= scale;
        }

    if( !mask.empty() )
        KeyPointsFilter::runByPixelsMask( keypoints, mask );

It would be more efficent to just search in the given mask for regions of interest, though. Maybe this problem can be fixed in the next version? I'm new to the issue tracker and hopefully this post helps.

Associated revisions

Revision fee81210
Added by Ivan Korolev over 11 years ago

Added regression tests for SURF/SIFT (related to #2892)

Revision b6769c9d
Added by Vadim Pisarevsky over 10 years ago

Merge pull request #2892 from PhilLab:patch-1

History

Updated by Alexander Smorkalov almost 12 years ago

What is your OS and compiler? If you use Visual Studio, write run time library type.

Updated by Philipp Paier almost 12 years ago

Win 7, VisStudio 2008 Express.

But this should happen independent of the system, as this seems to be an error in the algorithm itself in sift::operator(). Code like the following should fail almost anytime (except if keypoints have small characteristic scale, which then would result in incorrect use of the mask) with any mask, because at some point the algorithm tries to access a pixel in the mask but uses the scaled keypoint coordinates. Thats what I found out during debugging.

#include <vector>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/nonfree/features2d.hpp>
#include <opencv2/nonfree/nonfree.hpp>

...

int main(int argc, char *argv[])
{
cv::Mat img = imread("someImage.jpg", -1);
cv::Mat mask(img.size(),CV_8U, cv::Scalar(255));

cv::initModule_nonfree();
cv::Ptr<cv::FeatureDetector> detector = cv::FeatureDetector::create("SIFT");
std::vector<cv::KeyPoint> keypoints;
detector->detect(img,keypoints,mask); // this leads to a segfault
}

Updated by Andrey Kamaev almost 12 years ago

  • Target version changed from 2.4.5 to Next Hackathon

Updated by Ivan Korolev over 11 years ago

  • Assignee changed from Vadim Pisarevsky to Ivan Korolev
  • Affected version changed from 2.4.4 to 2.4.0 - 2.4.4

Updated by Ivan Korolev over 11 years ago

  • Pull request set to https://github.com/Itseez/opencv/pull/1000

Updated by Vladislav Vinogradov over 11 years ago

Fix was merged.

  • Status changed from Open to Done

Updated by Andrey Pavlenko over 11 years ago

  • Target version changed from Next Hackathon to 2.4.6
  • HW Platform set to Other
  • Operating System set to Any
  • Start date deleted (2013-03-16)

Also available in: Atom PDF