detectMultiScale undocumented call does not group rectangles properly (Bug #3064)

Added by Achal Dave about 4 years ago. Updated about 2 years ago.

Status:Done Start date:2013-06-01
Priority:Normal Due date:
Assignee:Achal Dave % Done:


Target version:Next Hackathon
Affected version:branch '2.4' Operating System:Any
Difficulty: HW Platform:Any
Pull request:


When detecting objects via a trained classifier, calling detectMultiScale via different prototypes results in very different results.

The documented prototype is so:

void CascadeClassifier::detectMultiScale( const Mat& image, vector<Rect>& objects,
                                          double scaleFactor, int minNeighbors,
                                          int flags, Size minObjectSize, Size maxObjectSize)

while the undocumented prototype is so:

void CascadeClassifier::detectMultiScale( const Mat& image, vector<Rect>& objects,
                                          vector<int>& rejectLevels,
                                          vector<double>& levelWeights,
                                          double scaleFactor, int minNeighbors,
                                          int flags, Size minObjectSize, Size maxObjectSize,
                                          bool outputRejectLevels )

The former just calls the latter with non-existent rejectLevels/levelWeights, and outputRejectLevels set to false.

Eventually, detectMultiScale calls groupRectangles. If outputRejectLevels was true, it passes a vector levelWeights and rejectLevels to groupRectangles; otherwise, it doesn't.

    if( outputRejectLevels )
        groupRectangles( objects, rejectLevels, levelWeights, minNeighbors, GROUP_EPS );
        groupRectangles( objects, minNeighbors, GROUP_EPS );

However, groupRectangles seems to have a different behavior when levelWeights exists. I can't figure out exactly where the issue is, but my suspicion is on the following line:

        int n1 = levelWeights ? rejectLevels[i] : rweights[i];

Variable n1 is later checked against a threshold which may be the issue. More importantly, rejectLevels and rweights seem to be very different things: rweights[i], I believe, contains the number of times the class i showed up in the rectangles. rejectLevels is copied from weights, which should be either the stage (of the cascaded classifier) at which an image was rejected, or the number of stages total (if the image was marked positive).

Unfortunately, I do not know the fix, nor do I fully understand what the issue is. Hopefully someone does.


outputRejectLevels set to true:
outputRejectLevels set to false:
(everything else was the same; the same training data; both were called with the undocumented prototype, except outputRejectLevels was set to true in one, and false in the other).


Updated by Achal Dave about 4 years ago

After looking at this further, it seems simply removing that ternary operator and setting

n1 = rweights[i]

is the fix. We are checking against groupThreshold, which is set to minNeighbors which was passed in from detectMultiScale. It does not make any sense to test rejectLevels (which is a measure of the stage it is at) vs minNeighbors. We are comparing a stage against a number of neighbors, which is nonsensical. This is also what results in no pruning of rectangles: with a high number of stages, all detected rectangles will be rejected at a stage greater than the minimum number of neighbors required.

Updated by Achal Dave about 4 years ago


However, this bug may still not be fixed. This may have only been one issue. Will update after testing.

Updated by Daniil Osokin about 4 years ago

Hi, Achal!
Thank you so much for contributing!

  • Pull request set to
  • Target version set to 2.4.6
  • Assignee set to Achal Dave

Updated by Peter Minin about 4 years ago

I've also tried to understand this code and it seems that this strange comparison is intentional. It is used only from this function:

//used for cascade detection algorithm for ROC-curve calculating
void groupRectangles(vector<Rect>& rectList, vector<int>& rejectLevels, vector<double>& levelWeights, int groupThreshold, double eps)
    groupRectangles(rectList, groupThreshold, eps, &rejectLevels, &levelWeights);
As the comment says, it's used for ROC-curves, so it doesn't need any objects to be rejected as it outputs the levels at which each object would be rejected.

Updated by Achal Dave about 4 years ago

From what I have seen, it is also called from detectMultiScale, and it is used to group the various detected windows. (Specifically, on this line )

Note that when outputRejectLevels is not specified, we call groupRectangles with weights and levels set to 0, which changes what groupRectangles does.

Of course, I might be wrong and perhaps this comparison is necessary; what I am sure of is that specifying outputRejectLevels = true returns very different (and incorrect, I believe) detections, as a direct result of this comparison.

Updated by Peter Minin about 4 years ago

Well, yes, that variant of groupRectangles is called from detectMultiScale when outputRejectLevels = true, but that just means that enabling outputRejectLevels is intended for ROC-curves calcutating, not for actual detection in any practical use. That may explain why this is not documented.

Updated by Achal Dave about 4 years ago

Oh, I suppose that could be a reason. However, if this is the case, the check here seems to not have a defined behavior (as it would depend on the total number of stages).

I do not fully understand why you wouldn't want to group rectangles for ROC curve plotting, but if that is so, you are correct and may wish to comment on the pull request so the reviewer notes it.

Updated by Achal Dave about 4 years ago

Actually, looking at this further, this does not seem to be correct. Even if you request outputRejectLevels, groupRectangles (currently) groups the rectangles anyway

The only thing it wouldn't do (in some cases) is that it may not remove a rectangle that is contained within another larger rectangle (and even then, this is not always true, it depends on the stage number). Hence, this still is a bug / an oversight.

Updated by Achal Dave about 4 years ago

Since I can't seem to find a way to edit comments:

this does not seem to be correct

refers to the idea that groupRectangles intentionally does not group when we try to plot ROC curves.

Updated by Kirill Kornyakov about 4 years ago

  • HW Platform set to Any
  • Operating System set to Any
  • Affected version changed from 2.4.5 (latest release) to branch '2.4'
  • Target version changed from 2.4.6 to Next Hackathon

Updated by Philip L about 2 years ago

PR was merged ages ago so this should be resolved.

  • Status changed from Open to Done
  • % Done changed from 0 to 100

Also available in: Atom PDF