Bug in cornerSubPix (cvFindCornerSubPix) (Bug #514)

Added by d v over 14 years ago. Updated over 14 years ago.

Status:Done Start date:
Priority:High Due date:
Assignee:Vadim Pisarevsky % Done:

0%
Category:imgproc, video
Target version:-
Affected version: Operating System:
Difficulty: HW Platform:
Pull request:

Description

cvFindCornerSubPix returns incorrect subpixel position of the chessboard corners.

Subpixel position of the corners differs by 0.5 pixel in x and y direction. This is because the subpixel position is computed from the middle of the subwindow and not from the upper left corner of the center pixel.

Example: (OpenCV 2.1)
I create an image of size 20 by 20 and a checker within. Corner coordinates are at (10, 10).

#include "cv.h" 
using namespace cv;
int main(void)
{
    Mat img = Mat::zeros(20, 20, CV_8U);
    img(Range(0,10),Range(0,10)) = Scalar(255);
    img(Range(10,20),Range(10,20)) = Scalar(255);
    vector<Point2f> corner(1, Point2f(10,10));
    cornerSubPix(img, corner, Size(5,5), Size(-1,-1), [[TermCriteria]](2, 0, 0.001));
    printf("Corner at (%g, %g)", cornerr0.x, cornerr0.y);
    std::getchar();
}

Output:
Corner at (9.50009, 9.50009)

Patch:

lines 237 and 238 in cornersubpix.cpp should be changed from

cI2.x = (float)(cI.x + InvAr0*bb1 + InvAr1*bb2);
cI2.y = (float)(cI.y + InvAr2*bb1 + InvAr3*bb2);

to

cI2.x = (float)(cI.x + 0.5 + InvAr0*bb1 + InvAr1*bb2);
cI2.y = (float)(cI.y + 0.5 + InvAr2*bb1 + InvAr3*bb2);

cornersubpix.patch (700 Bytes) Mitar M, 2010-08-20 04:02 pm

History

#1
Updated by d v over 14 years ago

I was to fast with the patch suggestion yesterday. The problem is in the procedure icvGetRectSubPix_8u32f_C1R on the line 185 in cornersubpix.cpp. It shifts the window by 0.5 pixel in each direction. One solution would be to change the line 185 to 187 from:

IPPI_CALL( icvGetRectSubPix_8u32f_C1R( (uchar*)src->data.ptr, src->step, size,
                                        src_buffer, (win_w + 2) * sizeof( src_bufferr0 ),
                                        cvSize( win_w + 2, win_h + 2 ), cI ));

to

IPPI_CALL( icvGetRectSubPix_8u32f_C1R( (uchar*)src->data.ptr, src->step, size,
                                        src_buffer, (win_w + 2) * sizeof( src_bufferr0 ),
                                        cvSize( win_w + 2, win_h + 2 ), cvPoint2D32f(cI.x - 0.5, cI.y - 0.5) ));

#2
Updated by Mitar M over 14 years ago

I am attaching a patch for this.

#3
Updated by Vadim Pisarevsky over 14 years ago

Indeed, after the patch the correct corner coordinates are returned. But if you, for example, run the standard OpenCV stereo calibration sample, with the original version it returns ~3.6 average squared backprojection error, and after applying the patch it returns ~4.7 error. Since camera calibration engine is our primary "user" of the findcornersubpix, I would prefer to study the question in more details before applying the patch. Let's keep this bug open for now.

  • Status deleted (Open)

#4
Updated by Mitar M over 14 years ago

How do you get this backprojection error? Is maybe there similar bug/problem/error/assumption? Like 0.5 in one direction and 0.5 in another?

#5
Updated by Vadim Pisarevsky over 14 years ago

Well, it's probably better to call it reprojection error. It's average distance between the original chessboard corners, found by cv::findChessboardCorners() and the projections of the ideal chessboard corners, given the geometry of the board, square size, the board pose in the particular frame and the camera parameters, as estimated by cv::calibrateCamera().

Actually, after studying this a little bit more, I must say, the current version works correctly. The integer pixel coordinates in OpenCV correspond to the centers of the pixels, rather than zero-area intersections of the lines, separating pixels one from another. In your sample you provided the corner is not in the center of the 9th pixel (counting from 0), neither in the center of the 10th pixel - it's between them, and thus its coordinate is (9.5, 9.5).

If you slightly change the geometry:

img(Range(0,10),Range(0,10)) = Scalar(255);
img(Range(11,20),Range(11,20)) = Scalar(255);
img.at<uchar>(10,10)=255;

then you will get corner at (10,10).

So, I'm closing the ticket.

  • Status set to Done
  • (deleted custom field) set to worksforme

Also available in: Atom PDF