getOptimalNewCameraMatrix is not optimal (Feature #1199)
| Status: | Done | Start date: | |
|---|---|---|---|
| Priority: | High | Due date: | |
| Assignee: |  Vadim Pisarevsky |
% Done: | 0% |
| Category: | calibration, 3d | ||
| Target version: | - | ||
| Difficulty: | Pull request: |
Description
getOptimalNewCameraMatrix always changes the principal point to the image center, which is not necessarily optimal. With alpha=0, the resulting rectified image may cover less of the scene than it could. Likewise, with alpha=1, the rectified image may have excess black bars on 1-2 sides, instead of framing the rectified original image exactly.
The attached patch changes the implementation to compute a pair of new camera matrices that exactly map the inner and outer rectangles to the viewport, and interpolates between those.
I've tested with the ROS camera_calibration tool, which has a slider to choose the alpha value post-calibration. The patch gives much more satisfying results.
Associated revisions
added centerPrincipalPoint=false to getOptimalNewCameraMatrix (ticket #1199)
Merge pull request #1199 from devernay:2.4-qtkit
History
Updated by Vadim Pisarevsky over 13 years ago
Users often assume that after undistortion and rectification the principal point is always at the center. It is especially critical for stereo pairs, there the principal points in both image must be the same or at least have the same y-coordinate.
So, the patch can not accepted in the current form. It should be a special flag in getOptimalNewCameraMatrix, which is set to "off" by default.
Updated by Patrick Mihelich over 13 years ago
Sure, for stereo the principal points must line up. But stereoRectify already has its own logic for aligning the principal points, and apparently chooses them to maximize overlap (not necessarily in the image center). Which makes perfect sense, but already breaks any user assumption that the principal point is centered post-rectification for the stereo use case where its position actually matters.
Is getOptimalNewCameraMatrix intended for stereo? My interpretation was that getOptimalNewCameraMatrix is for use with monocular cameras (where there's no inherent restriction on the principal point), and stereoRectify (which likewise takes an alpha zoom parameter) should always be used instead for stereo.
If the user does insist that the principal point be centered after rectification, they can always calibrate with CV_CALIB_FIX_PRINCIPAL_POINT in the first place.
It's documented that "By default, the undistortion functions in OpenCV (see initUndistortRectifyMap, undistort) do not move the principal point." Whereas there is no documented guarantee that getOptimalNewCameraMatrix centers it. I was actually surprised by the behavior; it was not what I expected based on the docs.
I'm happy to rework the patch and add a final bool centerPrincipalPoint argument to getOptimalNewCameraMatrix by analogy with getDefaultNewCameraMatrix. I'm less happy with setting it true by default; that's inconsistent with the default for getDefaultNewCameraMatrix, and arguably not the most useful behavior.
Updated by Vadim Pisarevsky over 13 years ago
Patrick,
sorry, I forgot that getOptimalNewCameraMatrix() is not used in stereoRectify(). You persuaded me. Let's add centerPrincipalPoint parameter, set to false by default, to make it consistent with getDefaultNewCameraMatrix().
Updated by Patrick Mihelich over 13 years ago
OK great. I've attached the new patch. The one thing I'm not sure about is how to expose the new centerPrincipalPoint argument in Python.
Updated by Vadim Pisarevsky over 13 years ago
thanks! put to trunk in r5918. Old Python interface has been updated. New Python interface will be updated automatically.
- (deleted custom field) set to fixed
Updated by Patrick Mihelich over 13 years ago
What are the chances of backporting this to the 2.3 branch? This would make the Electric version of camera_calibration work better.
Updated by Vadim Pisarevsky over 13 years ago
done. However, since we are releasing 2.3.1 next week, I copied the whole trunk to 2.3 branch, so together with getOptimalNewCameraMatrix() there is a whole bunch of other changes