Sometimes focal length is negative during bundle adjustment, it makes crash. (Bug #1545)
| Status: | Open | Start date: | |
|---|---|---|---|
| Priority: | Normal | Due date: | |
| Assignee: |  Alexey Spizhevoy |
% Done: | 0% |
| Category: | stitching | ||
| Target version: | - | ||
| Affected version: | branch '2.4' | Operating System: | |
| Difficulty: | HW Platform: | ||
| Pull request: |
Description
I detected that, this problem occurs in void BundleAdjusterRay::calcError(Mat &err) in motion_estimators.cpp
After some iterations, f1 or f2 may be negative, so mult = sqrt(f1 * f2) is NaN.
It makes the program crash in a further call of SVD to a NaN matrix.
Please investigate it.
Associated revisions
Merge pull request #1545 from alalek:ocl_test_fix_rng
Merge remote-tracking branch 'upstream/2.4' into merge-2.4
- #1538 from StevenPuttemans:bugfix_3283
- #1545 from alalek:ocl_test_fix_rng
- #1551 from alalek:cmake_install_win
- #1570 from ilya-lavrenov:ipp_warn_fix
- #1573 from alalek:perf_simple_strategy
- #1574 from alalek:svm_workaround
- #1576 from alalek:ocl_fix_cl_double
- #1577 from ilya-lavrenov:ocl_setto_opencl12
- #1578 from asmorkalov:android_fd_cp_fix
- #1579 from ilya-lavrenov:ocl_norm
- #1582 from sperrholz:ocl-arithm-additions
- #1586 from ilya-lavrenov:ocl_setto_win_fix
- #1589 from ilya-lavrenov:pr1582_fix
- #1591 from alalek:ocl_remove_cl_hpp_h
- #1592 from alalek:ocl_program_cache_update
- #1593 from ilya-lavrenov:ocl_war_on_double
- #1594 from ilya-lavrenov:ocl_perf
- #1595 from alalek:cl_code_cleanup
- #1596 from alalek:test_fix_run_py
- #1597 from alalek:ocl_fix_cleanup
- #1598 from alalek:ocl_fix_build_mac
- #1599 from ilya-lavrenov:ocl_mac_kernel_warnings
- #1601 from ilya-lavrenov:ocl_fix_tvl1_and_sparse
- #1602 from alalek:ocl_test_dump_info
- #1603 from ilya-lavrenov:ocl_disable_svm_noblas
- #1605 from alalek:ocl_fixes
- #1606 from ilya-lavrenov:ocl_imgproc
- #1607 from ilya-lavrenov:ocl_fft_cleanup
- #1608 from alalek:fix_warn_upd_haar
- #1609 from ilya-lavrenov:ocl_some_optimization
- #1610 from alalek:ocl_fix_perf_kalman
- #1612 from alalek:ocl_fix_string_info
- #1614 from ilya-lavrenov:ocl_svm_misprint
- #1616 from ilya-lavrenov:ocl_cvtColor
- #1617 from ilya-lavrenov:ocl_info
- #1622 from a0byte:2.4
- #1625 from ilya-lavrenov:to_string
Conflicts:
cmake/OpenCVConfig.cmake
cmake/OpenCVDetectPython.cmake
cmake/OpenCVGenConfig.cmake
modules/core/CMakeLists.txt
modules/nonfree/src/surf.ocl.cpp
modules/ocl/include/opencv2/ocl/ocl.hpp
modules/ocl/include/opencv2/ocl/private/util.hpp
modules/ocl/perf/main.cpp
modules/ocl/src/arithm.cpp
modules/ocl/src/cl_operations.cpp
modules/ocl/src/cl_programcache.cpp
modules/ocl/src/color.cpp
modules/ocl/src/fft.cpp
modules/ocl/src/filtering.cpp
modules/ocl/src/gemm.cpp
modules/ocl/src/haar.cpp
modules/ocl/src/imgproc.cpp
modules/ocl/src/matrix_operations.cpp
modules/ocl/src/pyrlk.cpp
modules/ocl/src/split_merge.cpp
modules/ocl/src/svm.cpp
modules/ocl/test/main.cpp
modules/ocl/test/test_fft.cpp
modules/ocl/test/test_moments.cpp
modules/ocl/test/test_objdetect.cpp
modules/ocl/test/test_optflow.cpp
modules/ocl/test/utility.hpp
modules/python/CMakeLists.txt
modules/ts/include/opencv2/ts.hpp
modules/ts/src/ts_perf.cpp
samples/android/face-detection/jni/DetectionBasedTracker_jni.cpp
History
Updated by about 13 years ago
- Description changed from I detected that, this problem occurs in void [[BundleAdjusterRay]]::calcError... to I detected that, this problem occurs in void BundleAdjusterRay::calcError(Mat... More
Updated by Alexander Shishkov about 13 years ago
- Category set to stitching
Updated by Andrey Kamaev almost 13 years ago
- Priority changed from Blocker to Normal
- Assignee changed from Vadim Pisarevsky to Alexey Spizhevoy
Updated by Alexey Spizhevoy almost 13 years ago
p x,
Could you attach here the dataset where that error occurs? Provide us, please, all the information to reproduce the bug.
Thanks.
Updated by Anatoly Baksheev almost 13 years ago
I think this reporter doesn't present at this tracker. Try to find his contacts on trac.
Updated by p x almost 13 years ago
I am trying to repro this bug and if it still exists now, some test cases will be given. Thank you for your reply.
Updated by Alexey Spizhevoy about 12 years ago
no response from the submitter
- Status changed from Open to Cancelled
Updated by Martin Ljungqvist almost 12 years ago
I have happened to reproduce this problem in BundleAdjusterRay::calcError(Mat &err) using the sample code stitching_detailed.cpp.
As described above; after some iterations, if f1 or f2 is negative, then mult = sqrt(f1 * f2) is NaN. This makes the program fail, the output image being just one pixel. If this 'mult' variable is removed then it works and the resulting image looks ok, but the refined focal lengths become quite high. This corresponds to the comments about bundle adjustment and residual weights in "Image alignment and stitching - A tutorial" by Richard Szeliski 2006, page 50, where this weight variable is explained.
This problem happens when running stitching_detailed.cpp with two images being very similar and hardly no movement of the camera but small changes inside the image. All settings default. The iteration in BundleAdjusterRay makes the focal lengths f1 and f2 to approach zero in this case, when one of them might eventually become a small negative value. So either the 'mult' weight variable could perhaps be changed in some way, or e.g the stitching_detailed algorithm could somewhere take precautions if the images are too similar.
Please send me a message if you need images for reproducing the problem.
Updated by Anna Kogan almost 12 years ago
Hello Martin,
Please attach the images for us to reproduce the issue. Thank you!
- Affected version set to branch '2.4'
- Status changed from Cancelled to Open
Updated by Martin Ljungqvist almost 12 years ago
Hello Anna,
Please find the attached files.
- File vid_00002.pgm added
- File vid_00001.pgm added
Updated by Martin Ljungqvist almost 12 years ago
Some speculation if BundleAdjusterRay() would assume not too similar images.. The function leaveBiggestComponent() sorts out images that are not matching at all or are too similar, depending on their confidence. The confidence is calculated in BestOf2NearestMatcher::match(), where there is a threshold for confidence which is set to 3 in the code. The attached images above give 2.66 confidence and fail. But some images can have 2.7 and still be stitched. So confidence alone seems not to discriminate. This is an interesting problem of how and where to handle this.
Updated by Maksim Shabunin over 9 years ago
Issue has been transferred to GitHub: https://github.com/Itseez/opencv/issues/4322