Open CV copyTo function ( and probably other copying functions) looses precision! (or stops copying after a large number of values) (Bug #4431)


Added by stav ares over 9 years ago. Updated over 9 years ago.


Status:Done Start date:2015-06-23
Priority:Blocker Due date:
Assignee:- % Done:

0%

Category:core
Target version:3.0
Affected version:branch 'master' (3.0-dev) Operating System:Windows
Difficulty:Hard HW Platform:x64
Pull request:https://github.com/Itseez/opencv/pull/4158

Description

[latest opencv built from source from github]
1) I read a binary file into an opencv Mat (lets say mat1).
2) I copy it into another mat (lets say mat2), with no alteration whatsoever, with the copyTo function.
3) i write both again into binaries.
4) I open both binary files in matlab (for its high precision) and do sum(sum(abs(mat1))) and sum(sum(abs(mat2))).
5) the two results are different by a significant amount !
6) In fact, i can see that the copying to the second mat seems to have "stopped" after some value. Why is that???
7) When the same copying is done by mapping both matrices as Eigen::Matrix and then copying mat1 to mat2 using the Eigen overloaded "+=+" operator, the result is 100% correct !

  • I would add the files, but they exceed 5GB in size. (working with big data).
  • The files have values of double precision, from -1 to 1.
  • I have to say that this issue is not reproduced for every such file, but happens only sometimes, like an undefined behaviour (although its consistent with this particular set of data).
  • If you can allow me to submit this 5gb file, you will see for yourselves.

History

Updated by stav ares over 9 years ago

Description

[latest opencv built from source from github]
1) I read a binary file into an opencv Mat (lets say mat1).
2) I copy mat1 into another mat (lets say mat2), with no alteration whatsoever, with the copyTo function.
3) i write both mat1 and mat2 again into binaries.
4) I open both binary files in matlab (for its high precision) and do sum(sum(abs(mat1))) and sum(sum(abs(mat2))).
5) the two results are different by a significant amount !
6) In fact, i can see that the copying to the second mat seems to have "stopped" after some value. Why is that???
7) When the same copying is done by mapping both matrices as Eigen::Matrix and then copying mat1 to mat2 using the Eigen overloaded "+=+" operator, the result is 100% correct !

I would add the files, but they exceed 5GB in size. (working with big data).
The files have values of double precision, from -1 to 1.
I have to say that this issue is not reproduced for every such file, but happens only sometimes, like an undefined behaviour (although its consistent with this particular set of data).
If you can allow me to submit this 5gb file, you will see for yourselves.

Updated by Maksim Shabunin over 9 years ago

Can you, please, check whether the PR 4148 (https://github.com/Itseez/opencv/pull/4148) fixes this problem?

Updated by stav ares over 9 years ago

Maksim Shabunin wrote:

Can you, please, check whether the PR 4148 (https://github.com/Itseez/opencv/pull/4148) fixes this problem?

Will it be enough to replace only the changed file (modules/core/src/copy.cpp) , or do i have to download latest source from github and rebuilt?

Updated by Maksim Shabunin over 9 years ago

The big contribution with IPP refactoring has been merged recently, so you will need to get full source snapshot: https://github.com/mshabunin/opencv/tree/fix-copyto-64

Also, please, note that there is a better solution for this problem being developed: https://github.com/Itseez/opencv/pull/4158

Updated by stav ares over 9 years ago

Turns out, i will have to wait at least until mid-july before i try anything, since i am on a time critical task. Sorry for the inconvenience.

Updated by Alexander Alekhin over 9 years ago

  • Status changed from New to Done
  • Pull request set to https://github.com/Itseez/opencv/pull/4158

Also available in: Atom PDF