int cvRound(double) doesn't correctly rounded when value is out of range (Bug #1160)
| Status: | Done | Start date: | |
|---|---|---|---|
| Priority: | High | Due date: | |
| Assignee: |  Vadim Pisarevsky |
% Done: | 0% |
| Category: | core | ||
| Target version: | - | ||
| Affected version: | Operating System: | ||
| Difficulty: | HW Platform: | ||
| Pull request: |
Description
CPU: AMD Phenom(tm) II X4 955 && Inel i7
OS: win7 64bits
lib: 32 bits
When I call the Sobel() function, there is a castOp operation(\modules\imgproc\src\filter.cpp line 2322), it will call the saturate_cast operation and then call cvRound() function.
\modules\core\include\opencv2\core\types_c.h
line 291
int cvRound(double value)
{
.......
int t;
+asm
{
fld value;
fistp t;
}
return t;
.....
}
if the value> INT_MAX or value < INT_MIN, all the bits of t is set to 1. The expected result should be INT_MAX or INT_MIN.
I'm not sure whether the castOp wrongly call the cvRound() function or the cvRound() function wrongly implements.
History
Updated by Vadim Pisarevsky over 13 years ago
Unfortunately, this is a property of the floating-point to integer conversion machine instructions that we use. If we add range checks before the conversion, that will kill the performance.
If you need a proper range check, you can compute Sobel in floating-point arithmetics, then run checkRange() and then Mat::convertTo().
- Status changed from Open to Done
- (deleted custom field) set to wontfix