Rollover when computing buffer size in TiffDecoder::readData() (Bug #2161)


Added by Griffin Myers over 12 years ago. Updated about 12 years ago.


Status:Done Start date:2012-07-11
Priority:Normal Due date:
Assignee:Andrey Kamaev % Done:

0%

Category:highgui-images
Target version:2.4.4
Affected version:pre 2.4 (deprecated) Operating System:
Difficulty: HW Platform:
Pull request:https://github.com/Itseez/opencv/pull/361

Description

Platform: Win7 64-bit (a colleague notes identical behavior in 64-bit Centos 5.X)
OpenCV: 2.3.1 (but issue is still evident in HEAD)
Compilation: MSVC 2008, I believe most optimizations were enabled (IPP, OpenMP, SSE, etc.) but I can not say for certain

I've observed an allocation failure in TiffDecoder::readData() (source:trunk/opencv/modules/highgui/src/grfmt_tiff.cpp@8618#L212) when the tile width/height is sufficiently large. I believe this is because of rollover due to signed (versus unsigned) math.

I'm working with 16384x16384 non-tiled TIFFs, but I believe it is the RowsPerStrip tag, or conceptually the number of strips in the image, that is really critical here (I'm by no means a TIFF expert, but this is what I have garnered by reading the TIFF documentation). I'm observing the following:
  1. On line 179 tile_width0 and tile_height0 are declared as int (signed 32-bit integers). tile_width0 is initialized to 16384
  2. On line 201 tile_height0 is set to TIFFTAG_ROWSPERSTRIP, which in my case is also 16384 because I only have one strip in this image.
  3. On line 212 AutoBuffer is allocated as:
    AutoBuffer<uchar> _buffer(tile_height0*tile_width0*8);
    Where the function definition for AutoBuffer is: (source:trunk/opencv/modules/core/include/opencv2/core/operations.hpp@8856#L2545)
    AutoBuffer(size_t _size)
  4. However, the computation of the buffer size is int*int*int, where the types of the tile sizes are as previously defined and the constant 8 is also int (http://www.cplusplus.com/doc/tutorial/constants/). So we have 16384*16384*8 = 2^31, which rolls over to -1 for the signed 32-bit data type.
  5. This -1 is now cast to a size_t which is unsigned and thus will roll over again to a positive value. In my case, size_t is unsigned 64-bit int so the value becomes 2^64.
  6. Then we try to allocate 2^64 bytes, which will almost surely always fail.

I think there are several solutions to this issue which all revolve around casting the math for computing the size of the buffer to unsigned. The simplest is perhaps:

AutoBuffer<uchar> _buffer(tile_height0*tile_width0*8u);
                                                    ^

I do not have an environment where I can easily rebuild OpenCV, so I have not been able to implement this change and test the results. Nor can I provide a simple set of test code, but anything that opens TIFF images should be able to replicate this issue. However, I am attaching two sample images generated with matlab which can be used for easily testing this issue. Both images are 16384x16384, LZW compressed (this should not matter), with all zero's pixels. One has RowsPerStrip=16383, so the image contains 2 strips, and this makes the math work out so it is just small enough to not roll over. The other image has RowsPerStrip=16384, so the image contains only 1 strip, and the rollover and allocation failure does occur here.


zeros_rps16383_00.tif - RowsPerStrip=16383 (this works) (193.9 kB) Griffin Myers, 2012-07-11 06:19 pm

zeros_rps16384_00.tif - RowsPerStrip=16384 (this fails) (193.7 kB) Griffin Myers, 2012-07-11 06:19 pm


Associated revisions

Revision b4d0dff4
Added by Andrey Kamaev about 12 years ago

Added minimal support for tiff encoder parameters and test for issue #2161

Revision 62ce8151
Added by Andrey Kamaev about 12 years ago

Fix rollover when computing buffer size in tiff decoder (bug #2161)

Revision ee331001
Added by Roman Donchenko about 11 years ago

Merge pull request #2161 from SpecLad:doc-req

History

Updated by Maria Dimashova over 12 years ago

  • Category set to highgui-images

Updated by Vadim Pisarevsky about 12 years ago

  • Affected version set to pre 2.4 (deprecated)
  • Target version deleted ()

Updated by Andrey Kamaev about 12 years ago

  • Assignee set to Andrey Kamaev

Updated by Andrey Kamaev about 12 years ago

  • Pull request set to https://github.com/Itseez/opencv/pull/361

Updated by Andrey Kamaev about 12 years ago

Fix is pushed to 2.4

  • Status changed from Open to Done

Updated by Kirill Kornyakov about 12 years ago

  • Target version set to 2.4.4

Also available in: Atom PDF