cv::Mat memory reallocation using push_back fails (Bug #4383)


Added by Jens Garstka almost 10 years ago. Updated over 9 years ago.


Status:Cancelled Start date:2015-06-05
Priority:Normal Due date:
Assignee:Vadim Pisarevsky % Done:

0%

Category:core
Target version:3.0
Affected version:branch 'master' (3.0-dev) Operating System:Linux
Difficulty: HW Platform:x64
Pull request:

Description

While successivly using push_back() from cv::Mat data may be lost when the matrix reaches a size somewhere above 4GB. At a given point, exactly at the same time when a resize of the matrix is performed, the resident memory shown in top drops from approximately 4.6GB to a approximately 0.6GB and the data gets invalid.

Interestingly one can make further push_back() calls without any problems. When the matrix reaches the point where the next resize of the matrix is required, i.e., at approximately 7GB, the resident memory flips back to the right size. The data, of course, remains invalid.

Here are some facts:

  • System: Linux (Debian GNU Linux 8)
  • OpenCV 3.0.0-rc1
  • System memory: 16GB RAM

You should be able to reproduce this behaviour with the attached code example.

The critical lines from output are:

7960000: virtual 6069.54M resident 4768.39M

7970000: virtual 6069.54M resident 4774.32M

7980000: virtual 8396.09M resident 684.29M

7990000: virtual 8396.09M resident 689.96M

... where the memory usage drops and:

11940000: virtual 8396.09M resident 2995.32M

11950000: virtual 8396.09M resident 3001.25M

11960000: virtual 11885.91M resident 7086.56M

11970000: virtual 11885.91M resident 7092.23M

... where the memory usage flips back to the expected size.

I hope that helps to find the Bug.

Cheers,
Jens


main.cpp (2 kB) Jens Garstka, 2015-06-05 09:43 am


History

Updated by Jens Garstka almost 10 years ago

For the sake of completeness: I tested the behaviour right now with OpenCV 3.0.0 (final). It is still the same.

Jens

Updated by Maksim Shabunin almost 10 years ago

Resident memory size (RSS) shows memory occupied in RAM, other part of allocated memory will be swapped on your hard drive. It does not mean the data is lost.
Please, take a look: http://en.wikipedia.org/wiki/Resident_set_size

If you have any other evidences of data becoming invalid at some point, feel free to reopen this issue with more detailed information.

  • Status changed from New to Cancelled

Updated by Sancho McCann over 9 years ago

I've experienced the same issue. When resident memory drops upon a push_back, the matrix being pushed to gets a bunch of zeros where there were none previously.

Also available in: Atom PDF