Crash is CUDA code (Bug #3807)


Added by T Abdullah over 10 years ago. Updated over 9 years ago.


Status:Incomplete Start date:2014-07-11
Priority:Normal Due date:
Assignee:Daniil Osokin % Done:

0%

Category:gpu (cuda)
Target version:-
Affected version:2.4.0 - 2.4.8 Operating System:Windows
Difficulty: HW Platform:x86
Pull request:

Description

I am doing object detection from video stream usig cascadeclassifer_gpu.detectmultiscale function. There are no crashes with one thread.

When I try to run multiple processing threads on GPU from the CPU. I have the following crashs in my program.

//-----------------------------------------------------------------------------------

OpenCV Error: Gpu API call (NCV Assertion Failed: cudaError_t=30, file=C:/opencv-2.4.6/modules/gpu/src/nvidia/NCVHaarObjectDetection.cu, line=1206) in unknown function, file C:\opencv-2.4.6\modules\gpu\src\cascadeclassifier.cpp, line 173

*ERROR Line from OpenCV code: * static void NCVDebugOutputHandler(const std::string &msg) { CV_Error(CV_GpuApiCallError, msg.c_str()); }

//-----------------------------------------------------------------------------------

Error Code: -217

Error Description: C:\Users\Abdullah\Downloads\opencv-2.4.6\modules\gpu\src\casc adeclassifier.cpp:173: error: (-217) NCV Assertion Failed: cudaError_t=30, file= C:/opencv-2.4.6/modules/gpu/src/nvidia/NCVHaarObjectDetection.cu, line=1206

ERROR Line from OpenCV code: ncvAssertCUDAReturn(cudaStreamSynchronize(cuStream), NCV_CUDA_ERROR);

//-----------------------------------------------------------------------------------

Error DescrOpenCV Error: Gpu API call (unknown error) in unknown function, file C:\opencv-2.4.6\modules\core\src\gpumat.cpp, line 1415

ERROR Function from OpenCV code: *
void mallocPitch(void
* devPtr, size_t* step, size_t width, size_t height) const {
cudaSafeCall( cudaMallocPitch(devPtr, step, width, height) );
}

//-----------------------------------------------------------------------------------

I ma using Windows/PThreads to spawn multiple-threads on the GPU. Each thread is independent of the others. It is processing a separate video file and loads its own cascadeclassifier.

I am facing these issues for last 3 months and could not find a solution for it. Could any body from development team help me out for these crashes?

I have asked these crashing issues at OpenCV answers forum and did not get a reply.

All of the reported crashes occur when I try to run multiple processing threads on GPU from the CPU.I use Windows/PThreads to spawn multiple-threads on the GPU. Each thread is independent of the others. It is processing a separate video file and loads its own cascadeclassifier. It will be nice to have answer of all or some of these questions:

1) http://answers.opencv.org/question/36913/occasional-crash-in/
2) http://answers.opencv.org/question/36602/multi-threading-support-in-opencv-gpu-module/
3) http://answers.opencv.org/question/36601/cascadeclassifier_gpu-with-cvgpustream/
4) http://answers.opencv.org/question/35170/gpu-threads-count-from-opencv-gpu-module/
5) http://answers.opencv.org/question/35943/gpu-real-time-monitoring-in-windows-7/
6) http://answers.opencv.org/question/34949/debug-cuda-code-from-opencv/
7) http://answers.opencv.org/question/34609/profiling-multi-threaded-opencv-gpu-module/
8) http://answers.opencv.org/question/34383/cuda-streams-in-opencv/
9) http://answers.opencv.org/question/34576/opencv-gpu-module-exceptions-from-its-cuda/
10) http://answers.opencv.org/question/34572/cudaerrorcudartunloading-in-opencv-gpu-module/

Regards,
Tariq Abdullah

P.S: Setting priority as Blocker since above issue is not blocking my further development progress.


001xds20140528154923.mp4 - sample video stream (5 MB) T Abdullah, 2014-07-14 04:08 pm


Associated revisions

Revision 192e630c
Added by Vadim Pisarevsky about 10 years ago

Merge pull request #3807 from guopei:patch-1

History

Updated by Daniil Osokin over 10 years ago

Hi, thahks for response! Please, attach a code sample (input video) to reproduce this problem.

  • Status changed from New to Incomplete
  • Priority changed from Blocker to Normal

Updated by Daniil Osokin over 10 years ago

  • Category set to gpu (cuda)

Updated by Daniil Osokin over 10 years ago

  • Assignee set to T Abdullah

Updated by T Abdullah over 10 years ago

Dear Daniil,

Thanks for your reply.I can upload the video stream. My project is quite big and includes a number of other modules.
This is what I am trying to achieve.

Create 2+ threads on CPU for processing the video streams on GPU using Windows threads.
Each thread loads its own cascade classifier and a separate video stream.

There is no crash with one processing thread. There is an occassional crash for 2 threads and the application crashes as soon as I start the 3rd processing thread on GPU.

I'll be happy to provide any further details for solving this problem.

Updated by Steven Puttemans over 10 years ago

I think that your crash is actually due to the scale space pyramid for detection being pushed to memory. Using multiple threads the will result into to much memory being allocated.
Is that possible?

Updated by T Abdullah over 10 years ago

I don't think so as I am not running out of memory on the GPU.

There is 4GB GPU memory on my Tesla K20C. Only around 500MB of GPU memory is used at the time of crash.

  • Assignee changed from T Abdullah to Daniil Osokin

Updated by T Abdullah over 10 years ago

Dear Daniil,
When would you be able to update?

Regards,
Tariq Abdullah

Daniil Osokin wrote:

Hi, thahks for response! Please, attach a code sample (input video) to reproduce this problem.

Updated by Maksim Shabunin over 9 years ago

Issue has been transferred to GitHub: https://github.com/Itseez/opencv/issues/4785

Also available in: Atom PDF