I transformed a small piece of code I found at the MATLAB calibration toolbox, regarding the initialization of intrinsic matrix for non planar objects. It might be useful for other people. I tested it with my data and the the reconstruction seems to be working.

<pre><code class="cpp">
CvMat [[CvMat]] *initIntrinsicMatrix(int width, int height, float fovAngle=35)
{
CvMat *intrinsicMatrix = cvCreateMat(3, 3, CV_32FC1);
cvSetIdentity(intrinsicMatrix);

float fc = ((float)width/2) / (float)tan(PI * fovAngle / 360.0f);

cvmSet(intrinsicMatrix, 0, 0, fc);
cvmSet(intrinsicMatrix, 1, 1, fc);
cvmSet(intrinsicMatrix, 0, 2, (float)width / 2);
cvmSet(intrinsicMatrix, 1, 2, (float)height / 2);

return intrinsicMatrix;
}
</code></pre>