Hi, I'm researching camera calibration and uncertainty propagation. I'm trying to understand how calibrateCameraExtended estimates stdDeviationsIntrinsics and stdDeviationsExtrinsics. [The docs](https://docs.opencv.org/ref/master/d9/d0c/group__calib3d.html#ga3207604e4b1a1758aa66acb6ed5aa65d) say very little.
So I go straight to the source code. Check [this lines of code](https://github.com/opencv/opencv/blob/00f36a261b09d6d6a89d41fb6268bceecd2e0dfd/modules/calib3d/src/calibration.cpp#L1798) where I understand the calculation is made.
It starts with `sigma2` (the "deviation of the noise") calculated as `norm(allErrors, NORM_L2SQR) / (total - nparams_nz);`
which is just the formula for the [unbiased estimator of the variance](https://en.wikipedia.org/wiki/Unbiased_estimation_of_standard_deviation). Ok so far.
And then it calculates each `s`-element of the vector of standard deviations `stdDevsM` by
stdDevsM.at(s) = std::sqrt(JtJinv.at(j,j) * sigma2);
Where `JtJinv` is the pseudo-inverse of the jacobian calculated a few lines above from `_JtJ` which in turn comes from the
LM solver invoked in [previous lines](https://github.com/opencv/opencv/blob/00f36a261b09d6d6a89d41fb6268bceecd2e0dfd/modules/calib3d/src/calibration.cpp#L1689).
**First question:** what exactly is `_JtJ`? I assume it must be the 1xN Jacobian of the projection error with respect to the parameters (there are N parameters). I've tried to trace the calculation of all the way to its origin, I got
[this far](https://github.com/opencv/opencv/blob/00f36a261b09d6d6a89d41fb6268bceecd2e0dfd/modules/calib3d/src/calibration.cpp#L809)
, but I'm not sure.
**Second question:** The moore-penrose of a 1xN matrix is a Nx1 matrix. So calling `JtJinv.at(j,j)` with two indices `j` confuses me.
**Third question:** I couldn't make sense of the formula itself and I think It's wrong. The code has the comment
//see any papers about variance of the least squares estimator for
//detailed description of the variance estimation methods
But my understanding from simple [uncertainty propagation](https://en.wikipedia.org/wiki/Propagation_of_uncertainty#Linear_combinations) for the case of uncorrelated parameters is that the Jacobian vector `_JtJ`, the variance of the projection error `sigma2` and the vector of parameters standard deviations `stdDevsM` should follow (in matlab-like pseudocode):
sigma2 = dot(_JtJ.^2, stdDevsM.^2)
Which is ill-conditioned, the simplest solution would be
stdDevsM = sqrt(sigma2) .*_JtJ ./ norm(_JtJ.^2)
Is my reasoning correct? and, where does the calculation implemented in OpenCV come from?
↧