Calibration parameters

Unified Spherical Model

The relationship between a point $\displaystyle \mathbf{P}$ in the 3D domain, and the corresponding pixel $\displaystyle \mathbf{u}$ in the recorded image is drawn with the help of the Unified Spherical Model. It consists of two steps:

1) Projection on the Sphere: the point $\displaystyle \mathbf{P}$ is first projected on the sphere of center $\displaystyle O$ and of radius $\displaystyle 1$ . The projected point is called $\displaystyle \mathbf{P}_s$ and is given by:

$\displaystyle \mathbf{P}_s = \frac{1}{||\mathbf{P}||} \mathbf{P} = \left[ \begin{array}{c} \frac{X}{\sqrt{X+Y+Z}}\\ \frac{Y}{\sqrt{X+Y+Z}} \\ \frac{Z}{\sqrt{X+Y+Z}} \end{array}\right]$

2) Perspective projection: the point $\displaystyle \mathbf{P}_s$ is then projected onto the camera sensor with a perspective projection of center $\displaystyle O_s$ (translated from $\displaystyle O$ with a distance of $\displaystyle \xi$ ) and of parameters $\displaystyle \mathbf{K}$ . The pixel position of the projection is called $\displaystyle \mathbf{u}$ and is given by:

$\displaystyle\left[ \begin{array}{c} u_x\\ u_y \\ 1 \end{array} \right] \equiv \mathbf{K} \left(\mathbf{P}_s +\left[ \begin{array}{c} 0\\ 0 \\ \xi \end{array} \right] \right) = \mathbf{K} \left[ \begin{array}{c} \frac{X}{\sqrt{X+Y+Z}}\\ \frac{Y}{\sqrt{X+Y+Z}} \\ \frac{Z}{\sqrt{X+Y+Z}} + \xi \end{array}\right]$

Note that the sign $\displaystyle \equiv$ means an equality in the homogeneous coordinate. Both vectors should be divided by their last element in the vector before the equality is verified.

Intrinsic parameters format

The parameters of the models have been estimated through a calibration (explained and available for download here). The intrinsic parameters are shared on the video downloading page or directly here. A distortion vector is also given (as a null vector). It is given in the convention of OpenCV omnidirectional calibration toolbox, but says that no distortion should be added to this model. These parameters are the same for all the cameras used in our captures.

Extrinsic parameters format

In order to keep a good calibration accuracy, we consider the two semi-spherical fisheye lenses that compose the Samsung Gear 360 camera, as two independent cameras. In other words, each one has its own calibration parameter, and thus its own reference system. The axis convention is drawn below.

Extrinsic parameters shared with the video data detail, for each camera, its translation and rotation (matrix or Rodrigues angle) with respect to the reference camera.

Let us take the simple example of the two lenses of a camera (considered as two cameras). As it can be seen the difference between the front and rear coordinate system is a simple negative translation along the z axis, and a rotation around the y axis. In other words, if the reference coordinate is $\displaystyle [x_{\rm f},\ y_{\rm f},\ z_{\rm f}]$ , then the translation vector is $\displaystyle [0 ,\ 0,\ -\delta]$ and the rotation angles are $\displaystyle [0 ,\ \pi,\ 0]$ (in the x-y-z convention).

Two types of the extrinsic calibration file are available on the download webpage. Both give, as a first information, the name of the reference camera:

 "reference_camera: 729_rear"

Another common point of the two files is that the translation camera parameters are given for each camera as follows (with respect to the reference camera):

camera_name: 719_front
position: [-0.0117221, 0.10309, -3.29996]

What differs between the two files is the convention used to represent the angles. On one file format the rotation is given as an Rodrigues angle (compliant with OpenCV):

camera_name: 719_front
orientation: [-0.0376518, 3.13225, 0.0147664]

Another one gives the rotation in its matrix form. If the angles are $\displaystyle [\alpha, \ \beta, \ \gamma ]$ , the rotation matrix is:

$\displaystyle \mathbf{R}=\left[\begin{array}{ccc}\cos\gamma & -\sin\gamma & 0\\ \sin\gamma & \cos\gamma & 0\\ 0&0&1\end{array}\right]\left[\begin{array}{ccc} \cos\beta & 0 & \sin\beta\\ 0&1&0\\ -\sin\beta &0& \cos\gamma\end{array} \right]\left[\begin{array}{ccc} 1&0&0\\0&\cos\alpha&-\sin\alpha\\0&\sin\alpha&\cos\alpha\end{array}\right]$

Here is an example of R matrix given in our calibration parameter files:

camera_name: 719_front
orientation: [-0.9996697906484845, -0.02407973187231039, 0.008970851563409689;
 -0.02399408075792513, 0.9996666183951203, 0.009536045042234481;
 -0.00919748625425762, 0.009317648814105252, -0.9999142901605016]