Acquisition Procedure

New system based on 360° cameras

Ultimate Free Viewpoint Navigation enables a user to freely change the position, $\displaystyle\mathbf{t} = [x, y, z] \in \mathbb{R}$ , and the angle, $\displaystyle\mathbf{r} = [\alpha, \beta, \gamma] \in[-\pi/2,\pi/2]\times[-\pi,\pi]\times[-\pi,\pi]$ of his viewpoint.

Naturally, it is impossible in practice to sample the light rays coming from every direction at every position. The challenge for an acquisition system is yet to make this sampling as dense as possible. For that purpose, perspective cameras have shown their limitation since each of them captures the light rays at one given position coming from some directions. Recently, omnidirectional (or 360°) cameras have been introduced in the public market. Their strength is that they are able to record the light rays at one given position coming from every direction.

This has motivated the following acquisition procedure, used to record the data made available on this website: a set of omnidirectional cameras have been spread inside a scene and synchronously film its content.

It is clear that, if a user is navigating through the recorded video, he is able to discretely translates in the scene, i.e., $\displaystyle\mathbf{t} \in \{ \delta_i\}$ , and at each translation position, he is able to visualize the angle he desires, i.e., $\displaystyle \forall \ i, \ \mathbf{r}(\delta_i) \in [-\pi/2,\pi/2]\times[-\pi,\pi]\times[-\pi,\pi]$ (see figure above).

A Capture in practice

In this website, by Capture, we consider the following steps:

— We position a certain number of omnidirectional cameras (typically 40) in a scene. Their distance with the neighbouring ones lies between 1m and 3m.

— We record one or several calibration sequences, in which a chessboard is moving in the scene. The recorded videos are then used to estimate the calibration parameters with an algorithm detailed and available for download here.

— We record several Sequences with the same camera arrangement (and thus the same calibration parameters). In each sequence, a small scene is acquired by all the synchronized cameras.

To summarize, the shared data has the following structure:
Capture → Sequence → Video

A Capture defines a set of Sequences that have been acquired with the same camera arrangement.
A Sequence corresponds to the recording of a given scene with several omnidirectional cameras.
A Video is the file recorded by one of the cameras in a Sequence.

For each Capture, the positions and orientations of each camera have been estimated and are given in the calibration data.

The data are available for download here.

Omnidirectional cameras

The acquisitions are done with Samsung Gear 360 cameras.
They are made of 2 lenses spanning a bit more than 180 degrees. The raw footage consists of the image captured by the two lenses, without any geometrical correction, written side by side on the same frame. The resolution is 360° (3840 x 1920)@30fps.
More information on the data format can be found here.