HyViD (Synthetic Hyperspectral Array Video Database)

Overview

Hyperspectral imaging aims at sampling the light spectrum for each pixel. To record these hyperspectral data cubes in a high resolution in each dimension, snapshot hyperspectral cameras are necessary. To achieve this, various sensing systems were developed, e.g., camera arrays and color filter arrays. These snapshot hyperspectral cameras always need a reconstruction process to recover high-resolution images since it is impossible to record a 3D data cube with 2D grayscale sensors without exploiting the temporal dimension, which sacrifices snapshot capabilities. Therefore, ground-truth hyperspectral video is necessary to evaluate the performance of these snapshot cameras. For this, a synthetic hyperspectral array video database was developed, which can be processed to simulate diverse hyperspectral snapshot cameras. Hence, each of the seven scenes is rendered from a three times three camera array for 30 frames. The approach to render hyperspectral images is to transform a classical RGB renderer by rendering each wavelength independently as grayscale image. Of course, this database also can be used for many other tasks, where a high resolution for each dimension is necessary, for example, spectral reconstruction, cross-spectral depth estimation, hyperspectral denoising algorithms and hyperspectral video coding.

Database

The synthetic Hyperspectral Video Database (HyViD) provides seven scenes rendered from 400 nm to 700 nm in 10 nm steps, resulting in 31 hyperspectral channels. The videos have a length of 30 frames. Furthermore, the scenes are rendered using a camera array using nine cameras arranged in a three times three grid. Depth maps for every camera and frame are provided as well. The camera array is arranged as follows:

[View from front onto the sensor]
0 - 1 - 2
|   |   |
3 - 4 - 5
|   |   |
6 - 7 - 8

Thus, camera 4 is the center camera.

The folder structure is:

camera -> channel -> frame


City	Family House	Forest	Indoor	Lab	Medieval Seaport	Outdoor

Source Code

Rendering Framework

If you execute main.py using Python3 everything should work as long as Blender is installed and executable in command line by blender. You can also execute „run.py“ within Blender if you want to model a new scene and execute it. If you add textures to objects, the name of an „Image Texture“ must be the same as the name of the folder. If you want to move the camera, please move the „cam_center“. This camera is used to calculate the positions of the other cameras. If you name a light source „sun“, then the spectrum of the sun is applied. Otherwise, if the name of a light source ends with ‚K‘, then the number in front of it determines the light spectrum according to the black body radiation, e.g., a light source with name 3200K will lead to a light spectrum of a black body at 3200K temperature.

Publication

If you use the dataset or source code for your research, you should cite the following paper:

Frank Sippel, Jürgen Seiler, André Kaup
Synthetic hyperspectral array video database with applications to cross-spectral reconstruction and hyperspectral video coding
Journal of the Optical Society of America A-Optics Image Science and Vision, vol. 40, num. 3, Mar 2023, pp. 479-491

DOI: 10.1364/JOSAA.479552
arxiv: https://arxiv.org/abs/2301.07551