Femto Mega Documentation

K4A Wrapper Recording File Format

The Matroska (.mkv) container format is used for recording sensor data to allow multiple tracks using various codecs. Recordings contain tracks for storing color, depth, IR images, and IMU data.

High level details on the .mkv container format can be found on the Matroska website.

Track Name Codec format
COLOR Mode dependent (MJPEG, NV12 or YUY2)
DEPTH b16g (16-bit grayscale, large character byte order)
IR b16g(16-bit grayscale, large character byte order)
IMU Customize the structure. See the following IMU Sample Structure

 

 

Using Third Party Tools

Tools like ffmpeg or the mkvinfo command from the MKVToolNix toolkit can be used to inspect and extract information from recording files.

For example, the following command extracts just the depth track as a 16-bit PNG sequence into the same folder:

ffmpeg -i output.mkv -map 0:1 -vsync 0 depth%04d.png

The -map 0:1 parameter extracts track index 1 which is depth for most recordings. -map 0:0 would be used if the recording did not contain a color track.

The -vsync 0 parameter forces ffmpeg to extract frames as is rather than trying to match 30fps, 15fps, or 5fps frame rates.

 

 

IMU Sample Structure

If extracting IMU data from the file without using the playback API, the data will be in a binary format. Below is the structure of the IMU data. All fields are little endian:

Fields Type
Acceleration sensor timestamp (µs) uint64
Acceleration sensor data (x, y, z) float[3]
Gyroscope timestamp (µs) uint64
Gyro data (x, y, z) float[3]

 

 

Identifying Tracks

It may be necessary to identify which track contains color, depth, IR, etc. when using third party tools to read Matroska files. Track numbers vary based on camera mode and which set of tracks is enabled. Tags are used to identify the meaning of each track.

The following tags will be attached to the specific Matroska elements that can be used to locate the corresponding track or attachment. Tools like ffmpeg and mkvinfo can be used to view these tags.

Marker Name Marker Target Marker Value
K4A_COLOR_TRACK Color Track Matroska Track UID
K4A_DEPTH_TRACK Depth Track Matroska Track UID
K4A_IR_TRACK IR Track Matroska Track UID
K4A_IMU_TRACK IMU Track Matroska Track UID
K4A_CALIBRATION_FILE Calibration Accessories Annex file name

 

Stay updated

Be the first to learn about our new
products and updates.