Contents
Orbbec SDK and Orbbec SDK K4A Wrapper
Human Skeleton Tracking SDK
Femto Bolt Hardware Requirements
Next Steps
Femto Bolt is an advanced iToF 3D camera jointly developed by Orbbec and Microsoft. The depth camera uses Microsoft’s latest advanced ToF sensing technology and has a working mode and performance that is completely consistent with the Microsoft AKDK depth camera.
The Femto Bolt 3D camera integrates multiple sensing methods including multi-mode depth camera, color video camera and inertial sensor, and can simultaneously realize data transmission and power supply to the camera through a single USB Type-C connection. At the same time, Microsoft recommends using Femto Bolt as an alternative to Azure Kinect DK. For details, please refer to:
https://techcommunity.microsoft.com/t5/mixed-reality-blog/microsoft-s-azure-kinect-developer-kit-technology-transfers-to/ba-p/3899122 Femto Bolt can be purchased at Femto Bolt Purchase Link.
The Femto Bolt development environment consists of several SDKs:
Orbbec SDK K4A Wrapper or Orbbec SDK for accessing the device.
Human Skeleton Tracking SDK for tracking 3D human bodies.
In addition, Cognitive Vision services can be used with the device RGB camera. The system architecture diagram using Orbbec SDK K4a Wrapper is as follows:
The system architecture diagram using Orbbec SDK directly is as follows:
Note: Using only Orbbec SDK without Orbbec SDK K4a Wrapper cannot use AKDK’s skeleton tracking algorithm.
Orbbec SDK and Orbbec SDK K4A Wrapper
Orbbec Femto Bolt is a reliable alternative product to Microsoft Azure Kinect DK. We provide two SDKs for this product: Orbbec SDK and Orbbec SDK K4A Wrapper.Since Orbbec cannot provide upgrades and maintenance of the original AKDK development tools, from a long-term perspective, we recommend that you try and use Orbbec SDK that Orbbec has been maintaining and updating as soon as possible. Using Orbbec SDK with Femto Bolt will give you richer and more powerful features.To help users who have developed applications using K4A get started with Orbbec Femto Bolt more quickly, and achieve seamless replacement of Microsoft AKDK. On top of Orbbec SDK, we encapsulated Orbbec SDK K4A Wrapper that is maximally compatible with K4A API. Using this Wrapper, users basically don’t need to modify their application code.Orbbec SDK and Orbbec SDK K4A Wrapper provide access to complete Femto Bolt hardware sensor and device configuration.For more information on Orbbec SDK and Orbbec SDK K4A Wrapper, please refer to 《Using Orbbec SDK K4A Wrapper》
Orbbec SDK Features
Orbbec SDK is the native SDK for Orbbec Femto Bolt and can use all the features of Femto Bolt:
Depth camera access, resolution and frame rate control (including passive IR mode)
RGB camera access, resolution, frame rate and image effect control (e.g. exposure, white balance, mirroring)
IMU (gyroscope and accelerometer) access, sampling rate and range control
Hardware synchronization of depth and color cameras and software synchronization (match by frame timestamp to achieve software synchronization) on the SDK side, configurable inter-camera delay
External device synchronization control, configurable inter-device delay offset
Free Run mode enables different frame rate configurations for depth and color cameras
Access camera frame metadata for handling image resolution, timestamps, etc.
Access to device calibration data
Utility filters such as color image format conversion, point cloud data generation
Depth data stream output aligned to color (i.e. D2C, output depth coordinate system and image resolution consistent with RGB)
Complete setting information acquisition (device name, firmware version, SN, device temperature, etc.)
Data recording function supports data compression
Native C/C++ API and Python, ROS1, ROS2, Android wrappers
In addition, Orbbec SDK also supports almost all Orbbec’s mainstream cameras.
Orbbec SDK K4A Wrapper Features
Orbbec SDK and Orbbec SDK K4A Wrapper provide the following features that can run on Femto Bolt once installed:
Depth camera access and mode control (passive IR mode, wide field-of-view and narrow field-of-view depth modes)
RGB camera access and control (e.g. exposure and white balance)
Motion sensor (gyroscope and accelerometer) access
Synchronized depth and RGB camera streams with configurable inter-camera delay
External device synchronization control with configurable inter-device delay offset
Access camera frame metadata for handling image resolution, timestamps, etc.
Access device calibration data
Femto Bolt SDK Tools
Orbbec SDK K4A Wrapper provides the following tools:
Viewer tool that can be used to monitor device data streams and configure different modes
Sensor recording tool and playback reader API using Matroska container format
Human Skeleton Tracking SDK
The human skeleton tracking algorithm SDK uses Microsoft’s human skeleton tracking algorithm, namely: Azure Kinect Body Tracking SDK.To use the human skeleton tracking algorithm on Femto Bolt hardware, please refer to:Azure Kinect Body Tracking Features
Access AKDK Applications with Femto Bolt
Femto Bolt supports accessing Azure Kinect Body Tracking SDK to obtain human body tracking features and effects completely consistent with using AKDK:
Provides human image segmentation
Skeletons of partial or full bodies within the FOV range
Provides unique identification for each human body
Can track human bodies in real time
Azure Kinect Body Tracking Tools
After accessing Azure Kinect Body Tracking SDK with Femto Bolt, the following functions can be supported using the viewer tool in this SDK:
The body tracker provides a viewer tool to see how it tracks 3D human bodies.
Femto Bolt Hardware Requirements
Femto Bolt is a high performance iToF 3D camera jointly developed by Orbbec and Microsoft. The camera uses the same depth module as Azure Kinect DK, continuing the same depth modes and performance. For details, please refer to Hardware Specifications.