Using NVidia GPU with C#, Azure Kinect Body Tracking and Femto Mega

Mabbasi · March 27, 2024, 2:49pm

HI,
I’m building a body tracking application in C# using the Orbbec Femto Mega and the Microsoft Azure Kinect Body Tracking SDK.

I bought the Femto Mega so that I can leverage the Nvidia GPU power for the Body Tracking.

However, the body tracking initialization fails when I try to use the CUDA setting. Body tracking passes only when I use the CPU setting for body tracking

Here is the line of code which initializes the body tracker:
tracker = Tracker.Create(calibration, new TrackerConfiguration() { ProcessingMode = TrackerProcessingMode.Cuda, SensorOrientation = SensorOrientation.Default });

I installed the correct version of CUDA (10.2) and other steps listed in this readme:

github.com

orbbec/Azure-Kinect-Samples/blob/master/body-tracking-samples/sample_unity_bodytracking/README.md

# Sample Unity Body Tracking Application

## Update

This sample project now support Orbbec Femto Bolt & Femto Mega, we put the libraries support these device in the Libs folder and modified MoveLibraryFiles.bat script, you will do nothing but follow steps below to run this bat script.

### Directions for getting started:

#### 1) First get the latest nuget packages of libraries:

Open the sample_unity_bodytracking project in Unity.
Open the Visual Studio Solution associated with this project.
If there is no Visual Studio Solution yet you can make one by opening the Unity Editor
and selecting one of the csharp files in the project and opening it for editing.
You may also need to set the preferences->External Tools to Visual Studio

In Visual Studio:
Select Tools->NuGet Package Manager-> Package Manager Console

On the command line of the console at type the following command:

This file has been truncated. show original

Can you advise what is required to benefit from the GPU using C#?

Thanks

obwh · March 28, 2024, 2:22am

Try to follow these steps:
1.Download cuda, cudnn and TensorRT, make sure these versions are match
2.Install cuda
3.Unzip cudnn, copy bin, include and lib folder to the same folder of cuda install path
4.Unzip TensorRT, copy all dlls to bin folder of cuda install path, if contains same dll, replace it
5.Edit System environment, make sure the bin and lib in PATH, then add libnvvp to PATH

6.Last, use nvcc --version command to check the version

obwh · March 28, 2024, 2:23am

The cuda, cudnn and TensorRT version i used:

Mabbasi · March 28, 2024, 2:26pm

HI,
thanks for the response., I really appreciate the support I followed the steps as you described them, but it still doesn’t work.
I installed Cuda v1.4, and copied all the cudnn bin, lib and include directories into the cuda install path (so the bin of the cudnn merged with the bin of cuda, etc).
Also, I put all the dirs you mentioned into the PATH.
Finally, I did the nvcc version and got the following:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Wed_Jun__2_19:25:35_Pacific_Daylight_Time_2021
Cuda compilation tools, release 11.4, V11.4.48
Build cuda_11.4.r11.4/compiler.30033411_0

I’m still getting the same error when I try to use CUDA for the Body Tracking.

Any other suggestions? I connected the camera via USB-C connector, and the camera device firmware (from the Kinect Viewer) shows me Depth camera: 1.1.5

I really appreciate if you can give me any other ideas on how to resolve this… Thanks

Mabbasi · March 29, 2024, 3:11pm

Hi
Just curious, did anyone here actually get the body tracking to work using the NVidia GPU integrated in the Femto Mega Camera? My PC does not have a GPU, I want to know it is even feasible, to leverage the Femto Mega integrated GPU for body tracking, then I can continue to debug things
thanks

robot_guy · March 29, 2024, 3:40pm

The specifications say the Mega has a Jetson Nano, which is well below the minimum requirements suggested for the Azure Body Tracking SDK (a GTX 1050). I would assume running the Azure Body Tracking on the Mega isn’t possible, but even if it were the performance would likely be terrible.

It might be possible to run some other 3rd party body tracking library. I would be interested to hear if anyone has had success doing this.

Mabbasi · March 31, 2024, 4:00pm

HI
Thanks for the response. Highly appreciated. So I will accept your assumption and give up on leveraging the GPU for Body Tracking.

I guess the main benefit of the GPU on the camera is to make the camera processing (2d-3d transformations) faster?

robot_guy · April 1, 2024, 1:18am

I believe the use case for the Mega is running your own custom depth/RGB processing locally on the device and then sending the processed results over the network. The Nano offers decent processing capability for a small embedded system, but not enough to run the Azure Body Tracking.

Unless I’m missing something, for any use case where the camera will be connected to a host machine via USB the Mega is a waste and you should just use the Bolt.

obwh · April 1, 2024, 1:32am

No, the body tracking not run on board intergrated in Mega, its run on your PC, so your PC must has Nvidia GPU to use GPU mode for body tracking