Leverage the Kinect v2 underlying depth buffer

One of the great new features in the API for new Kinect for Windows v2 developer preview is the ability to access the underlying buffer for the data sources. Normally, you can get by with using the CopyFrameDataToArray method but this option allows you to get a little closer to the metal and possibly squeeze out a bit more performance. Frankly, this is a good thing since we’re iterating over 217,088 pixels at a rate of 30 times per second – anything helps.

This is an early preview of the new Kinect for Windows, so the device, software and documentation are all preliminary and subject to change.

Also note, because C# does not support pointer arithmetic by default, you must mark your methods with the unsafe keyword.

To access the buffer, we use the AccessUnderlyingBuffer method on the DepthFrame which tells us how big the size of the buffer is and a pointer to the buffer.

unsafe
{
    uint size = 0;
    IntPtr ptr = IntPtr.Zero;

    depthFrame.AccessUnderlyingBuffer(out size, out ptr);

    // ptr now points to the buffer
}

Once we have a pointer to the buffer we can iterate through it’s contents. It’s up to us to understand how to parse the buffer and it’s not type-safe. We assume that the buffer is comprised of an array of ushort.

// this is also unsafe, so keep in the unsafe block above...
    
// obtain a pointer to the ushort array
ushort* data = (ushort*)ptr.ToPointer();

for(int i = 0; i < size; i++)
{
    ushort depth = data[i];

    // work with depth data
}

In my modest experimentation, switching to the buffer reduced cpu time between 0.5 – 1.0 milliseconds. Your mileage will obviously vary. Here’s a small clip of me toggling between the two approaches. The CPU counter in the bottom left is averaged over the number of frames per second and includes the time to render and update the image. There’s a subtle difference.

Happy Coding!