NVIDIA Unveils Real-Time GPU-Accelerated Gaussian Splatting Sample

NVIDIA has announced a groundbreaking advancement in real-time rendering technology with the introduction of vk_gaussian_splatting, a new Vulkan-based sample. This innovative approach utilizes Gaussian splatting to render complex 3D scenes, representing them as a collection of anisotropic Gaussians in 3D space, according to NVIDIA's blog.

Revolutionizing Real-Time Rendering

The technology behind Gaussian splatting is designed to enable real-time rendering of photorealistic scenes from small sets of images. This makes the technique particularly valuable for applications in gaming, virtual reality, and professional visualization. The vk_gaussian_splatting sample is part of NVIDIA's DesignWorks Samples, showcasing the potential of this cutting-edge volume rendering technique.

Exploring New Horizons with Vulkan

The NVIDIA DevTech team sees this new sample project as a platform to explore and compare various methods for real-time visualization of 3D Gaussian splatting (3DGS). By evaluating different techniques and optimizations, the team aims to provide insights into performance, quality, and implementation trade-offs when using the Vulkan API. The initial implementation is based on rasterization, demonstrating two approaches for rendering splats: one using mesh shaders and another utilizing vertex shaders.

Advanced Sorting Techniques

Gaussian splats require back-to-front sorting for correct alpha compositing. To achieve this, NVIDIA provides two alternative sorting methods: a GPU-based Radix Sort implemented in a compute pipeline and a CPU-based asynchronous sorting strategy using the multithreaded sort function from the C++ STL.

Comprehensive Profiling and Benchmarking

The vk_gaussian_splatting sample allows users to explore various aspects of this rendering technique. It offers several visualization modes, a complete benchmarking system for real-time profiling, and detailed insights into RAM and VRAM memory consumption. Additionally, it provides GPU timings for each stage of the different techniques investigated, offering an understanding of potential bottlenecks and workloads.

For developers interested in experimenting with Gaussian splatting rendering techniques and Vulkan-based optimizations, vk_gaussian_splatting serves as a valuable starting point. More information and access to the sample can be found on the NVIDIA blog.