Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis

We address the storage and rendering issue of 3D Gaussian Splatting (3DGS) by compressing the reconstructed scene parameters and rendering the compressed representation via GPU rasterization. To compress the scenes, we first analyze its components and observe that the Spherical Harmonics (SH) coefficients and the multivariate Gaussian parameters take up the majority of storage space and are highly redundant. Our compression pipeline consists of three steps:

• Sensitivity-aware clustering: We derive a sensitivity measure for each scene parameter by calculating its contribution to the training images. Color information and Gaussian parameters are encoded into compact codebooks via sensitivity-aware vector quantization.
• Quantization-aware fine-tuning: To regain information that is lost during clustering we fine-tune the scene parameters at reduced bit-rates using quantization-aware training.
• Entropy encoding: 3D Gaussians are linearized along a space-filling curve to exploit the spatial coherence of scene parameters with entropy and run-length encoding.

Gaussians are rendered in sorted order via GPU rasterization. For each Gaussian, one planar quad (a so-called splat) consisting of two triangles is rendered. A vertex shader computes the screen space vertex positions of each splat from the 2D covariance information. The size of a splat is set such that it covers the 99% confidence interval of the projected Gaussian. The vertex shader simply outputs the color computed in the pre-pass and the 2D splat center as input to the pixel shader. The pixel shader then discards fragments outside the 99% confidence interval. All remaining fragments use their distance to the splat center to compute the exponential color and opacity falloff and blend their final colors into the framebuffer.

The renderer is implemented with the WebGPU graphics API in the Rust programming language. Thus, it can run in a modern web browser on a large variety of devices. A demo can be found below.

The source code for our renderer can be found here.

We evaluate our method on 13 real scenes and the NeRF Synthesis dataset. Our method achieves a 26x compression rate on average on real scenes at minimal loss in visual quality. Our rasterization based renderer achieves a 3.5x speedup for the compressed scenes compared to the compute shader based renderer of Kerbl et al. The FPS measures shown below were obtained on the bicycle scene from Mip-NeRF360 on an NVIDIA GeForce A5000.