SUNDAE, a memory-efficient Gaussian field, addresses the high memory consumption issue associated with 3D Gaussian Splatting, a novel 3D representation known for its fast rendering speed and high rendering quality. The high memory footprint of well-trained Gaussian fields, which can utilize millions of Gaussian primitives and hundreds of megabytes of memory, is attributed to the lack of consideration for the relationship between primitives.
The proposed method introduces two key components: spectral pruning and neural compensation. Spectral pruning involves constructing a graph on the set of Gaussian primitives to model their relationship and designing a spectral down-sampling module to prune out primitives while preserving desired signals. Neural compensation, on the other hand, employs a lightweight neural network head to mix splatted features, effectively compensating for quality losses while capturing the relationship between primitives in its weights.
Extensive results demonstrate the impressive performance of SUNDAE. On the Mip-NeRF360 dataset, SUNDAE achieves a PSNR of 26.80 at 145 FPS using only 104 MB of memory, while the vanilla Gaussian splatting algorithm achieves a PSNR of 25.60 at 160 FPS using 523 MB of memory. This significant reduction in memory usage without compromising rendering quality and speed highlights the effectiveness of the proposed method.
SUNDAE’s ability to efficiently represent 3D scenes with reduced memory consumption has the potential to revolutionize various applications, such as virtual reality, gaming, and 3D visualization. By addressing the memory limitations of Gaussian splatting, SUNDAE paves the way for more accessible and scalable 3D rendering solutions.
By using advanced AI techniques, we can generate consistent character representations in neural compensation systems. This approach ensures more accurate and reliable results in applications involving spectrally pruned Gaussian fields.
