Novel Compression Strategies for Dynamic NeRF Plane Embeddings: Quantization, Pruning, and Spatiotemporal Decoupling

Abstract

Dynamic neural radiance fields (NeRF) have recently been introduced to extend NeRF’s capabilities to small videos and time-changing immersive experiences. Dynamic NeRF models the temporal changes in a 3D scene in addition to the 3D scene structure and appearance. To accomplish this, the size of these models is typically very large, even for short immersive experiences. This thesis investigates compression strategies for dynamic NeRF to enhance memory and communication efficiency while maintaining rendering quality for future immersive applications such as virtual and augmented reality. Focusing on the hybrid KPlanes representation, we first analyze the sparsity and redundancy of embeddings and then propose three novel techniques for compression and optimization. Our key contributions include quantization approaches that significantly reduce memory requirements while maintaining visual fidelity, and pruning strategies that eliminate less significant embeddings. We also introduce a combined pruning and quantization method that achieves substantial model size reductions. Additionally, we propose a concept of decoupling spatiotemporal embeddings to reduce their number and enhance scalability for longer dynamic NeRF representations. The findings highlight the potential for dynamic NeRFs to meet the demands of next-generation communication technologies and facilitate seamless immersive experiences, paving the way for their broader application in real-world scenarios.