PVD

Neural Radiance Fields (NeRF) methods have proved effective as compact, high-quality and versatile representations for 3D scense, and enable downstream tasks such as editing, retrieval, navigation, etc. Various neural architectures are vying for the core data structure of NeRF, including the plain Multi-Layer Perceptron (MLP), sparse tensors, low-rank tensors, hash tables and their compositions. Each of these representations has its particular set of trade-offs. For example, the hash table based representations admit faster training and rendering but their lack of clear geometric meaning hampers downstream tasks like spatial-relation-aware editing. In this paper, we propose Progressive Volume Distillation (PVD), a systematic distillation method that allows any-to-any conversions between different neural architectures, including MLP, sparse or low-rank tensors, hash tables and their compositions. PVD consequently empowers downstream applications to optimally adapt the neural representations for the task in hand in a post hoc fashion. The conversions are fast, as distillation is progressively performed on different levels of volume representations, from shallower to deeper. We also employ special treatment of density volume to deal with its specific numerical instability problem. Empirical evidences are presented to validate our method on the NeRF-Synthetic, LLFF and TanksAndTemples datasets. For example, with PVD, an MLP-based NeRF model can be distilled from a hash table-based Instant-NGP model at a 10X ~ 20X faster speed than being trained from scratch, while achieving superior level of sysnthesis quality.