MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes Anonymous Authors
Abstract
Neural radiance fields enable state-of-the-art photorealistic view synthesis. However, existing radiance field representations are either too compute-intensive for real-time rendering or require too much memory to scale to large scenes. We present a Memory-Efficient Radiance Field (MERF) representation that achieves real-time rendering of large-scale scenes in a browser. MERF reduces the memory consumption of prior sparse volumetric radiance fields using a combination of a sparse feature grid and high-resolution 2D feature planes. To support large-scale unbounded scenes, we introduce a novel contraction function that maps scene coordinates into a bounded volume while still allowing for efficient ray-box intersection. We design a lossless procedure for baking the parameterization used during training into a model that achieves real-time rendering while still preserving the photorealistic view synthesis quality of a volumetric radiance field.
Video
Real-Time Interactive Viewer Demos
Real Captured Scenes
Representation
For a location \(\mathbf{x}\) along a
ray: (1) We query its eight neighbors on a low-resolution 3D
grid; and we project it onto each of the three axis-aligned
planes, and then query each projection’s four neighbors on
a high-resolution 2D grid. (2) The eight low-resolution 3D
neighbors are evaluated and trilinearly interpolated while
the three sets of four high-resolution 2D neighbors are evaluated and bilinearly interpolated,
and the resulting
features are summed into a single feature vector
Piecewise-projective contraction
To model unbounded scenes we employ a contraction function. Existing works use spherical contraction, which maps straight lines to curves (left). This makes computing intersections between rays and axis-aligned bounding boxes intractable, which is required for empty space skipping. We propose a novel contraction function (right) that maps a line to a small number of segments. Intersections can be computed efficiently and thereby our contraction function is more suitable for real-time rendering.
SNeRG++ vs MERF






Acknowledgements
The website template was borrowed from Michaël Gharbi. Image sliders are based on dics.