NSTR: Neural Spectral Transport Representation for Space-Varying Frequency Fields

arXiv:2511.18384v2 Announce Type: replace-cross
Abstract: Implicit Neural Representations (INRs) have emerged as a powerful paradigm for representing signals such as images, audio, and 3D scenes. However, existing INR frameworks — including MLPs with Fourier features, SIREN, and multiresolution hash grids — implicitly assume a textitglobal and stationary spectral basis. This assumption is fundamentally misaligned with real-world signals whose frequency characteristics vary significantly across space, exhibiting local high-frequency textures, smooth regions, and frequency drift phenomena. We propose textbfNeural Spectral Transport Representation (NSTR), the first INR framework that textbfexplicitly models a spatially varying local frequency field. NSTR introduces a learnable emphfrequency transport equation, a PDE that governs how local spectral compositions evolve across space. Given a learnable local spectrum field $S(x)$ and a frequency transport network $F_theta$ enforcing $nabla S(x) approx F_theta(x, S(x))$, NSTR reconstructs signals by spatially modulating a compact set of global sinusoidal bases. This formulation enables strong local adaptivity and offers a new level of interpretability via visualizing frequency flows. Experiments on 2D image regression, audio reconstruction, and implicit 3D geometry show that NSTR achieves significantly better accuracy-parameter trade-offs than SIREN, Fourier-feature MLPs, and Instant-NGP. NSTR requires fewer global frequencies, converges faster, and naturally explains signal structure through spectral transport fields. We believe NSTR opens a new direction in INR research by introducing explicit modeling of space-varying spectrum.