Nature Communications 2026

Iterating the Transient Light Transport Matrix for Non-Line-of-Sight Imaging

Talha Sultan¹· Eric Brandt¹· Khadijeh Masumnia-Bisheh¹· Simone Riccardo²
Pavel Polynkin³· Alberto Tosi²· Andreas Velten¹

¹University of Wisconsin–Madison²Politecnico di Milano³University of Arizona

Paper Code Dataset Supplement

Recovering how light travels between hidden surfaces.

One wall measurement dataset. Independently focused virtual illumination and detection.

Experimental result

Choose a hidden illumination patch.

nWMirror

Hidden scene with virtual illumination focused on n; a red cross marks the target. — Focus location overlaid on the experimental scene

First n-focused transient frame, at 4.59 nanoseconds. — The first response is localized near the focus on n.

Key frames

Choose a focus, then step through time.

Selected experimental frames from Fig. 3. Each focus reuses the same acquisition. Separate light paths produce the delayed responses; these frames do not describe a single path through every object.

Watch the time-resolved response.

Focus on n

Supplementary Movie 4. Time-resolved transport for one fixed illumination patch.

Focus on W

Supplementary Movie 5. Time-resolved transport for one fixed illumination patch.

Each frame is rescaled to its own maximum to reveal weak indirect transport; displayed brightness is not directly comparable across time. Later residual energy near the focus includes temporal sidelobes.

See the n-focused scene and top-view explanation

User-provided n-focus illustration with a photographic reconstruction overlay and annotated top view. — Focus on n: reconstruction overlaid on the scene, with an annotated top view.

The idea

What happens to light
around the corner?

Photograph of the actual experiment: letter n at left, W in the center, a diffuse back wall, and a mirror at right. — The real hidden scene: n, W, a diffuse back wall, and a mirror. This photograph is taken from inside the hidden scene.

From seeing hidden objects to measuring the light between them.

Non-line-of-sight imaging uses reflections from a visible wall to reconstruct hidden objects. We ask a further question: if we virtually illuminate one hidden patch, where does its light go, and when does it arrive?

We measure transient light transport on the relay wall, then computationally focus illumination and detection at independently chosen hidden locations. The resulting TLTM-2 describes time-resolved transport between hidden-surface patches.

MeasuredTLTM-1Visible relay-wall patches

ReconstructedTLTM-2Hidden-surface patches

Research presentation

From wall measurements to virtual experiments
inside a hidden scene.

In preparationPresentation videoComing soon

Iterating the transient light transport matrix

How it works

A synthetic time-of-flight camera.

All focusing happens computationally,
after the measurements are captured.

Fast algorithms turn wall measurements into hidden-scene videos.

Fast Rayleigh–Sommerfeld diffraction (RSD) reconstruction combines the source measurements before imaging. FFT-based propagation then reconstructs each transient frame efficiently, making time-resolved virtual illumination and detection practical.

Measured TLTM-1 from a laser and SPAD array is transformed by fast RSD algorithms into TLTM-2 with independent virtual illumination and detection focusing. — From measured TLTM-1 to reconstructed TLTM-2 using fast RSD algorithms.

01
Capture the wall’s impulse responses
A dense pulsed-laser scan and a gated 16 × 16 SPAD array record the response for each sampled illumination–detection pair, forming TLTM-1.
02
Focus virtual illumination
After estimating the hidden surfaces, we synthesize a virtual pulse and apply delays and weights across wall samples to focus it onto a selected patch.
03
Reconstruct the response over time
Computational receiving focus reconstructs the transient response across the hidden scene. Illumination and detection locations are selected independently.
04
Build the hidden transport matrix
One illumination patch gives one transient video: a column of TLTM-2. Repeating over illumination patches builds the matrix.

Move the virtual illumination patch.

Each frame selects a different hidden source patch and sums its response over time. This movie scans focus locations; it does not show a single pulse evolving.

Supplementary Movie 2 · Watch without the photograph overlay (Movie 3)

What TLTM-2 reveals

Hidden interactions, resolved in time.

01 / Occlusion

A shadow from an indirect path.

Virtual illumination focused on a mirror reveals light reaching the back wall. A vase blocks part of this path and casts a soft shadow.

Removing the vase and its stand removes the shadow at the corresponding time. The control connects the reconstructed feature to a physical change in the scene.

Vase present and vase removed: mirror-focused fourth-bounce reconstructions show an indirect shadow disappearing when the vase and stand are removed. — Fourth-bounce shadowing. Fig. 4A.

02 / Scattering

Watch light propagate: water vs. milk.

Clear water produces distinct reflections from the tank’s surfaces. Adding milk produces persistent scattering within the volume and attenuates the back-wall reflection.

These transient differences reveal information about light–matter interactions beyond surface geometry.

Clear water

Hidden scene

Distinct front- and back-wall reflections. Open Supplementary Movie 6 ↗

Milk–water mixture

Hidden scene

Volumetric scattering and a weaker back-wall return. Open Supplementary Movie 7 ↗

Use the time labels inside each movie to compare propagation. Playback positions are independent.

Applications

More ways to inspect a hidden scene.

Use timing and spatial diversity
to change what the reconstruction shows.

Separate light by its path.

Geometry-dependent time gating separates the dominant single-hidden-surface return from later multi-bounce transport. Direct and indirect components use separate display ranges because the indirect signal is much weaker.

Direct and indirect light transport separated for illumination focused on n and W. — Direct–indirect separation. Fig. 5A. Residual focal energy includes reconstruction artifacts.

Current limits & outlook

Sharper focus.
Further iterations.

The prototype’s sparse SPAD sampling limits the virtual focus. Finite spatial and temporal resolution produce residual illumination that can overlap weak multi-bounce signals. Reliable surface estimates, calibration, and sufficient photons remain essential.

Denser sampling over a suitable relay aperture can improve virtual focusing.

16 × 16 SPAD array

Current prototype: sparse detector sampling limits focusing.

Denser SPAD array

Denser sampling supports a sharper virtual focus.

Sharper focusing, the same dominant reconstruction cost.

Denser SPAD sampling over a suitable relay aperture can sharpen a sampling-limited virtual focus without increasing computational complexity. For direct beamforming with fast RSD, the dominant reconstruction cost is O(kN³ log N) per illumination focus, independent of SPAD pixel count, where k is the number of transient frames and N³ is the number of voxels in the reconstruction grid. More pixels will still require additional data storage and handling.

Future direction

Seeing around two corners.

Further iteration could turn reconstructed hidden surfaces into new virtual relay surfaces, extending imaging around a second corner. The experiments here demonstrate TLTM-2; a two-corner reconstruction remains a future goal.

Proposed two-corner imaging geometry with two obstructors and a hidden scene, using a reconstructed surface as a virtual relay. — Proposed extension: use recovered light transport to form a new virtual imaging system beyond the first corner.

Acknowledgments

The UW–Madison authors acknowledge support from the Air Force Office for Scientific Research (FA9550-21-1-0341, FA9550-26-1-B169) and the Defense Advanced Research Projects Agency through the DARPA REVEAL Project HR0011-16-C-0025.

Reference

Cite this work.

Sultan, T., Brandt, E., Masumnia-Bisheh, K. et al. Iterating the transient light transport matrix for non-line-of-sight imaging. Nature Communications 17, 8951 (2026). doi:10.1038/s41467-026-75177-4

@article{Sultan2026,
  author  = {Sultan, Talha and Brandt, Eric and Masumnia-Bisheh, Khadijeh
             and Riccardo, Simone and Polynkin, Pavel and Tosi, Alberto
             and Velten, Andreas},
  title   = {Iterating the transient light transport matrix
             for non-line-of-sight imaging},
  journal = {Nature Communications},
  year    = {2026},
  volume  = {17},
  pages   = {8951},
  doi     = {10.1038/s41467-026-75177-4}
}