Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

High-resolution non-line-of-sight imaging employing active focusing

Nature Photonics volume 16, pages 462–468 (2022)Cite this article

Subjects

Abstract

Non-line-of-sight (NLOS) imaging is a rapidly developing research direction that has significant applications in autonomous vehicles, remote sensing and other areas. Existing NLOS methods primarily depend on time-gated measurements and sophisticated signal processing to extract information from scattered light. Here we introduce a method that directly manipulates light to counter the wall’s scattering. This method, termed unseen non-line-of-sight casted optical aperture visibility-enhanced return (UNCOVER) focusing, operates by actively focusing light onto the hidden target using wavefront shaping. By raster scanning that focus, we can actively image the hidden object. The focus thus formed is near diffraction limited and can be substantially smaller than the object itself, thereby enabling us to perform NLOS imaging with unprecedented resolution. We demonstrate that a resolution of ~0.6 mm at a distance of 0.55 m is achievable in UNCOVER, giving us a distance-to-resolution ratio of ~970.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Concept of the system setup.
Fig. 2: Step-by-step procedure for UNCOVER.
Fig. 3: System setup.
Fig. 4: UNCOVER results using different targets.
Fig. 5: UNCOVER scanning result of two targets whose reflection coefficients differ greatly.

Similar content being viewed by others

Data availability

The data that support the plots within this paper and other findings of this study are available from the corresponding author upon reasonable request.

Code availability

The code that supports the plots within this paper and other findings of this study are available from the corresponding author upon reasonable request.

References

  1. Vellekoop, I. & Mosk, A. Focusing coherent light through opaque strongly scattering media. Opt. Lett. 32, 2309–2311 (2007).

    Article  ADS  Google Scholar 

  2. Yaqoob, Z., Psaltis, D., Feld, M. & Yang, C. Optical phase conjugation for turbidity suppression in biological samples. Nat. Photon. 2, 110–115 (2008).

    Article  ADS  Google Scholar 

  3. Popoff, S., Lerosey, G., Fink, M., Boccara, A. & Gigan, S. Image transmission through an opaque material. Nat. Commun. 1, 81 (2010).

    Article  ADS  Google Scholar 

  4. Vellekoop, I. Feedback-based wavefront shaping. Opt. Express 23, 12189–12206 (2015).

    Article  ADS  Google Scholar 

  5. Xu, X., Liu, H. & Wang, L. Time-reversed ultrasonically encoded optical focusing into scattering media. Nat. Photon. 5, 154–157 (2011).

    Article  ADS  Google Scholar 

  6. Conkey, D., Caravaca-Aguirre, A. & Piestun, R. High-speed scattering medium characterization with application to focusing light through turbid media. Opt. Express 20, 1733–1740 (2012).

    Article  ADS  Google Scholar 

  7. Mosk, A., Lagendijk, A., Lerosey, G. & Fink, M. Controlling waves in space and time for imaging and focusing in complex media. Nat. Photon. 6, 283–292 (2012).

    Article  ADS  Google Scholar 

  8. Denk, W., Strickler, J. & Webb, W. Two-photon laser scanning fluorescence microscopy. Science 248, 73–76 (1990).

    Article  ADS  Google Scholar 

  9. Xu, C., Zipfel, W., Shear, J., Williams, R. & Webb, W. Multiphoton fluorescence excitation: new spectral windows for biological nonlinear microscopy. Proc. Natl Acad. Sci. USA 93, 10763–10768 (1996).

    Article  ADS  Google Scholar 

  10. Helmchen, F. & Denk, W. Deep tissue two-photon microscopy. Nat. Methods. 2, 932–940 (2005).

    Article  Google Scholar 

  11. Horton, N. et al. In vivo three-photon microscopy of subcortical structures within an intact mouse brain. Nat. Photon. 7, 205–209 (2013).

    Article  ADS  Google Scholar 

  12. Popoff, S., Lerosey, G., Fink, M., Boccara, A. & Gigan, S. Controlling light through optical disordered media: transmission matrix approach. New J. Phys. 13, 123021 (2011).

    Article  ADS  Google Scholar 

  13. Mounaix, M. et al. Spatiotemporal coherent control of light through a multiple scattering medium with the multispectral transmission matrix. Phys. Rev. Lett. 116, 253901 (2016).

    Article  ADS  Google Scholar 

  14. Faccio, D., Velten, A. & Wetzstein, G. Non-line-of-sight imaging. Nat. Rev. Phys. 2, 318–327 (2020).

    Article  Google Scholar 

  15. Bertolotti, J. et al. Non-invasive imaging through opaque scattering layers. Nature 491, 232–234 (2012).

    Article  ADS  Google Scholar 

  16. Katz, O., Heidmann, P., Fink, M. & Gigan, S. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations. Nat. Photon. 8, 784–790 (2014).

    Article  ADS  Google Scholar 

  17. Velten, A. et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012).

    Article  ADS  Google Scholar 

  18. O’Toole, M., Lindell, D. & Wetzstein, G. Confocal non-line-of-sight imaging based on the light-cone transform. Nature 555, 338–341 (2018).

    Article  ADS  Google Scholar 

  19. Xin, S. et al. A theory of Fermat paths for non-line-of-sight shape reconstruction. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 6800–6809 (IEEE, 2019).

  20. Liu, X. et al. Non-line-of-sight imaging using phasor-field virtual wave optics. Nature 572, 620–623 (2019).

    Article  ADS  Google Scholar 

  21. Reza, S., Manna, M., Bauer, S. & Velten, A. Phasor field waves: a Huygens-like light transport model for non-line-of-sight imaging applications. Opt. Express 27, 29380–29400 (2019).

    Article  ADS  Google Scholar 

  22. Gupta, M., Nayar, S., Hullin, M. & Martin, J. Phasor imaging: a generalization of correlation-based time-of-flight imaging. ACM Trans. Graph. 34, 156 (2015).

    Article  Google Scholar 

  23. Saunders, C., Murray-Bruce, J. & Goyal, V. Computational periscopy with an ordinary digital camera. Nature 565, 472–475 (2019).

    Article  Google Scholar 

  24. Horstmeyer, R., Ruan, H. & Yang, C. Guidestar-assisted wavefront-shaping methods for focusing light into biological tissue. Nat. Photon. 9, 563–571 (2015).

    Article  ADS  Google Scholar 

  25. Ruan, H., Jang, M., Judkewitz, B. & Yang, C. Iterative time-reversed ultrasonically encoded light focusing in backscattering mode. Sci. Rep. 4, 7156 (2014).

    Article  ADS  Google Scholar 

  26. Katz, O., Ramaz, F., Gigan, S. & Fink, M. Controlling light in complex media beyond the acoustic diffraction-limit using the acousto-optic transmission matrix. Nat. Commun. 10, 717 (2019).

    Article  ADS  Google Scholar 

  27. Judkewitz, B., Wang, Y., Horstmeyer, R., Mathy, A. & Yang, C. Speckle-scale focusing in the diffusive regime with time reversal of variance-encoded light (TROVE). Nat. Photon. 7, 300–305 (2013).

    Article  ADS  Google Scholar 

  28. Conkey, D. et al. Super-resolution photoacoustic imaging through a scattering wall. Nat. Commun. 6, 7902 (2015).

    Article  ADS  Google Scholar 

  29. Katz, O., Small, E. & Silberberg, Y. Looking around corners and through thin turbid layers in real time with scattered incoherent light. Nat. Photon. 6, 549–553 (2012).

    Article  ADS  Google Scholar 

  30. Starshynov, I., Ghafur, O., Fitches, J. & Faccio, D. Coherent control of light for non-line-of-sight imaging. Phys. Rev. Applied 12, 064045 (2019).

    Article  ADS  Google Scholar 

  31. Katz, O., Small, E., Guan, Y. & Silberberg, Y. Noninvasive nonlinear focusing and imaging through strongly scattering turbid layers. Optica 1, 170–174 (2014).

    Article  ADS  Google Scholar 

  32. Osnabrugge, G., Horstmeyer, R., Papadopoulos, I., Judkewitz, B. & Vellekoop, I. Generalized optical memory effect. Optica 4, 886–892 (2017).

    Article  ADS  Google Scholar 

  33. Freund, I., Rosenbluh, M. & Feng, S. Memory effects in propagation of optical waves through disordered media. Phys. Rev. Lett. 61, 2328–2331 (1988).

    Article  ADS  Google Scholar 

  34. Jang, J. et al. Complex wavefront shaping for optimal depth-selective focusing in optical coherence tomography. Opt. Express. 21, 2890–2902 (2013).

    Article  ADS  Google Scholar 

Download references

Acknowledgements

This research is supported by Caltech Sensing to Intelligence (S2I) (award no. 13520296) to R.C., F.G., B.B., J.X. and C.Y.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA

    Ruizhi Cao, Frederic de Goumoens, Baptiste Blochet, Jian Xu & Changhuei Yang

Authors
  1. Ruizhi Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frederic de Goumoens
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Baptiste Blochet
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Changhuei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

C.Y. conceived the idea. R.C., B.B. and J.X. developed the idea. R.C. designed and developed the experimental protocol and setup. R.C. conducted the theoretical analysis, conducted the experiments and wrote the simulation and experiment codes. F.G. helped with the simulation code. All the authors contributed to the preparation of the manuscript.

Corresponding author

Correspondence to Changhuei Yang.

Ethics declarations

Competing interests

The authors declare the following competing interests: in October 2020, California Institute of Technology filed a patent for UNCOVER (US application no. 63/090,429), which covered the concept and implementation of the UNCOVER system described here.

Peer review

Peer review information

Nature Photonics thanks Daniele Faccio, Allard Mosk and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–10 and Sections 1–8.

Supplementary Video 1

Optimized foci in UNCOVER at different initial-signal levels.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, R., de Goumoens, F., Blochet, B. et al. High-resolution non-line-of-sight imaging employing active focusing. Nat. Photon. 16, 462–468 (2022). https://doi.org/10.1038/s41566-022-01009-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41566-022-01009-8

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing