Photon-efficient camera with in-sensor computing

Image sensors with internal computing capabilities fuse sensing and computing to significantly reduce the power consumption and latency of machine vision tasks. Linear photodetectors such as 2D semiconductors with tunable electrical and optical properties enable in-sensor computing for multiple functions. In-sensor computing at the single-photon level is much more plausible but has not yet been achieved. Here, we demonstrate a photon-efficient camera with in-sensor computing based on a superconducting nanowire array detector with four programmable dimensions including photon count rate, response time, pulse amplitude, and spectral responsivity. At the same time, the sensor features saturated (100%) quantum efficiency in the range of 405-1550 nm. Benefiting from the multidimensional modulation and ultra-high sensitivity, a classification accuracy of 92.22% for three letters is achieved with only 0.12 photons per pixel per pattern. Furthermore, image preprocessing and spectral classification are demonstrated. Photon-efficient in-sensor computing is beneficial for vision tasks in extremely low-light environments such as covert imaging, biological imaging and space exploration. The single-photon image sensor can be scaled up to construct more complex neural networks, enabling more complex real-time vision tasks with high sensitivity.

Authors

Institutions

Publication Details

Journal
Nature Communications
Published
2025-04-03
DOI
https://doi.org/10.1038/s41467-025-58501-2
Citations
9
Primary Topic
Neural Networks and Reservoir Computing
Type
article
Field-Weighted Citation Impact
22.33

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Photon-efficient camera with in-sensor computing

Jian Chen, Xiaoqing Jia, Xuecou Tu, Xiangyang Ji et al.
9 citations
Nature Communications
Neural Networks and Reservoir Computing
22.33
article

Photon-efficient camera with in-sensor computing

Jian Chen, Xiaoqing Jia, Xuecou Tu, Xiangyang Ji, Zhuolin Yang, Labao Zhang, Peiheng Wu, Yanqiu Guan, Hao Wang, Qingyuan Zhao, Liang Ma, Qi Chen, Xiaohan Wang, Yi Zhang, Haochen Li, Lin Kang, Yuchen Qiu
article en
9 citations

Abstract

Image sensors with internal computing capabilities fuse sensing and computing to significantly reduce the power consumption and latency of machine vision tasks. Linear photodetectors such as 2D semiconductors with tunable electrical and optical properties enable in-sensor computing for multiple functions. In-sensor computing at the single-photon level is much more plausible but has not yet been achieved. Here, we demonstrate a photon-efficient camera with in-sensor computing based on a superconducting nanowire array detector with four programmable dimensions including photon count rate, response time, pulse amplitude, and spectral responsivity. At the same time, the sensor features saturated (100%) quantum efficiency in the range of 405-1550 nm. Benefiting from the multidimensional modulation and ultra-high sensitivity, a classification accuracy of 92.22% for three letters is achieved with only 0.12 photons per pixel per pattern. Furthermore, image preprocessing and spectral classification are demonstrated. Photon-efficient in-sensor computing is beneficial for vision tasks in extremely low-light environments such as covert imaging, biological imaging and space exploration. The single-photon image sensor can be scaled up to construct more complex neural networks, enabling more complex real-time vision tasks with high sensitivity.

Nature CommunicationsVol. 16(1)
University of Science and Technology of China (CN), Hefei University (CN), Institute of Electronics (CN), CAS Key Laboratory of Urban Pollutant Conversion (CN), Quantum Design (Germany) (DE), Nanjing University (CN), Tsinghua University (CN)
National Natural Science Foundation of China, Government of Jiangsu Province, Natural Science Foundation of Jiangsu Province, Nanjing University, Priority Academic Program Development of Jiangsu Higher Education Institutions
Affordable and clean energy
Openalex Percentile: Top 1%
Neural Networks and Reservoir Computing
22.33
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.