Physical sampling for computational photography

The standard model of photography is that lenses form images and focal planes capture those images. Over the past two decades, however, cameras have been transformed from devices that record focal images into analog-to-digital converters that transform massively parallel optical signals into serial electronic data. Under this new model, one may choose to maximize the quality and quantity of captured information, rather than focal image quality. Here we review the nature of the optical data stream and consider lens and focal plane designs that improve information capture capacity. After maximizing information capacity, the challenge of converting this information to digital data requires novel read-out and compression.
 We review strategies for approaching physical information limits under realistic size, power, and bandwidth constraints. Multiscale monocentric (spherical primary optics combined with arrays of secondary micro-cameras, each imaging a narrow subfield) and array lens architectures relax unfavorable geometric scaling by trading monolithic optics for co-designed system-level integration (jointly optimized optics, focal-plane, and readout subsystems). Metaoptic and mode-sorting focal plane filters implement richer projection operators that sample spectral, polarimetric, and coherence features without requiring exhaustive scanning. Finally, because readout and computation dominate energy at high throughput, we highlight architectures that perform dimensionality reduction before (or during) digitization, including integrated photonic encoders and multilayer optical projections coupled to detection and learned decoding.
 Together, these developments motivate a shift from cameras optimized to produce 2D images toward cameras engineered as end-to-end information channels, co-designing optics, focal-plane, and readout to deliver more task-relevant measurements per photon and per joule.