Super-resolution depth imaging via processing of compact single-photon histogram parameters

Area of Science:

• Computational imaging
• Sensor technology
• Computer vision

Background:

• Time-of-flight (ToF) sensors capture depth data using photon arrival time histograms.
• Compact ToF systems in consumer electronics reduce data by extracting key parameters (e.g., peak position, intensity).
• Current parameter extraction methods are suitable for low-resolution tasks but limit high-resolution depth imaging potential.

Purpose of the Study:

• To demonstrate that extracted key parameters from ToF sensors are sufficient for high-resolution depth image reconstruction.
• To develop a compact and data-efficient neural network for enhancing ToF sensor spatial resolution.
• To significantly reduce data requirements for ToF depth imaging without compromising performance.

Main Methods:

• Proposed a compact, data-efficient neural network architecture.
• Trained the network using only 3 key parameters per pixel, a 48x reduction from the original 144 histogram bins.
• Enhanced spatial resolution from a 4x4 pixel sensor to a 32x32 pixel output.

Main Results:

• Successfully reconstructed high-resolution depth images from limited key parameters.
• Achieved performance comparable to methods using full histogram data despite significant data reduction.
• Demonstrated a 48x reduction in data volume by using only 3 parameters per pixel.

Conclusions:

• Extracted key parameters from ToF sensors are sufficient for high-resolution depth reconstruction.
• The proposed neural network enables efficient, high-quality depth imaging in compact systems.
• This approach significantly reduces data transmission and processing requirements for ToF applications.