Hybrid CNN-Mamba network for single-pixel imaging

Area of Science:

• Computational imaging
• Deep learning for optics
• Signal processing

Background:

• Deep neural networks like CNNs and ViTs have advanced single-pixel imaging (SPI).
• Existing CNNs struggle with long-range dependencies due to local receptive fields.
• ViTs face computational challenges with high quadratic complexity in attention mechanisms.

Purpose of the Study:

• To address limitations in modeling long-range dependencies for single-pixel imaging.
• To develop a computationally efficient and high-performance imaging network.
• To integrate the strengths of convolutional neural networks and state space models for SPI.

Main Methods:

• Proposed a hybrid Convolutional Neural Network (CNN) and Mamba network named CMSPI for SPI.
• Integrated CNNs for local feature extraction and Mamba (SSMs) for efficient long-range dependency modeling.
• Employed complementary split-concat structure, depthwise separable convolution, and residual connections, along with a two-step training strategy.

Main Results:

• CMSPI demonstrated superior image reconstruction quality in simulations and real experiments.
• The proposed network achieved lower memory consumption compared to state-of-the-art methods.
• CMSPI exhibited a reduced computational burden, making it hardware-friendly.

Conclusions:

• CMSPI effectively models long-range dependencies in single-pixel imaging.
• The hybrid CNN-Mamba architecture offers improved imaging performance and computational efficiency.
• CMSPI presents a promising advancement for single-pixel imaging applications.