Deep Learning-Assisted Field-Effect Transistor for Polychromatic Light Sensing and Recognition

Abstract

image sensors have been widely utilized for image capture; however, they rely on color filtering and demosaicing algorithms for color reconstruction in each pixel. This single-photodetector, single-color architecture constrains light utilization and image resolution. In this study, we proposed a deep learning (DL)-assisted field-effect transistor (FET) to decouple mixed light components into their respective wavelengths and intensities simultaneously. Through sequential-bilateral-voltage driving, a single FET generates a series of drain current shifts (DCSs) driven by transient photoelectric effects. Using a convolutional neural network (CNN), the DCS map is decoded into multiple light components. Experiments were conducted with combinations of wavelengths-635 nm (lambda(red)), 510 nm (lambda(green)), and 450 nm (lambda(blue))-and intensities ranging from 0.1 to 0.9 W/cm(2). In monochromatic light experiments, the DCS-CNN achieved an average mean squared error (mse) of 0.0014 and mean absolute error (MAE) of 0.0216, outperforming the baseline flat DCS multilayer perceptron (MLP) by 86% in mse and 43% in MAE, respectively. In addition, our experiments confirmed the robustness of the DCS training method under laser source conditions, with and without 90(degrees) rotation. In polychromatic light experiments, the proposed DCS-CNN achieved 84.5% accuracy in detecting light from 64 distinct combinations. To deepen model understanding, we investigated the impact of transistor operation regions and transient current variation maps on light detection capabilities. Overall, this study demonstrates the potential of the DL-FET architecture for enabling single-shot color sampling in cameras without color filters.