Frame and Event-Based Neural Network based Vision Sensors

We provide an overview of the advancements made in the research of neuromorphic vision processors over the past few decades. We mainly focus on two types of vision processors: frame-based (FB) and event-based (EB) vision processors. These two types differ in system architecture, image sensing techniques, image coding, image processing algorithms, and design methodologies. Neuromorphic vision processors have the ability to overcome the processing and transmission bottlenecks that limit traditional image processing systems. They can perform high-speed image capture and real-time image processing operations.We introduce two representative vision processors, one from each category, to explain their architectures, image sensing schemes, image processing processors, and system operations. The FB neuromorphic reconfigurable vision processor includes a high-speed image sensor, a processing element array, and a self-organizing map neural network. The FB vision processor has advantages in high-resolution imaging, static object detection, time-multiplex image processing, and a smaller chip area. The EB neuromorphic vision processor system utilizes an event-driven multi-kernel convolution network and an address-event-representation image sensor. The EB vision processor has advantages in fast sensing, low communication bandwidth, brain-inspired processing, and high energy efficiency.