This hardware accelerates machine vision applications by using convolutional neural network operations to extract features from imagery at the image sensor. Smart image sensors have been incorporated with high-resolution cameras to analyze images in a wide variety of applications such as mobile device facial recognition. As such, the image sensor market is expected to reach $28 billion by 2025 . Traditionally, smart image sensors use time and energy intensive sequential image processing to evaluate images. However, there is a growing need for real-time, high-resolution video processing that cannot be met using sequential image processing methods. Researchers at the University of Florida have developed a hierarchical hardware architecture that accelerates machine vision applications and processes videos in real-time.
Enables secure, real-time video processing by applying convolutional neural network operations to extract local features from the landscape at the image sensor
This hierarchical hardware architecture applies convolutional neural network operations to complete low and mid-level image analyses at the image sensor. The design maintains hierarchical processing that begins at the local pixel level, offering pipelined execution, and an efficient data management system by storing and processing the data near the image sensor. After completing low and mid-level image analyses to extract local features at the sensor, the features are encrypted and can be sent to an external processor to complete image analysis.