Sony has introduced two new image sensors: IMX500 and IMX501. These are the first sensors that can process artificial intelligence tasks independently, i.e. without the use of coprocessors. IMX500 and IMX501 are the same sensors, but in the first case, it is supplied without a substrate.
The sensor has the optical format 1/2.3 inches with a resolution of 12.3 Mpix. So technically it is not something unusual. The novelty allows you to record videos in 4K resolution at 60 fps and Full HD at 240 fps.
The AI work is implemented in a special image processor with its own memory and works with both photos and videos. Such sensors can be used in devices with a weak processor or without a processor.
Initially, these sensors are not designed for smartphones and certainly not for consumer devices. For example, they can be equipped with surveillance cameras. In some cases, all data processing is done by such devices and in the cloud in general, so the new sensors from Sony save time and make the data more secure.
Sony compares its new sensor with some consumer devices. In the Google Clip camera or DJI Phantom 4 drone, for example, the image recognition algorithm works with a single video image for hundreds of milliseconds or even more, and the IMX500/IMX501 does it in 3.1 ms. However, Sony states that its sensors are designed to work with basic algorithms, so more complex tasks still require additional processing power.