A different sort of article today – a little update on the applications of artificial intelligence and more specifically deep learning in the real world.
The IMX500 is a joint venture between Sony and Microsoft. It’s a new camera chip, embedding artificial intelligence capabilities into Sony’s state-of-the-art imaging chips for their cameras.
Sony is the world leader in camera capture chips, and this partnership with Microsoft, one of the worlds biggest artificial intelligence development companies is a stride forward in practical applications of deep learning concepts.
This new chip has its own inbuilt AI processor and system memory capable of analyzing video caputre and providing metadata about the contents of the video – Microsoft Azure level analysis.
This is all done simultaneously whilst recording HD Video at 30 fps and is all completely internalised to the device. This eliminates the requirement of device transfers and syncs to the cloud. This is edge computing, handling complex AI and deep learning tasks at the “edge” of the network without having to send and receive data from remote cloud servers. Since this is aimed to eventually be a consumer product, Sony and Microsoft have also focused on taking the cost and efficiency of the chip into consideration.
This chip allows Microsoft Azure level artificial intelligence analysis within the device itself. This removes the need for the cloud computing “brain” to perform the AI analysis itself. Alongside allowing simulatenous analysis, this approach also improves the process on a security level. Since no data is being transmitted to and from a remote server for analysis, there is less opportunity for undesirable third-parties to hijack and intercept a signal and gain access to information not meant for them. This is the issue with current cameras which make use of AI analysis, as their analysis all happens on the cloud meaning it can be intercepted and cannot be simulatenous to recording.
The companies aim to pair this chip and devices containing this chip with a mobile app for customisation of the analysis, artificial intelligence hyperparameters , and for ease of access and use for its consumers. This functionality will also permit users to write and utilise their own machine and deep learning models with the chip, permitting high levels of adaptation to different tasks.
Due to this there are a lot of potential applications of this new chip. The IMX500 can be purposed for optimizing warehouses and factory automation through the use of Computer Vision and Convolutional Neural Networks. Being able to analyse which behaviours and sectors may lag behind production and impact efficiency increases the capability of decision makers to improve the system.
Thinking from a vehicular standpoint as well and taking into account the security aspect of internalizing the analysis, this chip is a step towards privacy-conscious monitoring. An example is in vehicles, where this chip could be used to detect perhaps when a driver may be drowsy and falling asleep as well as other similar real-time analysis applications.
The computer vision aspect is also applicable to self-driving cars. If these vehicles would be able to simulatenously process and analyse data whilst driving efficiently it optimizes the cost of image and video processing which is compute intensive for autonomous vehicles and also removes the load of the analysis from the electric control unit.
This technological development is a step towards a brighter and more technologically advanced world. With improvements it could lead to fully automated vehicles and, if applied to mobile technology, could move leaps and bounds forward in Virtual and especially Augmented Reality technological fronts as well as improve mobile camera qualities in general.
For more detailed information on the new chips visit this link here: https://www.sony.net/SonyInfo/News/Press/202005/20-037E/
What do you think about this new joint venture from Sony and Microsoft? What does it mean for the future of camera technology? And more interestingly, just how capable are mobile devices when it comes to artificial intelligence and deep learning analysis?
The Neural Network 