How a software update could add another amazing Tesla feature
In that sense, a very interesting feature developed by Andrea Ranieri, deep learning researcher at CNR-IMATI (@ 4ndr3r), and posted on Twitter yesterday, could potentially turn Tesla on-board cameras into different types of additional sensors, without any need. add other electronic equipment to vehicles: this would only be a software update (which would obviously require some adjustments and settings beforehand). In this case, we are talking about a deep learning AI, trained with @fastdotai as Ranieri explains, which essentially uses regular cell phone cameras to detect potholes and cracks on the roads in cement when you walk around. More details will be available as soon as he publishes an article he publishes: code, data and pre-trained models.
OH MY GOD! This one went really well!
– Andrea Ranieri (@ 4ndr3aR) November 5, 2021
This prompted immediate reactions on Twitter: “… the app for detecting structural damage to buildings, bridges, dams, etc. caused by earthquakes, hurricanes, etc. could be a game-changer. Consider the recent one. Florida condo collapse. lives. Learn about: Non-destructive testing techniques, “commented Dr. JT Kostman (@jt_kostman); to which Ranieri replied “… we would like to work on this too! We are submitting funding requests on this topic but so far they have all been rejected. In fact, with a bit of retraining, the model would be able to detect damage even on concrete and other materials. “
When asked if the technology would work on online or offline videos, he replied “… it works on images, so it can work on both offline videos or streaming videos. . This is an offline video and is processed at 7 frames per second on my RTX 2070. But the inference script is still not optimized, there is a lot of room for improvement! ” How many diapers did he use? To which he replied “… it’s a DeepLabv3 + with a ResNet-101 as a backbone. The training and video images are 540x540px, resolution matters when you want to see cracks in the asphalt.”
When asked if, by any chance, GPS would mark potholes, Andrea replied that “… no GPS at the moment, but we are working on a semi-permanent installation on a bus which should provide a GPS position. ” Another Twitter user, @GiorgioMantova, asked her if this would work “… so good on dark asphalt too?” (Dark road, dark pothole). “, To which he replied that” … No, on dark asphalt the model has a little more “uncertainty”. I will need to add more dark asphalt images or decrease the brightness as the data increase. “
Ranieri said he would write an article about it, and then he could do a summary / tutorial to share on Twitter; and that we will have to be patient. Finally, he said that the demo he posted is a video of a normal smartphone but that he would also be happy to try and merge the information from a LIDAR, but that he does not have the hardware. for the moment ; “… maybe for the next project.”
Just imagine applying this technology to millions of Tesla’s (Model 3, Model Y, S, and X, and the upcoming Cybertruck and Roadster) on the road, providing 24/7 information on a variety of categories (that’s i.e. not just potholes or cracks). on cement / asphalt roads) and creating a series of giant AI datasets that could then be recycled for a whole range of different uses in machine learning: it would be heaven on earth for Tesla engineers in AI, I bet.
What do you think? Please let us know in the comment section below.
All images in this article are courtesy of Tesla, Inc.
Nico Caballero is the vice-president of finance for Cogency Power, specializing in solar energy. He also holds a degree in electric cars from Delft University of Technology in the Netherlands and enjoys researching Tesla and EV batteries. He can be contacted at @NicoTorqueNews on Twitter. Nico covers the latest events from Tesla and EVs at Torque News.