Tesla FULL self driving improvements - explained and illustrated.

https://youtu.be/FwT4TSRsiVw

These presentations are getting better and better. The problems they solve to achieve full self driving, and the complexity of them, is mind boggling. You probably don't have 25 minutes to watch it, so here are a few key points to click on.

If you want to skip to the interesting part, look at the improvements in representing the virtual space around the car at 15:23. (The improvement comes from (13:53) asking the neural net to query nearby pixels from other cameras, in order to confirm what's expected to be there if the predicted object.)  Night and day, as he says. 

 I had not heard the term "jank" before, but it's well illustrated where he uses it at 16:19. 

"Jank refers to sluggishness in a user interface, usually caused by executing long tasks on the main thread, blocking rendering, or expending too much processor power on background processes." Executing these millions of calculations in real time is a huge underlying driver of these programming improvements.

 4D – incorporating time into predictions - At 17:09, he describes how they arrived at representing a "recent memory" of objects the car saw recently but may now be occluded, so that the system (21:20) has the power to determine when there's good, non-occluded data available, and to write selectively to this "recent memory" only at those times from that camera. At 23:10, he says that an occluded object that starts to reappear can give a lot of spurious information, so it now knows how to ignore the wildly erratic (orange) predictions.

He brings it all together at 24:10, illustrating the various levels of processing going on simultaneously to make the car aware of where it is.