jimmy_d shared another spectacular analysis of the NN on v9. His first post analyses the NN called AKNET_V9. Here are some details but you should go and read the original post on TMC:
- One unified camera network handles all 8 cameras vs. a sepate network per camera on previous versions
- Same weight file being used for all cameras (this has pretty interesting implications and previously v8 main / narrow seems to have had separate weights for each camera)
- Full resolution for the 3 front cameras and back camera (1280x960) and 1⁄2 the resolution for pillar and repeater cameras (640x480)
- All cameras analyze 2 frames at the same time (most likely so each camera can see motion)
- The size of the network makes jimmy_d wonder the amount of training data that Tesla is using in their backend and if they are manually tagging all the images since it seems a ton of manual labor, on his own words «there aren’t enough humans to label this much data»
- I really like his closing statement As a neural network dork I couldn’t be more pleased.»
In a later post jimmy_d mentions that AKNET_V9 might not be the network that’s currently driving the car since he thinks it’s too big to run on HW2 or HW2.5. His best guess is that it can only run at 3 fps and that doesn’t seem fast enough to be usable. However, it seems posible that this is the network that FSD is using since in HW3 it could run at 30 fps. The current firmware includes a number of different NN and it doesn’t seem easy to understand what is use and what is not, a bunch of the NN that are there are at evolution of what we got in 8.1 but AKNET_V9 seems a completely different beast.
As always, it is delightful to read you jimmy_d, keep up the good work!
Read more: TMC Forum