Road to RoboTaxi

Credits:
Kaz Barnes (CYBRLFT), US Navy Veteran & Content Creator. Follow him at his Youtube Channel.
Shop from Kaz’s here to support 

Robotaxi or what Tesla is calling Cyber Cab has been a topic of discussion for several years now. Put simply, this is a system of autonomous vehicles that will transport paying passengers at scale in the not-too-distant future.

Working in rideshare, along with being a staunch tech/science enthusiast, I knew exactly what I wanted to do when I gained access to Tesla’s FSD Beta program in October of 2021. My mission was simple yet daunting:

To catalogue and present FSD progress until the day Cyber Cab becomes a true reality.

Since October 2021 with FSD Beta version 10.2 I’ve done just that and have shared it with the online community for years.

RoboTaxi Results showing Trip Quantity (blue) and % of Success (yellow) taken over 6 versions of FSD starting with v11.4.4 to present day v12.3.6. 

Success is determined by whether or not the driver had to intervene in order to complete the trip. If intervention was necessary, the trip would be a failure. www.teslafsdtracker.com/CYBRLFT

This image featuring my RoboTaxi Results is a product of the years I’ve been collecting and presenting the data I’ve gathered in partnership with Elias Martinez on the X platform. Our goal has always been to bring an objective and thorough view of FSD’s performance to the greater audience both inside and outside of the vast Tesla community to show the proof of concept and potential of this radical technology.

In this regard, version 12 of the now FSD Supervised system has made some impressive leaps taking the average success rate from ~50% in previous version to an exciting ~83% success in this version. This lends credence to the big choice Tesla made to scrap their previous strategy entirely and focus on a purely video trained, end to end neural network. With the rapid pace of advancement in AI technology and Tesla’s objective lead in real world AI application of FSD over the years, this risk appears to have been the correct move.

The Causation Donut explains the reason for intervention during trips taken, both with and without customers, to better categorize what is causing the system to fail. What you’re seeing here are specifically the causes for disengagement on version 12.3.6.

The colors in the donut correlate with the labels seen at the bottom to keep it as simple as possible when comparing each point-release of FSD Supervised.
www.teslafsdtracker.com/CYBRLFT

As a longtime favorite, the Causation Donut has helped to illustrate what exactly is behind the failures that have led to the results you can see in the RoboTaxi Results figure. Over time Elias and I have worked together to revise and simplify the categories so as to avoid unnecessary noise or confusion when presenting these figures.

I think it is important at this point to make a clear delineation in the degree of severity in these disengagements. Some of you may be familiar with the terms Critical and Non-Critical Disengagements. To keep this as simple as possible, the following is how I’ve come to define the difference between these two.

Critical Disengagement

• Action taken to avoid the potential damage of persons or property.
• Action taken to prevent illegal maneuvers (traffic lights, stop signs, etc.)
• Action taken as a result of a system failure (forced red-wheel disengagement).

 Non-Critical Disengagement

• Everything else

Keeping this in mind, on the donut you can see Discomfort, Human, Maps, Safety, Skill Issue, Staging, System Error, and System Limitation. Of these, Safety and System Error are the critical categories whereas everything else is a non-critical disengagement.

Safety falls under the first bullet point regarding damage to persons or property and System Error falls under the third point, system failure, given the car tells the driver to take over immediately for largely unknown reasons, but likely things like loss of traction or massive glares from solar photons hitting the systems cameras.

The greater majority of these disengagements on FSD are by far in the “non-critical” category, but they are still very important points of improvement that Tesla continues to focus on with each update. The goal is for most of these labels to be as self-explanatory as possible, but I go into far more detail in my RoboTaxi Reports. 

After many conversations with the audience, it became increasingly clear that feedback from actual paying customers would be very valuable in gauging the likelihood of return-users for Cyber Cab. 

This image features the surveyed ratings taken on version 12 FSD Supervised based on the common 1-5 star system used by Uber and Lyft.
www.teslafsdtracker.com/CYBRLFT 

In the time I’ve spent testing this system, I’ve completed over 1,700 dedicated “RoboTaxi” grade trips where I’ve observed system competency from pick-up to drop-off and it became clear that I had been missing a big component of system success for future use. This component is customer satisfaction.

A gimmick or party trick, as one could label the current “smart summon” feature would be essentially useless in a world of customers that have rather simple but important needs to determine if they will incorporate a service into their daily lives. With this in mind, I began to survey customers while using version 12 FSD Supervised to see what they thought of the experience and the results were very reassuring.

As you can see 191 users to date have rated their experience at a 5-Star level, dwarfing even the 4-Star users and brought the point home with the most frequent comment being, “I would not have known it was driving itself without you telling me.”

With these results, Tesla’s FSD Supervised is already performing at a level on-par with human rideshare drivers like me and in a rather short amount of time will no doubt be superior to the greater majority of these drivers. In the least, drivers augmented by this FSD Supervised system will rise to the peak of customer care and satisfaction, proving yet again that it is those who are employing the use of AI that will disrupt those who are not.

*Tesla’s FSD Supervised system is not fully autonomous and requires the driver to be alert and ready to take over at all times. The operator of the vehicle is liable for any issues that arise due to the improper use of Tesla FSD Supervised.