Driving Automation Explained: Why (Existing) Teslas Will Never Get “There”

In mid-2021, Tesla removed RADAR from its Model Y and Model 3 fleet to transit to its "Tesla Vision" self-driving approach. My 2021 Model Y Long Range, delivered on the last day of 2020, was among the last that still has this sensor, which supposedly plays a part in Tesla's hyped-up "driving automation."

My take is Tesla Vision might have been Elon Musk's excuse for Tesla to keep shipping its cars even during a chip shortage—each sensor requires a computer chip—or to cut costs. I wouldn't be surprised if the company changed its mind and added RADAR back or even more types of sensors to its future cars.

The thing is, even with RADAR, existing Tesla cars will never get to the level of actual autonomous driving—as Elon Musk promised with an expensive "Full Self-Driving" add-on. It's just not possible.

Unlike the feature's intentionally vague name, driving automation is a science with well-defined levels. Let's find out what they are, and I'll add my own experience of what Tesla has been doing with its cars.

Driving Automation Explained: The concept

The whole concept of driving automation begins with the definitions of "driving" and "automation."

Since how a machine "drives" is very different from how a human does it, engineers need to subdivide the task into multiple parts, including moving straight, moving sideways, braking, etc.

Before a vehicle can handle more than one of those tasks simultaneously, it also needs to understand the road correctly, which makes things even more complicated and challenging in terms of engineering and technology.

And that brings us to a few unavoidable technical terms and their acronyms, including Automated Driving System (ADS), Dynamic Driving Task (DDT), Object and Event Detection and Response (OEDR), and Operational Design Domain (ODD).

The technical acronyms of autonomous driving

Since most of us don't want to spend time reading boring stuff, I have tried to translate these terms, per my understanding, into something easier to digest.

The cabinet below will give you the highlights. The information will be technical but not to the point of giving you a headache. Or you can skip it.

In any case, what we need to remember is this: automated driving is complicated.

On top of that, like any skill, it's available at different levels.

Driving Automation Explained: The 5 autonomous levels

According to the Society of Automotive Engineers (SAE), there are five levels of driving automation (LoDAs), ranging from LoDA 1 to LoDA 5.

The cabinet below shows how the engineers define these levels—it is only applicable if you have already read the technical terms above.

To simplify these levels, though, we'll need to add a pre-automation level. Let's call it level 0 (LoDA 0).

Driving automation level 0: No automation (at all)

Level 0 is the case of vehicles that require total manual control.

These vehicles might not be completely manual—there are driving-related systems, such as emergency braking or an auto signal light, to help the driver.

But the gist of this level is that the human behind the steering wheel does the driving, and any available assistance system does not involve the driving itself.

In short, with LoDA 0, a human drives the vehicle entirely at all times.

Driving automation level 1 or Driver Assistance

This level is where the driving automation begins.

This level is available in many cars that have cruise control, where the vehicle can keep itself at a fixed speed, or better yet, adaptive cruise control, where it can keep itself a safe distance from a moving object in its front.

At this level, the human driver has to handle other aspects of the driving, including braking and steering. Again, level 1 has zero environmental awareness or object and event detection and response (OEDR).

In short, with driving automation level 1, a human driver is required at all times.

Driving automation level 2 or Partial Driving Automation

Level 2 includes the automation of level 1. On top of that, the vehicle can also control steering and the speed of the car.

This level is often called "Super Cruise Control" or advanced driver assistance system (ADAS). So, level 2 can take care of a big part of the driving job and has a primitive object and event detection and response.

In short, with driving automation level 2, a human driver is required most of the time.

Driving automation level 3 or Conditional Driving Automation

Level 3 is a conditional self-driving. The car can drive itself, for the most part, in ideal or near-ideal conditions. Out of that, the driver has to take over.

That said, level 3 can self-drive and has a high level of environmental awareness, but it can only work in limited environment settings.

In short, with driving automation level 3, a human driver is required at critical times.

Driving automation level 4 or High Driving Automation

Level 4 is similar to level 3 with one significant improvement: The ADS can now take care of stations when things don't go as planned.

As a result, with level 4, a human driver is generally not needed, but the human manual override/driving option is readily available and might come in handy.

This level is where actual autonomous driving begins.

Driving automation level 5 or Full Driving Automation

This level is the holy grail of driving automation.

The vehicle drives itself entirely in all conditions. It completely takes over the driving job, has perfect environmental-awareness-based responses, and might not even have traditional controllers like the steering wheel and brake/accelerating pedals.

In short, with level 5, the human is the passenger.

---

To sum up, driving automation has a lot of degrees and nuances, even within each level itself.

But it includes two inseparable essential parts: The act of driving—or the dynamic driving task (DDT)—and the handling of the ever-changing real-time environment—or object and event detection and response (OEDR).

Generally, the former is impressive and relatively easy to program—it's what we humans can experience. The latter is where things get very complicated—it happens behind the scenes.

That's because the way a machine "sees" the environment is different from how we humans do it. It can be better at certain things and worse at others.

With that, let's see how a self-driving car manages the road.

Driving Automation Explained: The hardware

To self-drive, an autonomous vehicle uses object-detecting sensors to read the environment around it. Currently, there are four types, including Ultrasonic, RADAR, LiDAR, and cameras.

Ultrasonic sensor (or Sonar)

Ultrasonic is by far the most popular sensor in cars. They are the tiny round dots you see around the body and the reason the car beeps when you get close to an object, like another car or a wall.

An ultrasonic sensor emits high-pitched sound waves (higher than a human can hear) and waits for them to bounce back after hitting an object. It then measures the latency using the speed of sound to find out the distance between itself (therefore, the car) and the thing.

Ultrasonic sensors are cheap and reliable but have one major drawback: the short range.

Underwater, sonar can detect objects from tens (if not hundreds) of miles away because sound travels effectively in water—the environment is thick and bouncy with tightly packed water molecules.

The air particles are much farther apart—so we can breathe!—and that limits the range of ultrasonic soundwaves.

As a result, this type of detector can "see" objects no farther than 10 feet (3 meters) away. It's only good for detecting close objects, suitable for parking or blind spots, and not a reliable sensor for driving automation.

RADAR

RADAR is short for radio detection and ranging system. It works similarly to sonar but uses radio waves instead of sound—it's just like Wi-Fi.

As such, the signal can go much farther than Ultrasonic, though still not far enough, without being affected by the weather (snow, rain, dust, etc.).

A front-mounted RADAR sensor on a car can detect objects up to some 200 feet (70 meters) ahead—enough to slow down or apply the brakes on time.

Still, RADAR is far from perfect for autonomous driving—it has a long list of shortcomings, but the main ones are:

• It takes longer (than Ultrasonic) to lock on an object.
• It can't differentiate objects accurately, nor can it handle multiple objects well—it has very low resolutions, so to speak.
• It can't see colors or detect the composition of an object.

As a result, RADAR is never intended to be a complete detection system for self-driving, but rather, an additional detector.

LiDAR

LiDAR is short for light detection and ranging system. (Or you can think of it as Light + RADAR.)

LiDAR shares the same concept as RADAR, but now it uses light (laser) instead of radio waves.

A LiDAR sensor, generally placed on a vehicle's roof, emits millions of pulses of light (out of the spectrum humans can see) in a pattern and builds a high-resolution 3D model of the surroundings.

As a result, a LiDAR system can detect small objects and differentiate them with extreme accuracy. It can tell a bicyclist from a motorcyclist or even a person riding a skateboard with a helmet (or without one).

But LiDAR itself also has shortcomings:

• It can't handle weather as well as RADAR.
• The sensor, for now, is ostentatiously big and very expensive.
• It can't see colors.

So, one thing is for sure: LiDAR alone is not enough for a self-driving car, though it can replace RADAR.

Cameras

Cameras are by far the most important in autonomous driving because they are the only sensors on this list that can see anything—just like our eyes.

For a car to drive by itself, it needs to read signs, understand traffic lights, and so on. A camera can also see far, and when multiple units are in use, they can also build a 3D model of the environment—again, just like our eyes.

Cameras have been available in cars for a long time, initially to help with parking and driving in reverse. In this case, it's still the human driver who interprets what a camera can see. And that's simple.

For a car to drive itself, it must be able to understand the object by itself. And that's a whole different ball game. The car now needs a brain, too—again, just like what we have behind our eyes.

Unlike a human who can generally identify objects instantly, a car—or its computer, that is—wouldn't know if red means stop and green means go or if a big hole on the road is to avoid. And that's the simple stuff.

Making a machine able to see things and interpret them accurately (per human standards) is extremely hard.

For example, knowing that you must yield to a (fallen) bicycle but can run over a life-size painting of one on the road—often the sign for a bike lane—requires a lot of programming.

So a self-driving car needs to be trained via a network with a vast amount of accumulated data to interpret the environment and react correctly the way we want it to, in real-time. This type of artificial intelligence (AI) is no easy task—real life is fluid and full of unpredictability.

Cameras also have this major drawback: they need a line of sight and can, therefore, be easily blocked or obstructed.

So, cameras are also not reliable at all times. There's no way out of that.

---

The point is that there's no one-sensor-fits-all solution for self-driving. All autonomous cars need as many (types) of sensors as possible, working in tandem, to drive themselves accurately.

Or do they? Tesla's Elon Musk has suggested otherwise. And that brings us to Tesla and its never-ending game of Full Self-Driving chicken.

Driving Automation Explained: Why you can’t count on existing Tesla cars

For years, Tesla has been known for its hyped-up self-driving capability via the controversially (and rather dangerously) named features, including "Autopilot" and "Full Self-Driving."

In my experience, the former is an intelligent Cruise Control feature, and the latter adds a few gimmicks on top of that. They are much better than most existing gas cars' Adaptive Cruise Control, but neither lives up to its name.

And here's a fact: in all Teslas, the driver must be present and prepared to take over at any time. So, technically, none is qualified as real driving automation.

And that's because autonomous driving is challenging, and (existing) Tesla vehicles don't (yet?) have enough hardware, data, or intelligence for the job. The world needs more time on this front.

Just because Tesla is ahead of the game—if it's actually so—doesn't mean it's already at the finish line. And it's not even close.

What Tesla uses for its imperfect self-driving feature

Tesla is the only big car company that doesn't use LiDAR, so far. In 2019 Elon Musk famously said, "LiDAR is a fool's errand."

Tesla Model Y's tire: A real-world experience of driving without a donut

Before May 2021, all Tesla cars—at least those of the model year 2018 and later—come with a RADAR sensor mounted on its front, 8 cameras, and 12 Ultrasonic sensors around their body.

With that number of sensors, the Autopilot feature is generally impressive on a freeway—matching level 2 of driving automation (LoDA 2) mentioned above. And if you upgrade/subscribe to Full Self-Driving, you'll get a bit of level 3—to some extent, the car now handles lane changing and even turning on streets.

Apart from my model Y, I've tried Autopilot and FSD on a few Model S, X, and 3, too. They were all similar, if not the same, as I described in the piece about FSD. Tesla's self-driving feature sure is useful, mostly on freeways, but you can't count on it 100%.

That's because, in its current state, a Tesla's self-driving is still quite bad at avoiding small objects on the road, such as potholes or even a piece of a broken tire.

Even worse, it generally doesn't recognize accurately stationary objects, even large ones, including parked cars.

Indeed, with Autopilot engaged, my Model Y always insists on being in the middle of the lane and often gets too close to a parked lane-straddling vehicle. In fact, it likely would have even crashed a few times if I hadn't intervened.

Tesla crashes

And that's why you might have run into stories about Tesla crashing into a parked truck or even a police cruiser. The most famous is probably how a Model 3 crashed into an overturned truck on a Taiwanese freeway in the video above.

In this case, chances are the car's sensors saw the truck as a stationary object but interpreted it as an overpass or something that it had not been trained to perceive as a danger or didn't come to that conclusion fast enough.

(Keep in mind that all Teslas have Emergency Braking that's engaged by default even when Autopilot is not in use.)

And it hit close to home, too. A couple of days ago, a friend of mine crashed their Model S into a disabled truck in the middle of a freeway in the San Francisco Bay Area.

(Details of the incident are still unclear, but everyone walked away.)

And that brought us to a new thing that Tesla just did with its latest Model Y and Model 3: the removal of RADAR.

Tesla sans RADAR: What’s the deal?

Over Halloween, I took my kids trick-or-treating with a neighbor who had just bought a 7-seater Model Y for a month. Naturally, we chatted about our cars, and I was surprised at how displeased he was with his Autopilot.

"It's just terrible," he said and complained how the feature kept refusing to engage or disengaging for no reason rather frequently, or it would slow down "out of nowhere", by which he probably meant the said phantom braking. And that was when the removal of RADAR hit me.

Indeed, starting in May 2021, Tesla began its "Tesla Vision" approach to self-driving by relying solely on cameras and AI. In other words, it uses a computer system to interpret the car's 8 cameras' visions to detect objects as the only object-detecting solution.

In doing so, all Model Y and Model 3 shipped in May and later no longer have the front RADAR sensor.

Curious about the change, before this post, I test-drove briefly a new Model Y sans RADAR, and its Autopilot was indeed sub-par compared to mine.

It wasn't terrible—it's still far better than almost any ICE car's Cruise Control—and first-time buyers will still be impressed—but it was definitely worse than those with RADAR.

The new car often felt unsure of itself, and I often couldn't, on the first try, engage Autopilot when going at a certain speed or being on a particular patch of a road, which where I could on my RADAR-enabled Model Y.

There were also other subtle and not-so-subtle things—you'll notice if you have driven a Tesla with RADAR before. So, the Tesla Vision right now is a significant downgrade—it indeed has less hardware.

Tesla says the new approach will take some time for the system to learn. Data is being collected and analyzed each time a similar Tesla Vision car makes a "mistake." And that might be true—things will get better.

But even when or if Tesla Vision works 100%, keep in mind that, again, physically, cameras have many issues:

• They can't handle the weather, such as rain, snow, or dust.
• Mud, dead insects, small pieces of trash, or even bird drops can cover them at any given time. (This has all happened to my Model Y.) By the way, none of the cameras have a wiper.
• Cameras don't work well, if at all, when facing a bright source of light, such as the sun or high beams of the opposite traffic.

So the point is, you can't depend on Tesla Vision at all times. At best, it only works on beautiful days, and you can't count the best-case scenario as success.

In other words, if what Elon said about Tesla Vision were true, it'd be accurate only in certain situations—more than enough for some demos or YouTube fan videos. The car would need RADAR, LiDAR, or a new type of sensor for the rest.

Again, self-driving should be about leveraging multiple types of sensors to consistently deliver the best safety results, not betting on a single type combined with some software voodoo.

Otherwise, the human driver will still be required as a backup, and I don't want to have to keep my fingers crossed while driving.

The takeaway

Driving automation is a matter of degree.

On any well-balanced low-tech vehicle, you can count on it to "drive itself" for a few miles on a straight, empty road. But if you want a car to take you home reliably right now, even when you're sleeping, get a taxi! That's the automation level we don't have yet—it's just really hard.

That's especially true for all existing Tesla cars, despite the fact that the company is ahead of the game—or has managed to appear that way. So far, its Full Self-Driving feature (or even Autopilot) has been hype at best and a lie at worst, considering the names.

In the next couple of years, you might be able to have a Tesla drive by itself most of the time under your supervision—that's driving automation level 3 (out of 5). Anything more than that is simply impossible in all existing Teslas.

Don't get me wrong. I'm a Tesla fan, and I enjoy driving my Model Y. But I look at its Autopilot (or FSD) as what it is: A helpful driver-assistance feature. Nothing more.

In short, Tesla's self-driving feature on existing cars is an excellent tool if you don't count on it as driving automation—it likely never will be.

On the other hand, if you treat it as a driverless feature, you'll get yourself into trouble, possibly even worse, and you only have yourself to blame.

A machine doesn't feel pain. Humans do. Don't fool yourself till it hurts.