On January 12, according to foreign media reports, many media reports on automatic driving technology are focused on a few large companies with well-known brands, including Google, Uber, Tesla and general motors. But there’s another company that’s working on autonomous driving, and it may prove more important in the end. The company is Mobileye, an Israeli start-up acquired by Intel in 2017.
Mobileye doesn’t have the star power of Elon Musk, the US billionaire, or the resources of Google. But some of it can be said to be more important, which is the dominant position in the market of advanced driver assistance system (ADAS). As early as 2016, Mobileye broke up with Tesla’s autonomous driving strategy, but it is still doing business with many other automakers. Mobileye said it shipped 17.4 million systems last year, which means 17.4 million customers bought cars with Mobileye cameras, chips and software.
Amnon shashua, chief executive of Mobileye, made it clear in his speech at CES 2020 that this was a huge strategic advantage. He outlined Mobileye’s vision for the development of autonomous driving technology over the next five years and made it clear that he expected Mobileye to remain at the center of the industry.
Level 2 + automatic driving ability
In the past two years, many people have touted Cadillac’s “super cruise” function as the gold standard of ADAS system. There are two features that make super cruise stand out: first, it uses a camera facing the driver to verify that the driver’s eyes are on the road. If not, the system forces the driver to take over control of the car. This feature solves one of the biggest concerns of the ADAS system: they can make drivers so complacent that they can’t intervene in the event of a technical failure.
Second, Cadillac has pre mapped more than 210000 kilometers of highways in the United States and Canada. Its system will only run on those roads, which makes the system less likely to get confused and make dangerous mistakes.
In CES’s speech, Mobileye CEO Shashuya called the ADAS system with high definition map, such as super cruise function, called “ Level 2+ automatic driving system ” that is, more advanced than the conventional ADAS system (called &ldquo in the five stage autopilot frame; Level 2&rdquo). Many automakers have developed similar systems. Mr. shashuya said Mobileye provides technology to 70% of companies, including Nissan, Volkswagen and BMW systems.
When Mobileye sells technology to car manufacturers, it always bargains with them, hoping to get sensor data of customers’ vehicles. Mr. shashuya said Mobileye is already collecting data from VW, BMW and Nissan. He said three other unnamed automakers agreed to participate in the project. The scale of the project is huge. Mobileye says it collects six million kilometers of sensor data every day from vehicles on public roads. Mobileye expects to have more than 1 million cars in its European Fleet by the end of 2020 and more than 1 million US cars next year.
Mobileye uses all of this data to produce a detailed, high-definition map of the car’s driving areas. Mobileye said it already has software that can automatically generate high-definition road maps at speeds of more than 72 kilometers per hour. The company expects to expand this capability to all roads next year. Mobileye expects to map all of Europe by March and the U.S. to be fully mapped later this year.
This is significant because the collection of high-definition map data has always been a major obstacle to the deployment of autonomous driving technology. In the past, companies had to create maps by hand by hiring workers to drive maps on every street, and then having a second group manually annotate the collected data. If Mobileye can crowdsource and automate the process, the resulting data will easily be worth billions of dollars.
Once Mobileye combines all this data into a high-definition map, it can send the latest map back to its fleet. As a result, each of Mobileye’s partners, including Volkswagen, BMW, Nissan and other undisclosed companies, will be able to use the “level 2 + autopilot capability” similar to the super endurance feature without needing their own mapping team.
Autopilot with camera only
Mobileye seems to be firmly established in the dominant position of ADAS market in the United States. But Mobileye may also be caught up in ADAS’s old-fashioned dangers. Mobileye’s current business model is to sell chips, cameras and software to existing automakers. However, many experts believe that the first vehicles to deploy fully autonomous driving will be taxi fleets, not customer owned vehicles.
Shashuya also believes that the taxi fleet will first realize the full automatic driving function, and he is determined not to let Mobileye fall behind. So in addition to the “level 2 +” products, the company is also working on longer-term projects to build fully autonomous driving technology. This week, Mobileye showed a fully automatic driving vehicle based on 12 cameras. It is not equipped with radar, lidar or other sensors.
An impressive video shows the mobile Ye prototype driving for 20 minutes on Jerusalem’s chaotic streets. The car navigates at complex intersections, merges into crowded lanes, and cleverly avoids hitting other vehicles. Lidar skeptics may claim to be right here, but shashuya doesn’t plan to actually launch such a car without radar or lidar. On the contrary, this camera only car is part of the larger strategy of Mobileye to build a safe automatic driving system.
Mobileye’s plan is to build two fully automated driving systems: one is based entirely on cameras, the other is based on radar and lidar. Mobileye believes that if Mobileye can prove that each individual system can drive more than 10000 hours between two collisions, then a system with two sensors should be able to drive 100 million hours safely without a collision. The latter figure will make Mobileye’s autopilot safer than human drivers.
Mobileye seems to assume that the failure modes of the two systems are statistically independent. However, redundancy is an important principle in any safety critical system. The system based on camera and lidar will have some different failure modes. So building a separate autopilot stack around different sensors and running it in parallel should create a margin of safety, even if Mobileye’s statistics are too optimistic.
Mobileye’s belief in redundancy is evident in the design of its 12 camera autopilot system. The company has assembled a set of six different algorithms to detect objects around the car:
One of the algorithms is adjusted to identify the wheels and infer the vehicle position. Mobileye also has special algorithms to identify doors, because open doors usually represent potential security issues. Another algorithm uses “visual lidar”. By comparing images from different cameras, this algorithm can infer the distance of each pixel in the image. Then, the algorithm uses these estimates to generate a three-dimensional point cloud, just as the user gets from the lidar sensor. Then, Mobileye’s software applies standard software designed for lidar data, trying to identify objects in the scene. The third algorithm focuses on identifying which pixels correspond to the driveable road. Anything in the scene that doesn’t belong to the road is a potential obstacle and requires extra care.
Mobileye hopes that by processing images in many different ways, it can minimize the chance of missing any important object. Once an object is detected, Mobileye has four independent algorithms that try to place it precisely in three-dimensional space.
A key problem is, if different algorithms differ, how will the system deal with it? In his speech at CES, shashuya didn’t fully explain how it works. Nevertheless, the results seem to be self-evident. Mobileye’s 20 minute demonstration shows that its vehicles can handle complex traffic scenarios, and its progress in the field of autonomous driving is ahead of other companies.
The road to full automatic driving
As mentioned earlier, Mobileye never plans to launch a fully automatic driving vehicle with only webcam. On the contrary, Mobileye’s pure camera automatic driving system is destined to become part of its more complex autopilot system that it hopes to deploy over the next few years.
Mobileye is planning to build a separate autopilot stack based on lidar, radar and other non camera sensors. Mobileye plans to test the two systems separately, prove that they have achieved impressive performance, and then merge them into a super system, which Mobileye hopes will be much safer than any single system.
Shashua hopes to act quickly. Mobileye aims to deploy taxi fleets in three major cities by 2022, including Tel Aviv, Paris and Daegu, South Korea. Shashuya estimates that the initial hardware price for self driving taxis is $10000 to $15000 per vehicle. By 2025, his goal was “ the cost of autopilot system had been reduced to less than US $5000 &rdquo.
Shashuya’s speech convinced us that many industry observers, including many, have not taken Mobileye’s ambitions seriously enough. As a leading ADAS supplier, Mobileye has many very important strategic advantages. The company’s existing relationship with carmakers means there is little problem deploying new car technology in the real world.
Mobileye has successfully secured access to the customer’s car sensor data, enabling it to access a large number of valuable data sets that most automated driving companies (except automakers) can access. It has deep engineering expertise in building autopilot systems, and the impressive performance of its autopilot prototype proves this.
Mobileye’s open approach to developing autonomous driving technology is also laudable. Mobileye has developed a mathematical model for autonomous vehicle safety called responsibility sensitive safety, and is trying to promote it as an industry standard. Shashuya’s demonstration provides more information about the technical structure of its autopilot system than almost anyone in the autopilot industry. However, Mobileye still has a lot to do to convince the public that its technology is safe. Many of Mobileye’s competitors refuse to explain how their systems work, leaving the public unable to assess the security of their technology. (reviewed by Tencent technology / Jinlu)