As a leading enterprise in the ADAS industry, Mobileye currently holds more than 60% of the global market share of visual perception chips. At the same time, the closed visual perception ecosystem established by Mobileye has also become the mainstream form of the industry. Conditional open and relatively convergent perception API requirements have helped Mobileye establish standardized perception interface solutions and quickly promote products to the world, attacking cities and territories.
However, time has changed. As current visual ADAS capabilities continue to be upgraded and iterated, the industry expects more open solutions to help OEMs and Tier1 create differentiated and more adaptable ADAS capabilities.
The complexity and locality of driving scenarios call for open solutions
Once upon a time, a domestic OEM tried to develop an autonomous parking solution based on Mobileye’s visual perception solution. During the implementation of the project, it was necessary to identify the railing at the entrance of the parking lot. However, Mobileye’s closed solution did not support customers to update the perception algorithm independently, resulting in development difficulties.
This reflects the dilemma faced by OEMs. On the one hand, Mobileye’s visual perception solution is still the most mature solution at present, with many advantages in terms of product maturity, project risk control and quality management, which OEMs cannot refuse. On the whole, it is a situation of more than conservative and less aggressive. But on the other hand, this closed solution is like a black box. Because of the inability to differentiate and customize development, the performance is homogenized; moreover, in localized driving scenarios, it cannot fully meet the requirements. This limits OEMs to make more aggressive product solutions, which will reduce product competitiveness in the long run.
The particularity of China’s driving scene requires a localized sensing solution to meet the functional requirements of ADAS/autonomous driving.
At the same time, in order to meet the high reliability requirements of ADAS/autonomous driving for perception, more open perception solutions are also required. For example, if we can identify richer targets and more types of semantic segmentation, then we can obtain richer semantic information, so that different categories can be cross-validated. For example, road shoulders and sidewalks have a clear auxiliary verification function for the judgment of the drivable area. Fixed targets on the roadside, such as traffic signs and street lamps, are of great help in positioning.
If the road is covered with snow on a snowy day, how can you tell where the shoulder is? The movie "If You Are the One" describes a dialogue plot:
"Hey, you’ve been to [Hokkaido], you must know what the arrows on both sides of the road are for?" Answer: "Oh, in winter, the snow in Hokkaido is very thick, and the arrows instruct people not to drive out of the shoulder of the road."
If we can recognize the arrows on either side of the highway, we can reason about the boundaries of the drivable area. The development trend of perception technology requires more open solutions
The development of perception technology can be summarized as follows:
From simple scenes to complex scenes
From high-frequency targets to general targets
From 2D perception to 3D perception
From reality-oriented perception to prediction-oriented perception
All of these trends have further increased the richness and complexity of perception, making it impossible for vendors to use a standard perception solution to meet their needs.
In the era of software-defined cars, automakers need a more open approach
Intelligence is the core element of brand differentiation in the future, mainly through the addition of software functions. The post-deployment of software will be the general trend, which means that most software functions will be delivered after the car leaves the factory, and software iteration OTA will be the new normal. This trend is particularly important for mobility service operators. Service requirements in various scenarios need to be based on the functions of existing fleets and met by continuous upgrades and iterations.
In the future, the car delivered by OEMs will not be a product with solidified functions, but a robot that continues to evolve. Throughout the life cycle of the car, the hardware platform can continue to support software iterative upgrades. The efficiency of software development and differentiated functions will determine the success or failure of this intelligent competition.
In order to improve the efficiency of software development, from the perspective of system architecture, service-oriented system architecture (SOA) will become the mainstream, which requires the creation of a new perception solution that meets the requirements of four aspects: a highly open, consistent and complete tool chain, a strong computing power reserve, and strong scalability to meet the requirements of different levels of vehicle platforms.
Open perception intermediate results help domestic ADAS functions continue to evolve
At present, the perception algorithm API is still relatively closed in the industry. Many features are difficult to implement (such as the augmented reality display function of ADAS function in the infotainment domain), in part because the perception algorithm is only provided for internal use of the intelligent forward-looking camera module and is unwilling to be provided to other subsystems. And the Horizon perception intermediate result can be fully open. And because the low-level semantics of the Horizon algorithm are very rich, the fully open perception intermediate result can support customers to develop more complex functions at the application layer.
So far, the Horizon algorithm can support 10 types of dynamic targets and 53 types of static targets. Among them, dynamic targets include: adults, children, cyclists and other pedestrians, as well as cars, SUVs, vans, trucks, passenger cars, motorcycles and elderly scooters 7 categories; while static targets include 8 categories of lane lines, 2 types of traffic lights and 43 types of traffic signs. The richer perception information than Mobileye provides a solid perception foundation for customers to realize differentiated functions.
Rich Perceptual Information Provided by Horizon Algorithm
Open and comprehensive toolchain, practicing the concept of "deeply empowering"
In order to increase the agility of visual algorithm iteration and better support various extreme perceptual scenarios in China, Horizon has launched the AI chip tool chain Horizon OpenExplorer (Horizon "Tiangong Kaiwu"), including data, training and device deployment tools, such as model training tools, inspection and verification tools, compilers, emulators, embedded development kits, etc. (Figure 9), forming a closed loop. Data generates models, which can be deployed to devices for operation, and can guide the tuning of models during operation, and even collect new data. Such a self-evolving development model can improve development speed, lower development thresholds, and ensure development quality. Based on this model, development manpower can be reduced by about 30%, development time can be saved by 50%, and more importantly, because the development threshold is lowered, the scale of developers can even be expanded by an order of magnitude.
Horizon will continue to upgrade the power builder to provide customers with a semi-automated processing flow. Mainly include: closed-loop iteration between data tools and models, models and on-end devices; rich model/system reference prototypes, simple and easy to use intuitive and convenient interaction means; standardized development process, plus continuous testing, integration, and deployment mechanisms.
Horizon "Tiangong Kaiwu" chip tool chain
More specifically, Horizon’s model training tool can support mainstream deep learning frameworks such as TensorFlow to help users train their own models; the compiler supports converting the open-source training framework model format to the binary format on the chip; and the embedded development kit can support customers to call the algorithm library to develop their own applications, allowing customers to quickly deploy applications in chip manufacturers. The entire toolchain package can cover the complete development link (Figure 10). After optimization by the Horizon compiler, the memory access bottleneck of the algorithm can be greatly alleviated and the computing efficiency of the chip can be improved (Figure 11).
Development Process Based on "Tiangong Kaiwu" Tool Chain
Algorithms automatically optimized by the Horizon compiler provide significant performance improvements
Recently, Horizon partnered with South Korea’s SK Telecom to develop a dynamic crowdsourced high-definition mapping solution. SK used Horizon’s toolchain to develop a visual perception algorithm for Korean roads, demonstrating the ease of use and reliability of the toolchain.
In short, the opening of Horizon is a deep and multi-dimensional comprehensive opening from providing system reference solutions, to fully open sensing results, and then to full-stack solutions of toolchains, fully empowering the intelligent development of the automotive industry. If customers advocate the concept of division of labor and cooperation to maximize efficiency, Horizon provides software and hardware integrated chip solutions; if customers prefer to maximize capabilities and want to use their own algorithms, Horizon provides customers with pure chips and a complete toolchain to help customers achieve deep enough development freedom and practice the long-term commitment of "deep empowerment".
