Breakthrough in Brain-inspired Visual Perception Tsinghua University Develops World's First Complementary Visual Chip
TapTechNews May 30th news, a research team led by Professor Shi Luping from the Brain-inspired Computing Research Center of the Department of Precision Instruments at Tsinghua University proposed a new paradigm of brain-inspired visual perception based on visual primitives with complementary dual pathways, and accordingly developed the world's first brain-inspired complementary visual chip Tianmou Chip, achieving high-speed, high-precision, and high-dynamic-range visual information acquisition, breaking through the performance bottlenecks of traditional visual perception chips in terms of stability and security.
Today, this achievement was published as the cover article in the top academic journal Nature under the title Brain-inspired Complementary Visual Chip for Open-world Perception (TapTechNews attaches DOI: 10.1038/s41586-024-07358-4).
The official said that traditional visual perception chips are limited by the power wall and the bandwidth wall, and often face problems such as distortion, failure, or high latency when dealing with unexpected and accidental scenarios, seriously affecting the stability and security of the system.
In order to overcome these challenges, the brain-inspired computing research team of the Department of Precision Instruments at Tsinghua University proposed a new paradigm of brain-inspired visual perception based on visual primitives. This paradigm draws on the basic principles of the human visual system, disassembles the visual information of the open world into information representation based on visual primitives, and by organically combining these primitives, imitates the characteristics of the human visual system to form two visual perception pathways with complementary advantages and complete information.
Based on this new paradigm, the team further developed the world's first brain-inspired complementary visual chip Tianmou Chip, achieving high-speed of 10,000 frames per second, high-precision of 10-bit, and high-dynamic-range of 130 dB visual information acquisition at an extremely low bandwidth (reduced by 90%) and power cost.
Based on the Tianmou Chip, the team also independently developed high-performance software and algorithms, and verified the performance on an open environment vehicle-mounted platform. In a variety of extreme scenarios, the system achieved low-latency and high-performance real-time perception and reasoning, demonstrating its application potential in the field of intelligent unmanned systems.
It is worth noting that this is the second time that this team has appeared on the cover of Nature after the heterogeneous fusion brain-inspired computing Tianxin Chip, marking that China has made fundamental breakthroughs in both the important directions of brain-inspired computing and brain-inspired perception.