Professor Zhang Wenming's team at Shanghai Jiaotong University proposed the concept of acoustic hyperspheric surface to track weak sound sources in complex environments

On September 28th, Professor Zhang Wenming's team from the School of Mechanical and Power Engineering of Shanghai Jiaotong University published the paper "A wave-confining metasphere beamforming acoustic sensor for superior human-machine voice interaction" in Science Advances, proposing a new concept of acoustic hypersphere.
Based on the principle of local resonance and beamforming of hyperspherical defect cavity array, it is the first to design an acoustic hyperspherical sound sensor, which shows a number of functions such as omnidirectional pickup, sound pressure amplification, sound source tracking, high-performance audio cloning and speech recognition, which can locate and identify sound sources at adjacent angles even in a strong background noise environment, and achieve excellent human-machine acoustic interaction performance
.
Kejing Ma, PhD student in the School of Mechanical and Power Engineering, and Huyue Chen, PhD student at the Michigan Institute, are co-first authors, and Wenming Zhang and Lei Shao, associate professor of the Michigan Institute, are co-corresponding authors
.

Acoustic hyperspherical and acoustic-electrical coupling effects

Dialogue is the most common and easy way of interpersonal communication, and it is also an important development direction
of intelligent human-computer interaction technology.
This technology requires acoustic sensors with both ultra-high signal-to-noise ratios and sensitivity, as well as the ability to accurately identify, locate, and track multiple voices
in noisy environments.
At present, commercial microphones and emerging ultra-sensitive thin-film sensors cannot solve the fundamental problem of rapid dissipation of sound waves in space, and various smart speakers and conference room omnidirectional microphones are often difficult to effectively pick up
.
Acoustic metamaterials, on the other hand, have unlimited possibilities for modulating and manipulating sound waves and have been shown to be able to amplify and separate
sound waves.
However, how to use acoustic metamaterials to simultaneously achieve ultra-high signal-to-noise ratio and sensitivity parameters in the vocal frequency range, passive amplification, separation and positioning of multiple sound sources, and realize the use in practical application scenarios is an important challenge
in the field of speech sensing and interactive recognition.

Design concepts and physical mechanisms for acoustic hyperspheres

The research team proposed the concept of "acoustic superspheroid", and constructed an approximate orthodohedral acoustic metamaterial and defective cavity structure to verify its acoustic resonance characteristics
.
This strategy is based on the principle of local resonance, constrains the sound waves to the defect position in the center of each positive pentagon, passively guides and amplifies the sound waves, and can obtain twice the intensity of the sound field at the emission end, making up for the rapid and large dissipation
of sound waves in space under the long-distance perception scene.
Excellent signal-to-noise ratio (72 dB) and excellent sensitivity (137 mVpp/Pa or -26.
3 dBV) are also utilized with low-noise piezoelectric conversion performance in defective chambers
.
As a result, the research team achieved excellent audio cloning, authentication, and speech recognition for a variety of human-computer interaction functions
.

Conference assistance: Space with multiple sound sources and different angles

At the same time, based on the defect cavity array, combined with beamforming algorithm and machine learning algorithm, the research team also realized the real-time location and tracking of multiple sound sources, and demonstrated the powerful functions
in multiple application scenarios such as online conference assistance and factory patrol search and rescue.
The acoustic hyperspheric system not only successfully identifies multiple users who sound at the same time at adjacent angles in space, but also tracks human voices
drowned out by strong background noise.
Even in the case of heavy mixing of sound signals from multiple adjacent angles in space, acoustic hyperspheroids can distinguish different sound source information and orientation
based on normalized energy maps.

This research uses physical intelligence to construct a multi-functional spatial omnidirectional acoustic hyperspherical sensor, combined with a variety of intelligent algorithms to optimize the system function, to achieve multi-scenario excellent human-computer voice interaction system, providing new ideas
for the development of a new generation of intelligent robot hearing system and human-computer voice interaction technology.

Factory search and rescue: strong background noise and mobile vocal tracking identification

The research work has been funded
by the National Natural Science Foundation of China, the Youth Fund Project and the Hong Kong, Macao and Taiwan Science and Technology Cooperation Project of the Shanghai Municipal Science and Technology Innovation Action Plan.

Paper link: