Redefining Human Interaction with Wearable Tech and AI

Published March 12, 2025

Yang Bai, a graduate student advised by MC2 faculty member Nirupam Roy, is developing wearable technologies and AI systems that can sense and respond to their environment in real time, with multiple applications ranging from disaster relief to everyday communication.

Research and innovation combining wearable technologies and artificial intelligence (AI) could soon revolutionize how we engage with the world—creating smarter, more intuitive systems that not only understand their environment, but are able to respond to said environment in real time.

Yang Bai, a fifth-year computer science doctoral student at the University of Maryland, is active in this area, developing micro-robotic systems and advanced wearable devices that are based on sensing technologies. Her research, which consistently pushes the boundaries of what’s possible, could have a big impact, from making person-to-person communication easier to helping locate survivors in disaster situations.

One project involves earphones able to instantly detect the direction from which people are speaking—without relying on bulky microphone arrays or power-hungry systems. Instead, Bai’s devices use microstructures—various materials that can play a crucial role in sound generation and absorption—to track audio sources with remarkable accuracy, all in real time.

This technology might soon allow users to ask OpenAI’s ChatGPT or Meta AI’s Llama questions like: “Can you summarize the opinions of the person sitting on my left in this meeting?” With environmental data fed directly from wearable devices, the AI could provide live, context-aware summaries, effortlessly integrated into daily life.

Bai’s interest in combining sensing technology with AI began during her undergraduate studies in electrical engineering at Dalian Maritime University in China. However, it was not until earning her master’s degree at Rutgers University that she discovered her true calling.

The lightbulb moment came during a graduate course, wherein Bai learned how wireless signals could be used to sense the environment and track human movement.

“That’s when I realized the vast potential to change how we interact with technology,” she says.

This newfound passion led her to the University of Maryland, where she began her Ph.D. studies under the guidance of Nirupam Roy, an assistant professor of computer science with an appointment in the University of Maryland Institute for Advanced Computer Studies.

Roy’s expertise in acoustic sensing, particularly his work with microphone arrays, laid the foundation for Bai’s work. But it has been Bai’s innovative approach to complex problems and constant drive that have truly set her work apart.

“When I started this research, I realized that while large language models like ChatGPT are powerful, they lack an understanding of the physical world,” she explains. “They can process text and provide answers, but they don’t grasp what’s happening in the room. That’s where sensing technology comes in.”

Bai is also involved with micro-robots for disaster relief. Equipped with advanced navigation systems, these robots can navigate debris from collapsed buildings to locate survivors in areas too hazardous for human rescuers. Her initial research in this area earned the Best Paper Award at the 2022 ACM International Conference on Mobile Systems, Applications, and Services.

In Roy’s iCoSMoS Lab, Bai provides mentorship to undergraduates and junior Ph.D. students on several projects that include using microphone arrays for augmented reality glasses and improving security in voice-assisted devices. Roy praises Bai not only for her academic excellence, but for the example she sets for others.

“Yang’s innovative approach to wearable technology and her dedication to mentoring others highlight her exceptional skills and commitment to making a positive impact both in and outside the lab,” he says.

Bai, who is expected to graduate with her Ph.D. in May, is eager to apply her research to real-world applications. She recently accepted a position with Apple's applied sensing and health team, continuing the work she began during her summer internship there last year.

“I’m excited to take everything I’ve learned and apply it to products that can truly improve people’s lives,” she says. “Moving to California is a big step, but I’m ready for the adventure.”

—Story by Melissa Brachfeld, UMIACS communications group