HEAR ME: Empowering Communication for All – An Interview with Xiaobi Pan

HEAR ME is an innovative app that leverages AI to empower people with hearing and speech disabilities, making communication more accessible and life more fulfilling. In this in-depth interview, Xiaobi Pan, founder of HMW Design Studio, shares her journey, the inspiration behind the project, and the challenges faced in creating a truly transformative solution.

Award: Winner in Social Responsibility (C2A)
Project: HEAR ME: AI-Based Smart Personal Assistant for People with Hearing and Speech Disabilities
Firm: HMW Design Studio
Lead: Xiaobi Pan

Can you tell us a bit about your background and how it has influenced your creative practice?

Xiaobi Pan: I was born and raised in China. In 2017, I moved to the U.S. to study at Harvard GSD and the MIT Media Lab. After graduation, I began my career in New York and later transitioned to the Bay Area, where I’ve worked at the forefront of design and technology with companies like Mastercard Labs and Amazon.

My practice is rooted in exploring the intersections of emerging technologies such as XR, AI, and physical computing. I focus on creating experiences that bridge the physical and digital worlds, emphasizing empathy and fostering deep human connections. I believe that design isn’t just about aesthetics or function—it’s about creating meaningful, impactful moments that resonate emotionally and culturally.

Beyond my corporate work, I lead my own creative practice, HMW Design Studio, where we experiment with the edges of design, innovation, and social impact. We are committed to shaping a future where creativity serves as a catalyst for positive social change. With a focus on impact-driven design innovation, our studio crafts experiences that are both forward-looking and deeply grounded in today’s societal challenges.

What was the brief for your award-winning project, and what were the main goals?

Xiaobi Pan: HEAR ME is an app that leverages AI technology to empower people with hearing and speech disabilities in all aspects of their life. Users can have their selected avatar interpreters translate between spoken and sign languages for them during video calls, face-to-face conversations, movie/TV, etc. They can also book real-world interpreters on the app based on location and needs. Lastly, the app provides real-time mood tracking, and its AI chatbot assistant is there to cheer up and support the users 24/7.

More specifically, HEAR ME’s AI Avatar Sign Language Interpreter as well as On-Demand Human Interpreter features can translate between sign languages and spoken languages during face-to-face conversations, video chats, or TV/movies to help deaf people bridge the communication gap. The 24/7 Mood Tracking and Support feature helps them deal with emotional fluctuations. Meanwhile, the AI Chatbot Personal Assistant can answer their questions, give advice, perform tasks, or even tell jokes to cheer them up when needed. All these features are designed to help them communicate better and live better.

What inspired you to create HEAR ME?

Xiaobi Pan: This project stems from the real-life story of one of our studio members’ deaf and non-vocal friend. People with hearing and speech disabilities are a largely underserved community, and we wanted to use the insight gained from first-hand observation, interviews, and follow-up studies to help these individuals live an easier, happier, and more fulfilling life. We believe AI is the way to go since it provides the possibility for highly personalized and intelligent support for the deaf and non-vocal community.

What challenges did you face during the creation of HEAR ME, and how did you overcome them?

Xiaobi Pan: The main challenge when designing HEAR ME was integrating AI-driven features while prioritizing accessibility and inclusivity. Overcoming this required extensive research into the needs and preferences of the deaf and non-vocal community, as well as innovative technical solutions. Internal factors like historical stigma surrounding disability and external factors such as the evolving technology landscape and legal framework posed additional hurdles. However, with a multidisciplinary approach and dedication to user-centric design, we’re able to overcome these challenges and come up with a truly transformative solution that marries technology with user needs.

What sparked your passion for design, and how has your creative journey evolved?

Xiaobi Pan: I was an engineering student before I majored in industrial design in college. It stemmed from hearing a lecture by an esteemed professor from the industrial design department in our college, which was purely serendipitous.

My creative journey started with a focus on physical design, exploring how tangible materials and forms shape human interactions. Over time, I transitioned into the digital space, where I discovered the power of virtual tools like XR and AI to create immersive and boundaryless experiences.

Now, I work at the intersection of the physical and digital, blending the two to craft empathetic, human-centered designs. This approach allows me to push the limits of both realms, creating meaningful connections that feel both grounded in reality and open to new possibilities.

How do you stay updated on the latest trends, such as AI, and what impact do you see it having on the creative industry?

Xiaobi Pan: I follow a number of AI/creativity influencers on LinkedIn and other social media platforms to get inspired. I also do hands-on research on my own with side projects.

I’m deeply impressed by the creative capabilities of AI tools and I believe one of the biggest challenges as well as opportunities for designers of our time is to figure out our symbiotic relationship with AI design/creative tools. Actually, I’m working on synthesizing my learnings into a workshop form to share with fellow designers and creative technologists—so stay tuned.

What steps do you take to continue growing as a designer and creative professional?

Xiaobi Pan: I dedicate time to learning new tools, creating passion projects outside of work, as well as connecting with industry pioneers and researchers weekly to learn from others.

Also, I choose to collaborate with experts and creatives on a regular basis to force myself to step out of my creative comfort zone, expand my horizons, and avoid the echo chamber effect of an isolated creative workflow.

What inspires your imagination, and how do you nurture creativity in your daily life?

Xiaobi Pan: I find inspiration in nature, music, and interacting with animals—especially the connection with plants and animals (for example, my cat Bubble).

These interactions draw my attention to tiny details and engage my five senses, keeping me grounded in the physical world while nurturing my creativity.

Who are your biggest influences or the creatives you admire most in your field?

Xiaobi Pan: I draw inspiration from a diverse range of creatives, including musicians, artists, industrial designers, and graphic designers. I’m usually more inspired by versatile artists who make their own music, direct their own videos, design their own clothes, and even do their own choreography. I think creativity comes from connecting with your inner child, your consciousness, and connecting with nature and spirituality.

How do you balance your clients’ needs with your personal creative style?

Xiaobi Pan: I think design is at its core, a service. So I pride myself on solving the client’s needs creatively within constraints. I don’t really prioritize my personal style when I’m designing for a client. Instead, personal style comes naturally even when I’m not consciously thinking about it, and in my view, that’s the only way when it’s appropriate.

How do you feel about receiving a Creative Communication Award?

Xiaobi Pan: It’s a great recognition and a big motivation. I’m grateful for this opportunity and I look forward to continuing this amazing creative journey.

Designing the Future

HEAR ME serves as a powerful demonstration of how design can drive social change. By leveraging AI to empower a long-underserved community, this project not only transforms communication for people with hearing and speech disabilities—it also redefines what it means to design with empathy and innovation. This inspiring journey motivates us to continually push boundaries and create technologies that truly improve lives.