Coding Color Into Sound: Hacking for Assistive Technology

Have you ever thought about what a bird sounds like?

You’re probably assuming I’m referring to a bird’s audible song, but what if I told you there was more than one way to ‘hear’ an object? What if it was possible to hear things that don’t physically produce a sound – like a sunset, or the night sky?

It sounds a bit unconventional, but this type of outside-of-the-box thinking is exactly the kind of ‘hacking’ that Lenovo employees are up to these days.

In 2020, Lenovo’s PC & Smart Devices product development team partnered with the Office of Diversity and Inclusion to launch Lenovo’s Product Diversity Office (PDO). As the authority on inclusive product design, the PDO introduces new protocols Lenovo’s product planning, development, and execution phases, and consults with a broad range of diverse users to validate designs and provide feedback to ensure that our solutions are inclusive of a wide variety of users.

Perhaps the most prolific opportunity for inclusivity lies within the community of more than 1 billion people around the world who experience some form of disability or impairment, many of whom are underserved by the tools and ways of life that the abled population has adopted.

With more than 63,000 employees worldwide, Lenovo’s PDO has an enormous wealth of diverse insights, perspectives, and expertise at their fingertips to not only help inform their checkpoints for the product life cycle, but also to dare modern technology to think bigger.

Through sponsored hackathon events, employees like Barbara Xu are free to push the boundaries of ordinary devices to explore tech’s full, transformative potential.

As part of a recent “Smart Assistive Tech” hackathon lead by the Cloud and Software organization in partnership with the PDO, Barbara and her team of self-proclaimed “nerd crafters” are digging into the challenges associated with disability and the ways in which experience can be improved through innovation. Their product idea, which is currently in the concept development phases, explores the intersection of sound and color as means of assisting the visually impaired to see things in a different way.

“Most advances in aiding visual impairment focus on Braille scripts,” says Barbara. “But we feel that life is beyond words. It’s more than that. It has color, it has rhythm, it has emotion—it’s multi-dimensional.”

By mapping sound and color frequencies, Barbara and her colleagues demonstrate how the brilliant hues of a North American Blue Jay can be transformed into a bright and happy symphony of bell tones – a phenomenon they theorize could be engineered to fit most any visual scene.

A data scientist in software engineering, Barbara is no stranger to tinkering with the ‘what ifs’ of product design, but she’s also joined by colleagues from less technical backgrounds.

“That’s the fun part about hackathons,” says Bridgett Rogers, Director of Customer Experience and Product Diversity. “It brings out the entrepreneurs who are comfortable stepping outside the lines of their day-to-day roles and explore their passions through the creative medium of product design—you don’t have to be a developer to be involved in product design.”

By partnering closely with Lenovo’s employee resource group community, the PDO casts a wide net for employee engagement and brings together the best of both worlds – advocates and allies for historically under-represented groups, and the design experts who can help put their ideas into motion.

Aik Hong Goh, finance controller and leader of the WILL (Women in Lenovo Leadership) chapter in Singapore, is one such advocate.

“I personally enjoy coming up with new ideas; I’m a bit nerdy,” she says. “Science is what I enjoy on the side.”

Aik Hong and her team are thinking ahead to the needs of the late stages of the average human lifecycle, a critical juncture where the need for assistive technology greatly increases in response to the loss of memory, sight, and hearing that often comes with old age.

“There may come a day when we suffer from dementia and have difficulty remembering our daily tasks, or as we grow older our vision may naturally decline” she says. “It’s not just about providing solutions for immediate needs, but also recognizing that we all may encounter some degree of impairment in our lifetime; it’s very much assisting our future selves.”

Aik Hong views the act of bringing assistive tech to the market as part of a larger responsibility that Lenovo has in making the world a more inclusive place.

“With our reach as a global tech leader, we’re influencing our customers and partners to become more inclusive, not just with our words, but our actions,” she says. “We can really show the world what it is to stand behind our vision of smarter technology for all.”

“What is technology if it’s not assistive?” she asks. “A calculator assists us with our calculations, an elevator assists us in getting to our destinations –what enables us to call an innovation ‘smart’ or ‘assistive’ is that we can be assisting our future selves. At the end of the day, technology doesn’t just invest in the here and now.

“Whatever we’re investing in today is for our tomorrow – everyone’s tomorrow.”