Gaming Beyond Sight: Power of Sound

The Unseen Challenger: How Sound is Redefining the Limits of Gaming

In the high-stakes world of esports, Carlos Vasquez, known by his username Rattlehead, presents a unique challenge to his rivals. At tournaments across the United States, opponents quickly realise that he, in his own words, is “completely blind”. This discovery often leads them to lower their guard, assuming an easy match in the popular fighting franchise Mortal Kombat. Frequently, this assumption proves to be a costly mistake. Vasquez consistently triumphs, relying not on sight, but on an exceptionally keen sense of hearing to navigate the complex combat scenarios. By distinguishing the individual sound cues for each character, he executes both his attacks and his defensive actions with startling precision.

A Journey into Darkness

Hailing from the Texan city of Houston, Carlos Vasquez began his journey with video games in 1992 when he was only six years old. A life-altering diagnosis of glaucoma at eleven marked the beginning of a gradual decline in his vision. The loss was not sudden. For several years, he could still discern the general shapes of things, though the fine details were gone. Now thirty-seven, Vasquez recalls how his sight progressively worsened. Around the age of twenty-four, he was living in a world without light. This transition presented immense challenges, particularly in the world of gaming, which is predominantly a visual medium.

Adapting to a Silent World of Sight

During his twenties, Vasquez discovered that playing the majority of popular titles had become almost unplayable. The concept of gaming accessibility, which encompasses features designed to assist players with impairments, was largely non-existent. Such aids, like screen readers that verbally describe on-screen action, were not yet a staple of game design. Despite these limitations, he found he could still engage with certain fighting games, with a particular affinity for Mortal Kombat. The reason was not that Mortal Kombat possessed accessibility options back then, but rather that gamers such as himself could learn to interpret the game's distinct audio cues, allowing for a rich and competitive gameplay experience.

The Language of Sound

Vasquez mastered the art of listening. He learned to identify which side of the screen his character was on through subtle auditory differences. A fighter positioned on the screen's right side, for instance, would produce a grunt with a marginally sharper tone compared to the fighter on the left. This minute difference was enough for him to orient himself within the game's virtual space. This reliance on sound design, though born of necessity, highlights an often-overlooked aspect of game development. For Vasquez and others, the auditory landscape of a game is not just atmospheric; it is the entire world. His ability to decipher this world gave him a competitive edge that many sighted players failed to anticipate.

A New Era of Accessibility

In the current gaming landscape, the industry has made significant strides. An increasing collection of titles, including Street Fighter 6, The Last of Us, Diablo 4, and the most recent instalment of Mortal Kombat, now incorporate extensive accessibility features. The racing game Forza Motorsport contains a system named Blind Driving Assists. This system provides verbal information regarding upcoming corners and how severe they are. A range of audio signals also communicate the vehicle's speed, whether it is pointing the wrong way, and the correct moment for a gear shift. These innovations are transforming the gaming experience for players with vision loss around the globe.

Mortal Kombat's Auditory Overhaul

The newest Mortal Kombat has introduced a wide array of supplementary sound cues, setting a new standard for the fighting game genre. These features provide crucial information that was previously unavailable to visually impaired players. For instance, audio cues now provide updates on the space separating the combatants, allowing for more strategic positioning. Another sound alerts players if an adversary crouches, providing a critical piece of information in a fast-paced match. These additions, developed with input from consultants like Vasquez, demonstrate a significant shift in the industry's approach to inclusive design, ensuring that gameplay is accessible without sacrificing its competitive integrity.

The Science of Spatial Audio

Forza Motorsport also demonstrates the expanding application of a technology known as spatial audio. This innovative system employs sound projected from specific directions to assist players with vision loss in determining their precise position within the game’s environment. Professor Brian Smith, a leading authority on the subject from Columbia University in New York, explains that this technology mimics how humans perceive sound in the real world. When a sound occurs, the actual sound wave arrives at our two ears at slightly different moments and volumes. Our brain processes these subtle differences to pinpoint the sound's origin. Smith's work focuses on applying this principle to the sound design within a digital game.

Crafting a Virtual Soundscape    

Professor Smith’s research at the Department of Computer Science at Columbia University is at the forefront of this technological wave. His team works to translate the real-world, directional principles of hearing into the virtual environments of video games. The technology is compatible with both standard speakers and headsets, but its effects are enhanced with specialised hardware for spatial audio. Professor Smith also noted that it can be so potent in assisting those with visual impairments to navigate characters that a key challenge for developers is striking the right equilibrium. The goal is to provide accessibility without diminishing the inherent challenge that makes games compelling.

The Impact of Grassroots Innovation

While large development houses are now wholeheartedly adopting accessible design, the path was often paved by freelance, enthusiast creators. Many of these pioneers themselves have vision loss, driven by a personal understanding of the community's needs. Seven years ago, The AbleGamers Charity, an organisation that advocates for more accessible games, approached developer Rockstar. They inquired what the expense would be to render its blockbuster title Grand Theft Auto V accessible. The reported estimate was a staggering $128m, which the company deemed too expensive. This response could have been the end of the story.

A Mod for Change

In response to this challenge, a freelance software creator named Liam Erven took action. Erven, who has had no sight from birth, decided to create a custom software addition, often called a mod, designed to enhance the experience of Grand Theft Auto for those with sight loss. Working alongside a group of partners, he developed a mod called Grand Theft Accessibility. The features they added were transformative. Sound clues provide players with details about objects, people, and vehicles in their vicinity. Users also receive audio information regarding their position, direction, and selected weapon, all achieved for significantly less money than Rockstar's estimated cost.

Raising Industry Awareness

Liam stated that while the add-on has its imperfections, it remains exceptionally fun to experience, and it has successfully attracted a loyal following of players. Professor Smith asserts that these independent creators have had a major role in building recognition within the industry. They have helped in prompting major development houses to understand that making enhancements for gamers without sight is perhaps less difficult than they imagined. The success of grassroots projects like Grand Theft Accessibility has shown that innovation in this space is not only possible but also highly valued by the community it serves.

The Need for Representation

Despite this progress, Professor Smith notes that the gaming sector itself still has a notable lack of representation from blind individuals. This absence can lead to misconceptions among developers, who may hold the misguided notion that accessibility for players without sight is a self-contradictory idea. Without firsthand experience or direct consultation, it is easy for development teams to underestimate the capabilities of visually impaired players and the potential for innovative, non-visual gameplay mechanics. The inclusion of more diverse voices in the development process is essential for creating truly inclusive and engaging experiences for all players.

From Player to Pioneer

Nonetheless, circumstances are steadily advancing. Carlos Vasquez is a testament to this positive shift. Today, he is not just a formidable opponent in Mortal Kombat. He currently has a role with NetherRealm Studios, the company behind the game, where he provides his expertise on accessibility. In this capacity, he provides invaluable feedback, helping to shape the future of accessibility in one of the industry's most iconic franchises. He voiced his desire for gaming corporations to bring more individuals with his experience onto their teams, ensuring their lived experiences inform the design process from the ground up.

A Call for Consultation

Vasquez observes that, in earlier times, development teams would simply make assumptions about the needs of players with impairments instead of directly engaging them for advice. This approach, though well-intentioned, often missed the mark. He stresses the importance of connecting with individuals whose daily lives are shaped by these challenges to gather their insights. Whether through in-studio roles or early access testing, direct collaboration is key. This sentiment is echoed by many in the accessibility community, who argue that authentic inclusion requires more than just adding features; it demands a fundamental shift in how games are designed and who is involved in their creation.

The Future is Transparent           

Looking ahead, Vasquez believes the entire industry must improve its level of open communication regarding accessibility. Players should know before purchasing a game whether it will have the features they need to play. This transparency allows players to make informed decisions and encourages developers to prioritise accessibility from the outset. As more studios embrace this philosophy, the virtual worlds of gaming will become more welcoming to all. The journey towards a fully accessible gaming landscape is ongoing, but with advocates like Carlos Vasquez leading the charge, the future looks bright, and sounds clearer than ever.