Sound controls

Sony titles may feature sound generation based on user environment

VR and AR games have grown tremendously in recent years, becoming a new phenomenon in the gaming industry. The upcoming Sony VR experience Horizon Call of the Mountain and popular AR games like Pokemon Go are ideal examples of the technical capabilities of these new technologies.

Besides the staggering developments in VR and AR technology, immersion still seems to be lacking, but that may change soon in the future. An important aspect of video game immersion is sound generation, and game studios are finally struggling to improve the stagnant game mechanic.

We came across a new patent published by Sony named “IMPULSIVE RESPONSE GENERATION SYSTEM AND METHOD,” which will produce sounds in the game considering the real user environment. For example, for a player inside a room or park, the gameplay audio will be made to reflect their current background.


Main takeaways:

  • Sony has released a new patent that will produce in-game sounds based on the user’s current personal environment; it primarily deals with AR and VR titles, but expands beyond that.
  • It’s a big upgrade from the current ‘surround sound’ system which only makes you feel surrounded by speakers in a closed room. This system will instead mirror your unique background to provide an immersive experience.
  • Gameplay audio will differ depending on whether a player is in a closed or open space, and other similar variables will also be taken into account.
  • The VR or AR device (a phone or a headset) will capture images of the player’s current background to infer how the game audio should be produced.

Sony’s patent serves as an upgrade to the current one”surround sound” technology, which was created to enhance immersion. However, this method is not viable when the user’s environment has to be considered. Thus, the patent proposes a new system for AR and VR games to increase immersion at the user’s personal level.

Sony says:where augmented reality content is provided that is associated with an audio output, that audio output will be expected to sound different depending on the environment in which the augmented reality content is played.

Additionally, the patent cites the need for the proposed system to produce in-game audio to automatically reflect the player’s environment. He keeps on, “A user viewing this content in a small room at home will expect the audio to be different compared to a user viewing content in an echo cave.

The diagram shows a user playing an AR/VR game in front of a monitor and speakers.

It is impractical to manually generate audio considering the environment, as each player can indulge in various and diverse settings. Some users can play AR games in their bedroom, while others can sit on a bench in a park while playing.

He specifies, “for a popular product, it could be hundreds of thousands of different environments because each user is likely to view the content in a unique environment.“Additionally, manually collecting data for each environment will be very difficult and time-consuming, according to the patent.

The system will use something called a “dynamic impulse response generation process,which will be allowed for an environment to perform. He further continues, “This is enabled by using the properties of a known sound in the environment with a recording of the sound, the recording being captured in the considered environment.

It is dynamic because the process can be executed at any time. For instance, “whenever a user enters a new environment or whenever new content is initialized.“In addition, the process can be performed for a”number of slots in an environment.

Diagram showing the impulse response generation method and the hardware required for it.

Sony’s proposed system will work just fine for gaming, as noted. “The application may be particularly suitable since immersive games may provide content that is displayed in such a way as to give the impression of being in the room (i.e. environment) with the user.“The user will have the impression that the game takes place in his environment.

One of the most reviewed examples is AR and VR technology. The AR or VR device (which can be a headset or a phone) will capture images of the environment to infer the user’s background. After determining the captured images, the location will be identified to produce the audio corresponding to the setting.

When using an AR arrangement, there may be an outward-facing camera that can work to capture images of the environment. From these images, it may be possible to identify the size of the user’s local environment or a context that indicates a likely location (such as large green spaces indicating that the user is outside in a park).

Many variables will be measured, such as type of environment, size, etc. He keeps on, “Based on indicators such as size or type of environment, it may be possible to identify expected characteristics or impulse responses for the environment.

Additionally, dynamic impulse response generation can be controlled to keep it relevant. The patent continues, “deviations above a particular threshold can be considered to indicate that an incorrect impulse response is being generated. This can then be the trigger to repeat the impulse response generation process.

In some cases, the user may be prompted to make sounds, sometimes of varying pitches, to help identify the environment. The patent states:the user may be prompted to imitate a particular sound that is provided, for example, whistling or clapping (in some cases for the purpose of replicating a sound that is played to the user) […] which may allow more accurate generation of the impulse response.

The diagram shows a method for determining the impulse response of an environment.

Sony’s patent primarily discusses the patent in the context of games, but is stated to apply to other industries. Moreover, this “impulse response generation method” can be used in VR and AR games. However, the patent also hints at the inclusion of other games and interactive experiences, as stated explicitly.

The patent mentions “Such an implementation can be suitable for a wide range of applications, as a controller can be provided for use with a number of different VR, AR, and interactive experiences.

That’s not all; more and more studios have started to focus on upgrading overlooked mechanics like audio generation in video games. Sony is also exploring ways to revamp haptic feedback for VR titles. Additionally, using artificial intelligence, Activision attempts to generate unique game music for each player.

What do you think of Sony games, especially AR and VR titles, featuring in-game sound generation based on the user’s unique environment? Let us know your thoughts in the comments below.

Related readings: Kratos Vs Thor Leaked Fight Video Collects Over 2 Million Views In 24 Hours

Was this article helpful?

Thanks! Share your comments with us. ⚡

How could we improve this post? Please help us. ✍