Getting Your Sound Into the Game - A Process for Implementing Sound Design

Article by George Spanos

Sound design for video games is quite a different animal than sound design for other media. Film, television, and other linear formats require the sound design to unfold at a very deliberate and controlled pace. After all, the outcome of linear formats is always known.

The sound designer can therefore create sound-scapes and sound effects with the very deliberate intention of telling a story where the exact outcome is known to always happen the exact same way.

In video games we do not know the exact outcome of every scene. Sure, we know that the player has to perform 'A' to get to 'B' but we don't know how long the player may take to get from 'A' to 'B' (in most games). For example, the player may choose to go down a route that is an indirect way of reaching the end of a level, thereby encountering any number of enemies or alternate paths. The main thing to realize is that, in games we usually cannot control the exact sequence of events that a player may experience. This is obviously called randomness.

It is for this very reason that designing and implementing sound for a video game should not be based upon events that "happen next" but rather, "events that may happen next". This is why the great majority of games have their sounds broken down into chunks that are easily played back at the exact moment they are needed; at run time.

Sound Creation

The sound designer must first assess what sound that he or she is required to create for a particular scene. Many decisions are made at this stage (outlined in a rather simplistic fashion):

i. The type of sound needed. This can be organic or synthesized or a combination of the two.

ii. Does the sound currently exist in a custom or commercial library that the sound designer may be able to pull from, or does the sound need to be recorded.

iii. If the sound or component of the final sound, already exists then what other sounds and processing are required to create the new "hybrid" sound.

This is obviously a simplistic view of what runs through a sound designer's mind when he or she is at the conception stage of a new design. There are many other techniques that may be used that are not so "by the book", like random sound generation that will help to spark new and creative ideas for example. Another point to consider is that sometimes the key to great sound design is not to act in such a methodical way but rather, to rely on instinct which is built up over many years of experience. But for the purpose of this article, the above steps are a good rough outline.

Sound File Creation

Once the sound designer has made the required sound the next step is to either bounce it down or export it from the digital audio workstation (DAW) that is being used, generally into .wav format. Careful attention must be paid at this stage to ensure that if the sound is a one-off (to be triggered in game one at a time without repetition; for example an explosion) or if the sound is looping (to be played back in a loop; for example ambience) that the sound file adheres to some basic technical rules.

In the case of one-off files attention must be paid to ensure that the file is trimmed at the absolute start and end of the file so that the file is no larger in size than need be. Memory is at a premium even on today's consoles, even with compression. The smaller the file size, the better.

Generally it is advisable to export the sound file at the highest resolution possible (within reason, say 44.1 kHz or 48 kHz). This will enable the sound engine to compress the file as it sees fit, and possibly with parameters that the sound designer has set out.

If the sound is to be set as looping then careful attention must be paid to clicks or pops that may be introduced when the file is looped. An easy way to check this is to loop the audio file in question from within the DAW and listen for noise at the beginning and end of the looped file. If the engine being used converts the .wav files into .mp3 files then the game audio programmer will have to ensure a seamless loop. The encoding of audio into .mp3 files incorporates a form of "padding" into the file which will have to be accounted for in code, otherwise the file will not loop correctly.

Integrating Sound Into the Engine

Once the sound file has been created and has been trimmed and levels checked the next step is to import it into the game engine. There are many different game engines and methods out there for accomplishing this. It really all depends on what the game that is being developed is running on. It also depends on whether a third party application is being used to integrate audio into the engine. Third party software examples include Fmod and Wwise.

Generally though, a sound will be added and placed inside a folder that pertains to the specific sound set. For example "ZombieSteps" would include all the step and surface variations for the Zombies in the game. The sound is then compressed into a specified format depending on the console type that the game will be released on. XMA audio is generally used for the Xbox 360 system and MP3 audio is generally used for the Playstation 3 system. The engine or the third party software will include settings for compression ratios, looping or non looping flags, pitch, volume, randomness, positional settings, and a whole host of others.

What Now?

After the sound has been integrated into the engine it is time to hook it up to specific events. If the sound is to be played as part of a character's movements (for example footsteps) then animation notifications are placed onto the required animations. These flags will tell the engine when in the animation cycle the particular sound should be played.

If the sound is an ambience there will be sound triggers placed inside the level. These triggers along with radius parameters will determine over what area the ambience will be heard, so that the player will get the aural illusion of moving from room to room or world to world.

Obviously, there are more complex applications of sound placement and triggering. A good example is music that is acting as score or non-diegetic (music that is not playing for the characters in the world, such as stingers or theme music). Many games such as the Halo series use a form of music mixing that is non-repetitious. This can be called a "dynamic" system because the player will never hear the music played back in the same way twice. Musical elements are separated out into pieces such as percussion, brass, strings, and melody. These pieces are then re-assembled at run time into music that 'moves' with the player. For example, the player may be leading up to a battle so just the percussion elements will play. As the player enters battle the string parts and brass parts may be added. The bar and phrase structure can also be dynamically controlled to allow further customization.

Conclusion

At this point it is important for the sound designer to test the sounds that have been placed in game. The sounds should be tested on all console platforms to ensure that memory budgets are not exceeded and that compression ratios are not squashing too much life out of the sound. As in any other media, linear or not, it is imperative to test across multiple speaker set-ups. Large and small 5.1 and 7.1 setups should be tested. Stereo and mono speaker setups as well as headphones should be sanity-checked.

I have not covered some of the more detail-oriented and advanced steps in this article (stay tuned for an update!) but these guidelines will give you a good idea of what getting your sound into the game really means.

blog comments powered by Disqus