Industry Insider #4 – Mark Kilborn
All comments and opinions are those of the individuals in question and not representative of the developer or publisher
Welcome to the fourth part of the Industry Insider series. We’ve spoken to game designers, across varying platforms, and now turn our attention to the world of sound.
Mark Kilborn is Audio Director at Raven Software. Having worked on a range of high-profile titles, though not always as Audio Director or exclusively for Raven Software, he kindly agreed to answer our questions on audio design and the choices developers make.
Morning Mark, firstly, can you tell us how you started in the video game industry and where you are now?
I decided when I was very young, around five or six, that I wanted to get into games. I adored video games and fell in love with Metal Gear for the NES. At the same time (early 80s), I was listening to a lot of music and really liked the alternative music that was coming out of Europe at the time. Stuff like Depeche Mode, Pet Shop Boys, New Order. My mom was buying me cassettes of this stuff based on what I liked on the radio. I went through a pause-tape remix phase of sorts, where I was capturing sound effects from the BGM test menus of NES games to cassette, then assembling little scenes with them by copying back and forth on a dual cassette deck.
Fast forward 18 years or so, I’m working as a professional DJ in Dallas, Tx, playing keyboards in a Spanish industrial band, and occasionally doing remixes for indie bands. I decided I should probably grow up a bit, so I went to a tech school in Florida called Full Sail with the intent of getting into the games industry. This was around 2004. After I graduated, a friend, for whom I’d done a remix for, was listening to me complain about how impossible it was to get into the game industry and offered to help. He works in advertising, and he helped me land an internship at an audio post-production facility in Detroit.
I leveled up like crazy working for a group of grizzled old audio dudes, but the resolve to get into the games industry remained. I connected with a great guy in Dallas named Matt Piersall, I think through musical connections (he had a band too), and he asked if I wanted to help him out on a Tony Hawk game. I said yes and immediately quit my job.
Since then I’ve had a pretty wild ride. I’ve worked on a handful of titles that have received some major attention in the market, and I’ve worked with some incredibly talented people who have shared lots of knowledge with me. I’ve still got a long way to go, and I’m learning new things every day. I’m currently the Audio Director at Raven Software in Madison, WI, and the most recent titles we touched were Singularity and Call of Duty: Black Ops (though I wasn’t Audio Director on the latter, Brian Tuey was).
Did the degree help you land your current role?
I was already interested in sound and working in it, but I went to school to level up, so to speak. School was useful, although I found that it was as useful as I made it. I think a lot of other students didn’t get nearly as much out of it as I and a handful of others did. A group of us really threw ourselves into it. We were crunching at school.
I don’t think the degree really helped me land the job though, if that’s what you’re asking. And I should warn readers: this isn’t the case for every discipline in the games industry. But in audio, it’s not as much about the degree as it is about what you can do. I probably would have landed in the games industry even without my degree. I was determined to do it, and I put a hell of a lot of effort toward it.
As Audio Director do you look after all elements of sound production?
Yes. I’m responsible for everything that comes out of the speakers. I work internally with a team of sound designers who create all of the sound effects in the game. We work with external composers and voice production studios to generate music and VO (voice overs). These assets are then sent back to my team to integrate into the game. We’re also responsible for mixing the audio, adjusting the relative balance of sounds so that the player can hear what’s important at any given moment in play.
Is the sound team at the mercy of any in-game changes?
Absolutely. Sound is downstream of almost every other department, so we’re often chasing around trying to compensate for changes. If we hook sounds to an animation and that animation is re-timed, we have to modify our work. It can be frustrating at times, but it’s the nature of game audio development. We’re trying to paint a house that’s still being built.
Raven Software created a lot of games I loved in my childhood. Were you a fan of the early FPS’?
Honestly, not really, but I had a lot of exposure to them. I was a console kid, most of my friends growing up were PC kids. So they’d play Super Mario Bros or Final Fight at my house, I’d play Doom and Wolf-3D at theirs. I played a few of the early Raven games at some point in my youth, but I never owned them. When I applied for a position here, I tracked down the big titles and played them so I’d know what I was talking about.
The FPS genre is weird one for me. In my spare time, I rarely play these games. I’m much more likely to be playing handheld games, RPGs, adventure games, etc. But I seem to keep working on FPS titles. What I enjoy most about the genre is the technical challenge of it: the player’s listening position is in the player character’s head, generally these games are trying for realism (or at least “expected realism” within an unrealistic environment), and so you’re left creating these incredibly diverse, dynamic soundscapes. There are lots of complicated systems that drive the playback of the sounds, and getting those systems to play things in a realistic way while still applying some aesthetic control over the mix is a difficult feat.
As you mentioned, when gamers say ‘realism’ they often mean a consistent and ‘expected realism’ in the game world. How do you work to achieve this via sound?
You have to really bury yourself in the universe of the game. It sounds corny, but you have to find a way to mentally live in the world and try to feel out what things should sound like. Having a solid vision from a creative director is extremely helpful, and you should be in tight communication with your art and animation teams. This is why projects that bring audio people in toward the end rarely succeed with sound. It takes time for the sonic character of a project to evolve, and you need to start as early as possible (nevermind all the technical challenges that need to be evaluated early on).
Take a game like Bioshock. It’s a completely unrealistic scenario and environment, but there are things that make sense within it and things that don’t, and the clarity of creative vision (at least in the finished game, I didn’t work on it so I don’t know if it was consistent throughout the project) gives you a really good sense of how things should sound and feel. Rapture is its own reality. The Big Daddy sounds heavy and his groans are longing and sad because he’s looking for the sisters. All of that comes from narrative context. If his vocalizations were always aggressive or maniacal, it wouldn’t work as well. The turrets and mini helicopters sound antique and broken, very mechanical, with lots of springs. Knowing that Rapture was built in the late 40s gives the sound team clues as to how they should sound.
You have to immerse yourself in the world and imagine what it sounds like. You have to experiment. You find a few things that work and protect them, and then you create the new things and compare them. Over time it starts to come together, and by the time you’re well into production, provided the vision is consistent, you should have a very solid sense of what your world sounds like. Then it’s just iteration to ensure it all hits the right level of quality and authenticity. In the early days of a project it’s great to take concept art or animatic videos and do pure sound design to them to start this refining process. Build environment content packs, little audio scenes, etc, even without a game running, just to help move the process along.
Papa Sangre was an iOS title that relied on full 360 surround sound to tell the story (you’re blind in the game). Is this depth of sound production something that may be adopted more frequently by the industry?
Papa Sangre was cool. It tackled an idea I’ve heard from a number of sound guys in the past: make a game where all you have is sound. Congrats to them for pulling it off.
I don’t know if this is something you’ll hear more of. I’d certainly like to think so, as I enjoyed it. I do feel that surround mixing is being used more effectively as time goes on, especially in games like the recent Medal of Honor. You can really hear the depth of the world around you, and it becomes somewhat useful in a game context (i.e. how far away is the enemy that’s firing at me?). But games like this, that use sound as a critical component of gameplay, are much rarer. Maybe? I don’t know. I hope so.
For you, what game of 2010 had the best audio design and why?
2010 was a good year. I heard a lot of games that really impressed. Limbo was my favorite, because I loved the creativity of the sound. There were lots of textures and vibes in there that you don’t normally hear in a game. I also liked how powerful the sound was as a storyteller in that game. For example, you might enter a cave in the game, but the environment is so dark you can’t really see what the environment is. But you can hear it. You can hear the little water drips, you can hear the general atmospheric texture that makes you feel claustrophobic, etc. Extremely well done.
As far as realistic audio achievement, I felt Medal of Honor and Red Dead Redemption both sounded fantastic. They both had great content, a fantastic mix that felt dynamic and allowed for power where needed and solid VO. The RDR score was one of my favorites of the year as well. I really like that kind of score.
The RDR score fits the story and scenery beautifully, capturing moments of old Spaghetti Westerns. Does the music score in a game create just as valid a theme as the visual design?
Absolutely. Music can completely change the tone of an experience, more so than sound effects or even dialogue in some cases. Marty O’Donnell, Audio Director/Composer at Bungie, likes to say “the ear doesn’t blink”, and he’s right. Audio is a direct path to the brain of the player; music is about providing emotional context for the player rather than data, so the music of your game is a direct, uninterrupted path to the player’s emotional response. It’s critical.
This is a double edged sword though. It’s easy to fall into the traps of using too much music, or too little, using it at the wrong times or trying to sound like other games you like. You often hear games that have similar scores because composers are often pushed to create music that matches experiences from other games, i.e. “Wow, the music of Halo was fantastic, make my game sound like Halo too!” I dig Halo‘s music too (and picking on Mr. O’Donnell apparently), but I don’t want to hear a dozen games that sound like Halo. I want Halo to sound like Halo, I want the rest to sound like whatever they are.
I feel we should be shooting for more diversity in the music we put in our games. There are a lot of non-traditional ways to approach music for your games that can have a more dramatic and lasting effect on the player. Red Dead Redemption is a brilliant example of what can be achieved by doing something a little different. Killer 7 is another.
Independent filmmaker Buddy G. once said that ‘a film only looks as good as it sounds’. Would you agree that this applies to video games too?
Games are very cinematic these days, so if you’ve got horrible music or dialogue, it can cheapen the experience. But this isn’t always the case. There are games that have terrible music and dialogue, but the gameplay is so solid it doesn’t matter. In the end, gameplay trumps all: if a game is fun, just about anything can be forgiven. There are games that flaunt this. Godhand is an example that immediately jumps to mind.
I’d like to say that audio is so critical that a bad job can completely ruin a game, but I’d be overstating our importance a bit. As long as the design is there, most other things can be forgiven. But sound is a great ally to have on your side, because we can affect people in ways that other disciplines can’t. Dialogue can convey information, music can elicit emotional reactions, and sound effects can scare or amuse the player. I like to tell colleagues in other disciplines: “Audio can’t take a 3 up to 9, but we can take a 9 to 11.”
What are the differences between sound design for film and video game?
I’ve done a little of both and they’re very different beasts. In film, everything is fixed. You always know how a scene is going to play out. You always know exactly how many times a single sound effect is going to be played. If someone fires a rocket launcher, you can make it fit the scene perfectly. In a video game, someone can take that rocket launcher, walk up to a wall and fire it 20 times. Totally unpredictable.
In film, you can make a sound seem like it’s part of the environment by processing it as appropriate. In a game, you can’t always do that. A can falls on the floor in a small room, it makes a sound. In the film, it’s always in that room, it always falls in the same exact spot. In the game, that can may fall outside. It may fall onto a wood floor, or concrete, or carpet. The player may have impacted it in such a way that it bounces twice, or three times, or not at all.
In game audio, we have to develop systems that consider these variables and process sounds appropriately in real-time. And then we have to teach those systems how to behave in edge cases, so that they don’t do something to a sound that’s totally inappropriate. We have to create our content in a way that predicts how it might be used in the game and ensures it sounds good.
We also have to fight with memory and resource budgets: there are only so many sounds we can have loaded into RAM at a time in a game. In the film world this limitation simply doesn’t exist. In Brothers in Arms: Hell’s Highway, there was a mission that took place during a thunderstorm. The player is fighting through the streets of Veghel against axis forces while the storm is raging around them. Memory budget was tight, but this storm had to sound great for the 20 minutes the player was there. In the end I built it using real-time processing and approximately 90 seconds worth of sound effects. By creatively processing them, and making the game play them back smartly, I was able to assemble a 20-minute thunderstorm from 90 seconds of content.
This is what we do every day, and it’s far different from what our cousins in Hollywood do.
Regarding the master/mix of video game sound, is it more about making everything sound louder rather than creating a dynamic range of sounds?
That really depends on who you ask. For me it’s about creating a dynamic range of sound. There are others in the industry who share this attitude. But there are also some who are just trying to make the loudest game possible, cranking everything up to 11 as they say. The loudness wars are alive and well in the game industry.
I prefer greater dynamic range because I like to feel the power of a grenade that blows up next to me in a game. I like it when I commandeer a mounted turret and it feels larger than life, because it has so much contrast with the rest of the sounds in the game. Games mixed with the “everything at 11” philosophy feel flat and fatiguing to me.
Are libraries/sounds banks still commonly used in sound design?
Yeah, they are, but I think people are putting more effort into unique content these days. Whether they’re recording original stuff, or just layering and processing library stuff in different ways, the general state of things in the industry has improved. You’ll still hear the odd game that’s yanking tons of sounds from libraries, and it’s annoying to those that notice it, but it’s more the exception than the rule.
Most game audio people would love nothing more than to record completely original content for every project on which they work. But the realities of schedule, budget and geography usually prevent that from happening. Some of the best scenarios I’ve seen are on sequel projects, where there’s a huge content base from which to build, so you can pay more attention to revising old assets and creating new original stuff. It’s kind of a snowball effect over the course of several projects. But not everyone has that luxury, or even the desire to work on several iterations of the same game.
At Raven we make an effort to record original content whenever we can. We’ve recently built a foley room that’s ready to rock at a moment’s notice, and we’re currently filling it with every random piece of junk we can find. When things slow down a bit, the team takes field trips to record content. And we have a few portable recorders so people can record things while on family trips and such. For example, our awesome intern recently took a trip to Cuba and recorded some great wind and foliage sounds that we’re using on our current project.
What’s the current project?
I can’t comment on our current project, except to say that I’m extremely excited about what we’re working on.