Audio transitions: a smarter way to approach adaptive music?

This is by no means definitive, but it's an observation I (surprisingly) haven't brought up in clear-form before, so I'm going to: "adaptive music" is used to refer to music which responds to a player's actions (and related events) in a video/computer game. At a very simple level, think of how the music changes in Super Mario Bros. when you snag a star and get invisible for some seconds. At a more advanced stage, think of "exploring" music that crossfades into an uptempo "fight theme" when your character gets into conflicts. (Deus Ex is a fine, if earlier, example.)

Here's where it gets really intriguing: video has all sorts of transition effects. Not just crossfades, but irises, wipes with blurs, stutter-chop-ups, and, well, I can point you to the NewBlueFX line (disclosure: I consult for them ) for many fantastic examples. Audio doesn't emphasize this as much. I've heard some hardcore DSP glitchery, but audio transitions still primarily operate on the basis of crossfades — as a limit built into editing tools. At most, like in Sony Vegas, you'll get some curve shape changes. And you can get into VST effects and what-have-you to program your own switchovers. But automating creative audio transitions isn't as tight and seamless as it should be.

I'm familiar with many sonic variations used in DJ sets. There's the ol' "kill the low end until the beginning of the next 8 bars when the beat comes back in", but that's usually within a single track, not across 2. One of my faves is to sweep a delay (echo) send so it almost washes out the track. This is great if you then run it through a bitcrusher, so it sounds all filthy and retro… and I wish I had an audio sample onhand to show you. (I likely will if this thoughtline keeps invading my sleep.)

But the point is: I have to perform or program these moves each time without the luxury of a better "big picture" mode. I wish in more sequencers (and game dev. audio tools), there was a way to drag 'n' drop these transition modules that went beyond basic copy-and-paste, that was context-aware. I could stretch or shrink audio transitions (like video ones) just like other clips, without worrying about getting too fiddly and messing up automation curves. And best of all, this could be done live as you play a game, without prerendering. I don't know for sure but maybe it's possible with Flash 10 (after playing with Hobnox Audiotool). Of all the DAW apps out there on the market, it seems most well-suited for an elastic candidate like Ableton Live 8. Or 9.

I think it's pretty clever that some of the most popular video games right now are music ones, like Guitar Hero and its lineage stringing back to Dance Dance Revolution. Adding new tracks (Rez and FreQuency often get mentioned here) and playing synced cues as your performance increases is exciting. Locking to BPM and key makes it easy to match up music so it doesn't clash. Atomizing a production into stems, then loops, then individual parameters, etc. is even more freeing. But I think there's a lot we could do right now with tempo flexibility, and to go beyond crossfades.

(The above: semi-raw braindump.)

Tagged as: General