Sound controls

Immersed in sound

On an island of overgrown ruins, you place a luminous seed in the embrace of a plant whose thick black vines strangle the landscape. A cascading piano, on a bed of dreamy strings, accompanies the green color that suddenly pervades the giant plant, a cloud of purple flowers releasing into the air.

The Atmospheric Video Game 2020 Submerged: hidden depthsfrom Canberra-based indie studio Uppercut Games, is set in a world of glistening water, rusty structures and glittering skyscrapers rising from the sea. It’s such a polished game that at any moment you might be Distracted from your quest to find seeds hidden in the wreckage by a startling sight of golden sunlight reflecting off the water or a magical shimmer emanating from the plants reclaiming a once populated town.

A scene from the video game Submerged: Hidden Depths

A poignant score from Brisbane-based, BAFTA-winning composer and sound designer Jeff van Dyck – whose credits span the epic total war national puzzle hit series of 2021 Unpacking – reinforces the tone of quiet sadness that pervades this fallen world, in which you play as a brother and sister navigating the ruins in their boat.

While Van Dyck generally works with virtual instruments, for Submerged: hidden depths he recorded the string parts of the soundtrack with live musicians in Adelaide’s Brighton Concert Hall, engaging Adelaide-based composer, orchestrator, conductor and performer Nathan Cummins to help with transcription and preparing music.

It was the developers who specified the combination of piano and strings, Van Dyck tells me when I speak with him and Cummins ahead of High Score 2022, APRA AMCOS’ annual game sound and composition conference as part of Melbourne International Games Week. Cummins will join the online event from France for a panel titled From Screen to Stage: Game Music in Concert while Van Dyck will deliver the event’s closing keynote.

“They wanted something very moving, very melancholy – which suits me perfectly,” says Van Dyck.

It used the same instrumentation for the game’s predecessor Overwhelmed, released in 2015. “On the first one, we didn’t quite have the budget to use real players, it’s a shame, but I took advantage of it and did everything in MIDI and used a sequencer,” he says. “But for the second, we had the budget because Google was involved.”

Van Dyck’s journey begins the same whether or not live instruments are used in the recording. “It starts with playing here on the keyboard – raw output, capturing ideas, using technology to try to make it as real as possible,” he says. “But VSTs [Virtual Studio Technology] can only go so far.

That’s where Cummins comes in, transcribing MIDI files into sheet music for live musician performances.

Van Dyck met Cummins through Adelaide composer Christopher Larkin – responsible for the haunting music of the 2017 game hollow knight – who worked on music transcription and sound engineering for Submerged: hidden depths.

The process of converting MIDI files into usable scores requires knowledge of technology, experience, and intuition, because what sounds good when played back on the computer doesn’t always translate directly to what you want. a musician expects to see on the page. “There’s a point where you kind of have to massage the data to make it clean, so to speak,” says Van Dyck.

It can also be about figuring out what the composer wanted rather than relying on the output of the software. “I found that 90% of the time I was preparing the score, I was mostly listening to the recordings that Jeff had prepared,” Cummins says.

Cummins also had to find out which elements were idiosyncrasies of the sound library and what the composer’s creative decisions were. “On one of the sound samples, they might have a little crescendo in there, that’s not really what we want,” he explains.

“It really put me in control of my songwriting and composition,” laughs Van Dyck. “It was like all of your notes, all of your music was checked, quality checked or something.”

Although he’s been working in game music since the 1990s, on high-profile series like Fifa and Need of speed, the process got Van Dyck thinking about composing in a different way. “I was like, is that good enough?” he says. “Am I writing interesting stuff here or are these guys going to fall asleep during the session while they play?”

Van Dyck has only had the opportunity to work with live musicians on a soundtrack once before, he says, and not to the same degree as Submerged: hidden depths, which included a string ensemble led by violinist Belinda Gehlert. “It was very pleasant to work with Nathan and Chris, then also with the string players,” he says. “We pretty much had it all in a few takes.”

With free time in the studio, the band was able to experiment, including recording additional takes with muted strings, or “con sordino”. “We had a few places where we added con sord the second time around,” Cummins says. “I think a lot of them ended up using the con sord’ version.”

“It was good to have that option,” says Van Dyck. “Because when you record multiple takes like that, in the mix, you have one more chance to make a decision on which direction to go – one last tweak. With virtual instruments, you can tweak until you die, you are therefore spoiled by this flexibility, whereas once you record something, that’s it.

The team also recorded other experiences. “In Submerged: hidden depths, there are all these creatures that are natural creatures but they have been distorted by this thing called ‘The Mass’,” he says. “There is a whale, but the whale is green and there are strange things coming out of it, and there is a dolphin, but there is a little light on it.”

A scene from the video game Submerged: Hidden Depths

Van Dyck asked musicians to imitate these animals with their instruments. “I made a lot of really weird recordings,” he says. “And oddly enough, I was able to use a bunch of them in-game, which was pretty cool.”

Van Dyck also hopes his music will be performed live, so he has released the score. “It’s something I’ve always wanted to do,” he says. “It’s just one of those things on the to-do list.”

The score is again different from the parts used in the studio. “In a recording session, time is literally money,” says Cummins. “As an engraver, I try to answer any questions in advance to avoid the performers asking these questions.”

The Performance Edition, however, gives musicians more latitude for interpretation, although Cummins has included bows used in the studio in case performers want to achieve the same sound.

Which isn’t to say that the musicians who recorded didn’t bring their own interpretation to the music – and that was one of Van Dyck’s greatest pleasures. “The score is a suggestion of how to play it, at any given time the player is in control of it and how they interpret it is really up to them, and either you like it or you don’t. not,” he said. “Luckily everyone who played was fantastic.”

Having spent so much time with the MIDI version, Van Dyck admits it took some getting used to the sound of the live musicians – before the treasures were revealed. “At first it was really foreign when I heard it,” he says. “And then I go, ‘Oh, but that’s a nice little thing that player did’.”

“There was a time when I went back to the MIDI stuff and it sounded so flat,” he says. “It felt so uninteresting, boring and lifeless compared to the live game.”

“Just having the opportunity to sit in front of musicians, performers of this caliber, is just an incredible experience,” says Cummins. “You don’t get that with MIDI.”

This experience is now woven into the fabric of Submerged: hidden depthswhether you’re climbing ruins, scavenging artifacts, or simply gazing out to sea.

High Score 2022 takes place online and in person, October 1-2, as part of Melbourne International Games Week.

Sign up for the free Limelight newsletter