Released for PC in late 2018, Insurgency: Sandstorm is a multiplayer first-person shooter developed by New World Interactive (NWI) as a sequel to the studio’s 2014 hit Insurgency.
The game’s continued success compelled NWI to bring the title to Xbox and PlayStation consoles. “It was something we knew we wanted early on with Insurgency: Sandstorm” said John McBroom, Infrastructure Director at NWI. “We really wanted to expand our target audiences.”
However, launching a game for consoles was an unknown task for the studio. “Our focus for Insurgency: Sandstorm at launch was on PC via Steam” explains John. “We didn’t have a lot of console experience to pull from. On the game side we had concerns about getting performance to acceptable levels. And on the server side we had questions about player distribution and server density.”
Having already worked with NWI in 2019 to provide a bespoke game server orchestration solution for Insurgency: Sandstorm, our multiplayer experts were ready to support the complexity of game server operations for its launch to new platforms.
So in late 2020, NWI began talks with our team to plan a launch programme and work out approaches for a multiple console release in 2021.
|Studio New World Interactive||Game title Insurgency: Sandstorm|
|Game genre PvP and PvE FPS||Platforms PC, Xbox One and PS 4, with Xbox Series X|S and PS 5 compatibility|
The lack of knowledge was the real challenge for us. We were shipping to multiple console platforms for the first time.
To help test the console builds, gather data and optimise for efficient performance, our Improbable Multiplayer Services (IMS) team worked closely with NWI to establish and implement a beta launch programme to roll out in four phases, adding consoles, players and regions in each phase. The programme was designed to serve two important purposes:
One, it meant that NWI could forecast how much bare metal and cloud it would need and what the expected player counts would likely be for each region, based on the number of pre-orders or keys given out.
Two, it meant NWI could test its console development and builds, gather data about player numbers, interest and retention for each global region, and be sure that its teams were happy with progress.
“Our phases were locked down at the start of July” says John. “So we worked closely with IMS to make sure we were able to facilitate our operational needs and spin up the bare-metal servers in the short timeframe. By the end of July, the first beta phase was starting.”
This first beta phase was for Xbox consoles only, initially giving out 1000 keys across the US West and East regions, with phases two, three, and four running through to September 2021, each beta phase adding consoles, increasing player keys and introducing the European and Russian regions.
In preparation for the Insurgency: Sandstorm console launch, the phased beta programme effectively established a safe experimental space that allowed NWI to:
Scenario modelling gave us critical insight into the correct amount of server capacity to assign for our launch demand without paying excessive cloud costs.
“The big unknown to us was, if we’re selling ‘x’ number of copies, how many concurrent users can we expect in a certain region? And what will the region split be?” explains John. “Given this was our first venture into consoles, the scenario modelling gave us critical insight into the correct amount of server capacity to assign for our launch demand without paying excessive cloud costs.”
With the quality of the console player experience in mind, scenario modelling helped NWI to calibrate and optimise regional footprints for the console player base across each time zone. “Through the beta programme, we saw how we could consolidate certain regions to provide a better player experience” says Tom Hynes, account manager at IMS. “A nice byproduct of this integration was that it was also more cost effective for NWI.”
“It was something we were able to evaluate with the beta to see that it really was a viable option for release” says John of the regional allocations and consolidation. “Instead of releasing in seven independent regions, IMS data proved we were able to scale this back to five without compromising the player experience due to latency.”
Our live operations team is very small. One of the big challenges of the launch was having confidence that all the servers were being monitored around the clock, as we did not yet have a 24/7 presence.
At every stage of the beta launch, our professional services and incident response team constantly monitored live operations and reported on launch metrics, providing NWI with daily updates and weekly graphs tracking the CCU data.
With telemetry in place to constantly monitor launch operations, system health alerts allowed the team to foresee and tackle any issues – such as matchmaking or data centre downtimes – before they impacted players. This helped NWI to proactively mitigate risk and ensure the launch went smoothly.
We saw a much bigger spike in players than we were anticipating. It really was the biggest test of Insurgency: Sandstorm's multiplayer systems.
“With the release, we saw a much bigger spike in players than we were anticipating” John says of the CCU figures, which achieved a six-fold increase in player counts. “The fact we were exceeding it was a good surprise to have, and we were happy that the cloud scaling was covering all bursting. There weren't any concerns with the server capacity throughout the launch period despite huge demand, and we were confident throughout that the scaling was meeting all our needs.”
Bringing Insurgency: Sandstorm to Xbox and PlayStation is the realisation of many years of hard work for NWI. By partnering with IMS to support the complex server operations of launching to the new consoles, NWI gets to focus on delivering the highest quality products for its growing player communities around the world.
“Complexity has increased a fair bit for developers on the DevOps side, by nature of having to deliver to these different platforms” John observes of the challenges. “But knowing that the IMS team are monitoring and prepared to respond to server issues gives us the confidence needed to focus on expanding to further platforms and stores. We’re super excited with the way things are going.”