How gamification ruined your Next Fest stats (and why it doesn’t matter)

Celý článek

[The GameDiscoverCo game discovery newsletter is written by ‘how people find your game’ expert & company founder Simon Carless, and is a regular look at how people discover and buy video games in the 2020s.]

Good morning, evening, or whatever the time zone happens to be, wherever you are in the world. It’s time for us to return for another week of handy game discovery news and analysis - and thanks for coming along for the ride.

This time out, we start with an expansion on last Wednesday’s newsletter. Are players really hitting your Steam Next Fest demos to the tune of tens of thousands of CCUs? Are your wishlists numbers similarly affected? There’s bad news, and good news…

[Hey! Want to support us & get lots of extra PC and console game analysis and data? Our GameDiscoverCo Plus paid sub has a data-rich weekly game trends analysis, exportable Steam ‘Hype’/performance charts, eBooks, a member-only Discord & more. Sign up today!]

Gamification, Next Fest stats & signal vs. noise…

So we mentioned last Wednesday that “if you’re seeing [Steam] game demos get unlikely CCU numbers - such as Terra Nil, which briefly hit 95,000 CCUs, this is likely due to ‘badge farming’.” After that, we ended up being contacted by a few other regular Steam devs who were confused about their Next Fest demo stats.

We thought it was worth a quick ‘explainer’, then. How’s the ‘farming’ taking place, what effects might it have had on your Next Fest player stats, and why is this more of a temporary confusion factor rather than a long-term problem? Here’s what we know:

• Valve helped to incentivize Next Fest this time out with some badges: “Play any demo from Next Fest to unlock a badge you can feature on your Steam Profile.
  Level up your badge as you try out more demos.” Although the XP maxes out after you play 10 demos, you can see total demos played on your Steam profile - see this example with 1056 demos played (!). So some people are grinding, for fun.

• Rather than playing, Steam users are using third party apps like Archi’s Steam Farm to level up automatically. As the FAQ notes: “It doesn't have to download any of your game files, it can play games right away. Secondly, it's entirely independent of your normal Steam client - you don't need to have Steam client running or even installed at all.” So it’s passing ‘I’m playing this game!’ variables to Steam…

• Most of the time, you won’t obviously encounter ASF-related traffic on your regularly released, paid game. (It checks games that you’ve already bought for Steam trading card ‘drops’, and then idles until all the cards have dropped. So it might increase average playing time for your games, if you have trading cards - but that’s it.) However, in this case, individual Next Fest demos may see stats distortion.

Anyhow, here’s a useful real-world example, from Pavonis/Hooded Horse’s much awaited 4X title Terra Invicta. The game has 34,000 Steam followers - so hundreds of thousands of legit wishlists. Its Next Fest performance was top echelon, since it was #3 in the ‘Most Wishlisted’ chart. But its back-end demo stats are a little, uhh, screwy:

Do we really believe that nearly 250,000 of the 489,000 unique Next Fest demo players for Terra Invicta decided to play the game for less than 10 minutes? No, we don’t - that’s probably due to ASF & similar apps swiftly telling Steam your demo was loaded.

Hooded Horse’s CEO Tim Bender and myself were tipped that the odd spike at 3 hours is related to Steam ‘idling’ apps at work. Sometimes these apps are set as default to show you playing the game for longer - even if, as in this case, nothing happens.

But there was definitely an enthusiastic player base checking out the Terra Invicta demo, as can be seen from YouTube results. So we’re inclined to believe the 58,000 players who tried the game for more than 200 minutes - and all the in-between durational stats, actually. But this is data cherry-picking of a subjective kind.

Another example comes from Hellscreen’s Jamie D, who told us that 43% of his ‘downloads’ - and 80% of his ‘players’ for his game - were from the Netherlands. These are clearly not real players at that volume. (It’s perhaps due to a popular free VPN being there.) Unfortunately, not all ‘farmed’ plays of demos can be tied to a particular country and excluded, if we look at country data across many demos.

Anyway, the important message here is: don’t worry if your average Next Fest demo play time seems low, or suddenly dived. But also, definitely don’t get excited that you have lots more demo players than expected. Steam’s gamification has inadvertently led to stat-hungry players ‘playing’ every Fest demo - without actually loading them.

The good news here is that your game’s Steam wishlists are unaffected by these idling programs. Some Next Fest wishlists may be marginally more speculative than ‘organic’ ones. But there are no idling programs that automatically wishlist games as a matter of course. So believe your wishlists, and keep on keeping on!

Old World: lessons on a post-EGS Steam launch…

The ‘Epic Games Store exclusive coming to Steam later’ situation is starting to become more of an edge case, due to Epic’s decision to lean further into Epic Games Publishing for just a handful of games. But there’s still third party EGS exclusives coming to Steam later - and it’s pretty interesting to look at notable examples.

And one of these recently was Mohawk Games’ Old World, the quality turn-based historical strategy game from Civ IV lead designer Soren Johnson & colleagues. It launched on EGS in Early Access in May 2020, and finally arrived on Steam on May 19th, 2022. It even made the Top 10-grossing Steam SKUs for launch week.

We had a chance to chat to Hooded Horse’s CEO Tim Bender, who miraculously makes a second newsletter appearance today (!), and published Old World on Steam. And here’s some of the headline takeaways:

Think about localization languages carefully, since Steam is so global!

It’s trickier to gauge the regionality of EGS. But Tim noted that he expanded official translations to Chinese, Japanese, Spanish, Russian, French, and German for Steam launch, and particularly showcased Japanese as a good pick for strategy games.

He notes: “Old World has recorded around 8% of its Steam sales in Japan, even with the Japanese localization currently being in Beta until later this month, and needing to be enabled manually by players.” Here’s the Steam LTD unit % comparison point for Old World to German, a country we often think of as strategy game-hungry:

Tim adds: “Japan is home to a huge number of kind and supportive players excited about PC gaming, and I firmly recommend that every game seriously consider Japanese localization.” Our newsletter on this subject in early 2021 also recommended Japanese, actually.

Some games are outliers on conversion/review ratios - especially EGS to Steam ports?

If you were to look at various review-based methods of estimating unit sales on Steam, you might believe that Old World has done ‘fine’, but not spectacularly. However, Tim notes:

“Old World currently has a rather large sales-to-review ratio on Steam a few weeks after launch – almost 100 units sold per review. I expect that to even out some over time, but worth noting it has remained significantly outside the long-term estimates for quite a while, and maybe will never really align with estimates.”

And indeed, Old World’s GameDiscoverCo Plus-calculated ‘conversion rate’ on its Steam launch was 50% of expected. We presumed this was to do with players who had wishlisted on Steam, but then bought early on EGS instead. (Though this was unlikely in Old World’s case, since it was missing a Steam page until October 2021.)

But actually, it may just be that players are buying the game again on Steam, but not wanting to review it! Or any number of other reasons. It’s just a great reminder that, especially with games arriving on Steam from other platforms, the ‘20 to 55 sales per review’ guideline is for the majority - but there’s always edge cases that differ…

(BTW, we took a quick look at two sites that claim to scan Steam profiles to get a more accurate owner number, but they are also too low. You’ve gotta take all this stuff as ‘indicative’, not absolute, we guess?)

Steam reviews: delicate, but finesse-able with humility & care…

One big difference on Steam compared to EGS is the importance of written reviews. As Tim says, Old World’s debut - at 83% Positive launch day reviews - worked, as “the demand for strategy games on Steam and the strength of the game itself helped make for a successful launch.”

But with the game at 80% Positive right now (just on the cusp between Very Positive and Mostly Positive), it can be worth devs or publishers rolling their hands up and trying to ‘flip’ reviews from Negative to Positive - especially if the issues can be explained or addressed in a human way.

Tim went through “a rundown of a few of our experiences there” which we think are pretty interesting, so are reproducing in full. In his words:

• “One person left a negative review that essentially complained that the game was ‘not a replacement for the Civ franchise.’ I replied agreeing with them but pointing out that it wasn’t designed to be that, and asked whether they felt the negative review was really warranted given they said it was otherwise a good game. They switched their review to positive a few days later and left a nice comment.”

• “Sometimes players complained about something that could have been fixed by a setting change. I’d generally respond by pointing them to the setting. Likewise Soren Johnson, design director of Old World, replied to several negative reviews complaining about design decisions by explaining the logic behind the design. These replies didn’t always work, but several times the writer switched the review to positive after.”

• “Sometimes we didn’t convince the person leaving the review, but got a lot of comments from other players who seemed to like our reply - some even saying they decided to buy the game based on reading the reply.”

So that’s pretty neat! For some types of game, it’s not necessarily worth having detailed Steam user review discussions - especially if the reasons for the negative review are out of your control.

But for others, it can be worth having a dialogue - especially if those reviews are upvoted and appearing prominently on your Steam page! And thanks again to Tim for going through all these details for everyone.

The game discovery news round-up..

OK, quite a lot went down since last Wednesday - which is the last time we rounded up the game discovery and platform news. So let’s take a look at all of this news goodness, and coalesce it into one helpful whole:

• An opinion piece on why it doesn’t matter that Xbox doesn’t have big first-party 2022 games: “The games industry’s habitual annual rhythm, peaking in the last months of the year, is hard-wired into anyone who’s been playing console games for more than a few years. But that has been driven, for decades, by the needs of a retail industry that Microsoft is barely participating in any longer.”

• Having barely made it through ‘not-E3’ alive, Ars Technica rounded up the 25 best games it saw during all the showcases, Polygon also picked its favorites, and GameSpot also had a handy round-up. In case you actually have the stamina to look at more games, that is.

• Following up our piece on mobile games monetization and what’s ‘more acceptable’ in PC/console gaming, it’s very interesting to note Overwatch 2’s approach: “The new shooter will [ditch lootboxes] & now have two primary monetization points: A new battle pass for every season and an in-game storefront where players can directly purchase cosmetics, skins, and other trinkets.”

• Meta’s continued push to be ‘the AR/VR leaders’ continue with a showcase of lots of next-gen hardware prototypes, including one that lets you read the ‘20/20 vision’ line on an eye-chart, Plus: a super enthusiastic Mark Zuckerberg talking to Tested.com in a longform video interview about Meta’s plans. Get.. hype?

• Microlinks, Pt. 1: two in five Switch Joy-Cons affected by drift, says UK consumer watchdog report; all you need to know about Google's HTML5 platform Gamesnacks, which has had 35 million users; a larger Nintendo Direct (not the Xenoblade one) may be cropping up on June 29th, according to rumors.

• Sensor Tower has been poking around the Netflix mobile exclusives released so far, and say: “Netflix-published mobile games have generated more than 13 million downloads globally across the App Store and Google Play as the streaming giant ramps up its plans in the market… Stranger Things 1984 picks up approximately 2 million installs as Netflix’s most downloaded mobile title to date.”

• Two Epic announcements: Epic Game Store’s new game ratings tool is structured interestingly - they’ll “ask random players, who have played a game for more than two hours, to give a rating on a five point scale”, with polls for tags like ‘relaxing’ or ‘amazing boss battles’. And then “Epic Online Services crossplay now works seamlessly on Steam and Epic Games Store, enabling Steam players to search from over half a billion friends, connect and play.” Big crossplay plans here!

• An interesting Bloomberg newsletter notes that “the argument that video games spur mass shootings is losing steam”, with Stetson U’s Chris Ferguson commenting of U.S. politicians: “You see half-hearted attempts by Ted Cruz and a few other people bringing up video games, but there’s not much enthusiasm behind it… I don’t think anyone’s biting at the apple anymore.” Yup, it’s gun ownership, y’all.

• Hidden former Stadia exclusives? Did you know - via Axios - that “Google was involved in the creation of two notable video games [well-reviewed horror adventure The Quarry, and comedic sci-fi game High on Life] in the spotlight this month, before the downsizing of its Stadia gaming operation sent those projects elsewhere”?

• Microlinks, Pt.2: Amazon’s Prime Day (July 12-13) is going bigger with game giveaways via EA, including Mass Effect Legendary Edition, GRID Legends & more; the ID@Xbox Summer Game Fest demos - over 30 of them - are appearing from June 21st to June 27th; as Valve’s ‘Deckard’ next-gen VR hardware continues dev, a patent shows something-or-other, image wise.

And that’s it for this particular newsletter! See you back on Wednesday, when we’ll be talking about some large-scale, philosophical questions on derivative games & the game biz. Oh, and please support us with a Plus sub if you can!

[We’re GameDiscoverCo, an agency based around one simple issue: how do players find, buy and enjoy your premium PC or console game? We run the newsletter you’re reading, and provide consulting services for publishers, funds, and other smart game industry folks.]