How to become a video game superforecaster

Publikováno: 29.6.2022

You all want to become omniscient and omnipotent, right?

Celý článek

[The GameDiscoverCo game discovery newsletter is written by ‘how people find your game’ expert & company founder Simon Carless, and is a regular look at how people discover and buy video games in the 2020s.]

We have returned to your inbox, video game data and discovery nerds, to bring you the second free GameDiscoverCo newsletter of the week. This one’s about the traits we - and experts - claim you need, in order to think straight about the video game biz.

Meanwhile, thanks to all the new readers that signed up since Monday’s missive. It means a lot to us that we regularly get 10-12k actual readers of these pieces, given that this is actually a fairly niche B2B publication. (We’re hardly Tiger Beat here.)

[Care to get lots of extra PC and console game analysis and data? Yes? Now 25% off for the next 10 days, our GameDiscoverCo Plus paid sub has a data-rich weekly PC/console game trend analysis, interactive Steam ‘Hype’ charts, two eBooks, a member-only Discord & more.]

Can you, too, become a game ‘superforecaster’?

Recently, somebody dropped Tim Harford’s book The Data Detective (aka ‘How To Make The World Add Up’) in the Little Free Library outside my house. It’s subtitled ‘Ten Easy Rules To Make Sense Of Statistics’, and it’s a seriously great book on thinking critically about data.

While I recommend the whole thing, one section deals with Philip Tetlock’s Good Judgment project, which posed around 100 to 150 questions each year on geopolitical events to “enthusiastic amateurs” as well as self-professed experts, and saw who came out on top.

So it basically ranked the forecasters. And, it’s claimed by Harford: “there were a select group of people whose forecasts, while they were no means perfect, were vastly better than the dart-throwing chimp standard reached by the typical prognosticator. What is more, they got better over time rather than fading away as their luck changed.”

So what made them stand out? Interestingly, it wasn’t necessarily deep subject matter expertise, or intelligence. Though doubtless those help. It was the following - all of which I’ll explain how to apply to the video game space:

  • Training on basic statistics: This includes ‘how stats work’ in general. But Harford singles out ‘base rates’ - ie, the actual baseline numbers you’re basing your estimate on. For example, in Monday’s newsletter, we learned that about 15 games per month launch on Steam with 50,000 Steam wishlists or more. And we know about median wishlist conversion rates. Thus, you have a starting point for possible success for a game. Without it, you’re lost.

  • Keeping score: this is being honest about whether you were right or wrong, and iterating on it. A lot of game companies - especially those chasing big hits - will tend to ‘fire and forget’. But if you actually do a postmortem after each game launch, and try to improve tactics based on it? That can help a lot.

  • Update forecasts often as new information arrives: so you’ve discovered that - with a few exceptions - the Nintendo Switch market has become a bit oversaturated, and concentrating on PC-first games might be a better idea? That means you need to shift your portfolio approach based on that info.

  • ‘An open-minded personality’: Harford notes that those “willing to abandon an old view in the light of fresh evidence” are some of the most successful forecasters. For example, we thought that PC DLC was a bit pointless, and that VR game sales weren’t going to be significant. But we were shown proof this was wrong, and now GameDiscoverCo is covering these topics more. Not saying we’re superforecasters. But we are definitely willing to change our minds.

Anyhow, we find this all particularly interesting because it somewhat goes against the idea of the successful practitioner as the single-minded iconoclast or the rebel disruptor. That approach works if you’re exactly in the right place at the right time.

But in today’s game market - which is shifting business models swiftly - it seems to me that ‘listening to new information, and changing your tactics accordingly’ is one of the most important traits you can have. If you only have one approach… well, even a stopped clock is right twice per day, we guess?

Follow-up: ‘derivative games’, meta-mechanics, and indies..

Following our piece last week on whether ‘derivative games’ are the way to hit it big, we had some really good and robust discussion - both via email and on the GameDiscoverCo Plus Discord. So wanted to highlight some of the most important ones.

Firstly, Elliot Colp smartly noted that this isn’t just about game ‘clones’: “I think a key part of Kyle Kukshtel's article that kicked this all off is that this isn't just about derivative games vs. "micro-genres," but about certain mechanics and experiences that are becoming increasingly ubiquitous in games regardless of genre.

You'd be hard-pressed to find a AAA release these days without a crafting system, a skill tree, and randomized loot drops for example. These meta-mechanics are being tuned to maximize repeat engagement, with increasing influence from mobile and casino games.

As they become more standard, and players start to expect them in every game, it does feel like it's making it increasingly difficult to sell a game that doesn't fit the mold or easily allow you to layer those meta-mechanics on top of it. (‘depth’).”

This is a subtlety I think I missed in our analysis, which dealt more with subgenres than ‘portable’ game features. And Kyle Kukshtel himself - he’s a Plus subscriber, folks - turned up on our Discord to add:

“One bigger area that I didn't get into is how AAA is the biggest employer of games people, all skilled to make Assassin’s Creed-likes. So those people skill up in making AC-likes, crosspollinate to other AAA companies, and all sort of converge on making the same thing, as the talent pool becomes more and more specialized towards making that very specific type of game.”

It’s true - and alongside the homogenization, this is now the ‘depth’ hill that we now have to climb in order to compete with the big guys. And it can be pretty formidable, given the amount of features some of these franchises have built over time.

Finally, we got an interesting email from a different angle, thanks to Ryan Nicoletti. Ryan points out: "1. The risk / opportunity cost [for solo indies] isn't just ‘not getting paid’... it's also not getting paid programmer wages… The people that can make games have extremely valuable skills. 2. If ‘mimetics’ must drive my dev decisions to even have a chance, then I'm trading off the thing I want to buy with all the money I'm giving up from not doing a ‘boring’ business job: freedom to make the games I want to see succeed.”

In many parts of the world*, that’s the issue with small indies having to do ‘fast follow’ games. It just doesn’t map well with why they wanted to make games in the first place. Which was.. to be creative, have fun, and make the kind of game they truly want to play?

[*The U.S. in particular often has a massive gap between ‘boring programmer’ wages and indie game dev incomes. This is less true if you work at an actual game company, but… still true.]

The game discovery news round-up..

Finishing off for this week - and indeed month - there’s a lot to get through in terms of game discovery and platform news. So pull up a scrollwheel (?), and we’ll get through it all together:

Finally: we just spotted a new interview with former Kixeye CEO Will Harbin on his comeback. Which reminded us that the Facebook & mobile game dev (War Commander) was an ‘attitude outlier’, even in the San Francisco of the early ‘10s.

Exhibit A: Harbin telling the designer of Kixeye’s office to “make it more evil”: “The elevators open onto a lobby drenched in red light. Speakers overhead mutter boiler noises and bureaucratic murmurs - "Report to Lab B for testing," AOL dial-up sounds, "Ice cream social at 2:30 p.m." At the end of the lobby, an armed man in military fatigues sits at a desk, an iron chair at his side as if for an interrogation.” Is this… a discovery tactic?

Exhibit B? This ‘game dev recruitment video’, making fun of “competitors like Zynga (aka Janka), EA (aka EAARP), and Kabam (aka Kerpoop) at the time.” So yes, this was widely perceived to be dumb at the time. And… it’s still fascinatingly dumb today:

[We’re GameDiscoverCo, an agency based around one simple issue: how do players find, buy and enjoy your premium PC or console game? We run the newsletter you’re reading, and provide consulting services for publishers, funds, and other smart game industry folks.]

Nahoru
Tento web používá k poskytování služeb a analýze návštěvnosti soubory cookie. Používáním tohoto webu s tímto souhlasíte. Další informace