It is a common problem when you have a worthy hypothesis (at least you believe it's worthy) about new features or improvements, but the production costs are sky-high, and there is no solid proof it would work. Moreover, your team is skeptical about it and expressing concerns (or vice versa). In that case, the final decision "to do or not to do” usually relies on someone's expertise and beliefs — but we all know how those are, in fact, unreliable.

The simple part of this approach is that some features/improvements might be tested in a limited version; for example, if you think about rebalancing or narrative rewriting, you can change those just in the scope of the first 15-60min and just ignore the fact that it would lose consistency with a whole other game because the potential value of the experiment is much higher then any possible risk related to failed cohorts (and risk is probably exaggerated too).

It's not that much of a secret sauce, yet still, there is too much bias about avoiding risky experiments, even when the results might outweigh the expected risk a dozen times.

But sometimes, there is no way to make 5% of expensive features or improvements because their minimal viable realization is already expensive; then you try to:

The idea is that you may test not the feature itself but separately test some of it's effects / factors / risks or pre-interests. General approach: decompose the initial feature on risk/value factors, prioritize them, and brainstorm on how they might be tested separately as part of other, much simpler features.

In the best case, you can find something that affects the same kind of experience and metrics, probably with lesser impact, but with results that might be interpreted and extrapolated to evaluate the initial feature's worth, at least to some extent.

At least you can find something to test one of the most crucial risks, pre-interest, or to receive any kind of result that might convince you if the initial feature is worth it or not.

The simplest example: you have a game without guilds and ask yourself if this is a valuable feature. You can put “Guilds” button on the main screen and tell on tap that guilds require X level; also, there can be a mockup or list of guilds features (instead of simple "coming soon”). Next, you measure conversion from seeing this info to reaching X level versus those who didn't have the same button. The problem is that this kind of test tells you nothing about the value and impact of the Guild feature itself, but it's probably the only way to measure pre-interest (except cast-dev, but I can't advise relying on it). Such tricks shouldn't affect engagement much, so any result beyond statistical noise might be considered positive.

Less relevant but still a simple example to understand the idea: it's always a good practice to use offer segmentation / adaptive offers for most games, especially those that depend on whales. Wanna proof without making an actual feature? Try a double offer.

YES, it's not a fully correct example, it's a different feature, targeting different goals, BUT it's also proof of the basic idea of segmentation: that there are users that are ready to spend either less or more for the basically same goods, offered in the same point of the game and differs by only a value — in case you had concerns, ofc 🤷🏻

It's a rare case when you can actually test something on a much cheaper realization but still reliable results. I have one good example, but to provide that, I need more words and context to explain :)

Once, I've worked with a classic time-manager. Core loop provides you with time-sensitive encounters/orders that should be solved by a chain of interactions with available objects, eg "the client wants a coffee and burger > you run to the coffee machine, start it, wait for coffee, get it, run to the kitchen for a pre-cooked burger, bring both to the client before his timer runs out and receive a reward, etc."

Time managers have level progression and light meta-game. Basically, all you can do at the meta layer is upgrade the objects you interact with: the coffee machine will make coffee faster, the kitchen will make burgers faster, or increase max burger capacity. Upgrade costs you the same soft currency earned on level progression. Objects for each level are pre-setted and always available. Monetization includes renewable attempts ('health'), on-level boosters and soft/hard used primary to upgrade objects in metagame.

Those of you with a game design background should already guess what is wrong here. The main problem is that when you're creating economic pressure by creating levels that require more currency for upgrades than you previously allow to earn — your paywall is too fucking obvious. Plus, core mechanics provide you with a lot less variability and randomness than match3's or similar. And, yeah, you can add some renewable sources of currency, but either at the expense of giving away your economy or by increasing the costs of new content production.

I thought about a solution that would allow us to avoid content dependency by more effectively monetizing each separate piece of content and increasing liveops earn to cost ratio. Finally, I came up with those ideas:

So, you get the idea. Both my (publishing) and studio product people agreed it has the potential to boost monetization. But all of us understood its costs (despite in the long-term it targeted cost reduction), and on top of that, the studio saw significant risk in ruining mid/late-game. Their main concerns were:

Fair concerns, I should say. Still, I believed we would earn much more than might lose. But how to prove that? Obviously, all that staff costs like hell.

We discussed much, brainstormed, and at the end of the day, we mutually came up with the perfect idea: what if we test the main risks only? Not the whole feature with its gacha, adaptive balance, etc — but just the minimum that causes the same frustration, and let us know if that's a thing.

The idea was to put a new 'object' (Gum Machine) in all levels from 8 (~1-2d), lock it up on iap purchase (special offer), and add customers’ requirements for gum. Balance wasn't affected: you can ignore that object, fail all gum-orders, and win exactly as before. Buying GumMachine also gave minimal value and barely affected your winrate.

It's less than 10% prod-costs of the whole idea, but that simplification allows us to measure both the main risk that concerns the studio the most AND at least one of my hypotheses that visible-persistant objects should be sold well.

Now, try to guess how it affected the metrics of new users? No, I mean it — before opening a toggle, suggest some numbers, say it out loud, and only after that click below.

So, the hypothesis was proven, and the next step should be to implement the whole complex feature. And… it has never happened :) but that's a different story.