Dashboard averages hide the two populations under them

The most reassuring number on a live-ops dashboard is a metric that has not moved. Revenue per player flat, session length flat, conversion steady. Nothing to investigate.

That reading is right about as often as it is wrong. An average is a single number describing a distribution, and a distribution can change shape considerably while its centre stays put. In a live game the shape usually is changing, because the population feeding the average is changing underneath it every week.

The composition problem

Your player base is not a fixed group being measured repeatedly. It is a mix of cohorts arriving and leaving at different rates, and each cohort behaves differently.

Newer players convert at one rate and spend one amount. Players who have been with you a year convert at a different rate and spend a different amount. The average across both is a weighted blend, and the weights move whenever acquisition or churn changes.

This produces a specific and common illusion. If your veteran players are quietly getting more valuable while your intake of new players grows, the average can stay flat — more low-value newcomers diluting a rising veteran number. The dashboard says nothing happened. Two real things happened, in opposite directions, and they cancelled.

The reverse is more dangerous. If veterans are declining and you have grown acquisition to compensate, the average holds while the underlying game gets worse. You are refilling a leaking container faster, and the level looks stable right up until acquisition slows.

Neither situation is visible without splitting the number.

Averages over skewed distributions

The second failure is structural rather than compositional. Spending in mobile games is heavily skewed — a small group of players accounts for a large share of revenue, with a long tail of players spending little or nothing.

An average over that distribution is dominated by the top. Revenue per player can rise because a handful of high spenders spent more, while the number of players spending anything at all fell. Those are opposite stories about the health of the game, and the average reports them identically.

The practical consequence is that revenue per player is a poor early-warning metric. It responds late and it responds to the wrong things. The conversion rate — what share of players buy anything — moves earlier and describes the broader population. A game where conversion is falling while revenue per player rises is concentrating, and concentration is fragile: the fewer people your revenue depends on, the more a small change in their behaviour matters.

Reading the median alongside the mean makes this visible with almost no effort. When they move together, the whole distribution is moving. When the mean rises and the median does not, the change is at the top only.

The window problem

The third way an average misleads is by mixing time periods that should not be mixed.

Live games run events. An event week and a quiet week are different environments, and a monthly average across both describes neither. If your event calendar changed — more events, longer events, different spacing — a month-over-month comparison is comparing two different calendars, not two months of the same game.

The same applies to cohort age. A player's first week behaves nothing like their tenth. Comparing this month's average session length to last month's, when the population's average tenure shifted, measures the tenure shift as much as anything about the game.

The fix here is to compare like for like: cohort against cohort at the same age, event week against event week. That sounds obvious and is skipped constantly, because calendar-time reporting is what dashboards default to and cohort-time reporting takes deliberate setup.

See how we separate cohorts in live-game reporting →

What to split by, and when to stop

Splitting every metric by every dimension produces a dashboard nobody reads. The useful splits are the ones where you have a reason to expect the groups to behave differently.

Tenure is almost always one. New and established players are different populations in nearly every game, and the split between them explains more variance than most other cuts.

Paying status is another. Players who have bought something and players who have not respond to different things, and averaging them produces a number that describes nobody. This is particularly true for anything measuring engagement, because paying players engage more by definition.

Platform and market are worth splitting when your acquisition mix across them is moving, which it usually is. A shift in where your players come from will move a global average without anything about the game changing.

Beyond those three, add a split when you have a specific hypothesis about a group behaving differently — not as a general practice. A dashboard with too many cuts hides the signal as effectively as one with too few, because nobody looks at all of them.

The habit worth building is small: when a number has not moved, ask what would have had to happen underneath for it to stay still. Sometimes the answer is "nothing", and the stability is real. Often the answer is that two things moved in opposite directions, and one of them is the thing you needed to know about.

Talk to us about reading your live-game metrics →