10/23/2025
Why Winning in Every Group Can Still Mean Losing Overall
A mobile game studio’s analytics team faced a strategic decision: invest in Casual Mode for beginners or Ranked Mode for skilled players?
They evaluated Day‑7 retention (players active on the 7th day after install) for 1,342 players, segmented by skill.
Casual Mode (672 players)
Novices: 202 players, 143 active → 70.8%
Veterans: 470 players, 301 active → 64.0%
Overall: 444 / 672 = 66.1%
Ranked Mode (670 players)
Novices: 469 players, 328 active → 69.9%
Veterans: 201 players, 127 active → 63.2%
Overall: 455 / 670 = 67.9%
What’s surprising?
Within both subgroups, Casual > Ranked (70.8% vs 69.9% for novices; 64.0% vs 63.2% for veterans).
But overall, Ranked > Casual (67.9% vs 66.1%). That reversal is Simpson’s Paradox.
Why the reversal?
The confounder is skill mix.
In this dataset, novices retain better than veterans (≈71% vs ≈63%).
Ranked has many more novices (469/670 ≈ 70%).
Casual has far fewer novices (202/672 ≈ 30%).
Because the higher‑retention segment is concentrated in Ranked, the aggregate flips even though Casual wins within each skill group.
How the team responded
Interpreted results by segment to avoid misleading aggregates.
Improved Casual to onboard and engage new players.
Optimized Ranked to retain veterans.
Built dashboards that always segment key metrics (e.g., by skill, region, platform).
Takeaway
Always examine subgroup data before acting on overall metrics. Composition effects (mix of users) can change or even reverse the story.
At KYFEX, we help organizations design AI and analytics strategies that uncover deeper insights, ensure accurate interpretation, and guide effective decisions.
https://kyfex.com