Product Manager
ProductmediumFree
The Test That Lies
It won on conversion and lost on everything else.
FFinderk Originals 4.2 (349) 3,141 taken 30m Product Manager
The situation
Your new checkout flow lifted conversion 6%, but refunds ticked up, support contacts rose, and the result is barely significant with a week of data. Growth wants to ship it today; you suspect it's juicing a number at the cost of trust. The exec deck goes out tomorrow either way.
What you'll practice
Assess statistical significance and sample adequacy
Assess statistical significance and sample adequacy. Show it clearly — with evidence a reviewer can point to.
Weigh the primary metric against guardrail metrics
Weigh the primary metric against guardrail metrics. Show it clearly — with evidence a reviewer can point to.
Make a defensible ship / hold / iterate decision
Make a defensible ship / hold / iterate decision. Show it clearly — with evidence a reviewer can point to.
Communicate uncertainty honestly to leadership
Communicate uncertainty honestly to leadership. Show it clearly — with evidence a reviewer can point to.
The room
3 autonomous AI coworkers, each with their own agenda. They won't all agree.
A
Aisha Bello
Growth PM
Wants: Wants to declare the win and roll out immediately
Style: Confident, momentum-seeking
M
Mara Lindqvist
Analyst
Wants: Worried about significance and the refund signal
Style: Cautious, statistically literate
T
Theo Novak
Head of Product
Wants: Wants a clear ship/no-ship call, not hedging
Style: Decisive, impatient with ambiguity
Your workspace
Real tools, pre-seeded with context. You're not roleplaying, you're working.
Docs / wiki Kanban board Team chat Email
Scored on
Problem framingPrioritizationEvidence usageStakeholder alignment