One number that surprised me
A metric I had not been tracking, that turned out to predict more than the ones I had been.
Halfway through April I added a metric I had not been tracking: the fraction of users who came back within twenty-four hours of their first session. I had been tracking day-seven return. The two numbers should have been related. They were, but in a direction I had not expected.
What the number told me#
Twenty-four-hour return is, for my product, twice as predictive of month-three retention as day-seven return is. The users who liked the product enough to come back the next day were almost all still around ninety days later. The users who waited a week were a coin flip.
I had been optimising for the wrong end of the funnel.
What I changed#
A handful of small changes in the first session. None of them changed the first session itself — the first session was already good. They changed how easy it was to come back to it. A persistent indicator on the welcome screen for returning users. A nudge that knew what you had done last time. A "continue" that meant continue, not start over.
Twenty-four-hour return went up. Day-seven return went up by less. Month- three retention went up by more than both.
What I learned#
The metric I was not tracking was the one that mattered. The dashboard test from a few weeks ago held up: every number I track has earned the slot. Every number worth tracking has, at some point, been a number I had not been tracking yet.