28 APR 2026
David asks: How do we measure retention signal cleanly when users span the cohort boundary?
David questioned whether a retention analysis comparing the 24–48 hour cohort to the 0–24 hour cohort was biased by users who were active during both windows and asked how to isolate the real signal.
After Kerra's launch push to Notre Dame, David was tracking how many users who chatted with Kerra or spent time in workspaces during a 48–24h ago window returned in the 0–24h window. When the analysis came back, he challenged its validity:
okay but the analysis needs to be honest like it cant be biased by a user being active during the crossover boundary of now-24h to 48h-24h like how do we measure the signal from the noise here? thoughts?
This led to a discussion about cohort window methodology — specifically ensuring that the analysis counted only users whose first active session was entirely within the 48–24h window, not users who straddled the boundary. David then caught another issue when the data didn't match his priors:
wait this analysis doesnt make sense cause i know my user (david kariuki) has come back, no?
The analysis was rerun with corrected cohort boundaries.