On the data
Seeded synthetic data for a fictional direct-to-consumer beverage brand. No employer or client data appears anywhere in this project. The generator is modelled on how social data actually behaves, with heavy-tailed reach, format and day-part effects, a compounding follower base and paid-boost distortion, so the analysis code meets the same shapes it would meet in production.
Why it was hard
Most social reporting fails in the same four ways, and all four produce a number that looks perfectly reasonable on a slide.
Definition failures
- "Your engagement rate doesn't match mine"
- Rate on reach and rate on followers are both correct and roughly three times apart. Both are computed, named separately, and defined once in one module.
- The headline rate disagrees with itself
- Averaging per-post rates lets a 200-reach post count as much as a 200,000-reach one. Aggregate only: total engagement divided by total reach.
Sample-size failures
- "Best time to post: Friday 9pm" from five posts
- Day and hour cells hold two or three posts each, so one viral post crowns an empty slot. Recommendations run on day-parts with a 20-post floor, and thin cells render blank.
- "Carousels beat Reels" on a difference that is noise
- No significance test, and formats pooled across platforms. Mann-Whitney U with effect size, computed within each platform.
That last one is the subtle one. "Short video gets 2.5 times the reach share" sounds like a format finding, but short video only exists on TikTok, and TikTok engages at 2.4 times X's rate. A pooled format ranking is mostly measuring the platform. Every format comparison here is within-platform for exactly that reason.
The averaging trap, measured
The mean of per-post engagement rates on this account is 4.73%. The aggregate rate, total engagement divided by total reach, is 5.32%. Same posts, same month, an 11% gap.
Worth stating plainly: the direction of that gap is not fixed. It is commonly assumed that averaging inflates the headline, because small posts tend to show high percentage rates. Here it deflates, because on this account the high-reach posts also engage well. Which way it breaks depends on how reach and rate correlate in your data, and the only safe move is to stop averaging rates at all rather than to memorise a rule of thumb.
What it found
Half the account's reach depends on one recommendation algorithm, and that platform also engages best, so the dashboard reads it as a win.
TikTok carries 48.4% of reach and the highest engagement rate. A performance dashboard reports that as the account's strongest channel, which it is. The generated stakeholder narrative flags it as a concentration risk instead, because both facts are true at once. Half the account's reach sits behind a single recommendation algorithm that nobody in the room controls.
The opposite problem sits two rows down. X takes 715 posts, more than any other platform, and returns 11.7% of reach at the lowest engagement rate on the account. That is the clearest effort-against-return finding in the set, and it is invisible on any chart that ranks platforms by a single metric.
On Instagram, Reels earn 1.5 times the reach their calendar space costs. Stories earn a quarter of theirs.
What I would report
One engagement rate on the front page, reach-based and aggregate, with the follower-based figure available and labelled rather than hidden, so the second definition stops arriving as a surprise mid-meeting.
Timing guidance moves to day-parts with a stated sample floor, and cells under the floor render blank rather than ranked. LinkedIn's weekend column stays mostly empty because the brand barely posts there, which is information, not a gap to fill with a guess.
The concrete recommendation on this account is a calendar reallocation, not a vibe. Move Story and Static slots into Reels on Instagram, and audit whether 715 posts a year on X is defensible against 11.7% of reach. Follower charts stay on independent scales per platform, because X's slowing growth is a real story that an Instagram-sized axis would hide entirely.
Limitations, and what I would do differently
Reach is a platform-reported number and platforms define it differently. Comparing reach across four of them is directionally useful and precisely wrong, and no amount of careful arithmetic fixes that. The format multiplier also treats calendar slots as interchangeable units of effort, which they are not. A Reel costs considerably more to produce than a Static, so a 1.5 times reach return is not automatically a 1.5 times return on effort.
The 20-post floor is a judgement I picked for stability, not a derived threshold. It should be set from the variance in each account rather than copied across, and on a low-volume account it would blank almost everything. At that point the honest output is "we cannot answer this yet" rather than a lower floor.
Given more time I would add paid-against-organic splits throughout, since boosted posts distort every reach-based rate on the page, and bring in a saves-and-shares-weighted quality measure. A save is worth considerably more than a like, and a flat engagement count treats them alike.
Artefacts
Source and tests on GitHub, the metric dictionary and QA checklist, and 28 tests passing on a clean checkout.
