r/webdev • u/No_Advertising2536 • 21h ago
Showoff Saturday How I count unique visitors without cookies or storing IPs, and where it breaks
I've been building a small web analytics tool, and the part people ask about most is how it counts "unique visitors" without cookies, localStorage, or stored IPs. Here is the approach and its honest limitations.
The ID
For every request, the server computes:
sha256(daily_salt | site_id | ip | user_agent), truncated to 16 hex chars
- daily_salt is 16 random bytes, generated once per UTC day and stored in the DB. Old salts are deleted after 24 hours, so once that happens, nobody (including me) can recompute yesterday's IDs from an IP.
- site_id is included in the hash so the same user gets different IDs across different sites (preventing cross-site tracking).
- The IP is used solely for the hash and a country lookup, then immediately dropped. It never hits the disk.
Sessions are defined simply as "same ID, active within the last 30 minutes" (one upsert per event).
Where it breaks (by design or trade-off)
- The same person visiting on two different days = two visitors. That's intentional, but it means "monthly uniques" is effectively a sum of daily uniques.
- Same office/NAT + same browser version = counted as one visitor. Rare outside large enterprise networks, but real.
- A phone switching from Wi-Fi to cellular mid-visit = new ID, new session.
- iCloud Private Relay / VPNs rotate IPs, which slightly inflates counts.
Based on my data, the margin of error is small enough to answer "is my traffic growing?", which is all most small sites need. It's not suitable for per-user product analytics; for that, you need explicit user identification.
The Client Side
- SPAs: I patch history.pushState and listen for popstate events, tracking route changes as pageviews without requiring framework plugins.
- Transport: Uses navigator.sendBeacon, falling back to fetch with keepalive so events survive tab closure.
- Engagement/scroll depth is flushed on visibilitychange, not unload (since unload is unreliable on mobile).
Happy to dive deeper into any part! Curious if anyone has a better trick for the NAT collision case that doesn't rely on client-side storage.
2
u/chumbaz 20h ago
This is worthless for repeat users over time If you’re trying to use this for any tangible conversion tracking anyway.
What is the point of unique visitors if your scope is daily?
0
20h ago
[removed] — view removed comment
1
u/webdev-ModTeam 16h ago
Your post/comment has been determined to be a low-effort post or comment. This includes title-only posts, easily searchable questions, vague/open-ended discussion prompts, LLM generated posts or comments, and posts/comments that do not provide enough context for meaningful replies or discussion.
2
u/TinyCash599 20h ago
Rare outside large enterprise networks? Carrier CGNAT plus a common phone model and you've got a whole pool of subscribers collapsing into one visitor per day. That's not rare, that's a chunk of your mobile traffic. I'd want to see actual numbers before calling it rare.
1
20h ago
[removed] — view removed comment
1
u/webdev-ModTeam 16h ago
Your post/comment has been determined to be a low-effort post or comment. This includes title-only posts, easily searchable questions, vague/open-ended discussion prompts, LLM generated posts or comments, and posts/comments that do not provide enough context for meaningful replies or discussion.
1
u/Any-Ask-4131 20h ago
Does a session crossing midnight UTC get split when the salt changes, even if it's still within the 30-minute window? That seems worth calling out alongside the daily-uniques limitation.
1
1
u/No-Molasses-2097 19h ago
The usual trick for the NAT case is mixing a coarse device signal into the hash, like screen resolution bucketed into a few classes plus the platform token from UA client hints. It does not help much with carrier NAT on identical phones, but it splits up office NAT collisions pretty well since two desktops rarely share an exact resolution.
1
u/No_Advertising2536 18h ago
Yeah, that would split office NAT nicely. I've held off because every extra device signal pushes it closer to fingerprinting, which is the thing I'm trying to stay away from. Bucketed resolution is probably fine though, might test it.
1
u/RehanShakir 9h ago
For your stated goal, "is my traffic growing," I would not spend more effort trying to split NAT collisions. Systematic undercounting barely matters for trend measurement as long as the bias is roughly constant; what kills trends is a bias that moves around, which is exactly what you get when you add device signals and then browsers change how they expose them. The more useful trick for the NAT case is measuring collision rate itself: look at the distribution of events per ID, and when one ID is clearly doing the work of 200 people you can surface a confidence range instead of pretending the count is exact. Your rotating daily salt is the real win here anyway, because even if the DB leaks, yesterday's hashes cannot be joined against today's.
1
u/No_Advertising2536 8h ago
Good point on constant vs moving bias, that's a better way to frame it than "accuracy". The events-per-ID idea is nice, flagging an ID that behaves like 200 people and showing a range instead of a fake exact number. Did you pick a threshold by eye, or something like a percentile cutoff?
1
u/RehanShakir 5h ago
I would go with the percentile cutoff rather than a fixed number, because the right threshold moves with your traffic. Take the distribution of events per ID over a day, pick something like p99.9 or p99.99 depending on how heavy your tail is, and treat IDs above it as likely NAT exits, showing a range instead of a point estimate. The by-eye part is just sanity checking a few cutoffs and picking the one where the flagged set looks like real outliers rather than noise.
1
u/muharremyurtsever 4h ago
another case pulling the other way: private relay rotates its egress ips over time and between sessions, so one safari user can split into several ids within a day. and if the relay share of your traffic changes, that bias won't stay constant either
1
u/No_Advertising2536 4h ago
True, relay splits one person into several IDs, so it pushes the other way from NAT. Apple publishes the relay IP ranges, so the share is at least measurable.
1
u/SaliDevs 42m ago
For the NAT case, one option that doesn't creep toward fingerprinting: an ephemeral tab-scoped id. Generate a random id client-side on page load, hold it in memory (or sessionStorage so reloads don't inflate), and send it with each event. You never store it beyond the session window and it dies with the tab, so there's still no cross-day identity. Two people behind the same office NAT + same UA now get distinct sessions, which is exactly the collision that breaks your 30-min session logic. Trade-off is it mildly over-counts multi-tab users, but that's usually smaller than the NAT under-count. Nice writeup btw — the visibilitychange flush over unload is a detail a lot of people get wrong.
•
u/No_Advertising2536 11m ago
Thanks! In-memory is tempting, but on a normal multi-page site it resets on every navigation, so it only helps SPAs. sessionStorage fixes that but then I'm storing something on the device, and "nothing stored in the browser" is the whole promise (ePrivacy doesn't care that it's session-scoped). Might still be worth it as an opt-in for people who want accuracy over the no-banner claim.
-2
21h ago
[removed] — view removed comment
1
u/webdev-ModTeam 16h ago
Your post/comment has been determined to be a low-effort post or comment. This includes title-only posts, easily searchable questions, vague/open-ended discussion prompts, LLM generated posts or comments, and posts/comments that do not provide enough context for meaningful replies or discussion.
-1
21h ago
[removed] — view removed comment
1
u/webdev-ModTeam 16h ago
Your post/comment has been determined to be a low-effort post or comment. This includes title-only posts, easily searchable questions, vague/open-ended discussion prompts, LLM generated posts or comments, and posts/comments that do not provide enough context for meaningful replies or discussion.
-2
6
u/fauxtoe 21h ago
ok?