CertifiedData.io
Cryptographic certification for synthetic datasets — prove y
Visit Website
April 7, 2026
How do you actually prevent PII from leaking into test data?
We’ve been bitten by this multiple times—real emails ending up in CSV fixtures and getting committed to repos.
What we’ve learned:
Manual review doesn’t catch it.
“Just be careful” doesn’t work.
Once it’s in a PR, it’s already too late.
The only thing that’s held up is treating it like a build failure:
scan in CI
fail on high-risk patterns
require explicit override if needed
We built a small local CLI to enforce this:
deterministic pattern matching
no network calls
exits non-zero on high-risk findings
Curious what’s actually working for others—especially at scale.
1 Comment
After dealing with this a few times, the main takeaway for us was:
This isn’t a review problem—it’s a build-time problem.
Manual checks miss it, and once it’s in a PR it’s already too late.
The only thing that’s held up consistently is:
CI gating
fail on high-risk patterns
explicit override if someone really needs to push
Curious if others have found anything more robust at scale.