Adversarial Methodology
A live stress test on a complex forensic passage. Adjust the confidence threshold. Click any redaction to inspect the rule. Download the audit log.
How Confidence Scores Drive Decisions
Confidence scores are not metrics. They are decision inputs. Each score maps to exactly one of four outcomes.
Direct identifiers and near-certain indirect identifiers. No human review required.
High-confidence identifiers with contextual or combination risk. Logged with flag for audit.
Ambiguous identifiers. System halts automatic elision. Reviewer receives context and rule.
Insufficient confidence for elision. Field retained in output. Decision documented in audit log.
One Messy Example. Eight Steps. Nothing Skipped.
This is not a clean case. One identifier is genuinely uncertain. Watch how the system handles it.
"The evaluee is a 41-year-old bilingual male of Puerto Rican descent, currently residing in the Eastside neighborhood of Austin. He was referred by his public defender, Ms. Gabriela Reyes, following a competency evaluation ordered in Cause No. 2024-CR-04471 in Travis County District Court."
Raw forensic passage ingested. Document received in memory. Not written to disk. Parser isolates text from any metadata structure.
Named Entity Recognition pass identifies candidates: "41-year-old" (AGE), "bilingual male" (DEMOGRAPHIC), "Puerto Rican descent" (ETHNICITY), "Eastside neighborhood of Austin" (GEOGRAPHIC), "Gabriela Reyes" (PERSON), "2024-CR-04471" (CASE_NUMBER), "Travis County District Court" (JURISDICTION).
First pass: pattern recognition + NER model. Seven candidate identifiers flagged. Each assigned preliminary type classification.
"Eastside neighborhood of Austin" — geographic subdivision raises combination risk question. Austin population: ~978,000. Eastside: ~85,000. Combined with age, ethnicity, and legal proceeding: population estimate narrows significantly. Is this a direct identifier? No. Is it a combination risk? Possibly.
Second pass: contextual evaluation. System identifies that geographic subdivision + demographic combination + specific court proceeding creates potential re-identification pathway. Confidence: 76%. Below auto-elide threshold.
AGE (41): 0.94 → auto-elide. DEMOGRAPHIC (bilingual male): 0.83 → auto-elide + flag. ETHNICITY (Puerto Rican): 0.91 → auto-elide. GEOGRAPHIC (Eastside, Austin): 0.76 → human review. PERSON (Gabriela Reyes): 0.99 → auto-elide. CASE_NUMBER: 1.00 → auto-elide. JURISDICTION (Travis County): 0.88 → auto-elide.
Each candidate scored against protocol threshold. Decision table applied. One field (GEOGRAPHIC) falls in the human review band. All others auto-resolved.
Six identifiers auto-elided per threshold. One identifier ("Eastside neighborhood of Austin", confidence 0.76) routed to human review queue with full context: field value, type classification, confidence score, governing rule, and combination risk flag.
Pipeline halts automatic elision for the flagged field. All other decisions finalized. Output held pending human review decision on the geographic field.
Reviewer receives: field value, risk classification (COMBINATION_RISK), confidence score (76%), governing rule (Expert Determination — geographic subdivision combined with demographic and jurisdictional data), and the three contributing fields creating the combination risk. Reviewer decision: ELIDE. Reasoning: combination with ethnicity and specific court creates population below acceptable threshold.
Human reviewer sees exactly what the system saw. Decision documented. Reviewer identity and timestamp logged. Reasoning captured in audit trail.
"The evaluee is a [AGE REDACTED] bilingual male of [ETHNICITY REDACTED] descent, currently residing in [GEOGRAPHIC REDACTED]. He was referred by his public defender, [NAME REDACTED], following a competency evaluation ordered in [CASE NUMBER REDACTED] in [JURISDICTION REDACTED]."
De-identified output generated. All seven identifiers elided. Clinical content fully preserved. Document structure intact. Re-identification risk: below Expert Determination threshold per reviewer attestation.
7 elision events logged. 6 auto-resolved by system. 1 escalated to human review. Reviewer decision, identity, timestamp, and reasoning captured. Decision source noted for each entry: RULE_ENGINE or REVIEWER. Reproducibility statement appended. Log available for IRB submission, deposition, or peer review.
Complete decision chain documented. Every elision traceable to its source: rule-based, model-based, or human reviewer. Audit log signed with protocol version and threshold applied.
On Human Judgment
Not as a safety net. As a structural decision. The pipeline does not claim omniscience. It surfaces uncertainty with full context and stops. A human clinician, researcher, or reviewer makes the call. That decision is documented as a first-class event in the audit trail — with identity, timestamp, reasoning, and the information the system provided at the point of escalation.
Confidence Threshold
Client is a ████████ Hispanic female referred by her attorney, ████████ of ████████ in ████████. She presents with reported symptoms of PTSD following a motor vehicle accident on ████████, on ████████. Client reports that her ████████, who works at ████████, drove her to the initial ER visit at ████████. She currently resides at ████████, and can be reached at ████████. Her treating therapist, ████████, has documented twelve sessions of EMDR treatment since the incident.
Items Below Threshold — Require Clinical Judgment
The system does not claim omniscience. It surfaces uncertainty. Human judgment determines the final boundary.
Machine-Readable Audit Trail
Every elision decision logged with its rule, confidence, and flag. The audit trail is the methodology. Exportable for IRB review or deposition.
Elider flags and scores. It does not decide. The pipeline is sufficient for direct identifiers and high-confidence indirect identifiers. Contextual, relational, and combination-risk identifiers require human clinical judgment at the review boundary. This is not a limitation. This is the design.
The Standard Is Not “Did We Remove Identifiers”
Elider evaluates four distinct risk categories. Each requires different detection logic. Each requires different documentation. All four appear in the audit log.
A single field that identifies an individual without additional information. Names, addresses, phone numbers, SSNs, dates of birth.
Two or more fields that, individually, would not identify an individual but, in combination, create a re-identification pathway. The canonical failure mode of Safe Harbor-only analysis.
A field that is not an identifier in isolation but becomes one given the context of the document, the population, or the proceeding. Geographic specifics, institutional references, incident descriptions.
Re-identification achieved by linking the document to external datasets. Occupation, institutional affiliation, and unique family structures can create a population of one when linked to publicly available records.
The Case Safe Harbor Alone Would Pass
No single field is a direct identifier. The combination is.
Evaluee is a 52-year-old male, employed as a senior electrical engineer, who sustained a traumatic brain injury in a workplace accident. He resides in a rural county in Central Texas with a population under 8,000. He is the only employee at his company with his job title and tenure. His treating neurologist noted that he has an identical twin sibling, also a professional in the same field, which complicates baseline cognitive comparison.
No HIPAA Safe Harbor identifier appears in this passage. All 18 categories are technically absent. Under Safe Harbor alone, this document would pass. Under Expert Determination, it fails. The linkage of occupation, geography, organizational uniqueness, and family structure reduces the re-identification population to a near-certain individual. This is what Elider's two-pass architecture is designed to catch.