Compliance, legal & trust-and-safety teams
Trust & Safety Triage
PilotA detect-and-escalate pipeline that classifies content and routes cases to human reviewers. It never takes enforcement action on its own.
The challenge
Compliance teams face growing volumes of content to screen against policy. Fully automated enforcement is legally risky, but fully manual review doesn't scale.
Our approach
The pipeline classifies public metadata against versioned, human-authored policy rules, assembles a deduplicated case file, and puts it in a review queue. Classifier accuracy is measured against a labeled evaluation set before thresholds change. Humans make every enforcement decision.
Architecture
A supervisor and its specialists.
Metadata Collector
Fetches public metadata under hard rate and scope caps.
Classifier
Gemini-based classification with confidence thresholds from versioned config.
Policy Mapper
Maps signals to versioned policy rules reviewed by counsel.
Case Builder
Assembles deduplicated, idempotent case files for reviewers.
Vertex AI / GeminiPythonPostgreSQLAlembicFastAPI review UI
Guardrails, enforced structurally
- Detects and escalates only. No automated enforcement
- Never stores raw disallowed media
- Thresholds and policy rules are versioned, not hard-coded
- Classifier validated against a labeled evaluation set
What it changes
- Reviewers spend their time on real cases, not triage
- Every escalation carries the policy version and evidence behind it
Next suite
Contract Review Agent
Want the Trust & Safety Triage for your team?
We'll adapt the specialists, data connections and approval rules to your systems in a scoped pilot.