Chat & RAG audit
Live foundationFind confident answers that the knowledge cannot support.
Start with the Northstar reference harness, then use its evidence contract to onboard an external customer agent safely.
Authenticated live scan
Run the reference support harness.
This makes real Gemini calls, grades every response, writes the run and case evidence to Supabase, and emits Langfuse traces when enabled.
Customer account required
Authentication prevents anonymous users from creating model costs or viewing evidence.
Sign in or create an accountReady0%
Target: ProveWorks reference RAG harness · no external customer system is contacted.Preserved demonstration
Replay the deterministic reference run.
This replay never calls a model and never creates usage charges.
Customer scan experience
Watch the evidence build, phase by phase.
This preview replays the preserved run. It does not call a model or create charges. Production will use the same interface with persisted worker events.
Ready
Ready to replay the saved audit.
Prepare & lock scope
QueuedValidate the target, cases, rubric, and evidence contract.
Baseline scan
QueuedRun the current agent without crediting any remediation.
Diagnose & apply controls
QueuedMap failures to retrieval, prompting, safety, or application logic.
Verification scan
QueuedRepeat the identical suite against the fixed configuration.
Publish evidence
QueuedSeal the report, score provenance, timings, and audit ledger.
Failure coverage
Six ways support agents break.
Ten cases per category, with explicit hard and soft assertions.
out of scope hallucination
stale conflicting info
adversarial instruction injection
ambiguous phrasing
numerical precision
tone brand consistency
JSON suite → lexical RAG → deterministic or Gemini model adapter → rule-based grader → reproducible JSON/Markdown report.