Your eval tool measures it. A person decides what passes.

Most teams that ship an AI system already run evals. The experiments sit in Langfuse, LangSmith, Braintrust or W&B Weave, and the numbers are good. What is usually missing is everything around the numbers. Who decided that 0.93 was good enough? Was the release that shipped the one that was tested? When a supervisor or an auditor asks, where is the record?

Eval tools record what the model did. They were never meant to record who was allowed to let it ship. That gap is what Compliance Mode is for, and as of this week it reads evidence from all four of those tools.

Four connectors, one rule

Langfuse, LangSmith, Braintrust and W&B Weave are each connected once by your organisation, on Admin → Integrations. After that, a build step names an experiment or an evaluation, and Deliverd reads its scores from the tool's own API.

If you would rather not connect a tool, a pipeline can send the tool's own report instead, and Deliverd reads it on the server. That works for Weave and Braintrust, and for promptfoo, Inspect, garak and JUnit too. A pulled run and a sent report are mapped by the same function, so one threshold meets either.

Either way, the result is the same thing: scored evidence on a governed task, with a link back to the run in the tool that produced it.

Why it is built as a human-in-the-loop system

It would have been simpler to let the pipeline post its scores and say "passed". We did not, for four reasons.

Scores are evidence, not decisions. Deliverd records what the tool measured, under the name the tool gave it, and invents nothing. It adds no severity the tool did not give and no verdict on an aggregate. Whether 0.93 passes is the organisation's call.

Whatever submits the results does not decide what passes. Thresholds such as accuracy >= 0.9 are set by a person, in the app. An API key is refused, including a key that belongs to that same person. An organisation can set one default bar for every governed task, and a person can override it on a single task. Either way, the build step that sends the evidence cannot move the bar it is judged against.

The agent never holds the eval tool's key. A connector reads Langfuse, LangSmith, Braintrust or Weave with your organisation's key. An owner or an administrator connects it, in the app, and the key is tried against the tool before anything is kept. It is then sealed, and no member's session can read it back. The build step asking for evidence holds a Deliverd key with permission to submit evidence, and that is all.

The newest run decides. Once a control has thresholds, only its newest scored evidence counts. A good run last week does not hold the gate open after a bad one today. And when the model, the prompt or the guardrails change, the decision reopens and the evaluation has to be met again.

Why it is built as infrastructure

We built every connector on the same parts our customers use.

One reader, everywhere. Each tool's mapping lives in one shared package. The CLI, the MCP server, the REST API and both SDKs send the same request, and the server turns it into the same results. A fix to a reader reaches every client at once.

A build step is just another agent. It authenticates with a key, asks for evidence, and asks the gate before it deploys. In a pipeline that is two commands with the CLI:

npx deliverd governance evidence "$TASK" --langfuse credit-memo-v12 --control model_evaluation
npx deliverd governance check "$TASK"   # exits 1 when the gate is closed

The same pull is one call from the TypeScript or Python SDK, and submit_evidence on MCP takes the same pull.

Every call leaves the same way. Each pull goes out over https only, never to a private address, follows no redirects, and has a timeout and a size cap. A stored key is only ever sent to the address it was connected for.

Only scores cross. No connector reads an item's, a run's or a span's inputs or outputs. Those can hold customer data and the model's answers, and the evidence does not need them. It needs the scores and a way back to each failure.

What this is not

Deliverd does not run your evals, and it does not replace your eval tool. The experiment, the traces and the dataset stay where they are, and every result links back to them.

It also does not make anyone compliant. A framework pack is a starting point for your own review. What Deliverd gives you is the record: what was tested, the scores, who set the bar, who approved, and that nothing has changed since.

Try it

Compliance Mode is on Business and Enterprise. Connect a tool under Admin → Integrations (how), add a threshold to a governed task, and point one build at it.

Deliverd is human-in-the-loop infrastructure for AI agents. Evidence, thresholds and the gate are documented in the Compliance Mode guide.