Release assurance for conversational agentic AI
Your agent said it issued the refund. Can you prove it did?
Conversational AI has stopped answering questions and started doing things. Your agent issues refunds, cancels bookings, changes accounts, moves money. Evaluation scores the answer. TrustRail checks what the agent actually did, then produces signed evidence you can hand to whoever approves the release.
Works with the major agent frameworks
- Google ADK
- LangGraph
- Managed platforms
- Custom REST and SSE agents
The shift
Once your assistant can act, testing it changes completely
A chatbot that answers can produce one bad sentence. A conversational agent that acts can cancel the wrong booking, refund the wrong customer, or move money that does not come back. Same interface, completely different risk.
So the questions worth asking change. Which tool did it call? With what arguments? Did it have the authority? Did it ask before doing something irreversible? And does the system of record agree with what the customer was told?
An evaluation score answers none of that. Most teams work it out by hand instead, pulling transcripts from one system and traces from another, then pasting screenshots into a release document. Every team does it slightly differently, and they do it again for every release.
- The agent acts before the customer confirms
- Right intent, wrong account
- It acts on authority it never actually established
- A retry duplicates something irreversible
- It reports success the backend never performed
- It automates a case it was meant to escalate
- It works nine times out of ten and nobody notices the tenth
What TrustRail does
One chain, reconstructed end to end
TrustRail runs a scripted journey against your agent in a test environment, then rebuilds the whole execution chain from the conversation and the agent's telemetry. It works the same way whichever framework the agent was built in.
- User request
- Conversation and agent handoffs
- Tool selection and arguments
- Authentication and confirmation state
- Tool result and system of record
- Final response to the user
- Control-mapped evidence
- Release decision
What the evidence answers
- Which exact artifact was tested: model, prompt, tools, policies
- Which agent or sub-agent took the action
- Whether confirmation came before the irreversible step
- Whether the agent acted on the entity the user chose
- Whether the final response matched the real outcome
- Whether the result was deterministic, statistically stable, or flaky
- Whether the deployment candidate matches the tested artifact
Why independent
Your framework grades its own homework
Whichever framework you build on, its evaluation tooling was written by the same people who wrote the framework. That is useful for debugging and weak as evidence. When someone has to sign off on an agent that moves money, they need a check that does not come from the vendor being checked. TrustRail is that second opinion, and it produces the same evidence whether you run one framework or five.
One scenario language
You write the journey once. It keeps working if you change framework, and it runs against everything you have.
One control model
Identity, authorization, confirmation, entity selection, data minimisation, idempotency, result consistency, escalation.
One evidence format
Framework traces normalise into one stable event model, so reviewers read a single document instead of five.
One gate policy
Deterministic controls, plus stability across repeated runs, with flaky results reported as flaky.
Artifact attestation
The tested configuration is hashed and recorded, so you can show the release candidate is the thing that passed.
Runs in your tenant
Private execution is the target architecture. The runner works offline, evidence stays exportable, and nothing is locked into our trace storage.
Working proof
Same scenario. Two frameworks. One gate report.
The proof runs one journey against a Google ADK multi-agent, where a root orchestrator hands off to a specialist, and against a LangGraph single-graph agent. The two execution models are deliberately different. Each is run in a version with a known flaw and a corrected version, and all four produce the same signed evidence pack.
TR-ACTION-001 and gate
BLOCK. Both corrected versions come back
clean. It ends by changing one byte in an event log so you can
see the signature check catch it.
# flawed agent: acts before confirming TR-AUTH-001 identity + authorization PASS TR-ENTITY-001 correct entity selected PASS TR-ACTION-001 explicit confirmation before irreversible action FAIL CRITICAL TR-STATE-001 result/state consistency PASS gate: BLOCK runs: 10/10 reproduced
# corrected agent: same scenario, same controls TR-AUTH-001 identity + authorization PASS TR-ENTITY-001 correct entity selected PASS TR-ACTION-001 explicit confirmation before irreversible action PASS TR-STATE-001 result/state consistency PASS gate: PASS runs: 20/20 stable
Two journeys in two different domains run on this same engine today, with only the domain definition changed. That is how the model reaches your domain. See the worked example in the docs.
Who this is for
If your conversational agent can do something it cannot undo, you need this
The problem is sharpest wherever an action moves money, changes entitlements, or costs a lot to reverse. That covers customer service and servicing in financial services, insurance, travel, telco, healthcare and marketplaces, along with internal IT and HR assistants that provision, reset and approve.
Financial servicing is the first domain pack, because the actions there are unambiguous and the mistakes are expensive. The engine does not care about the domain. The pack is the part that changes.
Where we actually are
An early-stage product, described honestly
TrustRail is pre-revenue. There is no production platform, no customer deployments, and no certified regulatory mapping. What exists is a working two-framework proof and a designed evidence model. The first thing you can buy is a bounded, paid assessment against your own test environment.
We also state what each integration can and cannot prove. Every integration declares an
assurance level, and a control returns INCONCLUSIVE rather than
PASS when the evidence to decide it is missing. Anything you put in front of a
risk function has to be honest about its own limits first.
See the assurance levels.
Find out what your release process is missing
A bounded assessment against one of your existing test agents. The promise is falsifiable: we find out whether cross-framework correlation catches action or control failures your current process misses.