Register every agent, who owns it, what it may touch, and the targets its owner has signed.Judge them against your own definition of correct, across quality, security and load, with graders an expert has checked.Turn failures into one scoped change at a time, each proving itself before a person lets it ship.
Agent registry
7 agents, each with a named owner
- Returns assistantRetail ops, over MCPActive
- Order lookupFulfilment, over HTTPIn review
- Refund approverFinance, over MCPCanary
- Claims triageInsurance ops, over MCPActive
- Invoice matcherFinance, over HTTPActive
- KYC document checkCompliance, over HTTPIn review
- Policy explainerCustomer care, over HTTPDraft
An agent gets an owner and signed targets before it gets a score.
Example data
Agents reach production faster than anyone can prove they still work
Teams stitch together a tracing tool, an eval library, a load tester and a spreadsheet of test prompts. None of them agree on what good means.
See what one agent scoresScores you cannot trust
Automated graders nobody has checked against human judgement. A pass rate can be a made-up number.
Security tested once, if at all
Prompt injection, leaked instructions and echoed card numbers get found by customers, not before launch.
Reliability drifts
Models and prompts change underneath the agent. Without scheduled re-tests, regressions surface as incidents.
Fixes are risky
A prompt change that fixes one failure can quietly break another, or be tuned to the test set.
Nothing is on record
Risk and audit teams get screenshots, not evidence anyone can verify.
What “reliable” actually means
Most tools sell one rung of this ladder. AlignX owns the top two.
Covered by existing tools
Where AlignX lives
- 01
Observability
Tells you what happened.
Covered by existing tools
- 02
Evaluation
Tells you whether it was good.
Covered by existing tools
- 03
Reliability
Tells you whether you can trust it to happen again.
Where AlignX lives
- 04
Continuous improvement
Makes it better over time.
Where AlignX lives
Seven objectives, signed per agent
Not one vanity score. A readiness score rolls them up and counts objectives with no data against it, so a gap never reads as a pass.
5 of 7 objectives counted as met
- Task success
- Share of tasks completed correctly against the golden standard.
- Consistency
- Share of tasks that succeed on every repeated run, not just once.
- Golden-standard correlation
- How closely answers track your reference answers over time.
- Policy safety
- Security, PII and policy violations per 1,000 runs.
- Release stability
- Share of releases that regress any objective.
- Recovery
- Time from a detected drop to a fixed or rolled-back agent.
- Efficiency
- Cost and p95 latency per successful task.
Every change passes through one loop
A score only counts once an expert has checked the grader behind it. From there a failing target sends the agent back round, and every verdict is kept, rejected fixes included, so nothing is learned twice.
- 01
Evaluate
Run the golden set, the security probes and a load test. Every hop is traced.
- 02
Label
An expert marks answers Pass or Fail, with a failure mode and a note.
- 03
Validate
Graders are scored against those labels. Until one is validated, task success reads not measured.
- 04
Report
Signed reports as PDF and JSON, checkable offline with a public key.
- 05
Diagnose
Failures cluster by mode and topic, each with a real example. Expert labels outrank the graders.
- 06
Improve
One scoped change at a time: safety first, then quality, then cost.
- 07
Approve
Ten checks, then a named owner decides. No change ships without one.
- 08
Canary
The change meets a slice of live traffic. A bad canary rolls back on its own.
- 09
Watch
CI gates, production traces and scheduled runs keep watching every approved version.
One harness, five capabilities
Govern runs across the others rather than sitting beside them: everything below is judged against the agents you have registered and the targets their owners signed.
A failing target in production sends the agent back to Test.
Govern
Know what is running and who answers for it.
A registry of agents with owners, an ontology of topics, code-checked rules and protected terms, and reliability targets signed by a named person.
Not yetSingle sign-on, role-based access, audit logs and risk reporting are planned.
Test
Quality, security and load, against your own golden set.
A golden set with a held-out slice, evaluation with repeats and confidence intervals, 30 OWASP probes where a timeout counts as unknown rather than a pass, load testing, and graders validated against expert labels.
Deploy
A release has to pass the targets its owner signed.
A CI gate that answers 200 or 422 against the signed targets, a ten-step promotion gate, an approval queue, and a canary that rolls itself back.
Monitor
Every approved version stays under watch in production.
Hop-by-hop traces of every action, tool call and reasoning step, production answers scored for policy violations, alerts that open an incident with its runbook, and evaluation every six hours with a daily security scan.
Not yetBehaviour drift detection and stability tracking are planned. Live answers are checked for safety, not quality.
Improve
Failures become scoped fixes that have to prove themselves.
Failures clustered by mode and topic with expert labels outranking graders, an improver that works safety first then quality then cost, and a memory that keeps rejected fixes so nothing is proposed twice.
Not yetJudge-ranked search over candidate fixes is planned.
Your agents. Your frameworks.
One reliability layer.
Halo tests an agent from the outside, with no access to its code — which is why it also works on an agent a vendor built for you.
- MCP (Streamable HTTP) or plain HTTP
- POST /evaluate answers 200 or 422
- POST /traces, scored for policy violations
- The agent's own stack is never touched
Eight roles, each with a job in the loop
One app with a view per role. Labelling, approval and incident work all happen in it, not in chat or spreadsheets.
Who works here
AI platform owner
Connects agents, runs the fleet, sets the scheduler and the alert rules.
Where they act in the loop
Who works here
Agent owner
Signs the agent's targets and approves changes before they ship.
Where they act in the loop
Who works here
QA / domain expert
Labels answers, which is what validates the graders.
Where they act in the loop
Who works here
Domain owner
Writes the ontology: the topics, the checked rules, the protected terms.
Where they act in the loop
Who works here
Engineer
Wires the CI gate and production traces into their pipelines.
Where they act in the loop
Who works here
Approver
Decides on candidates the gate has passed.
Where they act in the loop
Who works here
Operations
Works incidents and runbooks, resolves or rolls back.
Where they act in the loop
Who works here
Risk and audit
Reads signed reports and verifies them offline.
Where they act in the loop
What changes once an agent is under assurance
You know what is running
Every agent, its owner, and the targets it is held to.
Scores that survive scrutiny
No number counts until an expert has checked the grader behind it.
Releases that cannot slip through
A change that misses the signed targets cannot merge.
Evidence, not screenshots
Signed reports a risk team can verify without access to AlignX.
Client spotlight
InfraGPT — natural language to production infrastructure
We built InfraGPT so a DevOps team could ship infrastructure faster. Engineers describe what they need in plain English; it writes the Terraform, applies organization policy, estimates the spend, and raises a pull request for a human to approve.
Engineer describes the infrastructure in plain English
Org policy applied: cost code, department, tagging
Spend estimated before anything is created
Pull request raised in Azure DevOps for approval
Merge deploys it to Azure
- Manual Terraform edits required
- 0
- Human approval on Azure DevOps
- PR-gated
How we partner with you
From idea to reliable production AI — we build, instrument and validate alongside your team.
- 1
Build
Custom AI agents and workflows tailored to your domain, like InfraGPT for DevOps.
Services - 2
Instrument
Connect any agent over its API, including ones you did not build.
Platform - 3
Validate
Halo test suites and evaluations so your AI ships reliably and stays reliable.
AlignX Halo
Prove your agents still work. Every release, every run.
Connect it, score it against your golden set, and get back a signed reliability report.