Agent Reliability Infrastructure

Ship Agents That Hold

Maulfind Holdsmeasure Vigilprevent

01 · find failures

Maul

How does the agent behave under failure?

An adversarial OpenAI-compatible proxy. Point the agent's base_url at Maul to inject controlled failures and record how the workflow responds.

Artifact reliability_report.json

See how Maul holds ↓ Open on GitHub ↗

terminal · maul

$ maul test --config maul.yaml \
  --agent "python agent.py"

scenario        force_429
faults_injected 3
recovered       2
report written  artifacts/reliability_report.json

02 · measure quality

Holds

Did the agent actually solve the job?

A suite-first evaluation harness. Run representative tasks repeatedly, grade real outcomes, and stop regressions before they reach production.

Artifacts holds.yaml · baseline.json

See how Holds measures ↓ Open on GitHub ↗

terminal · holds

$ holds run --suite holds.yaml

tasks        12
pass_rate    96%
consistency  92%
threshold    passed

$ holds compare --baseline baseline.json

03 · prevent incidents

Vigil

Are unsafe or expensive behaviors stopped?

A runtime policy SDK for explicit operational controls. Define budgets, allowlists, circuit breakers, routing, and redacted audit behavior.

Artifacts vigil.yaml · audit.jsonl

See how Vigil stops it ↓ Open on GitHub ↗

vigil.yaml

budgets:
  max_llm_calls_per_workflow: 12
  max_cost_usd_per_workflow: 0.75

controls:
  - type: circuit_breaker
    condition: repeated_equivalent_request
    max_occurrences: 3

Use cases