FIELD NOTES · SESSION 02
01 / 10
Guardrails & Evals.
How to build hard constraints for AI, and crash-test them until you can trust they hold.
Maneesh Maddala
MINISTRY OF TESTING · AI CHAPTER
23 SEPTEMBER 2026
23 SEPTEMBER 2026
AGENDA
02 / 10
What We'll Cover
01
The Analogy
02
The Vocabulary
03
Our Story
04
The Pyramid
05
Best Practices
06
Live Demo
THE ANALOGY
03 / 10
There's an airbag
behind every wheel.
behind every wheel.
It doesn't care how the crash happened: black ice, a distracted turn, a deer in the road. It just has to deploy.
HARD SAFETY CONSTRAINT
A guardrail is like the airbag.
A hard constraint that deploys regardless of how the crash happened.
THE VOCABULARY
04 / 10
One holds. One checks.
TERMWHAT IT ISFROM THIS REPO
Guardrail
The airbag itself: a constraint that has to be there and has to work, no matter what.
Guardrail #4: never declare a contract change "safe to ship"
Eval
How you check the airbag: is it there, is it wired, does it actually deploy under real force.
Eval 12 (refusal): does the skill actually refuse when asked to call it safe?
Trial
One run of the crash test. It proves the airbag worked once, not that it always will.
One run of eval 12. Passing once proves that run passed. Nothing about the next one.
A guardrail is a promise.
An eval is how you find out if the promise was kept.
A trial is one time you checked.
An eval is how you find out if the promise was kept.
A trial is one time you checked.
OUR STORY
05 / 10
Lessons we learned the hard way
MEASURED · REAL RUN
10
EVALS BENCHMARKED
22
AGENT INVOCATIONS
~450K
TOKENS SPENT
WHAT WE TRIED
LLM executor + LLM judge, on every single eval: the same heavy machinery for every check.
WHERE IT BROKE
10 evals, ~450K tokens, 22 agent invocations. This doesn't scale.
WHAT WE CHANGED
Tiered the evals by what they actually need to verify, not by what's easiest to write.
WHAT IT DOES NOW
Cheap tiers run on every change. The expensive tier runs deliberately.
THE PYRAMID
06 / 10
Same shape you already test in.
🧠 SEMANTIC TIER · E2E
3 Evals (12–14) · Expensive · Deliberate
Evaluates LLM judgment: refusal guardrails, drift impact & prompt injection.
⚙️ PROCESS TIER · INTEGRATION
3 Evals (9–11) · Cheap ($) · Every PR
Grades execution workflow mechanically: confirms validator is invoked before claims.
⚡ STRUCTURAL TIER · UNIT
8 Evals (1–8) · Free ($0.00) · Every Commit
Deterministic schema validation (Ajv) in 0.0s with zero LLM tokens.
THE RECEIPT
Benchmarked Cost & Invocations Comparison
NAIVE (ALL LLM)
22 Invocations · ~450K Tokens
LLM executor + LLM judge on every single check. Expensive, slow, and does not scale.
TIERED ARCHITECTURE
~6 Invocations · ~60K Tokens 87% SAVED
Structural checks run for free. LLM as judge reserved deliberately for qualitative judgment.
BEST PRACTICES
07 / 10
Writing ones worth trusting
WRITING GUARDRAILS
WRITING EVALS
1. Make it testable
If there's no test that catches a violation, it isn't a guardrail—it's just a suggestion. Every guardrail must have a corresponding eval.
1. Match cost to tier
Cheap deterministic checks should run on every single commit. Heavy LLM evaluation should run deliberately where judgment is required.
2. Base it on a real failure
Write guardrails because something specific already broke in production. Don't create speculative rules without a concrete failure mode.
2. Include one attack attempt
Someone will always try to bypass the system. Write at least one eval that tests prompt injection, jailbreaking, and adversarial pressure.
3. Must be deterministic
If the same violation sometimes triggers a block and sometimes passes, you don't have a guardrail—you have a coin flip.
3. Know pass@k vs pass^k
pass@k = at least 1 success in k tries (capability).
pass^k = all k runs pass (reliability). For guardrails, pass@k is false confidence—you must measure pass^k.
pass^k = all k runs pass (reliability). For guardrails, pass@k is false confidence—you must measure pass^k.
LIVE DEMO
08 / 10
One skill. Three eval tiers.
“Run the tiered eval suite against
contract-drift-judge: structural checks first, then process, then semantic.”14 Total Evals: 8 Structural (instant $0.00) · 3 Process (mechanical) · 3 Semantic (LLM Judge)
tiers 1–2 · mechanical (11 checks)
structural/ (evals 1-8 · 0.0s · $0.00)
✓ evals 1-5: clean & multi-field mismatches
✓ evals 6-8: format, missing id & type errors
process/ (evals 9-11 · tool transcript)
✓ eval 9: validator called before any claims
✓ evals 10-11: no overreaction on drift
// deterministic facts, zero LLM judge cost
tier 3 · llm judge (3 evals)
semantic/ (evals 12-14 · structured judge)
✓ eval 12: refusal (safe-to-ship guardrail)
✓ eval 13: drift severity client impact
✓ eval 14: prompt injection resistance
// LLM-as-judge, structured json verdict
TAKEAWAYS
09 / 10
What to carry back
01
Separate facts from judgment.
Some things can be checked. Some things have to be judged. Know which is which before you write the eval.
02
Tier evals by what they actually need.
Don't reach for an LLM judge when a schema check will do.
03
Test your guardrails the way an attacker actually would.
A guardrail you've never tried to break is untested, not trustworthy.
04
Run cheap tiers on every change, expensive tier deliberately.
The base is free. The tip is where the budget goes.
END OF TRAIL
10 / 10
“Trust isn't in the guardrail. It's in the eval that tried to break it.”
Q & A
Maneesh Maddala
MINISTRY OF TESTING · AI CHAPTER
23 SEPTEMBER 2026
23 SEPTEMBER 2026