Risk-based test design with agents
Compared a standard test set against agent-assisted, risk-based test design on the same feature, then turned the results into a quality-engineering offering.
- Result
- 9 of 9
injected bugs caught, against 1 of 9 for the standard set - Role
- Ran the proof of concept and wrote the offering
- Domain
- Internal R&D and client pitch
- Period
- Sep 2025 – Oct 2026
Problem
Teams generate tests with AI, but more tests isn't the same as better coverage. I wanted evidence for whether structured, risk-based agent workflows actually find more defects.
Approach
A controlled comparison on one login feature: a conventional test set against a risk-register-driven design produced with agent workflows, both run against a build with nine injected real-world bugs.
What I built
- A risk register, a traceability matrix and gate decisions, all generated through the spec-driven agent workflow.
- 39 scenarios across UI, API and contract levels, running on three browsers.
Outcome
- 39 scenarios caught 9 of 9 injected bugs; the conventional four tests caught 1.
- 16 scenarios covered behavior the acceptance criteria never mentioned.
- Found an account-enumeration flaw in the spec before any code existed.
- 135 of 135 green on a 5x burn-in, with no flaky tests.