Automation coverage, measured from source
Replaced spreadsheet tracking with a zero-dependency pipeline that reads 20+ team test repositories and publishes a weekly executive report.
- Result
- 81.7%
regression automation coverage across 21 teams, now measured weekly - Role
- Designed and built the pipeline and reports
- Domain
- National telecom carrier
- Period
- Oct 2026 – present
Problem
Leadership reported test-automation progress from hand-maintained spreadsheets. Every team organized its repository differently, used different tags, and counted "automated" in its own way. The numbers in the steering deck couldn't be traced back to code.
Approach
Measure from the source of truth: the test code itself. Mirror every team repository, parse the specs, and map each team's conventions through one config instead of asking 21 teams to change.
What I built
- A shared scanner and spec parser that collapses copied tests and understands per-team folder and tag conventions, later extended with a Java-aware parser for mobile TestNG suites.
- Four report generators: regression coverage, build-verification coverage, functional-flow coverage, and an open-PR dashboard (age, AI-review status, mergeability).
- Week-over-week change markers, a release filter, and an email-safe variant so the report reads correctly in Outlook.
- Weekly publishing to an internal CDN plus an executive email.
Outcome
- 21 teams measured from code, not self-reporting: 81.7% regression automation (2,352 of 2,879 cases).
- 272 automatable manual cases identified, setting a 91.1% target.
- Per-team spread (66.9% to 98.5%) made it obvious where to invest.