Skip to content

Instantly share code, notes, and snippets.

@jaredpar

jaredpar/roslyn-ci-low-test-runs.md

Created June 5, 2026 20:16
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save jaredpar/3480f6187bdc07b648458fe2f31e4452 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save jaredpar/3480f6187bdc07b648458fe2f31e4452 to your computer and use it in GitHub Desktop.
Download ZIP
Roslyn-CI builds with suspicious low test count runs (last 48h)
Raw
roslyn-ci-low-test-runs.md

Roslyn-CI Builds with Suspicious Low Test Count Runs (Last 48h)

Scanned 77 roslyn-CI builds from 2026-06-03 to 2026-06-05. Found 17 test runs with < 10 total tests (excluding macOS empty runs).

Failed Retry Runs (0.0s duration, all failed)

These appear to be AzDO retry summary runs that aggregate failures without real test execution.

Build Run ID Run Name Total Failed Duration
1450839 40169782 Test Windows Desktop Debug 64 1 1 0.0s
1450136 40164614 Test Windows CoreClr Release 9 9 0.0s
1450136 40164708 Test Windows Desktop Debug 64 8 8 0.0s
1450067 40163672 Test Windows Desktop Debug 64 2 2 0.0s
1449582 40153220 Test Windows CoreClr Debug 3 3 0.0s
1449108 40133218 Test Windows 1 1 0.0s
1449106 40133118 Test Windows 1 1 0.0s
1448739 40127162 Test Windows 1 1 0.0s
1448694 40127800 Test Windows 1 1 0.0s
1447704 40116062 Test Windows 3 3 0.0s
1447704 40116096 Test Windows 2 2 0.0s
1448176 40114818 Test Windows 1 1 0.0s

Single-Test Passed Runs (~30-50s duration)

These legs normally run 126K-150K tests but only reported 1 test passed. The 30-50s duration suggests real execution occurred but was cut extremely short.

Build Run ID Run Name Total Passed Duration
1450153 40154690 Test_Windows_CoreClr 1 1 41.2s
1448212 40114342 Test_Windows_Desktop_Release_32 1 1 47.1s
1448212 40114400 Test_Windows_Desktop_Debug_64 1 1 53.8s
1447704 40105532 Test_Windows_CoreClr 1 1 32.9s
1447787 40107056 Test_Windows_CoreClr 1 1 33.3s

Notes

  • The "failed retry" runs have short names (spaces, no underscores) like "Test Windows Desktop Debug 64" vs the normal full names with underscores like "Test_Windows_Desktop_Debug_64_windows.amd64.vs2026.pre.scout.open"
  • The "single-test passed" runs use the full underscore naming convention, suggesting they are real Helix test legs that somehow only executed 1 test
  • All affected builds ultimately reported as succeeded because retries or other legs passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment