Fail the gate when a test unexpectedly skips #50
Labels
No labels
Compat/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
ethan-hann/SavePoint#50
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
From the test-suite review (WARN-002,
.tools/reviews/test-suite/code-review-test-suite.md).If a skip probe breaks, a group of real tests turns into skips and the gate still reports a pass. That happens when a probe such as
TestLinks.TryCreateFolderLink(tests/Savepoint.Core.Tests/Git/Engine/TestLinks.cs) starts returning false. The same holds forHonorsFileModes. ForIgnoresCase, only an always-false break hides tests, becauseFileFolderCollisionTestscatches an always-true one on case-sensitive systems.Write the test report the gate already allows (
--report-xunit), compare its skipped test names with a checked-in list for each OS, and fail on any skip the list does not name. Report an expected skip that now runs as a note. Key the list by test name, not by count, so adding a platform-specific test is a one-line edit next to it.Today's counts: 26 skips on the Windows gate and 75 in Linux CI (CI runs since 2026-10-23, as an ordinary user). The local WSL gate (
build/gate-linux.sh) skips more because its 9p mount cannot hold sockets or permissions, so it needs its own list or a switch that says it is not CI. Bothbuild/gate.ps1andbuild/gate.shneed the step.Done when making
TryCreateFolderLinkreturn false everywhere fails the gate and names the tests that started skipping.