Prove the layout, accessibility, and job-pill audits can fail #49

Closed
opened 2026-10-24 00:10:38 -05:00 by ethan-hann · 0 comments
Owner

From the test-suite review (WARN-001, .tools/reviews/test-suite/code-review-test-suite.md).

If someone breaks one of three audit helpers, about 300 test rows keep passing while they test nothing. That happens whenever a helper edit makes it report no problems. The helpers change as screens grow, so this is a realistic failure.

The helpers are LayoutAudit.Problems (with IsReachable) in tests/Savepoint.App.Tests/Theme/LayoutAudit.cs, AutomationAudit.Problems in tests/Savepoint.App.Tests/Theme/AutomationAudit.cs, and the job-pill Covered check in tests/Savepoint.App.Tests/Features/EndToEnd/JobPillPlacementTests.cs. A search of all of tests/ found no test that feeds any of them a bad tree; every caller asserts an empty list. StyleGalleryBehaviorTests.The_audit_flags_an_icon_button_without_a_name already does this for one other helper and is the model to copy.

Add one small test per helper with a hand-built tree, and assert which control each problem names:

  • Layout audit: a Button outside the window, one inside a scroll area, a squeezed scroll area, and a ComboBox placed off-screen. The ComboBox matters: dropping it from IsReachable is a real mutation that a tree of buttons does not catch. Dropping ToggleSwitch is not, because it already derives from Button (unverified against the Avalonia assembly).
  • Automation audit: a nameless Button, one named after its type, one named only by an icon glyph, and one named correctly.
  • Job pill: one control under the pill and one clear of it. If Covered stays private, move it beside LayoutAudit.

Done when each new test fails under the mutation it targets (make the helper report nothing, drop ComboBox, make NeedsName return false, make TakesTheClick return false) and passes on the real helpers.

From the test-suite review (WARN-001, `.tools/reviews/test-suite/code-review-test-suite.md`). If someone breaks one of three audit helpers, about 300 test rows keep passing while they test nothing. That happens whenever a helper edit makes it report no problems. The helpers change as screens grow, so this is a realistic failure. The helpers are `LayoutAudit.Problems` (with `IsReachable`) in `tests/Savepoint.App.Tests/Theme/LayoutAudit.cs`, `AutomationAudit.Problems` in `tests/Savepoint.App.Tests/Theme/AutomationAudit.cs`, and the job-pill `Covered` check in `tests/Savepoint.App.Tests/Features/EndToEnd/JobPillPlacementTests.cs`. A search of all of `tests/` found no test that feeds any of them a bad tree; every caller asserts an empty list. `StyleGalleryBehaviorTests.The_audit_flags_an_icon_button_without_a_name` already does this for one other helper and is the model to copy. Add one small test per helper with a hand-built tree, and assert which control each problem names: - Layout audit: a `Button` outside the window, one inside a scroll area, a squeezed scroll area, and a `ComboBox` placed off-screen. The `ComboBox` matters: dropping it from `IsReachable` is a real mutation that a tree of buttons does not catch. Dropping `ToggleSwitch` is not, because it already derives from `Button` (unverified against the Avalonia assembly). - Automation audit: a nameless `Button`, one named after its type, one named only by an icon glyph, and one named correctly. - Job pill: one control under the pill and one clear of it. If `Covered` stays private, move it beside `LayoutAudit`. Done when each new test fails under the mutation it targets (make the helper report nothing, drop `ComboBox`, make `NeedsName` return false, make `TakesTheClick` return false) and passes on the real helpers.
ethan-hann added this to the 1.0 milestone 2026-10-24 00:10:38 -05:00
Sign in to join this conversation.
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
ethan-hann/SavePoint#49
No description provided.