Run the App and Core test modules side by side in the gate #51

Closed
opened 2026-10-24 00:16:42 -05:00 by ethan-hann · 0 comments
Owner

From the test-suite review (deferred item S3, .tools/reviews/test-suite/code-review-test-suite.md).

The gate runs one test module at a time (--max-parallel-test-modules 1 in build/gate.ps1 and build/gate.sh). That rule came from a machine-load problem that is now gone: the process leak was fixed on 2026-10-21, and recent gates end with about 64 GB free of 95 GB on a 16-core, 32-thread machine.

The cost is large. The App module runs one test at a time for 315 of its 336 s (its UI tests share one headless UI thread), while Core uses four threads. Running both modules together means about five busy threads and should take the Windows gate from about 9.5 minutes to about 6. The estimate comes from the xunit reports; it is not measured.

The risk is flaky timing-sensitive tests, not memory or CPU:

  • the delete-responsiveness test with its 250 ms wall-clock threshold (SUGG-009);
  • the two cancel-mid-write tests that skip when git finishes first (SUGG-003);
  • the render settle sleeps in ComposedApp.Render;
  • launch tests with real FakeGame processes and grace windows.

Both modules also share one temp root (.tools/tmp/<os>, tests/Shared/TestEnvironment.cs). Each test makes its own folder, so a collision is unlikely, but check that no fixture uses a fixed name.

Plan:

  1. Set --max-parallel-test-modules 2 in both gate scripts. Keep below-normal priority and 4 Core test threads.
  2. Run the Windows gate three times in a row. Record each duration and every failure or new skip.
  3. If a test flakes, fix that test rather than revert the setting.
  4. Update the load rules in AGENTS.md and docs/developer-guide.md, and record the decision in docs/decisions.md.

Linux CI runs on the Unraid runner and can take the same change.

Done when three consecutive Windows gates pass with both modules in parallel, with the measured times in this issue, and the docs say the new rule.

From the test-suite review (deferred item S3, `.tools/reviews/test-suite/code-review-test-suite.md`). The gate runs one test module at a time (`--max-parallel-test-modules 1` in `build/gate.ps1` and `build/gate.sh`). That rule came from a machine-load problem that is now gone: the process leak was fixed on 2026-10-21, and recent gates end with about 64 GB free of 95 GB on a 16-core, 32-thread machine. The cost is large. The App module runs one test at a time for 315 of its 336 s (its UI tests share one headless UI thread), while Core uses four threads. Running both modules together means about five busy threads and should take the Windows gate from about 9.5 minutes to about 6. The estimate comes from the xunit reports; it is not measured. The risk is flaky timing-sensitive tests, not memory or CPU: - the delete-responsiveness test with its 250 ms wall-clock threshold (SUGG-009); - the two cancel-mid-write tests that skip when git finishes first (SUGG-003); - the render settle sleeps in `ComposedApp.Render`; - launch tests with real FakeGame processes and grace windows. Both modules also share one temp root (`.tools/tmp/<os>`, `tests/Shared/TestEnvironment.cs`). Each test makes its own folder, so a collision is unlikely, but check that no fixture uses a fixed name. Plan: 1. Set `--max-parallel-test-modules 2` in both gate scripts. Keep below-normal priority and 4 Core test threads. 2. Run the Windows gate three times in a row. Record each duration and every failure or new skip. 3. If a test flakes, fix that test rather than revert the setting. 4. Update the load rules in `AGENTS.md` and `docs/developer-guide.md`, and record the decision in `docs/decisions.md`. Linux CI runs on the Unraid runner and can take the same change. Done when three consecutive Windows gates pass with both modules in parallel, with the measured times in this issue, and the docs say the new rule.
ethan-hann added this to the 1.0 milestone 2026-10-24 00:16:42 -05:00
ethan-hann removed this from the 1.0 milestone 2026-10-24 00:17:55 -05:00
ethan-hann added this to the 1.0 milestone 2026-10-24 00:27:08 -05:00
Sign in to join this conversation.
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
ethan-hann/SavePoint#51
No description provided.