A snapshot's details list only a few of the changed files and say nothing while the game folder copies #108

Closed
opened 2026-10-24 23:10:16 -05:00 by ethan-hann · 1 comment
Owner

First snapshot of A Plague Tale: Innocence on 0.9.7 (Windows, 44.6 GB, 222 files in the game folder, done in about 6 minutes). The Activity details showed the saves folder's file and its count, then nothing about the game folder for minutes while the bar moved; when the job finished they listed only four game files (APlagueTaleInnocence_x64.exe, RTC/LV12_INVASION.DPC, RTC/LV12_TRANSITION.DPC, zlibd.dll) before "222 files were new, changed or removed in the game folder." The count comes from git's full list, so the names were read but most never became detail lines: progress reports are throttled to about twenty a second and the file list arrives from git in a burst at the end (D50), so most names are dropped with the skipped reports. Since #91 a path is also added only once.

Make the details list every changed file once (newest 200 kept, as the recent lines already are) independent of progress throttling, and while a folder copies with no names yet, say so in plain words (for example the folder and how much of it is done) instead of going quiet. Check switches and Free up space for the same gap.

First snapshot of A Plague Tale: Innocence on 0.9.7 (Windows, 44.6 GB, 222 files in the game folder, done in about 6 minutes). The Activity details showed the saves folder's file and its count, then nothing about the game folder for minutes while the bar moved; when the job finished they listed only four game files (APlagueTaleInnocence_x64.exe, RTC/LV12_INVASION.DPC, RTC/LV12_TRANSITION.DPC, zlibd.dll) before "222 files were new, changed or removed in the game folder." The count comes from git's full list, so the names were read but most never became detail lines: progress reports are throttled to about twenty a second and the file list arrives from git in a burst at the end (D50), so most names are dropped with the skipped reports. Since #91 a path is also added only once. Make the details list every changed file once (newest 200 kept, as the recent lines already are) independent of progress throttling, and while a folder copies with no names yet, say so in plain words (for example the folder and how much of it is done) instead of going quiet. Check switches and Free up space for the same gap.
ethan-hann added this to the 1.0 milestone 2026-10-24 23:10:16 -05:00
Author
Owner

Approach (discussed 2026-10-24). Streaming git's output to a file instead of a pipe would not help: git buffers its stdout whenever it is not a terminal, so names would reach the file in the same late chunks. Options, in order:

  1. Try GIT_FLUSH=1 in the capture's git environment and check whether git add -v then writes each name as it goes (cheap; one setting).
  2. If not, hand git the changed files in batches (for example --pathspec-from-file per few hundred files or about 1 GB), so each batch's names arrive when it ends; a few dozen extra git starts on a big game is small next to a multi-minute copy.
  3. Either way, while one large file is being stored git prints nothing, so the details show the file being copied and how far it is, from the byte sampler SavePoint already runs.
    Separately, every name git does print must become a detail line regardless of progress throttling (the burst at the end is what drops most of them today).
Approach (discussed 2026-10-24). Streaming git's output to a file instead of a pipe would not help: git buffers its stdout whenever it is not a terminal, so names would reach the file in the same late chunks. Options, in order: 1. Try `GIT_FLUSH=1` in the capture's git environment and check whether `git add -v` then writes each name as it goes (cheap; one setting). 2. If not, hand git the changed files in batches (for example `--pathspec-from-file` per few hundred files or about 1 GB), so each batch's names arrive when it ends; a few dozen extra git starts on a big game is small next to a multi-minute copy. 3. Either way, while one large file is being stored git prints nothing, so the details show the file being copied and how far it is, from the byte sampler SavePoint already runs. Separately, every name git does print must become a detail line regardless of progress throttling (the burst at the end is what drops most of them today).
Sign in to join this conversation.
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
ethan-hann/SavePoint#108
No description provided.