GOG cover art from Galaxy's cache; release packages built in CI on a version tag #98

Merged
ethan-hann merged 10 commits from integration-098 into main 2026-10-24 21:00:39 -05:00
Owner
  • GOG cover art (#26): games installed by GOG get their cover from GOG Galaxy's local web cache (%ProgramData%\GOG.com\Galaxy\webcache\<account>\gog\<id>\*_glx_vertical_cover.webp), with no network and no access to Galaxy's database. The product id comes from the install folder's goggame-*.info files (most common rootGameId, base game wins ties). Precedence: own image, a Steam id the player stored, the GOG product id, a detected Steam id, the tint, because GOG and retail builds carry other apps' Steam ids. Reparse points in the cache are not followed, files are read with a size cap, and the install folder is read once per run. Windows only for now; macOS and Heroic are follow-ups (#95, #97). D30 updated.
  • Release packages in CI (#19): pushing v<Version> runs .forgejo/workflows/release.yml: the Linux job builds the AppImage in the container, the Windows job builds the Windows packages on the self-hosted Windows runner, and a last job writes SHA256SUMS.txt and links the bundled git's sources release. Everything goes to a draft pre-release; publishing stays a person's step. Files over 95,000,000 bytes are split into 90 MiB parts for Cloudflare, re-runs replace assets and drop stale parts, and only the upload steps see the token. The developer guide describes it.

Both branches had an Opus review and the findings are fixed here.

Closes #19
Closes #26

- **GOG cover art (#26):** games installed by GOG get their cover from GOG Galaxy's local web cache (`%ProgramData%\GOG.com\Galaxy\webcache\<account>\gog\<id>\*_glx_vertical_cover.webp`), with no network and no access to Galaxy's database. The product id comes from the install folder's `goggame-*.info` files (most common `rootGameId`, base game wins ties). Precedence: own image, a Steam id the player stored, the GOG product id, a detected Steam id, the tint, because GOG and retail builds carry other apps' Steam ids. Reparse points in the cache are not followed, files are read with a size cap, and the install folder is read once per run. Windows only for now; macOS and Heroic are follow-ups (#95, #97). D30 updated. - **Release packages in CI (#19):** pushing `v<Version>` runs `.forgejo/workflows/release.yml`: the Linux job builds the AppImage in the container, the Windows job builds the Windows packages on the self-hosted Windows runner, and a last job writes SHA256SUMS.txt and links the bundled git's sources release. Everything goes to a draft pre-release; publishing stays a person's step. Files over 95,000,000 bytes are split into 90 MiB parts for Cloudflare, re-runs replace assets and drop stale parts, and only the upload steps see the token. The developer guide describes it. Both branches had an Opus review and the findings are fixed here. Closes #19 Closes #26
Adds the offline pieces for GOG artwork: GogProductIdDetector reads the
goggame-*.info files in an install folder and names the base game (rootGameId,
else gameId), GogLocalCache finds Galaxy's cached poster (vertical_cover) and
background (bg_top_padding_7) for a product in any account's webcache folder,
and GogGalaxyCacheLocator finds that webcache folder on Windows. Nothing is
wired into the artwork lookup yet; ArtworkSource gains Gog.

Real data inspected read-only on one PC: three .info files in the Cyberpunk
2077 folder (base game plus two add-ons, all with rootGameId 1423049311, the
add-on file sorts first), older .info files without rootGameId for Deus Ex and
Soldier of Fortune II, and 930 product folders in Galaxy's webcache, 927 with
one 342x482 lossy WebP poster.

Observed reds (against a stub returning null):
- GogProductIdDetectorTests.The_base_game_is_named_even_when_an_add_ons_file_sorts_first:
  Assert.Equal() Failure: Expected "1423049311", Actual null
- GogProductIdDetectorTests.A_file_that_is_not_json_is_skipped_for_one_that_is: same
- GogProductIdDetectorTests.A_file_with_a_byte_order_mark_is_read: Expected
  "1228964594", Actual null; the first implementation also failed it because
  JsonDocument refuses a byte order mark, so the parser strips it
- GogLocalCacheTests.The_vertical_cover_is_the_tall_image_and_the_background_is_the_wide_one,
  The_newest_image_wins_across_accounts,
  A_file_that_is_not_a_whole_image_is_passed_over_for_an_older_good_one:
  Assert.Equal() Failure: Expected a path, Actual null

Mutation checks (broken on purpose, tests failed, restored): the digits-only
id check, the info file size cap, the digits-only account folder filter, the
digits-only product id guard (caught by the ../ id case), the image check, the
Windows-only guard and the file system error handling in the locator.
A v* tag now starts .forgejo/workflows/release.yml: the Linux job runs build/package.sh in the job container and the Windows job runs build/package.ps1 on the windows runner, each after checking that the tag equals v<Version> from Directory.Build.props. Each uploads its release folder to the Forgejo release for the tag through build/release-assets.ps1, which creates a draft pre-release (tag message as placeholder body) when none exists and never publishes or un-drafts one. Files over 100 MB go up as .partN pieces of 90 MiB because Cloudflare refuses bigger bodies. A last job downloads what the release holds, uploads SHA256SUMS.txt (every file, every part, and each joined file) and adds a "Source code of the bundled git <version>" attachment with external_url pointing at the git-sources release. Source archives are not packed into the app release.

The token is RELEASE_TOKEN when the repository has that secret, else the job's GITHUB_TOKEN.

Observed red: all 10 ReleaseWorkflowTests failed on assertions ".forgejo/workflows/release.yml does not exist." / "build/release-assets.ps1 does not exist.". After the files existed all 10 passed; mutation check (needs: [linux], --sources on package.sh, draft = $false) failed Nothing_publishes_or_undrafts_a_release, The_checksums_job_waits_for_both_package_jobs and Source_archives_are_not_packed_into_a_release_because_they_have_a_release_of_their_own.

Also run against a local mock of the Forgejo release API: draft creation, a 200 MB file split into 3 parts, rerun replacing earlier assets, SHA256SUMS.txt whose joined hash matched, the sources link.
A game whose install folder holds goggame-<id>.info files now gets artwork
from the images Galaxy already downloaded, whether it came from the library
import or was added by hand. Order: the user's own image, Steam's images (the
Steam client's cache, as before), then Galaxy's cache, then Steam's servers
when online lookup is on. A game whose art was found on this computer makes no
request. Galaxy's poster is the tall image and its darkened background is the
wide one; they are kept under gog-wide and gog-square in the game's artwork
folder in the format they came in (WebP). The lookup memory records the GOG
product the kept images belong to, so a folder re-pointed at another product
drops the old images. A miss in Galaxy's cache is not remembered across runs,
since it is only a folder read away. The editor shows "From GOG Galaxy".

Backends in tests point Galaxy's cache at a fake so none reads the real
machine. ArtworkOnlineTests.A_slow_server_ends_the_lookup_with_nothing_found_
after_ten_seconds_a_request counted requests after a read that starts another
lookup, which raced; it now counts before that read.

Observed reds (before the lookup used Galaxy's cache):
- ArtworkGogLookupTests.A_game_installed_by_gog_gets_both_images_from_galaxys_cache_copied_into_its_folder:
  Assert.Equal() Failure: Values differ (Source None, expected Gog)
- ArtworkGogLookupTests, the same assertion on: The_images_are_found_again_after_a_restart_without_galaxy,
  Nothing_in_galaxys_cache_is_no_artwork_and_the_cache_is_looked_at_again_after_a_restart,
  Look_again_finds_images_galaxy_downloaded_since,
  A_game_pointed_at_another_gog_product_drops_the_old_images_and_takes_the_new_ones,
  An_install_folder_that_cannot_be_read_keeps_the_images_found_before,
  Galaxys_images_are_used_when_a_steam_id_is_known_but_steams_cache_has_nothing_and_no_request_is_made,
  A_game_with_only_one_of_the_two_images_in_the_cache_has_one_path_and_a_null_for_the_other
- GameEditorArtworkTests.Gog_artwork_shows_in_the_preview_as_from_gog_galaxy_and_is_not_removable:
  Expected "From GOG Galaxy", Actual "No artwork found"

Mutation checks (broken on purpose, tests failed, restored): skipping the
online lookup only when Galaxy had the art, trying Galaxy after Steam found
art, keeping Galaxy's images beside Steam's, and the two places that drop
images kept for another product.
The release script no longer needs .NET 9 (ToHexString plus ToLowerInvariant replaces ToHexStringLower) and asks for PowerShell 7.4. A rerun deletes assets left under names it no longer sends (old parts when a file is now whole, extra parts when fewer), using a pure Get-StaleAssetNames; files split above 95,000,000 bytes because the multipart form adds to the body. RELEASE_TOKEN and GITHUB_TOKEN are set only on the three steps that run the release script, not on the job. The token header reaches curl through a private temp file written without a line ending, so no CR rides along on Windows. The PowerShell dotnet tool is installed at 7.6.8. The script has a Load action that only defines its functions, so tests can run them.

The tests now run pwsh on Split-Asset, Get-StreamHash, Get-AssetNames, Get-StaleAssetNames and Assert-PartsNumbered (skipping when pwsh is missing, listed with ~ in build/expected-skips), tie the upload folders and the three required names to what package.sh and package.ps1 write, check token scope per step, and reject a draft:false or PATCH/PUT in the script or workflow, case-insensitively and with --request.

Observed reds (assertion failures): Parts_that_a_run_will_not_send_again... expected "A.part1 A.part2 A.part3" actual ""; A_file_is_split_only_when_it_is_over_95_million_bytes... expected "x.bin" actual ""; Parts_numbered_with_a_gap_are_refused expected "The parts of f are not numbered 1 to 2." actual "no"; The_tokens_are_set_only_on_the_steps..., The_token_goes_to_curl_through_a_header_file..., The_hash_is_made_without_a_method_newer..., The_PowerShell_tool_is_installed_at_a_pinned_version failed Assert.DoesNotContain / Assert.Matches. Mutation checks on first-run passes: part read length +1 failed A_file_is_cut_into_numbered_parts; dropping ToLowerInvariant failed the hash test; renaming a required asset failed The_upload_directories_and_required_names.
Fixes from review of the GOG artwork. Order is now: the user's image, a Steam
id the user stored (Steam's cache, then its servers), the GOG product id
(Galaxy's cache), a Steam id only detected, then none. GOG and retail builds
carry other apps' ids, so a detected id yields to GOG's own and a stored one
does not.

- A stored Steam id no longer treats Galaxy's images as done: storing the id
  starts a Steam lookup, Steam's images replace Galaxy's, and the servers are
  asked even when Galaxy has art.
- A detected Steam id, and Steam images kept for it, no longer block Galaxy's
  art; once Galaxy's images exist the Steam ones are dropped.
- A readable install folder with no goggame-*.info drops the GOG images; a
  folder that cannot be listed keeps them.
- The product id tie-break never picks an add-on: most files naming the root
  win, then a root with a base-game file of its own (gameId equals rootGameId,
  or play tasks for older files that have no rootGameId, as in the real
  Deus Ex, Soldier of Fortune II and Cyberpunk files), then the smallest id.
- The cache root, <account>\gog and <account>\gog\<id> are never followed
  through links, and cache files are read through a stream that stops at the
  size limit.
- The install folder is read once per run for each game and path, and again
  on Look again.
- Identical GOG bytes are not rewritten and no change is announced for them.

Observed reds (assertion failures):
- GogProductIdDetectorTests.In_a_tie_between_older_files_the_one_with_play_tasks_is_the_base_game:
  Expected "1000000009", Actual "1000000001"
- ...In_a_tie_the_root_with_a_file_of_its_own_is_the_base_game_not_an_add_on:
  Expected "1000000009", Actual "1000000005"
- ...In_a_tie_with_nothing_to_tell_them_apart_the_smallest_id_wins:
  Expected "1000000007", Actual "1000000010"
- GogLocalCacheTests.A_webcache_folder / A_gog_folder / A_product_folder_that_is_a_link_is_not_followed:
  Expected null, Actual a path (three tests)
- ArtworkGogLookupTests.Storing_a_steam_id_replaces_galaxys_images_with_steams_when_steams_cache_has_them:
  Expected Steam, Actual Gog
- ...A_stored_steam_id_still_asks_steams_servers_when_galaxy_has_the_art:
  Assert.NotEmpty() Failure: Collection was empty
- ...Galaxys_images_replace_steam_images_kept_for_an_id_that_was_only_detected:
  Expected Gog, Actual Steam
- ...A_readable_install_folder_with_no_info_file_any_more_drops_the_images_found_before:
  Expected None, Actual Gog
- ...The_install_folder_is_read_once_per_run_until_look_again:
  Expected None, Actual Gog
- ...Looking_again_at_unchanged_images_rewrites_nothing_and_announces_nothing:
  Collections differ, Expected ["alpha-1"], Actual ["alpha-1", "alpha-1"]

Mutation checks (broken on purpose, tests failed, restored): the most-files
rule in the detector; the GogProductId comparison in the lookup-due check
(fails A_lookup_file_from_before_gog_support_does_not_hold_back_galaxys_images);
the per-run cache of the install folder; the stored-id exception to GOG-first.
The stream size cap has no direct test (a 25 MB fixture); it replaces the
previous length check.

Two existing tests that change info files mid-run now restart the backend,
as the per-run cache requires.
# Conflicts:
#	docs/developer-guide.md
Merge branch 'issue-26-gog-artwork' into integration-098
All checks were successful
Gate / linux (pull_request) Successful in 11m56s
Gate / windows (pull_request) Successful in 25m0s
Gate / linux (push) Successful in 12m19s
Gate / windows (push) Successful in 25m52s
0c4d336d4d
Sign in to join this conversation.
No reviewers
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
ethan-hann/SavePoint!98
No description provided.