Steam app id detection that matches what Steam writes #43

Merged
ethan-hann merged 2 commits from issue-39-steam-app-id-detection into main 2026-10-23 22:52:53 -05:00
Owner

Fixes #39. Stacked on #42 (its first three commits are #42's); merge #42 first.

Checked against the real Steam install on one PC first (174 installed games, libraries on three drives):

  • 38 games keep a steam_appid.txt: 25 in the install folder, 13 only in a subfolder (Binaries/Win64, Binaries/Win32, Binaries, Build/Final, x64, bin, Launcher, Classic, Online/Binaries/Win32).
  • Of the 25 install-folder files, 13 end in LF + NUL, 1 in space + CRLF, 11 are bare digits. Trim() kept the NUL, so 13 were rejected.
  • Mafia III (398870 vs manifest 360430), House Flipper (1471280 vs 613100), and Fallout 4 GOTY (377160 vs 1946160) carry another app's id in the file.

Changes:

  • Detection order is now launch link, library manifest, then steam_appid.txt, so the manifest wins inside a library.
  • The file parser accepts digits surrounded by whitespace and NUL bytes, nothing else.
  • With no file in the install folder, subfolders are searched up to 3 levels (shallowest first, at most 200 folders, never through a link).
  • SteamCdn accepts image/png and image/webp as well as image/jpeg; the bytes must still decode.
  • D30 records the order, the parser, and the evidence.

Real games before and after: the detected id equals the manifest id for 170 of 174 installed games before and 172 after; for the files alone (as if copied outside a library) 10 of 38 before and 34 of 38 after.

Fixes #39. Stacked on #42 (its first three commits are #42's); merge #42 first. Checked against the real Steam install on one PC first (174 installed games, libraries on three drives): - 38 games keep a steam_appid.txt: 25 in the install folder, 13 only in a subfolder (Binaries/Win64, Binaries/Win32, Binaries, Build/Final, x64, bin, Launcher, Classic, Online/Binaries/Win32). - Of the 25 install-folder files, 13 end in LF + NUL, 1 in space + CRLF, 11 are bare digits. Trim() kept the NUL, so 13 were rejected. - Mafia III (398870 vs manifest 360430), House Flipper (1471280 vs 613100), and Fallout 4 GOTY (377160 vs 1946160) carry another app's id in the file. Changes: - Detection order is now launch link, library manifest, then steam_appid.txt, so the manifest wins inside a library. - The file parser accepts digits surrounded by whitespace and NUL bytes, nothing else. - With no file in the install folder, subfolders are searched up to 3 levels (shallowest first, at most 200 folders, never through a link). - SteamCdn accepts image/png and image/webp as well as image/jpeg; the bytes must still decode. - D30 records the order, the parser, and the evidence. Real games before and after: the detected id equals the manifest id for 170 of 174 installed games before and 172 after; for the files alone (as if copied outside a library) 10 of 38 before and 34 of 38 after.
Newer games are cached as library_capsule.jpg / library_capsule_2x.jpg (tall) and library_header.jpg (wide) in hashed subfolders, so the local lookup found nothing for them. The tall kind now also accepts the capsule names, the wide kind library_header.jpg; the newest complete image among all accepted names still wins. Hero, blur, logo and icon files are never taken.

Observed reds before the change: the hashed-subfolder, double-size capsule and newest-across-names tests failed on Assert.Equal (expected a path, actual null). The unfinished-new-name and never-taken tests passed first run; a mutation (probe check removed, library_hero.jpg added to the wide names) made both fail.
Newer games are not served at the plain steamstatic addresses (all 404), only under a hashed path that the store API lists. When both plain addresses say not found for a kind, SteamCdn now asks api.steampowered.com once per lookup (a StoreAssetsMemo is shared by the wide and tall fetch) and fetches the named asset from shared.akamai.steamstatic.com: tall takes library_capsule_2x else library_capsule, wide takes header. The answer is capped at 256 KB and parsed with System.Text.Json; the URL format must start with steam/apps/<same id>/ and hold the placeholder once, and asset names must match an optional 40-hex folder plus a plain .jpg name, else the kind counts as not found and nothing is followed. A store answer that is not a plain 200, is bad JSON, is about another app or breaks the format rule is Failed, so no miss is remembered. Older games still cost no extra request.

Observed red before the change: 58 of the 66 new SteamCdnStoreTests failed on assertions (outcome, URL list, request counts, Failed/NotFound outcomes) with the pre-change fetch. Eight passed first run; mutations (always take the store path; treat a missing assets object as Failed; drop the memo in ArtworkApi) made the plain-failed-asks-nothing, no-assets, and shared-request tests fail. Existing tests that relied on a bare 404 from every address now answer the store with an empty item list, since a 404 from the store is a Failed.
docs: D30 names Steam's newer cache names, the hashed image paths, and the store API host
Some checks failed
Gate / linux (pull_request) Successful in 13m26s
Gate / linux (push) Has been cancelled
f70a9e665c
Closes #38
Real data first (read only; Steam on C:, D:, X:; N: offline): 174 installed games, each with an appmanifest naming its install folder. 38 keep a steam_appid.txt somewhere (136 keep none). 25 have it in the install folder: 13 end in LF + NUL (0A 00), 1 ends in space + CRLF (20 0D 0A), 11 are bare digits. 13 more keep it only in a subfolder, found at: Binaries/Win32, Binaries/Win64, Binaries, Build/Final, x64, bin, Launcher, Classic, Online/Binaries/Win32, and SteamVR's tools/steamvr_environments/game/bin/win64 (5 deep, a tool, not a game). Nine of the ten usable subfolders are 1 to 2 deep, the deepest game case is 3. Three root files name an app other than the manifest: Mafia III 398870 (manifest 360430), House Flipper 1471280 (613100), Fallout 4 GOTY 377160 (1946160, shares a folder with 377160). House Flipper's subfolder copies say 480.

Changes: inside a library the manifest's id now wins and steam_appid.txt is the fallback; the file parser trims whitespace and NUL bytes (and nothing else) around the digits; with no file in the install folder, up to 3 levels of subfolders are searched breadth first in name order, at most 200 folders, never through a link; SteamCdn accepts image/png and image/webp as well as image/jpeg (ImageProbe still has to decode the bytes) and says so in its Accept header.

Observed reds (all assertion failures): file with "377160\n\0", "489830\0", "\0\0 489830 \n\0\0" returned null instead of the id; eight subfolder shapes (Binaries/Win32, Binaries/Win64, Binaries, Build/Final, x64, bin, Launcher, Online/Binaries/Win32) returned null; the Mafia III shape returned 398870 instead of 360430; a file in the install folder beat the manifest; the three-level stop, shallower-wins and root-beats-subfolder tests failed on null or the wrong id; PNG and WebP answers came back Failed instead of Found; the Accept header lacked image/png and image/webp.

Mutation checks for tests that passed first run: removing ReparsePoint from the skipped attributes fails the symbolic-link test; raising the folder budget to 2000 fails the many-subfolders test; replacing the trim with Replace("\0","").Trim() fails the "48<NUL>9830" case; making the image probe always succeed fails the PNG/WebP-over-HTML test and the two existing not-an-image tests.

Real installed games, detector id equal to the manifest id: in place 170 before, 172 after of 174 (the other two share an install folder with a lower-numbered app, so only the lowest id can match); copies holding only the steam_appid.txt files outside any library 10 before, 34 after of 38 (the four left: SteamVR is 5 folders deep, and Mafia III, House Flipper and Fallout 4 GOTY carry another app's id).
docs: D30's app id detection order and image types after checking real Steam installs
All checks were successful
Gate / linux (pull_request) Successful in 13m19s
Gate / linux (push) Successful in 14m22s
ab69b354c3
Closes #39
ethan-hann deleted branch issue-39-steam-app-id-detection 2026-10-23 22:52:54 -05:00
Sign in to join this conversation.
No reviewers
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
ethan-hann/SavePoint!43
No description provided.