The MapGenie icon step lived apart from the crawl and was forgotten by hand often enough to ship a game in fallback glyphs; the crawl now runs it itself once a game's maps are captured, and a machine without node is told exactly what to run later rather than failed -- the tiles and pins are already safely on disk. The Piggyback importer stops implying a generality it does not have: each Piggyback map projects through a transformation buried in its own scripts, only Cyberpunk's is verified, and an unverified game is refused at the door, where a wrong lens would have moved every pin and said nothing. The README catches up with all of it: the three sources, the composition, the provenance, and the workbench beside the reader.
- commit
- 3cb3b8481c50dab2f9cf7f9bf499af2915feec46
- parent
- cb57f1120adb0b58f5accf7dee7e0eb2fa7ea966
- author
- Dami <dami.etoile@gmail.com>
| M | README.md | 44 | +++++++++++++++++++++++++++--- |
| M | cmd/ | 6 | +++- |
| M | tools/ | 27 | ++++++++++++++++++ |
| M | tools/ | 21 | +++++++++++--- |
@@ -62,6 +62,40 @@ photographic maps retain smooth interpolation.
6262 [SCRAPER_PROMPT.md](SCRAPER_PROMPT.md) contains a ready-to-use prompt for6363 preserving MapGenie's text-display and zone fields in the upstream archive.6464 65+## Capture sources66+67+Three crawlers fill the archive, all through `tools/crawl`:68+69+```sh70+go run ./tools/crawl -game skyrim # MapGenie71+go run ./tools/crawl -ign cyberpunk-2077/night-city # IGN's native wikimaps72+go run ./tools/crawl -piggyback cyberpunk-2077/night-city # Piggyback (Cyberpunk only)73+```74+75+Each stores a content-addressed capture -- an unchanged re-crawl records no76+new version -- plus the tile pyramid and the category icons. MapGenie's77+icons resolve from an icon font, which the crawl now renders automatically78+once a game's maps are captured (`tools/icons/render-icons.mjs`, rerunnable79+by hand on a machine without node). The IGN mode refuses maps that merely80+embed MapGenie, and the Piggyback importer is deliberately game-specific:81+each Piggyback map projects through a transformation buried in its own82+scripts, only Cyberpunk's is verified, and an unverified game is refused83+rather than captured through the wrong lens.84+85+Sources are additive components of one game. `tools/generate` composes every86+capture of a game slug into a single bundle: pins of shared ground resolve87+one-to-one through an affine fitted on the places both sources name88+(projections differ between sources, so the shared names -- not coordinates89+-- are the basis of locality; a fit that will not close is refused), lesser90+rasters ride along as aligned variants, maps no other source draws join91+whole, and every map's payload carries its provenance: an origin account92+naming its source, and a ledger per donor of what matched, was added,93+adopted, held, or rejected. `tools/maturity` reports every build of every94+game along the axes this evolution is judged by, and `cmd/cartograph`95+(`go run ./cmd/cartograph`) serves a local workbench over the same96+collection: builds, ledgers, diffs, and the source registry with97+crawl/tiles/generate runnable from the page.98+6599 ## Building the game bundles66100 67101 With `../gamemap` beside this repository:@@ -131,10 +165,12 @@ manifest: game identity, version stamp, tile grid, map list), `maps/<slug>`
131165 payloads in three parts, `tiles/<pyramid>/z/x/y` rasters stored uncompressed132166 for byte-range serving, and `icons/`. The format is Atlas's own and carries133167 nothing specific to any capture source; `internal/bundle` owns reading,134-writing, and validation. Two bundles naming the same game slug are two135-versions of that game -- the newest by capture time wins, so updating a game136-is dropping in a newer file, and an older file dropped beside a newer one is137-simply shadowed. File names carry the version for people and for cheap168+writing, and validation. Provenance still travels: each map's payload names169+the source it came from and accounts for every other source folded into it.170+Two bundles naming the same game slug are two versions of that game -- the171+newest by capture time wins, ties broken by the build policy revision and172+then the stamp, so updating a game is dropping in a newer file, and an older173+file dropped beside a newer one is simply shadowed. File names carry the version for people and for cheap138174 existence checks; ordering always comes from the manifest inside.139175 140176 ## Renderer architecture@@ -156,10 +156,12 @@ var sources = []Source{
156156 crawlSource{157157 name: "Piggyback",158158 slug: "piggyback",159- description: "The official guide house -- Cyberpunk 2077's first-party map, with prose on nearly every pin. " +159+ description: "The official guide house's Cyberpunk 2077 map, with prose on nearly every pin. " +160+ "A deliberately game-specific importer: each Piggyback map projects through a transformation " +161+ "buried in its own scripts, and only Cyberpunk's is verified -- other games are refused. " +160162 "Its CDN answers only requests carrying a Referer, and the free tier ceilings pyramids at zoom 7.",161163 flag: "-piggyback",162- hint: "gameSlug/mapSlug, e.g. cyberpunk-2077/night-city",164+ hint: "cyberpunk-2077/night-city (the one supported game)",163165 pair: true,164166 components: fullSet,165167 },@@ -19,6 +19,7 @@ import (
1919 "flag"2020 "fmt"2121 "os"22+ "os/exec"2223 "path/filepath"2324 "sort"2425 "strconv"@@ -134,9 +135,35 @@ func run(o options) error {
134135 return fmt.Errorf("%s / %s: %w", game.Title, target.Title, err)135136 }136137 }138+ if !o.dryRun {139+ renderIcons(ctx, o, game)140+ }137141 return nil138142 }139143 144+// renderIcons resolves the game's icon font into the archive once its maps145+// are captured. MapGenie ships icons as a font behind a stylesheet, which the146+// Node script under tools/icons untangles; it is quick, idempotent, and was147+// forgotten by hand often enough that the crawl now runs it itself. A machine148+// without node is told what it is missing rather than failed: the tiles and149+// pins are already safely on disk, and icons can be rendered later.150+func renderIcons(ctx context.Context, o options, game *apiGameFull) {151+ script := filepath.Join("tools", "icons", "render-icons.mjs")152+ if _, err := os.Stat(script); err != nil {153+ fmt.Printf(" icons not rendered (no %s here); run it from the atlas checkout:\n"+154+ " node %s --archive %s --game %s\n", script, script, o.archive, game.Slug)155+ return156+ }157+ command := exec.CommandContext(ctx, "node", script,158+ "--archive", o.archive, "--game", game.Slug)159+ command.Stdout = os.Stdout160+ command.Stderr = os.Stderr161+ if err := command.Run(); err != nil {162+ fmt.Printf(" icons not rendered: %v; run when ready:\n"+163+ " node %s --archive %s --game %s\n", err, script, o.archive, game.Slug)164+ }165+}166+140167 // crawlMap captures one map completely: its metadata snapshot, then its tiles.141168 // Both are idempotent, so an interrupted run resumes by simply being repeated.142169 func crawlMap(@@ -40,11 +40,17 @@ type pbPage struct {
4040 typeLabels map[string]string4141 }4242 43-// pbTransform is the transformation Piggyback's Cyberpunk map projects44-// through: game coordinates spanning ±8192 onto the unit tile at zoom zero.45-// It lives in the map's scripts rather than its data, so the crawler asserts46-// it instead of reading it; the capture records what was assumed, and a game47-// drawn to a different rule would need this looked up in its own scripts.43+// This is the Cyberpunk importer, and says so. Piggyback publishes guides44+// for many games, but each map's coordinate transformation lives in that45+// map's own scripts rather than its data: the crawler asserts Cyberpunk's --46+// game coordinates spanning ±8192 onto the unit tile at zoom zero -- and the47+// capture records what was assumed. Another Piggyback game would need its48+// transformation read out of its own scripts and verified; until someone49+// does that work, an unknown game is refused at the door rather than50+// captured through the wrong lens, where every pin would land somewhere51+// plausible and nothing would say it was wrong.52+const pbSupportedGame = "cyberpunk-2077"53+4854 var pbTransform = pbmap.Transform{A: 0.015625, B: 128, C: -0.015625, D: 128}4955 5056 func runPB(ctx context.Context, fetcher *fetcher, o options) error {@@ -52,6 +58,11 @@ func runPB(ctx context.Context, fetcher *fetcher, o options) error {
5258 if !ok || gameSlug == "" || mapSlug == "" {5359 return errors.New("-piggyback wants gameSlug/mapSlug, as the address bar spells it")5460 }61+ if gameSlug != pbSupportedGame {62+ return fmt.Errorf("the Piggyback importer knows only %s: %s draws through a "+63+ "transformation this crawler has not verified, and a wrong one moves every pin",64+ pbSupportedGame, gameSlug)65+ }5566 pageURL := fmt.Sprintf("https://maps.piggyback.com/%s/maps/%s", gameSlug, mapSlug)5667 5768 page, err := fetchPBPage(ctx, fetcher, pageURL)