SaveToFile walked _keysList twice - once to count the map's chunks, then again
through a ChunkEnumerator closure to emit them - because Write needed the count up
front to size its index array. It now collects the map's chunks into a
PooledRefList in one pass and hands Write a span, so the count is just span.Length.
That deletes the ChunkEnumerator delegate, the closure over the list enumerator,
and both InvalidOperationException throws, which existed only to police the
delegate's "yield exactly chunkCount chunks" contract - a contract a span makes
unrepresentable.
Write now patches the header's IndexOffset by seeking back to it rather than
reaching into the writer's live buffer with BinaryPrimitives. This also removes the
IndexOffsetFieldPosition constant, which hard-coded the field's byte offset and had
to be kept in sync with the header layout by hand.
Tests: StepCacheFileV6/V7/V8Tests were named for the format version that introduced
each transform, and the format is now v9 - so all three names described formats the
loader rejects outright. Each file also carried its own copy of the chunk builders,
the round-trip plumbing, and the assertion helper, plus a near-identical
"an older version is rejected" test whose comment cited a MinSupportedVersion that
had since moved.
They are now one StepCacheFileFormatTests named for the behavior it covers -
predictive-Z elision, compression, the compact index - with shared builders, a
single AssertIdentical that checks the swim and strata trailers too (the old
AssertBaseEqual silently skipped both), and the three rejection tests folded into
one theory that also covers a future version. Added a zero-chunk case, which the
old delegate-based writer had no coverage for.
Verified non-vacuous: injecting an off-by-one into the IndexOffset patch fails 15
of the 123 tests.
Also reattaches a doc comment in StepCacheFileTests that had drifted off
LazyReaderHit_BypassesMissTrackerOnFirstTouch and stacked onto the test above it,
and drops its reference to the long-gone "v3 file format".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The pathing and step-cache comments had accumulated as development notes rather
than documentation: internal phase jargon ("Tier 4", "the Phase-2 synthesizer"),
change narration aimed at a reviewer ("which the old ComputeStandingZ anchor
missed", "legacy behavior"), benchmark anecdotes ("benchmarked as near-optimal",
"a ~20 ns lookup"), and multi-paragraph blocks restating what the code says.
Rewritten to keep the rationale a reader cannot derive from the code - why the
source-Z guard cannot be loosened, why multis fall through, why the promotion gate
counts Finds instead of calls, why the fingerprint hashes files rather than the
live tile tables - and to drop the history that got us there.
Also corrects comments that had gone stale:
- CacheEvictionTimer and CacheStats documented a class named
StaticWalkabilityCache, which no longer exists; it is StepCache.
- StepCacheFile's header said "File layout v8" while FormatVersion is 9, and the
body called the current record layout "the v6 layout" throughout. The layout
descriptions are now unversioned, since they describe whatever FormatVersion
currently is.
- StepProbe.ComputeStandingZ claimed StepCache uses it to bake SourceZ. It hasn't
since the baker moved to the clearance-aware ComputeStandableSurfaceZs; only a
parity test calls it now.
Two small code changes came along with the comment work, both behavior-preserving:
_lazyReaders now uses a collection expression like its neighbours, and
TryLoadFromLazyReader collapses to an expression body once its inline comment moved
to the doc comment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Summary
Fixes the pathfinding step-cache (`.swb`) prebake so it bakes **once** and skips when a valid cache already exists, instead of re-baking on every boot. The root cause was the staleness fingerprint hashing mutable in-memory tile data rather than the on-disk files. This PR makes the fingerprint a pure function of the client data files and separates dynamic multis (houses/boats) from the static cache.
> This branch builds on the `ConfigurePrompts` first-boot-prompt unification (commit `5df8d0bd`, also included here) — that commit accounts for the `ServerConfiguration.cs` and `dev-docs/server-lifecycle.md` changes in the diff.
## The bug
With `pathfinding.prebakeMaps` set, the cache re-baked on **every** boot. The `.swb` staleness fingerprint hashed the live `TileData.LandTable`/`ItemTable` flags, which the server patches at runtime (`ItemFixes`, `LOSBlocker`, `PotionKeg`, `CTF`) at nondeterministic lifecycle points (Initialize-phase methods share a priority; static ctors fire lazily). So a fingerprint stamped at runtime (`[PathBake`) never matched the one recomputed during startup `Initialize()`, and the cache rebaked every time.
## Changes
**1. Fingerprint the files, not the in-memory tables** (`fix`)
Hash `tiledata.mul` (cached, computed once) plus the per-map `.mul`/`.uop` files — never the runtime-mutated `TileData` tables. The fingerprint is now lifecycle-stable. Existing `.swb` files rebake once after deploy, then stay stable.
**2. Compute the fingerprint once per boot** (`refactor`)
`Configure()`'s `AutoLoadAtStartup()` already opens and fingerprint-validates a reader for every up-to-date `.swb`. `Initialize()` now skips baking any map that already has an open reader (`StepCache.HasLazyReader`) instead of recomputing the fingerprint a second time.
**3. Bake static-only; route multis to the live path** (`refactor`)
Multis (houses/boats) are dynamic, so they're no longer baked into the static chunk cache — they were tagged with `BuiltMultisVersion`, a non-persisted session counter, which made persisting them unsafe (false matches / wasted re-bakes).
- Chunks bake land + `statics.mul` only.
- At query time, any cell whose sector (or its 1-cell halo) contains a multi routes to `Fallthrough_Multi` → the existing live, multi-aware `CheckMovement` path. The halo prevents a cell proposing a walkable edge into a neighbouring wall; interior (multi-free) cells pay one sector lookup.
- Adds `Sector.HasMultis` (one engine accessor); `.swb` format → v9 (rejects old multi-baked files); new `Fallthrough_Multi` telemetry.
- Behaviour-preserving: multis use the same live path the engine used before the cache existed.
**4. Comment polish** (`style`) — no behaviour change.
## Testing
All green: **92** pathfinding (incl. a new fingerprint-stability test and a multi-halo fallthrough test), **423** UOContent, **708** Server.
## Follow-ups (not in this PR)
- **Background bake worker** — make `[PathBake` and the boot prebake non-blocking (game thread serves tile reads to an off-thread worker).
- **Per-multi MCL cache** — cache walkability in each multi's own frame (keyed by multiID, movement-invariant) so houses/boats get a fast path instead of the live fallback.
- **House-pathfinding equivalence tests** — the one area not yet covered by a dedicated automated test; multi pathing is currently correct by delegation to the live path.
## Summary
Phase #3b (final roadmap item), stacked on #2470. Compacts the index trailer from 20 to 8 bytes/chunk.
Trammel: 19.2 MB → 17.9 MB. Roadmap total: 565 MB → 17.9 MB (−96.8%).
## Details
- Trailer stores `{ u32 packedKey = (ChunkX << 16) | ChunkY, u32 recordLength }` per chunk, in record write order; the file offset is dropped and reconstructed by cumulative recordLength from HeaderSize.
- No record reordering, no varint; fixed-stride, TryReadChunk unchanged.
- Also simplifies the accumulated `.swb` code comments across the stack.
- Format v8; v7 files rejected and re-baked once.
## Tests
v8 multi-chunk round-trip (cumulative offset reconstruction) + the v6/v7 suite; full pathfinding suite green; Release build clean.
## Summary
Phase #3a, stacked on #2469. Compresses each chunk record independently with libdeflate (random access preserved).
Trammel: 124.7 MB → 19.2 MB (−85%).
## Details
- Whole-record framing `[u32 UncompressedLen][payload]`; records that don't shrink (tiny Uniform) are stored raw, detected as payload length == UncompressedLen.
- Codec chosen by full-Trammel spike: libdeflate VeryHigh (16.5 MB, 1.83 µs/chunk decompress) over zstd L19/22 (17.4 MB) and managed Brotli q11 (16.4 MB) — best native ratio, fastest decompress, already the repo's packet codec (no new dependency).
- Reuses cached thread-static bindings: `Deflate.Maximum` for bake, `Deflate.Standard` for reads.
- Compression is bake-time only; decompression is one-time per chunk (LRU-cached).
- Format v7; v6 files rejected and re-baked once.
## Tests
v7 unit tests + the v6 suite run through the compression path; full pathfinding suite green; Release build clean.
## Summary
Phase #2 of the `.swb` step-cache size-reduction roadmap (after #2465, v5 uniform elision). Stores the 16 base directional Z arrays as masked residuals against each cell's own SourceZ and omits any array that matches its prediction. Lossless, byte-identical reconstruction.
Trammel: 231.9 MB → 124.7 MB (−46%).
## Details
- Predictor: `predict = mask bit ? SourceZ : 0` (matches the baker's 0 on unwalkable directions); residual `Z − predict` via unchecked two's-complement (byte-exact for all inputs); reconstruct `Z = predict + residual`.
- A `u16 ZArrayMask` flags which of the 16 base arrays differ from prediction; matching arrays are omitted and synthesized from mask + SourceZ at read.
- Serializer-layer only: StepChunk, the cache, the algorithm, and the baker are unchanged.
- Format v6; v5 files rejected and re-baked once.
## Tests
21 v6 unit tests; full pathfinding suite green; Release build clean.
## Summary
Sub-project #1 of the `.swb` step-cache size-reduction roadmap (`dev-docs/pathfinding.md` § Future work). Adds **uniform-chunk elision** to the `StepCacheFile` format, bumping it **v4 → v5**.
A fully-uniform 16×16 chunk — no strata, **no swim layer**, all 19 base arrays constant (open ocean, Green Acres, void) — serializes to a **~28-byte record** (`KindUniform`) instead of ~5,393, and reconstructs **byte-identically** via `Array.Fill`. Non-uniform chunks use the existing v4 body (`KindFull`) with the swim-layer and strata trailers **fully preserved** — the Kind byte is just prepended.
## Calibrated result (measured, not projected)
Baked Trammel via `SaveToFile`:
| | |
|---|---:|
| Chunks | 114,688 |
| Uniform (swim-aware) → elided | 62.7% |
| Swim-layer chunks (stay Full) | 8.9% |
| Strata chunks (stay Full) | 1.8% |
| Baseline (full records) | 592.2 MB |
| **Actual v5 `.swb`** | **231.9 MB (−61%)** |
The residual is ~150 MB of non-uniform land Z-blocks (targeted by #2 predictive-Z) + ~81 MB of swim-layer trailers (#2/#3). #2 and #3 are separate follow-up PRs.
## Implementation
- `StepChunk.IsUniform()` — false if it has strata **or a swim layer**, else true only when all 19 base arrays are constant (the "all-same" check uses the SIMD-accelerated `ContainsAnyExcept`).
- `StepCacheFile` v5 — `Kind` byte (`KindFull=0`/`KindUniform=2`, 1 reserved); uniform write/read; `FormatVersion`/`MinSupportedVersion` → 5 (v4 files rejected on open and re-baked). No `StepCache`/algorithm/index changes; fingerprint logic untouched.
## Tests
7 `StepCacheFileV5Tests` (uniform round-trip + `<200 B` compactness, varied-full, swim-layer-full, strata-full, swim+strata combined, v4 version-gate rejection) + the existing StepCache/pathfinding suite — **70 pass**, including the prior `SwimLayer_RoundTrips`. An independent review verified write/read symmetry, cast round-tripping, swim/strata preservation, and the version gate (READY TO MERGE).
## Summary
Closes the Cold-cache regression flagged in PR #2450. `StepCache.TryGetMask` no longer eagerly runs `BuildChunk` on the first miss for a chunk that isn't in a `.swb` lazy reader. Instead it returns `Fallthrough_NotBuilt` and the caller (`BitmapAStarAlgorithm`) takes the per-cell slow path. The chunk is only promoted to the bitmap fast path after the **second** miss within a 30-second window, filtering single-touch pass-throughs.
This makes BitmapAStar's worst-case (cold cache + short hops) collapse from **12–47× slower** than FastAStar to **roughly the same**, which is the floor the slow path can deliver. Steady-state warm performance (the actual deliverable) is unchanged from PR-5 — it was always the cache fast path.
## The pet-follow scenario this fixes
A mounted player at ~4 tiles/sec with a pet/hireable following will trigger an NPC pathfind every 100–300 ms. Each pathfind is 1–6 tiles. As the player crosses chunk boundaries (~4 sec/chunk), the pet's first pathfind in the new chunk under the previous behavior triggered a full ~700 µs `BuildChunk` for a chunk the player would leave shortly after. At 50–100 mobiles per shard, this exceeded the 8 ms tick budget. PR-5 BDN data showed scenarios 6–9 (2–8 tile NPC perception) at 2,300–3,700 µs Cold vs FastAStar's 80–200 µs.
Under the new gate:
- First miss → `Fallthrough_NotBuilt` → caller uses slow path (~30–50 µs short path). No `BuildChunk`. No allocation.
- Player keeps moving → chunk never gets a second touch within window → never promoted, no rot.
- NPC patrolling a fixed territory → repeatedly hits the same chunks → second touch within window → promote → cache fast path on subsequent calls.
## What changed
- **`CacheHitKind.Fallthrough_NotBuilt = 6`** + **`CacheStats.FallthroughNotBuilt`** counter. `IsHit=false`, so the caller routes to slow path.
- **`StepCache._chunkMissTracker`** — `Dictionary<long, ChunkMissState>` capped at 4096 entries. State is `(byte missCount, uint lastMissTickStamp)` keyed by chunk key. Window-expired entries reset count to 1; capacity overflow prunes window-old entries first.
- **`StepCache.MissPromotionThreshold`** (default `2`) and **`StepCache.MissPromotionWindowMs`** (default `30_000`) — tunable, can be wired through `ServerConfiguration` if shards want different policy. Setting threshold to `1` restores legacy eager-build behavior (used by tests that prime chunks via single `TryGetMask` call).
- **`StepCache.TryGetMask` miss branch** — try lazy reader first (file-loaded chunks bypass the tracker entirely; an `.swb` represents an explicit prior decision to keep the chunk warm). Otherwise consult the tracker.
- **`BitmapAStarAlgorithm.GetSuccessorsSlowPath`** now layers `IsBlockedByDynamic` on top of `CalcMoves.CheckMovement`. Previously the slow path only ran for `CanFly` creatures and rare cache fallthroughs — `CheckMovement` doesn't iterate same-cell mobiles, so the bitmap fast path's `IsBlockedByDynamic` was the only mobile-blocking check. Now first-touch pathfinds run through the slow path, so the gap had to close.
## Tests
50 pathfinding tests pass (was 47). New / updated:
- **`TryGetMask_FirstTouchOnUnbuiltChunk_DefersBuildAndReturnsFallthrough`** — single TryGetMask call returns `Fallthrough_NotBuilt`, no chunk built, no allocation.
- **`TryGetMask_SecondTouchWithinWindow_PromotesAndBuilds`** — second call inside the 30s window builds + serves.
- **`TryGetMask_SecondTouchAfterWindow_RestartsCounterAndDefers`** — second call outside the window restarts the count, returns Fallthrough again.
- **`TryGetMask_DistinctChunks_TrackedIndependently`** — counters are per-chunk; one touch on each of two adjacent chunks both stay in fallthrough.
- **`LazyReaderHit_BypassesMissTrackerOnFirstTouch`** — open `.swb` + first touch hits without consulting the tracker. Production with `.swb` loaded skips the gate entirely.
- **`MultisVersion_Bump_TriggersDirtyRebuild`** — updated to reflect the new 3-step flow (Fallthrough → Miss_NotBuilt → Miss_DirtyRebuild).
- Tests that prime chunks via a single `TryGetMask` call (multi-Z, Tier4, lifecycle, parity, BitmapAStar uses-cache) set `MissPromotionThreshold = 1` to opt into eager behavior.
## Expected BDN impact
The Cold column from PR-5's BDN should change as follows once the bench's submodule pointer is updated to this branch:
| # | Scenario | Cold (PR-5) | Cold (PR-6 expected) | FastAStar Cold |
|--:|-----------------|-------------:|---------------------:|---------------:|
| 2 | sewer corridor | 1,627 µs | ~36 µs | 36 µs |
| 4 | causeway | 1,533 µs | ~39 µs | 39 µs |
| 6 | pet 2-tile | 2,364 µs | ~80 µs | 81 µs |
| 8 | npc 5-tile | 3,708 µs | ~140 µs | 141 µs |
| 9 | npc 8-tile | 2,386 µs | ~200 µs | 197 µs |
WarmNoFile and LazyWarm rows should be unchanged — they were always cache-warm. The miss tracker only fires when neither resident chunks nor the lazy reader can satisfy the request.
## Future work (not in this PR)
- **Background-thread bake**: builds outside the game thread so even promoted chunks don't pay the 700 µs build cost on the main thread. Rule 10 (no Task.Run) applies, so this needs careful design — the bake is a pure data transform but main-thread synchronization on chunk-state transitions has to be threaded through. Defer to a follow-up.
- **Long-traverse BDN scenario**: a multi-Find benchmark simulating 50 pet repaths across chunk transitions. Requires restructuring the bench harness; the existing 10-scenario corpus + Cold provider already exercises the gate.
- **Swim sourceZ bake**: scenario 5 (sea serpent) shows 56 B alloc on warm paths because the cache's SourceZ is computed under default-walker rules. Swim creatures fall through to slow path. Independent of this PR.
## Summary
Multi-Z cells (bridges, stairs, paver-over-ground, multi-floor structures) now carry **per-stratum walkability data** in the cache instead of falling through to the slow path. The data is computed at chunk-build time, persisted in the `.swb` file, and selected at query time by matching the request's `sourceZ` against each stratum's `zCenter` (within `StepHeight` tolerance).
This is the Tier 4 strata feature, deferred from PR #2447 / PR #2448 / PR #2449. Builds on PR #2449's lazy backing store and public bake helpers.
## Wire format change (v1 → v2)
`StepCacheFile.FormatVersion = 2`. `MinSupportedVersion = 2`. v1 `.swb` files are silently rejected at open time (treated as missing) and overwritten on the next `SaveToFile` / `BakeMap`. **No migration** — older files just get re-baked.
The `MinSupportedVersion` sentinel is the model going forward: bump the constant when an incompatible change lands; admins re-bake on the next deploy. No matrix of v1↔v2↔v3 migration logic to maintain.
## What changed
- **`StepProbe.ComputeStrataAt(map, x, y)`** — enumerates walkable standing-Zs at the cell (one per land surface plus one per walkable static), collapses Zs within `2*StepHeight`, runs `ComputeMaskAt` at each surviving Z. Returns `null` for single-Z cells (caller uses the chunk's main mask).
- **`StepChunk`** — replaces the old `MultiZCells` bitmap with a **strata storage pair**:
- `ushort[256] StrataOffsetByCell` (sentinel `NoStrata = 0xFFFF` = "no strata for that cell")
- `byte[] StrataData` packed: `u8 stratumCount`, then `count × 19-byte stratum`
- `sbyte zCenter, byte walkMask, byte wetMask, sbyte walkZ_N..NW (8), sbyte swimZ_N..NW (8)`
- `IsCellMultiZ` derives from `StrataOffsetByCell[cell] != NoStrata` — same semantics, single source of truth.
- **`StepCache.BuildChunk`** — populates strata for cells flagged multi-Z via `SetStrata`. Chunks with zero multi-Z cells pay zero strata overhead (offset array + data array stay null).
- **`StepCache.TryGetMask`** — for multi-Z cells, scans strata with `TryStratumHit`; returns the matching one with `HitKind=Hit`. Falls through to slow path only when no stratum matches the query `sourceZ`.
- **`StepCacheFile`** — v2 serialization with strata trailer per chunk + `recordLength` in index entry. Lazy reader sizes scratch per-chunk-record using the recorded length, growing on demand for multi-Z-heavy chunks. Patches `IndexOffset` on `w.Buffer` (BufferWriter's current backing array) since it grows during variable-size chunk writes.
## File layout v2
```
Header (48 bytes):
u32 Magic = 0x42575300 ('SWB\0')
u32 Version = 2
u32 MapId
u64 Fingerprint XxHash3 over LandTable + ItemTable flags + map files (mapX.mul/.uop, staidxX.mul, staticsX.mul)
u64 BakeTimestamp informational
u32 ChunkCount
u64 IndexOffset position where chunk index begins
Per chunk (variable size):
u16 ChunkX, ChunkY
u32 BuiltMultisVersion
u8 HasStrata 0 = no strata trailer; 1 = strata trailer follows
byte WalkMask[256], WetMask[256]
sbyte SourceZ[256], WalkZN..WalkZNW[256], SwimZN..SwimZNW[256]
// Strata trailer (only when HasStrata == 1):
u16 StrataOffsetByCell[256] // NoStrata sentinel = 0xFFFF
u32 StrataDataLength
byte StrataData[StrataDataLength]
Per multi-Z cell: u8 count, then count × Stratum (19 bytes)
Index trailer (20 × ChunkCount bytes):
per chunk: { u64 chunkKey, u64 fileOffset, u32 recordLength }
```
## Summary
Adds two pieces of pathfinding tooling on top of PR #2448's lazy `.swb` infrastructure:
- **`PathfindRecorder`** — admin-toggled JSONL telemetry capture; one record per `BitmapAStarAlgorithm.Find` call. Output format matches the BDN harness corpus, so production traffic can be captured and replayed in benchmarks without an adapter.
- **Public bake helpers on `StepCache`** — `ComputeLiveTileDataHash`, `TryReadTileDataHashFromFile`, `BakeMap`, `ClearResidentChunks`. Lets the benchmark project (and any future bake utility) drive cache fill + persist without exposing internal types.
The companion BDN harness update lives in [ModernUO-Benchmarks#kb/pathfinding-pr4-bench](https://github.com/modernuo/ModernUO-Benchmarks/tree/kb/pathfinding-pr4-bench): porting `Benchmarks/PathfindInGame/` from the `kb/ai_pathfinding` branch to the API shipped in #2446–#2448.
## What's in this PR
### `PathfindRecorder` (`PathfindRecorder.cs`)
- Holds a single `StreamWriter` open while recording; its internal buffer absorbs per-record writes without per-call `File.AppendAllText`.
- Single `bool` check on the hot path; cheap when disabled.
- Disabling flushes + disposes; an IO failure during write also disables the recorder.
- Server config:
- `pathfinding.recorder.enable` — bool, default `false`. Read on boot via `GetOrUpdateSetting`.
- `pathfinding.recorder.path` — default `<basedir>/Data/Pathfinding/recordings/pathfinds.jsonl`.
- Hooked into `BitmapAStarAlgorithm.Find` — runs once per call, does nothing when disabled.
- Admin command: `[PathRecord [on|off|flush|status]` (default `status`).
### Public cache helpers
- `static ulong StepCache.ComputeLiveTileDataHash()` — wraps the file module's hash function for staleness checks.
- `static bool StepCache.TryReadTileDataHashFromFile(string, out ulong)` — peeks at a `.swb` file's hash field (20 bytes).
- `int StepCache.BakeMap(int, string)` — walks every chunk in the map, populates resident set, saves. Offline / fixture use; blocks for many seconds on a full-map walk.
- `void StepCache.ClearResidentChunks()` — drops chunks + zeros counters but keeps lazy readers open. Lets benchmark loops measure "first query after boot" cost across iterations without the lazy-reader reopen overhead.
## Summary
Adds a binary disk format + lazy reader so the step cache can warm-start from a precomputed file without paying chunk-build cost on the first pathfind through a region. **Resident memory stays bounded by `MaxResidentChunks` regardless of file size** — opening a `.swb` reads only the header + chunk-offset index (~16 bytes per indexed chunk), and individual chunks are seeked + deserialized only when `ResolveMissingChunk` asks for them.
The lazy design (vs. an eager bulk load): a 250 MB bake on a RAM-constrained shard never materializes more than the LRU cap (~40 MB at the default 8192-chunk cap), and unwanted regions never enter memory at all.
Builds on PR #2447.
## What changed
- **`StepCacheFile`** — binary reader/writer module. Writer emits header → chunks (offsets recorded) → index trailer, then patches the header's `IndexOffset` field. Reader is `OpenForLazy(path)` returning a `LazyReader` that holds an open `FileStream` + offset dictionary.
- **`StepCacheFile.LazyReader`** — `TryReadChunk(chunkX, chunkY)` does a single seek + bulk read for one record. `Dispose` releases the underlying stream. Files are opened with `FileShare.Read | FileShare.Delete` so admin tooling can replace them.
- **TileData fingerprint via XxHash3.** The `.swb` header carries a hash of `LandTable + ItemTable` flags. Load rejects any file whose hash doesn't match the running server. Computed via `HashUtility.ComputeHash64` (engine-blessed hasher) — adds a `ReadOnlySpan<byte>` overload alongside the existing `ReadOnlySpan<char>` one for parity.
- **`StepCache.SaveToFile(path, mapId)`** — writes resident chunks for the given map.
- **`StepCache.TryOpenLazyReader(path, mapId)`** — opens the file, validates header, holds the reader for the map's lifetime.
- **`StepCache.ResolveMissingChunk`** — now consults the lazy reader before invoking the runtime baker. A loaded chunk whose `BuiltMultisVersion` doesn't match the live sector falls through to the baker (snapshot was made before a multi was added/removed in that sector).
- **`StepCache.Clear` closes lazy readers.** Test cleanup can delete `.swb` files cleanly.
- **Auto-load at startup.** `PathCacheCommands.Configure()` opens `Data/Pathfinding/<mapId>.swb` as a lazy reader for every map.
- **`[PathCacheSave`** / **`[PathCacheLoad`** — admin commands for the same workflow.
- **`pathfinding.maxResidentChunks` shard-tunable.** Read from `server.cfg` at boot via `ServerConfiguration.GetOrUpdateSetting` (default 8192 ≈ 40 MB). Small shards can tune down; large shards with substantial bakes can tune up to reduce eviction churn. Default is written back to `server.cfg` on first boot, matching the engine pattern used by other settings.
## File layout (v1)
```
Header (48 bytes):
u32 Magic = 0x42575300 ('SWB\0')
u32 Version = 1
u32 MapId
u64 TileDataHash XxHash3 over LandTable + ItemTable flags (HashUtility)
u64 BakeTimestamp informational
u32 ChunkCount
u64 IndexOffset file position where the chunk index begins
Chunk records (fixed size, ~5,393 bytes each, +32 if multi-Z):
u16 ChunkX
u16 ChunkY
u32 BuiltMultisVersion
u8 HasMultiZ
byte WalkMask[256], WetMask[256]
sbyte SourceZ[256], WalkZN..WalkZNW[256], SwimZN..SwimZNW[256]
[byte MultiZCells[32] when HasMultiZ == 1]
Index trailer (16 × ChunkCount bytes):
(u64 chunkKey, u64 fileOffset)
```
## Memory math
| Scenario | Disk file | RAM at boot | Notes |
|---|---|---|---|
| Empty / no `.swb` files | — | 0 | Silent; cache builds on demand. |
| Admin-curated towns (5K chunks) | 25 MB | 0 + per-query | Index ≈ 80 KB. Resident grows to the configured cap under steady-state queries. |
| Full-map bake (50K chunks) | 250 MB | 0 + per-query | Index ≈ 800 KB. Same configured cap. Cold areas never load. |
| All 5 maps fully baked | 1.25 GB | 0 + per-query | Index ≈ 4 MB total. Same configured cap. |
## Hash choice (FNV-1a → XxHash3)
The original draft used inlined FNV-1a-64. Switched to XxHash3 via `HashUtility`:
- ~30× faster on this workload (~30 GB/s SIMD vs ~2 GB/s byte-by-byte). Boot-time only, so absolute saving is microseconds — the real wins are elsewhere.
- Stronger collision resistance and distribution.
- Drops ~25 lines of inlined hash code; matches the rest of the codebase's hashing pattern.
- Hash is stable as long as `HashUtility`'s `xxHash3Seed` constant doesn't change (already marked `// DO NOT CHANGE THIS NUMBER`).
Kamron Batman