ModernUO/Projects/UOContent
Kamron Batman 7c9215d97c
feat(pathfinding): lazy .swb backing store for the step cache (#2448)
## Summary

Adds a binary disk format + lazy reader so the step cache can warm-start from a precomputed file without paying chunk-build cost on the first pathfind through a region. **Resident memory stays bounded by `MaxResidentChunks` regardless of file size** — opening a `.swb` reads only the header + chunk-offset index (~16 bytes per indexed chunk), and individual chunks are seeked + deserialized only when `ResolveMissingChunk` asks for them.

The lazy design (vs. an eager bulk load): a 250 MB bake on a RAM-constrained shard never materializes more than the LRU cap (~40 MB at the default 8192-chunk cap), and unwanted regions never enter memory at all.

Builds on PR #2447.

## What changed

- **`StepCacheFile`** — binary reader/writer module. Writer emits header → chunks (offsets recorded) → index trailer, then patches the header's `IndexOffset` field. Reader is `OpenForLazy(path)` returning a `LazyReader` that holds an open `FileStream` + offset dictionary.
- **`StepCacheFile.LazyReader`** — `TryReadChunk(chunkX, chunkY)` does a single seek + bulk read for one record. `Dispose` releases the underlying stream. Files are opened with `FileShare.Read | FileShare.Delete` so admin tooling can replace them.
- **TileData fingerprint via XxHash3.** The `.swb` header carries a hash of `LandTable + ItemTable` flags. Load rejects any file whose hash doesn't match the running server. Computed via `HashUtility.ComputeHash64` (engine-blessed hasher) — adds a `ReadOnlySpan<byte>` overload alongside the existing `ReadOnlySpan<char>` one for parity.
- **`StepCache.SaveToFile(path, mapId)`** — writes resident chunks for the given map.
- **`StepCache.TryOpenLazyReader(path, mapId)`** — opens the file, validates header, holds the reader for the map's lifetime.
- **`StepCache.ResolveMissingChunk`** — now consults the lazy reader before invoking the runtime baker. A loaded chunk whose `BuiltMultisVersion` doesn't match the live sector falls through to the baker (snapshot was made before a multi was added/removed in that sector).
- **`StepCache.Clear` closes lazy readers.** Test cleanup can delete `.swb` files cleanly.
- **Auto-load at startup.** `PathCacheCommands.Configure()` opens `Data/Pathfinding/<mapId>.swb` as a lazy reader for every map.
- **`[PathCacheSave`** / **`[PathCacheLoad`** — admin commands for the same workflow.
- **`pathfinding.maxResidentChunks` shard-tunable.** Read from `server.cfg` at boot via `ServerConfiguration.GetOrUpdateSetting` (default 8192 ≈ 40 MB). Small shards can tune down; large shards with substantial bakes can tune up to reduce eviction churn. Default is written back to `server.cfg` on first boot, matching the engine pattern used by other settings.

## File layout (v1)

```
Header (48 bytes):
  u32  Magic           = 0x42575300 ('SWB\0')
  u32  Version         = 1
  u32  MapId
  u64  TileDataHash    XxHash3 over LandTable + ItemTable flags (HashUtility)
  u64  BakeTimestamp   informational
  u32  ChunkCount
  u64  IndexOffset     file position where the chunk index begins

Chunk records (fixed size, ~5,393 bytes each, +32 if multi-Z):
  u16  ChunkX
  u16  ChunkY
  u32  BuiltMultisVersion
  u8   HasMultiZ
  byte WalkMask[256], WetMask[256]
  sbyte SourceZ[256], WalkZN..WalkZNW[256], SwimZN..SwimZNW[256]
  [byte MultiZCells[32] when HasMultiZ == 1]

Index trailer (16 × ChunkCount bytes):
  (u64 chunkKey, u64 fileOffset)
```

## Memory math

| Scenario | Disk file | RAM at boot | Notes |
|---|---|---|---|
| Empty / no `.swb` files | — | 0 | Silent; cache builds on demand. |
| Admin-curated towns (5K chunks) | 25 MB | 0 + per-query | Index ≈ 80 KB. Resident grows to the configured cap under steady-state queries. |
| Full-map bake (50K chunks) | 250 MB | 0 + per-query | Index ≈ 800 KB. Same configured cap. Cold areas never load. |
| All 5 maps fully baked | 1.25 GB | 0 + per-query | Index ≈ 4 MB total. Same configured cap. |

## Hash choice (FNV-1a → XxHash3)

The original draft used inlined FNV-1a-64. Switched to XxHash3 via `HashUtility`:

- ~30× faster on this workload (~30 GB/s SIMD vs ~2 GB/s byte-by-byte). Boot-time only, so absolute saving is microseconds — the real wins are elsewhere.
- Stronger collision resistance and distribution.
- Drops ~25 lines of inlined hash code; matches the rest of the codebase's hashing pattern.
- Hash is stable as long as `HashUtility`'s `xxHash3Seed` constant doesn't change (already marked `// DO NOT CHANGE THIS NUMBER`).
2026-05-06 01:32:44 -07:00
..
Accounting feat: add young_player_system feature flag to disable Young player system (#2445) 2026-05-05 20:09:49 -07:00
Assistants chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
Commands perf(messages): mechanical interpolation cleanups (#2436) 2026-05-03 18:26:49 -07:00
Compression feat: Implements new robust/pluggable backup/archive system. (#2388) 2026-03-22 19:51:20 -07:00
Configuration fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Console feat: Adds console commands (#1714) 2024-03-30 09:40:43 -07:00
Context Menus fix: Moves ContextMenu out of core, streamlines code, fixes bugs (#1873) 2024-07-20 21:33:23 -07:00
Engines feat(pathfinding): lazy .swb backing store for the step cache (#2448) 2026-05-06 01:32:44 -07:00
Gumps perf: Migrate RewardGump to DynamicGump (#2432) 2026-05-03 10:22:52 -07:00
Holiday Stuff perf: Migrate Veteran Reward gumps to DynamicGump/StaticGump (#2415) 2026-04-25 20:09:27 -07:00
Items perf: Zero-alloc interpolation for SendMessage/Overhead APIs (#2434) 2026-05-03 18:04:02 -07:00
Migrations feat: Adds rope teleporter for New Haven Mines (#2439) 2026-05-03 17:23:31 -07:00
Misc perf(messages): mechanical interpolation cleanups (#2436) 2026-05-03 18:26:49 -07:00
Mobiles feat: add young_player_system feature flag to disable Young player system (#2445) 2026-05-05 20:09:49 -07:00
Multis perf: Migrate vendor management gumps to DynamicGump/StaticGump (#2431) 2026-05-03 10:19:12 -07:00
Network feat: Adds robust speed hack detection and movement throttling (#2266) 2026-03-07 11:44:37 -08:00
Regions fix: Fixes criminals having guards called on them (#2348) 2026-03-13 17:35:47 -07:00
Skills perf: Migrate NPC and Skill UI gumps to DynamicGump/StaticGump (#2414) 2026-04-25 19:51:42 -07:00
Special Systems feat: Add zero-alloc interpolation handler to ValueStringBuilder, replace all StringBuilder usage (#2387) 2026-03-22 14:23:44 -07:00
Spells feat: Adjusts fame and karma system with era gates for OSI accuracy (#2389) 2026-04-25 11:19:57 -07:00
Systems/JailSystem fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Targets chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
Text fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Utilities chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
World Saves perf(messages): mechanical interpolation cleanups (#2436) 2026-05-03 18:26:49 -07:00
Module.cs fix: Fixes sending packets and sidesteps a major issue with stackalloc and PGO in .NET 8 (#1607) 2023-11-21 12:18:20 -08:00
UOContent.csproj fix: Bumps dependencies to address vulns. (#2408) 2026-04-19 13:41:27 -07:00