ModernUO/Projects/Server
Kamron Batman 7c9215d97c
feat(pathfinding): lazy .swb backing store for the step cache (#2448)
## Summary

Adds a binary disk format + lazy reader so the step cache can warm-start from a precomputed file without paying chunk-build cost on the first pathfind through a region. **Resident memory stays bounded by `MaxResidentChunks` regardless of file size** — opening a `.swb` reads only the header + chunk-offset index (~16 bytes per indexed chunk), and individual chunks are seeked + deserialized only when `ResolveMissingChunk` asks for them.

The lazy design (vs. an eager bulk load): a 250 MB bake on a RAM-constrained shard never materializes more than the LRU cap (~40 MB at the default 8192-chunk cap), and unwanted regions never enter memory at all.

Builds on PR #2447.

## What changed

- **`StepCacheFile`** — binary reader/writer module. Writer emits header → chunks (offsets recorded) → index trailer, then patches the header's `IndexOffset` field. Reader is `OpenForLazy(path)` returning a `LazyReader` that holds an open `FileStream` + offset dictionary.
- **`StepCacheFile.LazyReader`** — `TryReadChunk(chunkX, chunkY)` does a single seek + bulk read for one record. `Dispose` releases the underlying stream. Files are opened with `FileShare.Read | FileShare.Delete` so admin tooling can replace them.
- **TileData fingerprint via XxHash3.** The `.swb` header carries a hash of `LandTable + ItemTable` flags. Load rejects any file whose hash doesn't match the running server. Computed via `HashUtility.ComputeHash64` (engine-blessed hasher) — adds a `ReadOnlySpan<byte>` overload alongside the existing `ReadOnlySpan<char>` one for parity.
- **`StepCache.SaveToFile(path, mapId)`** — writes resident chunks for the given map.
- **`StepCache.TryOpenLazyReader(path, mapId)`** — opens the file, validates header, holds the reader for the map's lifetime.
- **`StepCache.ResolveMissingChunk`** — now consults the lazy reader before invoking the runtime baker. A loaded chunk whose `BuiltMultisVersion` doesn't match the live sector falls through to the baker (snapshot was made before a multi was added/removed in that sector).
- **`StepCache.Clear` closes lazy readers.** Test cleanup can delete `.swb` files cleanly.
- **Auto-load at startup.** `PathCacheCommands.Configure()` opens `Data/Pathfinding/<mapId>.swb` as a lazy reader for every map.
- **`[PathCacheSave`** / **`[PathCacheLoad`** — admin commands for the same workflow.
- **`pathfinding.maxResidentChunks` shard-tunable.** Read from `server.cfg` at boot via `ServerConfiguration.GetOrUpdateSetting` (default 8192 ≈ 40 MB). Small shards can tune down; large shards with substantial bakes can tune up to reduce eviction churn. Default is written back to `server.cfg` on first boot, matching the engine pattern used by other settings.

## File layout (v1)

```
Header (48 bytes):
  u32  Magic           = 0x42575300 ('SWB\0')
  u32  Version         = 1
  u32  MapId
  u64  TileDataHash    XxHash3 over LandTable + ItemTable flags (HashUtility)
  u64  BakeTimestamp   informational
  u32  ChunkCount
  u64  IndexOffset     file position where the chunk index begins

Chunk records (fixed size, ~5,393 bytes each, +32 if multi-Z):
  u16  ChunkX
  u16  ChunkY
  u32  BuiltMultisVersion
  u8   HasMultiZ
  byte WalkMask[256], WetMask[256]
  sbyte SourceZ[256], WalkZN..WalkZNW[256], SwimZN..SwimZNW[256]
  [byte MultiZCells[32] when HasMultiZ == 1]

Index trailer (16 × ChunkCount bytes):
  (u64 chunkKey, u64 fileOffset)
```

## Memory math

| Scenario | Disk file | RAM at boot | Notes |
|---|---|---|---|
| Empty / no `.swb` files | — | 0 | Silent; cache builds on demand. |
| Admin-curated towns (5K chunks) | 25 MB | 0 + per-query | Index ≈ 80 KB. Resident grows to the configured cap under steady-state queries. |
| Full-map bake (50K chunks) | 250 MB | 0 + per-query | Index ≈ 800 KB. Same configured cap. Cold areas never load. |
| All 5 maps fully baked | 1.25 GB | 0 + per-query | Index ≈ 4 MB total. Same configured cap. |

## Hash choice (FNV-1a → XxHash3)

The original draft used inlined FNV-1a-64. Switched to XxHash3 via `HashUtility`:

- ~30× faster on this workload (~30 GB/s SIMD vs ~2 GB/s byte-by-byte). Boot-time only, so absolute saving is microseconds — the real wins are elsewhere.
- Stronger collision resistance and distribution.
- Drops ~25 lines of inlined hash code; matches the rest of the codebase's hashing pattern.
- Hash is stable as long as `HashUtility`'s `xxHash3Seed` constant doesn't change (already marked `// DO NOT CHANGE THIS NUMBER`).
2026-05-06 01:32:44 -07:00
..
Buffers feat(buffers): add :L lowercase format spec to RawInterpolatedStringHandler (#2440) 2026-05-03 18:24:57 -07:00
Client feat: Add zero-alloc interpolation handler to ValueStringBuilder, replace all StringBuilder usage (#2387) 2026-03-22 14:23:44 -07:00
Collections fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Comparers fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Compression fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Configuration fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Console fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
ContextMenus fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Events fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Exceptions fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
GarbageCollection chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
Geometry fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Items perf(messages): mechanical interpolation cleanups (#2436) 2026-05-03 18:26:49 -07:00
Json fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Localization feat: Add zero-alloc interpolation handler to ValueStringBuilder, replace all StringBuilder usage (#2387) 2026-03-22 14:23:44 -07:00
Maps feat: Replace FastAStarAlgorithm with BitmapAStarAlgorithm (#2446) 2026-05-05 21:53:43 -07:00
Menus fix: Cleans up core code (#1187) 2022-10-10 21:47:08 -07:00
Migrations fix: Moves containers/bods/traps/etc to serialization generator (#2310) 2026-01-07 21:55:55 -08:00
Mobiles perf(messages): restructure format-string call sites (#2437) 2026-05-03 18:29:11 -07:00
Network perf: Zero-alloc interpolation for SendMessage/Overhead APIs (#2434) 2026-05-03 18:04:02 -07:00
PropertyList fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Random fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Regions fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Serialization feat: Refactor Poison system, implement Darkglow & Parasitic effects (#2385) 2026-03-21 21:27:22 -07:00
Targeting chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
Text feat: Add zero-alloc interpolation handler to ValueStringBuilder, replace all StringBuilder usage (#2387) 2026-03-22 14:23:44 -07:00
TileMatrix fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Timer feat: Adds dynamic thread idle to address CPU usage (#2370) 2026-03-13 23:53:49 -07:00
TimeZones fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Utilities feat(pathfinding): lazy .swb backing store for the step cache (#2448) 2026-05-06 01:32:44 -07:00
World perf: Zero-alloc interpolation for SendMessage/Overhead APIs (#2434) 2026-05-03 18:04:02 -07:00
AssemblyHandler.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Attributes.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
CityInfo.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Commands.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Effects.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
EntityFinalizationTracker.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
EventLoopTasks.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
ExpansionInfo.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
FeatureFlags.cs feat: Adds robust speed hack detection and movement throttling (#2266) 2026-03-07 11:44:37 -08:00
Guild.cs perf: Migrate Old Guild System gumps to DynamicGump (#2420) 2026-04-26 10:44:11 -07:00
HuePicker.cs fix: Cleans up core code (#1187) 2022-10-10 21:47:08 -07:00
IAccount.cs fix: Fixes looking up accounts that were renamed. (#2078) 2025-01-18 17:11:43 -08:00
IEntity.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Interfaces.cs feat: Add CanSpawnMobile overload with props Z-range support. (#2293) 2025-12-27 17:01:15 -08:00
IVirtualCheckGump.cs feat: Moves gumps out of the core (#1916) 2024-08-09 19:07:32 -07:00
KeywordList.cs fix: Cleans up core code (#1187) 2022-10-10 21:47:08 -07:00
Main.cs feat: Adds dynamic thread idle to address CPU usage (#2370) 2026-03-13 23:53:49 -07:00
MessageType.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Module.cs fix: Fixes sending packets and sidesteps a major issue with stackalloc and PGO in .NET 8 (#1607) 2023-11-21 12:18:20 -08:00
Party.cs fix: Cleans up core code (#1187) 2022-10-10 21:47:08 -07:00
Poison.cs feat: Refactor Poison system, implement Darkglow & Parasitic effects (#2385) 2026-03-21 21:27:22 -07:00
Prompt.cs fix: Cleans up core code (#1187) 2022-10-10 21:47:08 -07:00
Race.cs fix: Fixes TryParse for Race (#1244) 2022-11-12 00:31:25 -08:00
SecureTrade.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Serial.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Server.csproj fix: Bumps dependencies to address vulns. (#2408) 2026-04-19 13:41:27 -07:00
Skills.cs chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
TileData.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
TileList.cs fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00