feat(pathfinding): lazy .swb backing store for the step cache (#2448)
## Summary Adds a binary disk format + lazy reader so the step cache can warm-start from a precomputed file without paying chunk-build cost on the first pathfind through a region. **Resident memory stays bounded by `MaxResidentChunks` regardless of file size** — opening a `.swb` reads only the header + chunk-offset index (~16 bytes per indexed chunk), and individual chunks are seeked + deserialized only when `ResolveMissingChunk` asks for them. The lazy design (vs. an eager bulk load): a 250 MB bake on a RAM-constrained shard never materializes more than the LRU cap (~40 MB at the default 8192-chunk cap), and unwanted regions never enter memory at all. Builds on PR #2447. ## What changed - **`StepCacheFile`** — binary reader/writer module. Writer emits header → chunks (offsets recorded) → index trailer, then patches the header's `IndexOffset` field. Reader is `OpenForLazy(path)` returning a `LazyReader` that holds an open `FileStream` + offset dictionary. - **`StepCacheFile.LazyReader`** — `TryReadChunk(chunkX, chunkY)` does a single seek + bulk read for one record. `Dispose` releases the underlying stream. Files are opened with `FileShare.Read | FileShare.Delete` so admin tooling can replace them. - **TileData fingerprint via XxHash3.** The `.swb` header carries a hash of `LandTable + ItemTable` flags. Load rejects any file whose hash doesn't match the running server. Computed via `HashUtility.ComputeHash64` (engine-blessed hasher) — adds a `ReadOnlySpan<byte>` overload alongside the existing `ReadOnlySpan<char>` one for parity. - **`StepCache.SaveToFile(path, mapId)`** — writes resident chunks for the given map. - **`StepCache.TryOpenLazyReader(path, mapId)`** — opens the file, validates header, holds the reader for the map's lifetime. - **`StepCache.ResolveMissingChunk`** — now consults the lazy reader before invoking the runtime baker. A loaded chunk whose `BuiltMultisVersion` doesn't match the live sector falls through to the baker (snapshot was made before a multi was added/removed in that sector). - **`StepCache.Clear` closes lazy readers.** Test cleanup can delete `.swb` files cleanly. - **Auto-load at startup.** `PathCacheCommands.Configure()` opens `Data/Pathfinding/<mapId>.swb` as a lazy reader for every map. - **`[PathCacheSave`** / **`[PathCacheLoad`** — admin commands for the same workflow. - **`pathfinding.maxResidentChunks` shard-tunable.** Read from `server.cfg` at boot via `ServerConfiguration.GetOrUpdateSetting` (default 8192 ≈ 40 MB). Small shards can tune down; large shards with substantial bakes can tune up to reduce eviction churn. Default is written back to `server.cfg` on first boot, matching the engine pattern used by other settings. ## File layout (v1) ``` Header (48 bytes): u32 Magic = 0x42575300 ('SWB\0') u32 Version = 1 u32 MapId u64 TileDataHash XxHash3 over LandTable + ItemTable flags (HashUtility) u64 BakeTimestamp informational u32 ChunkCount u64 IndexOffset file position where the chunk index begins Chunk records (fixed size, ~5,393 bytes each, +32 if multi-Z): u16 ChunkX u16 ChunkY u32 BuiltMultisVersion u8 HasMultiZ byte WalkMask[256], WetMask[256] sbyte SourceZ[256], WalkZN..WalkZNW[256], SwimZN..SwimZNW[256] [byte MultiZCells[32] when HasMultiZ == 1] Index trailer (16 × ChunkCount bytes): (u64 chunkKey, u64 fileOffset) ``` ## Memory math | Scenario | Disk file | RAM at boot | Notes | |---|---|---|---| | Empty / no `.swb` files | — | 0 | Silent; cache builds on demand. | | Admin-curated towns (5K chunks) | 25 MB | 0 + per-query | Index ≈ 80 KB. Resident grows to the configured cap under steady-state queries. | | Full-map bake (50K chunks) | 250 MB | 0 + per-query | Index ≈ 800 KB. Same configured cap. Cold areas never load. | | All 5 maps fully baked | 1.25 GB | 0 + per-query | Index ≈ 4 MB total. Same configured cap. | ## Hash choice (FNV-1a → XxHash3) The original draft used inlined FNV-1a-64. Switched to XxHash3 via `HashUtility`: - ~30× faster on this workload (~30 GB/s SIMD vs ~2 GB/s byte-by-byte). Boot-time only, so absolute saving is microseconds — the real wins are elsewhere. - Stronger collision resistance and distribution. - Drops ~25 lines of inlined hash code; matches the rest of the codebase's hashing pattern. - Hash is stable as long as `HashUtility`'s `xxHash3Seed` constant doesn't change (already marked `// DO NOT CHANGE THIS NUMBER`).
This commit is contained in:
parent
9066e8fd00
commit
7c9215d97c
6 changed files with 861 additions and 4 deletions
|
|
@ -50,6 +50,22 @@ public static class HashUtility
|
|||
return result;
|
||||
}
|
||||
|
||||
public static ulong ComputeHash64(ReadOnlySpan<byte> bytes)
|
||||
{
|
||||
if (bytes.Length == 0)
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
var hasher = _xxHash3 ??= new XxHash3(unchecked((long)xxHash3Seed));
|
||||
hasher.Append(bytes);
|
||||
|
||||
var result = hasher.GetCurrentHashAsUInt64();
|
||||
hasher.Reset();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
public static uint ComputeHash32(ReadOnlySpan<char> str)
|
||||
{
|
||||
if (str == ReadOnlySpan<char>.Empty)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue