7c9215d97c
## Summary Adds a binary disk format + lazy reader so the step cache can warm-start from a precomputed file without paying chunk-build cost on the first pathfind through a region. **Resident memory stays bounded by `MaxResidentChunks` regardless of file size** — opening a `.swb` reads only the header + chunk-offset index (~16 bytes per indexed chunk), and individual chunks are seeked + deserialized only when `ResolveMissingChunk` asks for them. The lazy design (vs. an eager bulk load): a 250 MB bake on a RAM-constrained shard never materializes more than the LRU cap (~40 MB at the default 8192-chunk cap), and unwanted regions never enter memory at all. Builds on PR #2447. ## What changed - **`StepCacheFile`** — binary reader/writer module. Writer emits header → chunks (offsets recorded) → index trailer, then patches the header's `IndexOffset` field. Reader is `OpenForLazy(path)` returning a `LazyReader` that holds an open `FileStream` + offset dictionary. - **`StepCacheFile.LazyReader`** — `TryReadChunk(chunkX, chunkY)` does a single seek + bulk read for one record. `Dispose` releases the underlying stream. Files are opened with `FileShare.Read | FileShare.Delete` so admin tooling can replace them. - **TileData fingerprint via XxHash3.** The `.swb` header carries a hash of `LandTable + ItemTable` flags. Load rejects any file whose hash doesn't match the running server. Computed via `HashUtility.ComputeHash64` (engine-blessed hasher) — adds a `ReadOnlySpan<byte>` overload alongside the existing `ReadOnlySpan<char>` one for parity. - **`StepCache.SaveToFile(path, mapId)`** — writes resident chunks for the given map. - **`StepCache.TryOpenLazyReader(path, mapId)`** — opens the file, validates header, holds the reader for the map's lifetime. - **`StepCache.ResolveMissingChunk`** — now consults the lazy reader before invoking the runtime baker. A loaded chunk whose `BuiltMultisVersion` doesn't match the live sector falls through to the baker (snapshot was made before a multi was added/removed in that sector). - **`StepCache.Clear` closes lazy readers.** Test cleanup can delete `.swb` files cleanly. - **Auto-load at startup.** `PathCacheCommands.Configure()` opens `Data/Pathfinding/<mapId>.swb` as a lazy reader for every map. - **`[PathCacheSave`** / **`[PathCacheLoad`** — admin commands for the same workflow. - **`pathfinding.maxResidentChunks` shard-tunable.** Read from `server.cfg` at boot via `ServerConfiguration.GetOrUpdateSetting` (default 8192 ≈ 40 MB). Small shards can tune down; large shards with substantial bakes can tune up to reduce eviction churn. Default is written back to `server.cfg` on first boot, matching the engine pattern used by other settings. ## File layout (v1) ``` Header (48 bytes): u32 Magic = 0x42575300 ('SWB\0') u32 Version = 1 u32 MapId u64 TileDataHash XxHash3 over LandTable + ItemTable flags (HashUtility) u64 BakeTimestamp informational u32 ChunkCount u64 IndexOffset file position where the chunk index begins Chunk records (fixed size, ~5,393 bytes each, +32 if multi-Z): u16 ChunkX u16 ChunkY u32 BuiltMultisVersion u8 HasMultiZ byte WalkMask[256], WetMask[256] sbyte SourceZ[256], WalkZN..WalkZNW[256], SwimZN..SwimZNW[256] [byte MultiZCells[32] when HasMultiZ == 1] Index trailer (16 × ChunkCount bytes): (u64 chunkKey, u64 fileOffset) ``` ## Memory math | Scenario | Disk file | RAM at boot | Notes | |---|---|---|---| | Empty / no `.swb` files | — | 0 | Silent; cache builds on demand. | | Admin-curated towns (5K chunks) | 25 MB | 0 + per-query | Index ≈ 80 KB. Resident grows to the configured cap under steady-state queries. | | Full-map bake (50K chunks) | 250 MB | 0 + per-query | Index ≈ 800 KB. Same configured cap. Cold areas never load. | | All 5 maps fully baked | 1.25 GB | 0 + per-query | Index ≈ 4 MB total. Same configured cap. | ## Hash choice (FNV-1a → XxHash3) The original draft used inlined FNV-1a-64. Switched to XxHash3 via `HashUtility`: - ~30× faster on this workload (~30 GB/s SIMD vs ~2 GB/s byte-by-byte). Boot-time only, so absolute saving is microseconds — the real wins are elsewhere. - Stronger collision resistance and distribution. - Drops ~25 lines of inlined hash code; matches the rest of the codebase's hashing pattern. - Hash is stable as long as `HashUtility`'s `xxHash3Seed` constant doesn't change (already marked `// DO NOT CHANGE THIS NUMBER`).
70 lines
3.2 KiB
C#
70 lines
3.2 KiB
C#
namespace Server.Engines.Pathing.Cache;
|
|
|
|
/// <summary>
|
|
/// Per-chunk storage backing StepCache. Holds raw walk + swim masks and destination Z
|
|
/// values for each of 256 cells in a 16x16 chunk, plus build-time metadata (multis
|
|
/// version, multi-Z bitmap) and LRU bookkeeping.
|
|
/// </summary>
|
|
internal sealed class StepChunk
|
|
{
|
|
public const int CellsPerChunk = 256; // 16 x 16
|
|
|
|
/// <summary>Bit i of WalkMask[c] = "default walker can step from cell c to neighbor (Direction)i". Raw — no diagonal corner-cut applied here.</summary>
|
|
public readonly byte[] WalkMask = new byte[CellsPerChunk];
|
|
|
|
/// <summary>Bit i of WetMask[c] = "swim-only mob can step from cell c to neighbor (Direction)i". Layered with WalkMask via canSwim/cantWalk capability flags.</summary>
|
|
public readonly byte[] WetMask = new byte[CellsPerChunk];
|
|
|
|
public readonly sbyte[] SourceZ = new sbyte[CellsPerChunk];
|
|
|
|
public readonly sbyte[] WalkZN = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] WalkZNE = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] WalkZE = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] WalkZSE = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] WalkZS = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] WalkZSW = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] WalkZW = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] WalkZNW = new sbyte[CellsPerChunk];
|
|
|
|
public readonly sbyte[] SwimZN = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] SwimZNE = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] SwimZE = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] SwimZSE = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] SwimZS = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] SwimZSW = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] SwimZW = new sbyte[CellsPerChunk];
|
|
public readonly sbyte[] SwimZNW = new sbyte[CellsPerChunk];
|
|
|
|
/// <summary>
|
|
/// 32 bytes = 256 bits when allocated. Lazy: most chunks are entirely single-Z,
|
|
/// so we only pay the 32 bytes on chunks that actually need it.
|
|
/// </summary>
|
|
private byte[] _multiZCells;
|
|
|
|
/// <summary>Snapshot of Sector.MultisVersion at the time this chunk was built.</summary>
|
|
public int BuiltMultisVersion;
|
|
|
|
/// <summary>Updated on every cache hit/miss. Used by LRU fallback eviction.</summary>
|
|
public long LastTouchedTicks;
|
|
|
|
public bool IsCellMultiZ(int cellIndex) =>
|
|
_multiZCells != null && (_multiZCells[cellIndex >> 3] & (1 << (cellIndex & 7))) != 0;
|
|
|
|
public void MarkCellMultiZ(int cellIndex)
|
|
{
|
|
_multiZCells ??= new byte[32];
|
|
_multiZCells[cellIndex >> 3] |= (byte)(1 << (cellIndex & 7));
|
|
}
|
|
|
|
/// <summary>
|
|
/// Serialization hook for <see cref="StepCacheFile"/>: returns the multi-Z bitmap,
|
|
/// or null if no cells in this chunk are multi-Z. Read-only — callers must not mutate.
|
|
/// </summary>
|
|
internal byte[] GetMultiZCellsForSerialization() => _multiZCells;
|
|
|
|
/// <summary>
|
|
/// Deserialization hook: assigns the multi-Z bitmap from a file load. Caller is
|
|
/// responsible for passing a 32-byte array (or null for "no cells multi-Z").
|
|
/// </summary>
|
|
internal void RestoreMultiZCellsFromSerialization(byte[] multiZ) => _multiZCells = multiZ;
|
|
}
|