Started as an allocation pass over `StepCache` and grew into a cleanup of the surrounding pathing engine. Four commits, each independently reviewable; net **−560 lines**. Build clean (0 warnings). All 122 `Server.Tests.Pathfinding` tests pass. --- ## 1. `perf`: pool the strata buffer, cut a hot-path dictionary lookup **The headline is that `TryGetMask` — the actual hot path — was already allocation-free.** `StepMask` is a readonly struct, `StaticTileEnumerable` is a `ref struct`, `ChunkMissState` is a struct in a `Dictionary`. So most of this is a bake-throughput and GC-churn win, with one exception noted below. `BuildChunk` accumulated packed multi-Z strata into a `List<byte>` that grew by doubling (256 → 512 → 1024 → …) and then paid a final `ToArray()`. A full map bake runs it ~114k times. It now writes into a `byte[]` rented from `STArrayPool<byte>.Shared` through a span writer, and hands the chunk one exact-size copy. **This required fixing a latent out-of-bounds guard.** The record-fit check reserved headroom for **8** strata (`StratumByteLength * 8`) while `ComputeStandableSurfaceZs` can return up to **16** — so a cell could write 305 bytes starting from a 65,383-byte offset. Against a `List` that was benign (it just grew past 64 KB, and emitted offsets stayed under the `NoStrata` sentinel). Against a fixed-size rented buffer it is an out-of-bounds write, so tightening it was a *prerequisite* for the pooling, not a drive-by. The guard is now exact, which additionally proves no emitted offset can collide with `NoStrata == ushort.MaxValue`. **One genuine query-path win:** `ShouldPromoteAfterMiss` did *two* dictionary lookups per miss — a `TryGetValue`, then an indexer assignment that re-hashes and re-probes. It now mutates in place via `CollectionsMarshal.GetValueRefOrNullRef`. This runs on every uncached chunk touch during A* expansion. The window-expiry branch keeps its explicit early return, so `MissPromotionThreshold == 1` still resets rather than promoting. Also dropped `StepProbe.ComputeStrataAt` / `ComputedStratum` (dead code, zero callers) and collapsed six 18-argument `new StepMask(0, 0, …, kind)` blocks into `Fallthrough(kind)`. **Considered and rejected:** pooling the `Direction[]` that `Find` returns. It *escapes* the call — `MovementPath` holds it across ticks while `PathFollower` walks `m_Index` through it — so it cannot be rented-and-returned, and it cannot be borrowed from the shared `BitmapAStarAlgorithm.Instance` without one creature clobbering another's in-flight path. `CheckPath` rate-limits repaths to one per 2s per creature, putting this at roughly 60 KB/sec at 1,000 pathing creatures. Not worth a public API break plus a use-after-return footgun. ## 2. `docs`: rewrite the comments for publication The comments had accumulated as development notes: internal phase jargon (`Tier 4`, `the Phase-2 synthesizer`), change narration aimed at a reviewer (`which the old ComputeStandingZ anchor missed`, `legacy behavior`), benchmark anecdotes (`benchmarked as near-optimal`, `a ~20 ns lookup`), and paragraphs restating the code. Rewritten to keep the rationale you cannot recover by reading the code — why the source-Z guard cannot be widened, why multis fall through with a halo, why the promotion gate counts Finds rather than calls, why `ComputeFingerprint` must hash the *files* and not the live tile tables — and drop the history that got us there. Three comments were **factually wrong**, not just wordy: - `CacheEvictionTimer` and `CacheStats` documented a class called `StaticWalkabilityCache`. No such class exists — it is `StepCache`. - `StepCacheFile` declared `File layout v8` while `FormatVersion` is 9, and called the current record layout "the v6 layout" in four places. The layout descriptions are now unversioned so they cannot drift again. - `StepProbe.ComputeStandingZ` claimed `StepCache` uses it to bake `SourceZ`. It has not since the baker moved to the clearance-aware `ComputeStandableSurfaceZs`; only a parity test calls it. ## 3. `refactor`: simplify `StepCacheFile.Write`, consolidate the format tests `SaveToFile` walked `_keysList` **twice** — once to count the map's chunks, then again through a `ChunkEnumerator` closure to emit them — because `Write` needed the count up front to size its index array. Both loops had the same root cause. Passing a **span** collapses them: the count is just `span.Length`. That deletes the `ChunkEnumerator` delegate, the closure over the list enumerator, and **both `InvalidOperationException` throws**, which existed only to police the delegate's "yield exactly `chunkCount` chunks" contract — a contract a span makes unrepresentable. `Write` now patches the header's `IndexOffset` by seeking back to it rather than reaching into the writer's live buffer with `BinaryPrimitives`. That also retires `IndexOffsetFieldPosition`, a hand-maintained byte offset that had to track the header layout, and sidesteps the stale-array hazard that motivated the manual patch (`BufferWriter` reallocates on growth). **Tests:** `StepCacheFileV6/V7/V8Tests` were named for the format version that introduced each transform — and the format is now **v9**, so all three names described formats the loader rejects outright. Beyond triplicated builders and plumbing, two things were actually broken: - The three near-identical rejection tests each cited a `MinSupportedVersion` that had since moved (`"version 5 < MinSupportedVersion 6"`, `"6 < 7"`, `"7 < 8"`). They passed for the wrong reason. - `AssertBaseEqual` (used by V7 and V8) **silently skipped the swim and strata trailers**. A regression dropping either would not have failed those tests. Now one `StepCacheFileFormatTests`, named for behavior — predictive-Z elision, compression, compact index — with a single `AssertIdentical` that does check both trailers, the three rejection tests folded into one theory that also covers a future version, and a zero-chunk case the delegate-based writer never had coverage for. ## 4. `test`: consolidate the parity and lifecycle tests Three files tested "parity" and none of the names said *which*. They were three different layers, and the seams are the useful part, so they are now one `StepCacheParityTests` that names them: | Test | Compares | Answers | |---|---|---| | `ProbeMatchesSlowPath` | StepProbe vs MovementImpl | Is the bake right? | | `CacheMatchesProbe` | StepCache vs StepProbe | Is it stored and returned intact? | | `CacheServesReachableWalkStates` | StepCache vs MovementImpl | End to end, over the states A* visits | Merging removed a duplicated stub `Mobile`, duplicated region seeds, and a filename/class mismatch (`StepProbeParityTests.cs` declared `StaticWalkabilityParityTests`). `SwimBake_ProducesWetCells` moved with it — it lived in the cache parity file but never touched the cache. Tests reached into `StepCache._chunks` via `GetField` in **9 places**, each rebuilding the key encoding and cell-index arithmetic by hand. `StepCache` now exposes `GetResidentChunk` and `ResidentIndexInSync` alongside the internal test hooks it already had (`LazyReaderHasChunk`, `CurrentFindGeneration`), and the shared arithmetic moved to `PathingTestSupport`. All 9 reflection blocks are gone. `StepCacheLifecycleTests` is regrouped by what it covers — promotion gate, fallthrough routes, strata, swim layer, eviction — with the `Tier4*` names dropped. Removed `Singleton_IsAvailable`, which asserted an inline-initialized static property was not null; that is the entire 123 → 122 test-count delta. --- ## Verification Tests were mutation-checked rather than just run, since round-trip and parity tests can pass while a transform silently no-ops: - Injecting an off-by-one into the `IndexOffset` patch fails **15 of 123** — the format tests are load-bearing. - Offsetting the cache's cell index by one fails **7 of 10** parity cases, and the 3 that stay green are exactly the ones that do not touch the cache. The layering localizes a fault rather than just reporting one.
308 lines
13 KiB
C#
308 lines
13 KiB
C#
using System;
|
|
using System.Diagnostics;
|
|
using System.IO;
|
|
using Server.Engines.Pathing.Cache;
|
|
using Server.Logging;
|
|
|
|
namespace Server.Engines.Pathing;
|
|
|
|
/// <summary>
|
|
/// Admin commands for inspecting and operating the pathfinding step cache.
|
|
/// [PathCacheStats — resident-chunk count and hit/miss/eviction telemetry.
|
|
/// [PathCacheClear — drop all cached chunks, close the .swb readers, zero the counters.
|
|
/// [PathBake — build a map's full static cache and save it.
|
|
/// [PathCacheSave — persist the resident chunks to Data/Pathfinding/<mapId>.swb.
|
|
/// [PathCacheLoad — open those files as backing stores. Also runs at startup.
|
|
/// [PathRecord — toggle capture of pathfind telemetry.
|
|
///
|
|
/// None of this is required: the cache builds chunks on demand as creatures path, with or without
|
|
/// a .swb on disk. Baking one is purely an optimization that trades disk and a few minutes of bake
|
|
/// time for the removal of first-pathfind-after-boot latency.
|
|
/// </summary>
|
|
public static class PathCacheCommands
|
|
{
|
|
private static readonly ILogger logger = LogFactory.GetLogger(typeof(PathCacheCommands));
|
|
|
|
// When set, Initialize() bakes any missing or stale .swb at startup. ConfigurePrompts() asks
|
|
// for it on first boot.
|
|
private const string PrebakeSetting = "pathfinding.prebakeMaps";
|
|
|
|
private static string PathFor(int mapId) =>
|
|
Path.Combine(Core.BaseDirectory, "Data", "Pathfinding", $"{mapId}.swb");
|
|
|
|
public static void Configure()
|
|
{
|
|
// Resident-chunk cap, shard-tunable — the default works out to roughly 40 MB. Written back
|
|
// to server.cfg on first boot so it's discoverable.
|
|
StepCache.Instance.MaxResidentChunks = ServerConfiguration.GetOrUpdateSetting(
|
|
"pathfinding.maxResidentChunks",
|
|
8192
|
|
);
|
|
|
|
PathfindRecorder.Configure();
|
|
|
|
CommandSystem.Register("PathCacheStats", AccessLevel.Administrator, OnPathCacheStats);
|
|
CommandSystem.Register("PathCacheClear", AccessLevel.Administrator, OnPathCacheClear);
|
|
CommandSystem.Register("PathBake", AccessLevel.Administrator, OnPathBake);
|
|
CommandSystem.Register("PathCacheSave", AccessLevel.Administrator, OnPathCacheSave);
|
|
CommandSystem.Register("PathCacheLoad", AccessLevel.Administrator, OnPathCacheLoad);
|
|
CommandSystem.Register("PathRecord", AccessLevel.Administrator, OnPathRecord);
|
|
AutoLoadAtStartup();
|
|
}
|
|
|
|
/// <summary>
|
|
/// Asks, once, whether to pre-bake the .swb cache; <see cref="Initialize"/> does the work later.
|
|
/// The answer persists, so the question is never repeated, and it's skipped entirely when input
|
|
/// is redirected — a headless or CI boot sets <see cref="PrebakeSetting"/> directly instead.
|
|
///
|
|
/// Runs in the ConfigurePrompts phase because that's the one window where content can prompt:
|
|
/// assemblies are loaded, but Serilog hasn't started, so console output won't interleave with
|
|
/// async log writes.
|
|
/// </summary>
|
|
public static void ConfigurePrompts()
|
|
{
|
|
if (ServerConfiguration.GetSetting(PrebakeSetting, (string)null) != null || Console.IsInputRedirected)
|
|
{
|
|
return;
|
|
}
|
|
|
|
Console.WriteLine();
|
|
Console.WriteLine("Pre-bake the pathfinding cache for your selected maps now?");
|
|
Console.WriteLine(" Bakes each map's .swb so there is zero first-pathfind-after-boot latency.");
|
|
Console.WriteLine(" Takes several minutes and ~tens of MB of disk per facet. You can also do");
|
|
Console.WriteLine(" this later at runtime with [PathBake.");
|
|
Console.Write("Pre-bake now? [y/N] ");
|
|
|
|
var answer = Console.ReadLine()?.Trim();
|
|
var prebake = answer?.StartsWith("y", StringComparison.OrdinalIgnoreCase) == true;
|
|
|
|
ServerConfiguration.SetSetting(PrebakeSetting, prebake);
|
|
}
|
|
|
|
/// <summary>
|
|
/// Bakes any map whose <c>.swb</c> is missing or stale, when <see cref="PrebakeSetting"/> is
|
|
/// set. Runs in the Initialize phase, once the tile matrix and world are loaded. An up-to-date
|
|
/// cache makes it a no-op, so the cost lands only on a first boot or after a client or map
|
|
/// update moves the fingerprint.
|
|
///
|
|
/// A map is judged up-to-date by whether it has an open reader. <see cref="AutoLoadAtStartup"/>
|
|
/// already ran in the earlier Configure phase and only opens a reader for a .swb whose
|
|
/// fingerprint validates, so an open reader is proof of a good bake — no need to fingerprint
|
|
/// the map a second time here.
|
|
/// </summary>
|
|
public static void Initialize()
|
|
{
|
|
if (!ServerConfiguration.GetSetting(PrebakeSetting, false))
|
|
{
|
|
return;
|
|
}
|
|
|
|
var baked = 0;
|
|
for (var i = 0; i < Map.Maps.Length; i++)
|
|
{
|
|
var map = Map.Maps[i];
|
|
if (map == null || map == Map.Internal)
|
|
{
|
|
continue;
|
|
}
|
|
|
|
if (StepCache.Instance.HasLazyReader(map.MapID))
|
|
{
|
|
continue; // already has a fingerprint-valid .swb open
|
|
}
|
|
|
|
var path = PathFor(map.MapID);
|
|
|
|
logger.Information(
|
|
"PathBake: pre-baking map {MapId} (pathfinding.prebakeMaps) — this can take several minutes...",
|
|
map.MapID
|
|
);
|
|
StepCache.Instance.BakeMap(map.MapID, path);
|
|
StepCache.Instance.ClearResidentChunks();
|
|
baked++;
|
|
}
|
|
|
|
if (baked > 0)
|
|
{
|
|
logger.Information("PathBake: pre-bake complete ({Count} map(s) written).", baked);
|
|
AutoLoadAtStartup(); // reopen what we just wrote
|
|
}
|
|
}
|
|
|
|
/// <summary>
|
|
/// Opens Data/Pathfinding/<mapId>.swb as a backing store for every map. Only the header and
|
|
/// index are read up front; chunk records are fetched as the cache asks for them, so resident
|
|
/// memory stays bounded by the LRU cap however large the files are.
|
|
/// </summary>
|
|
private static void AutoLoadAtStartup()
|
|
{
|
|
for (var i = 0; i < Map.Maps.Length; i++)
|
|
{
|
|
var map = Map.Maps[i];
|
|
if (map == null || map == Map.Internal)
|
|
{
|
|
continue;
|
|
}
|
|
StepCache.Instance.TryOpenLazyReader(PathFor(map.MapID), map.MapID);
|
|
}
|
|
}
|
|
|
|
[Usage("PathCacheStats")]
|
|
[Description("Reports StepCache resident-chunk count and hit/miss/eviction telemetry.")]
|
|
private static void OnPathCacheStats(CommandEventArgs e)
|
|
{
|
|
var stats = StepCache.Instance.GetStats();
|
|
var from = e.Mobile;
|
|
|
|
from.SendMessage($"StepCache: {stats.ResidentChunks} chunks resident");
|
|
from.SendMessage($" builds={stats.BuildsTotal} hits={stats.Hits}");
|
|
from.SendMessage($" miss(notBuilt)={stats.MissesNotBuilt} miss(dirty)={stats.MissesDirtyRebuild}");
|
|
from.SendMessage($" fallthru(multiZ)={stats.FallthroughMultiZ} fallthru(offMap)={stats.FallthroughOffMap} fallthru(srcZ)={stats.FallthroughSourceZMismatch}");
|
|
from.SendMessage($" fallthru(multi)={stats.FallthroughMulti} fallthru(notBuilt)={stats.FallthroughNotBuilt}");
|
|
from.SendMessage($" multiLocalHits={stats.MultiLocalHits}");
|
|
from.SendMessage($" evictions(lruCap)={stats.EvictionsByLruCap}");
|
|
}
|
|
|
|
[Usage("PathCacheClear")]
|
|
[Description("Drops all StepCache resident chunks and zeros the telemetry counters.")]
|
|
private static void OnPathCacheClear(CommandEventArgs e)
|
|
{
|
|
var residentBefore = StepCache.Instance.GetStats().ResidentChunks;
|
|
StepCache.Instance.Clear();
|
|
e.Mobile.SendMessage($"StepCache cleared: {residentBefore} chunks dropped, counters reset.");
|
|
}
|
|
|
|
[Usage("PathBake [mapId]")]
|
|
[Description("Walks every chunk of the given map (or all loaded maps) building the full static step cache, then saves it to Data/Pathfinding/<mapId>.swb so a future boot has zero first-pathfind latency. WARNING: blocks the game loop for several seconds and transiently uses hundreds of MB per map — run during maintenance, not peak hours.")]
|
|
private static void OnPathBake(CommandEventArgs e)
|
|
{
|
|
var from = e.Mobile;
|
|
int? only = e.Arguments.Length > 0 && int.TryParse(e.Arguments[0], out var parsed) ? parsed : null;
|
|
|
|
from.SendMessage("PathBake: building the static step cache. The server will pause briefly per map...");
|
|
|
|
var totalChunks = 0;
|
|
var totalMaps = 0;
|
|
var sw = Stopwatch.StartNew();
|
|
|
|
for (var i = 0; i < Map.Maps.Length; i++)
|
|
{
|
|
var map = Map.Maps[i];
|
|
if (map == null || map == Map.Internal || only.HasValue && map.MapID != only.Value)
|
|
{
|
|
continue;
|
|
}
|
|
|
|
// BakeMap leaves every chunk it built resident. Drop them between maps so peak memory
|
|
// is one map's worth rather than all of them, and the footprint afterwards is back
|
|
// under the LRU cap.
|
|
var written = StepCache.Instance.BakeMap(map.MapID, PathFor(map.MapID));
|
|
StepCache.Instance.ClearResidentChunks();
|
|
|
|
if (written > 0)
|
|
{
|
|
totalChunks += written;
|
|
totalMaps++;
|
|
from.SendMessage($" map {map.MapID}: {written} chunks → {PathFor(map.MapID)}");
|
|
}
|
|
}
|
|
|
|
sw.Stop();
|
|
|
|
if (totalMaps == 0)
|
|
{
|
|
from.SendMessage(only.HasValue ? $"PathBake: map {only.Value} not loaded." : "PathBake: no maps to bake.");
|
|
return;
|
|
}
|
|
|
|
// Reopen what we just wrote, so the bake is usable immediately without a restart.
|
|
AutoLoadAtStartup();
|
|
from.SendMessage($"PathBake: {totalChunks} chunks across {totalMaps} map(s) in {sw.Elapsed.TotalSeconds:F1}s; lazy readers reopened.");
|
|
}
|
|
|
|
[Usage("PathCacheSave")]
|
|
[Description("Persists resident StepCache chunks for every loaded map to Data/Pathfinding/<mapId>.swb.")]
|
|
private static void OnPathCacheSave(CommandEventArgs e)
|
|
{
|
|
var totalChunks = 0;
|
|
var totalMaps = 0;
|
|
for (var i = 0; i < Map.Maps.Length; i++)
|
|
{
|
|
var map = Map.Maps[i];
|
|
if (map == null || map == Map.Internal)
|
|
{
|
|
continue;
|
|
}
|
|
var path = PathFor(map.MapID);
|
|
var written = StepCache.Instance.SaveToFile(path, map.MapID);
|
|
if (written > 0)
|
|
{
|
|
totalChunks += written;
|
|
totalMaps++;
|
|
e.Mobile.SendMessage($" map {map.MapID}: {written} chunks → {path}");
|
|
}
|
|
}
|
|
e.Mobile.SendMessage($"StepCache saved: {totalChunks} chunks across {totalMaps} map(s).");
|
|
}
|
|
|
|
[Usage("PathCacheLoad")]
|
|
[Description("Opens Data/Pathfinding/<mapId>.swb as a lazy backing store for every map. Chunks are fetched on demand, so RAM stays bounded by the LRU cap regardless of file size.")]
|
|
private static void OnPathCacheLoad(CommandEventArgs e)
|
|
{
|
|
var openedMaps = 0;
|
|
for (var i = 0; i < Map.Maps.Length; i++)
|
|
{
|
|
var map = Map.Maps[i];
|
|
if (map == null || map == Map.Internal)
|
|
{
|
|
continue;
|
|
}
|
|
if (StepCache.Instance.TryOpenLazyReader(PathFor(map.MapID), map.MapID))
|
|
{
|
|
openedMaps++;
|
|
}
|
|
}
|
|
e.Mobile.SendMessage(
|
|
$"StepCache: opened {openedMaps} map(s) for lazy loading (total readers open: {StepCache.Instance.OpenLazyReaderCount})."
|
|
);
|
|
}
|
|
|
|
[Usage("PathRecord [on|off|flush|status]")]
|
|
[Description("Toggles PathfindRecorder. With no arg, reports state. 'on' enables JSONL capture of every Find call; 'off' disables and flushes; 'flush' forces a buffer flush without disabling.")]
|
|
private static void OnPathRecord(CommandEventArgs e)
|
|
{
|
|
var arg = (e.Arguments.Length > 0 ? e.Arguments[0] : "status").ToLowerInvariant();
|
|
var from = e.Mobile;
|
|
switch (arg)
|
|
{
|
|
case "on":
|
|
{
|
|
PathfindRecorder.SetEnabled(true);
|
|
from.SendMessage(PathfindRecorder.Enabled
|
|
? $"PathRecord: ON, writing to {PathfindRecorder.OutputPath}"
|
|
: "PathRecord: enable failed (see server log)");
|
|
break;
|
|
}
|
|
case "off":
|
|
{
|
|
PathfindRecorder.SetEnabled(false);
|
|
from.SendMessage("PathRecord: OFF");
|
|
break;
|
|
}
|
|
case "flush":
|
|
{
|
|
PathfindRecorder.Flush();
|
|
from.SendMessage($"PathRecord: flushed ({PathfindRecorder.RecordsWritten} records this session)");
|
|
break;
|
|
}
|
|
default:
|
|
{
|
|
from.SendMessage(
|
|
$"PathRecord: {(PathfindRecorder.Enabled ? "ON" : "OFF")}, "
|
|
+ $"path={PathfindRecorder.OutputPath}, records={PathfindRecorder.RecordsWritten}"
|
|
);
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
}
|