ModernUO/Projects/UOContent/Engines/Pathing/PathDiag.cs
Kamron Batman b852bca41e
perf(pathing): pool the StepCache strata buffer, then clean up the pathing engine around it (#2523)
Started as an allocation pass over `StepCache` and grew into a cleanup of the surrounding pathing engine. Four commits, each independently reviewable; net **−560 lines**.

Build clean (0 warnings). All 122 `Server.Tests.Pathfinding` tests pass.

---

## 1. `perf`: pool the strata buffer, cut a hot-path dictionary lookup

**The headline is that `TryGetMask` — the actual hot path — was already allocation-free.** `StepMask` is a readonly struct, `StaticTileEnumerable` is a `ref struct`, `ChunkMissState` is a struct in a `Dictionary`. So most of this is a bake-throughput and GC-churn win, with one exception noted below.

`BuildChunk` accumulated packed multi-Z strata into a `List<byte>` that grew by doubling (256 → 512 → 1024 → …) and then paid a final `ToArray()`. A full map bake runs it ~114k times. It now writes into a `byte[]` rented from `STArrayPool<byte>.Shared` through a span writer, and hands the chunk one exact-size copy.

**This required fixing a latent out-of-bounds guard.** The record-fit check reserved headroom for **8** strata (`StratumByteLength * 8`) while `ComputeStandableSurfaceZs` can return up to **16** — so a cell could write 305 bytes starting from a 65,383-byte offset. Against a `List` that was benign (it just grew past 64 KB, and emitted offsets stayed under the `NoStrata` sentinel). Against a fixed-size rented buffer it is an out-of-bounds write, so tightening it was a *prerequisite* for the pooling, not a drive-by. The guard is now exact, which additionally proves no emitted offset can collide with `NoStrata == ushort.MaxValue`.

**One genuine query-path win:** `ShouldPromoteAfterMiss` did *two* dictionary lookups per miss — a `TryGetValue`, then an indexer assignment that re-hashes and re-probes. It now mutates in place via `CollectionsMarshal.GetValueRefOrNullRef`. This runs on every uncached chunk touch during A* expansion. The window-expiry branch keeps its explicit early return, so `MissPromotionThreshold == 1` still resets rather than promoting.

Also dropped `StepProbe.ComputeStrataAt` / `ComputedStratum` (dead code, zero callers) and collapsed six 18-argument `new StepMask(0, 0, …, kind)` blocks into `Fallthrough(kind)`.

**Considered and rejected:** pooling the `Direction[]` that `Find` returns. It *escapes* the call — `MovementPath` holds it across ticks while `PathFollower` walks `m_Index` through it — so it cannot be rented-and-returned, and it cannot be borrowed from the shared `BitmapAStarAlgorithm.Instance` without one creature clobbering another's in-flight path. `CheckPath` rate-limits repaths to one per 2s per creature, putting this at roughly 60 KB/sec at 1,000 pathing creatures. Not worth a public API break plus a use-after-return footgun.

## 2. `docs`: rewrite the comments for publication

The comments had accumulated as development notes: internal phase jargon (`Tier 4`, `the Phase-2 synthesizer`), change narration aimed at a reviewer (`which the old ComputeStandingZ anchor missed`, `legacy behavior`), benchmark anecdotes (`benchmarked as near-optimal`, `a ~20 ns lookup`), and paragraphs restating the code.

Rewritten to keep the rationale you cannot recover by reading the code — why the source-Z guard cannot be widened, why multis fall through with a halo, why the promotion gate counts Finds rather than calls, why `ComputeFingerprint` must hash the *files* and not the live tile tables — and drop the history that got us there.

Three comments were **factually wrong**, not just wordy:

- `CacheEvictionTimer` and `CacheStats` documented a class called `StaticWalkabilityCache`. No such class exists — it is `StepCache`.
- `StepCacheFile` declared `File layout v8` while `FormatVersion` is 9, and called the current record layout "the v6 layout" in four places. The layout descriptions are now unversioned so they cannot drift again.
- `StepProbe.ComputeStandingZ` claimed `StepCache` uses it to bake `SourceZ`. It has not since the baker moved to the clearance-aware `ComputeStandableSurfaceZs`; only a parity test calls it.

## 3. `refactor`: simplify `StepCacheFile.Write`, consolidate the format tests

`SaveToFile` walked `_keysList` **twice** — once to count the map's chunks, then again through a `ChunkEnumerator` closure to emit them — because `Write` needed the count up front to size its index array. Both loops had the same root cause. Passing a **span** collapses them: the count is just `span.Length`.

That deletes the `ChunkEnumerator` delegate, the closure over the list enumerator, and **both `InvalidOperationException` throws**, which existed only to police the delegate's "yield exactly `chunkCount` chunks" contract — a contract a span makes unrepresentable.

`Write` now patches the header's `IndexOffset` by seeking back to it rather than reaching into the writer's live buffer with `BinaryPrimitives`. That also retires `IndexOffsetFieldPosition`, a hand-maintained byte offset that had to track the header layout, and sidesteps the stale-array hazard that motivated the manual patch (`BufferWriter` reallocates on growth).

**Tests:** `StepCacheFileV6/V7/V8Tests` were named for the format version that introduced each transform — and the format is now **v9**, so all three names described formats the loader rejects outright. Beyond triplicated builders and plumbing, two things were actually broken:

- The three near-identical rejection tests each cited a `MinSupportedVersion` that had since moved (`"version 5 < MinSupportedVersion 6"`, `"6 < 7"`, `"7 < 8"`). They passed for the wrong reason.
- `AssertBaseEqual` (used by V7 and V8) **silently skipped the swim and strata trailers**. A regression dropping either would not have failed those tests.

Now one `StepCacheFileFormatTests`, named for behavior — predictive-Z elision, compression, compact index — with a single `AssertIdentical` that does check both trailers, the three rejection tests folded into one theory that also covers a future version, and a zero-chunk case the delegate-based writer never had coverage for.

## 4. `test`: consolidate the parity and lifecycle tests

Three files tested "parity" and none of the names said *which*. They were three different layers, and the seams are the useful part, so they are now one `StepCacheParityTests` that names them:

| Test | Compares | Answers |
|---|---|---|
| `ProbeMatchesSlowPath` | StepProbe vs MovementImpl | Is the bake right? |
| `CacheMatchesProbe` | StepCache vs StepProbe | Is it stored and returned intact? |
| `CacheServesReachableWalkStates` | StepCache vs MovementImpl | End to end, over the states A* visits |

Merging removed a duplicated stub `Mobile`, duplicated region seeds, and a filename/class mismatch (`StepProbeParityTests.cs` declared `StaticWalkabilityParityTests`). `SwimBake_ProducesWetCells` moved with it — it lived in the cache parity file but never touched the cache.

Tests reached into `StepCache._chunks` via `GetField` in **9 places**, each rebuilding the key encoding and cell-index arithmetic by hand. `StepCache` now exposes `GetResidentChunk` and `ResidentIndexInSync` alongside the internal test hooks it already had (`LazyReaderHasChunk`, `CurrentFindGeneration`), and the shared arithmetic moved to `PathingTestSupport`. All 9 reflection blocks are gone.

`StepCacheLifecycleTests` is regrouped by what it covers — promotion gate, fallthrough routes, strata, swim layer, eviction — with the `Tier4*` names dropped. Removed `Singleton_IsAvailable`, which asserted an inline-initialized static property was not null; that is the entire 123 → 122 test-count delta.

---

## Verification

Tests were mutation-checked rather than just run, since round-trip and parity tests can pass while a transform silently no-ops:

- Injecting an off-by-one into the `IndexOffset` patch fails **15 of 123** — the format tests are load-bearing.
- Offsetting the cache's cell index by one fails **7 of 10** parity cases, and the 3 that stay green are exactly the ones that do not touch the cache. The layering localizes a fault rather than just reporting one.
2026-07-12 20:02:29 -07:00

197 lines
8.8 KiB
C#

using System;
using System.Diagnostics;
using System.IO;
using Server.Engines.Pathing.Cache;
using Server.PathAlgorithms;
using Server.Targeting;
namespace Server.Engines.Pathing;
/// <summary>
/// Diagnoses why the step cache does or doesn't serve a given route. Stand where the creature
/// would start, run <c>[PathDiag</c>, target the goal; the full report lands in
/// <c>Logs/pathdiag.log</c> and a summary goes to the client. It reports the tile makeup of the
/// start and goal cells alongside the standable surfaces the baker anchors to, the cache
/// hit/fallthrough breakdown for one warm Find, and warm timings.
///
/// The fallthrough fraction is the number to read: a high one means the cache is paying for a
/// lookup on every cell and then taking the slow path anyway. That usually points at
/// static-over-land geometry — dungeon walkways, bridges, stairs, stacked floors — baking at the
/// wrong Z, which is why this is most useful when bringing up a custom map or facet.
///
/// The promotion gate is forced eager for the duration, so the numbers reflect the cache's best
/// case rather than an artifact of chunks not having been built yet.
/// </summary>
public static class PathDiag
{
private const int TimingIterations = 200;
private static string LogPath => Path.Combine(Core.BaseDirectory, "Logs", "pathdiag.log");
public static void Configure()
{
CommandSystem.Register("PathDiag", AccessLevel.Administrator, OnPathDiag);
}
[Usage("PathDiag")]
[Description("Diagnoses the step cache for a route (results appended to Logs/pathdiag.log): target a tile to record start/goal tile makeup, the per-Find cache hit/fallthrough breakdown, and warm timing.")]
private static void OnPathDiag(CommandEventArgs e)
{
var start = e.Mobile.Location;
e.Mobile.SendMessage("PathDiag: target the goal tile.");
e.Mobile.BeginTarget(-1, true, TargetFlags.None, (from, targeted) => OnTarget(from, start, targeted));
}
private static void OnTarget(Mobile from, Point3D start, object targeted)
{
if (targeted is not IPoint3D p)
{
return;
}
var map = from.Map;
var goal = new Point3D(p.X, p.Y, p.Z);
if (!Utility.InRange(start, goal, 38))
{
from.SendMessage("PathDiag: goal is outside the A* search window (38 tiles); aborting.");
return;
}
var cache = StepCache.Instance;
var previousThreshold = cache.MissPromotionThreshold;
cache.MissPromotionThreshold = 1; // build eagerly, so we measure the cache's best case
StreamWriter log = null;
try
{
Directory.CreateDirectory(Path.GetDirectoryName(LogPath)!);
log = new StreamWriter(new FileStream(LogPath, FileMode.Append, FileAccess.Write, FileShare.Read));
log.WriteLine($"===== [{Core.Now:yyyy-MM-dd HH:mm:ss}] PathDiag ({start.X},{start.Y},{start.Z}) -> ({goal.X},{goal.Y},{goal.Z}) on {map} =====");
DumpCell(log, map, start.X, start.Y, start.Z, "start");
DumpCell(log, map, goal.X, goal.Y, goal.Z, "goal");
var find = RunInstrumentedFind(log, from, map, start, goal);
var (minUs, avgUs) = TimeWarm(log, from, map, start, goal);
log.WriteLine();
from.SendMessage($"PathDiag ({start.X},{start.Y},{start.Z})->({goal.X},{goal.Y},{goal.Z}): {find.result}");
from.SendMessage($" cache fallthrough {find.fallthroughPct:F1}% of {find.total}; warm min={minUs:F1}us avg={avgUs:F1}us");
from.SendMessage($" full report appended to Logs/pathdiag.log");
}
catch (IOException ex)
{
from.SendMessage($"PathDiag: failed to write {LogPath}: {ex.Message}");
}
finally
{
log?.Dispose();
cache.MissPromotionThreshold = previousThreshold;
}
}
/// <summary>
/// Dumps one cell's tiles and the surfaces the baker anchors to. A wide gap between the query Z
/// and every standable surface is the signature of a cell the cache can't serve.
/// </summary>
private static void DumpCell(TextWriter log, Map map, int x, int y, int queryZ, string label)
{
map.GetAverageZ(x, y, out var landZ, out var avgZ, out var landTop);
var landTile = map.Tiles.GetLandTile(x, y);
var landFlags = TileData.LandTable[landTile.ID & TileData.MaxLandValue].Flags;
var landImpassable = (landFlags & TileFlag.Impassable) != 0;
var landWet = (landFlags & TileFlag.Wet) != 0;
Span<sbyte> surfaces = stackalloc sbyte[16];
var surfaceCount = StepProbe.ComputeStandableSurfaceZs(map, x, y, surfaces);
log.WriteLine($"{label} cell ({x},{y}) queryZ={queryZ}:");
log.WriteLine($" land: avgZ={avgZ} landZ={landZ} landTop={landTop} impassable={landImpassable} wet={landWet} ignored={landTile.Ignored}");
var sb = new System.Text.StringBuilder();
for (var i = 0; i < surfaceCount; i++)
{
sb.Append(i == 0 ? "" : ",").Append(surfaces[i]);
}
log.WriteLine($" standable surfaces={surfaceCount} [{sb}] -> {(surfaceCount >= 2 ? "multi-Z (strata)" : "single-Z anchor")}");
log.WriteLine(" static/multi surfaces:");
foreach (var tile in map.Tiles.GetStaticAndMultiTiles(x, y))
{
var data = TileData.ItemTable[tile.ID & TileData.MaxItemValue];
log.WriteLine($" id=0x{tile.ID:X4} z={tile.Z} top={tile.Z + data.CalcHeight} h={data.Height} surface={data.Surface} impass={data.Impassable} bridge={data.Bridge} wet={data.Wet}");
}
}
/// <summary>
/// Runs one Find against a warm cache and reports the counter delta it produced — the
/// hit/fallthrough mix for that single pathfind.
/// </summary>
private static (string result, double fallthroughPct, long total) RunInstrumentedFind(
TextWriter log, Mobile from, Map map, Point3D start, Point3D goal
)
{
var cache = StepCache.Instance;
// Build every chunk the route touches first, so the measured Find below reports steady-state
// behaviour rather than first-touch misses.
for (var i = 0; i < 3; i++)
{
BitmapAStarAlgorithm.Instance.Find(from, map, start, goal);
}
var before = cache.GetStats();
var path = BitmapAStarAlgorithm.Instance.Find(from, map, start, goal);
var after = cache.GetStats();
var served = after.Hits - before.Hits
+ (after.MissesNotBuilt - before.MissesNotBuilt)
+ (after.MissesDirtyRebuild - before.MissesDirtyRebuild);
var fallthrough = after.FallthroughMultiZ - before.FallthroughMultiZ
+ (after.FallthroughSourceZMismatch - before.FallthroughSourceZMismatch)
+ (after.FallthroughOffMap - before.FallthroughOffMap)
+ (after.FallthroughNotBuilt - before.FallthroughNotBuilt);
var total = served + fallthrough;
var pct = total == 0 ? 0 : 100.0 * fallthrough / total;
var result = path == null ? "NO PATH" : $"{path.Length} steps";
log.WriteLine($"warm Find: {result}");
log.WriteLine($" cache-served={served} fallthrough={fallthrough} ({pct:F1}% of {total} probes)");
log.WriteLine($" fallthrough breakdown: multiZ={after.FallthroughMultiZ - before.FallthroughMultiZ} " +
$"srcZ={after.FallthroughSourceZMismatch - before.FallthroughSourceZMismatch} " +
$"offMap={after.FallthroughOffMap - before.FallthroughOffMap} " +
$"notBuilt={after.FallthroughNotBuilt - before.FallthroughNotBuilt}");
if (path == null)
{
log.WriteLine(" NO PATH: goal unreachable within the 38-tile window (or not standable). This is an A* scope limit, independent of the cache.");
}
return (result, pct, total);
}
private static (double minUs, double avgUs) TimeWarm(TextWriter log, Mobile from, Map map, Point3D start, Point3D goal)
{
var sw = new Stopwatch();
var minTicks = long.MaxValue;
long totalTicks = 0;
for (var i = 0; i < TimingIterations; i++)
{
sw.Restart();
BitmapAStarAlgorithm.Instance.Find(from, map, start, goal);
sw.Stop();
totalTicks += sw.ElapsedTicks;
if (sw.ElapsedTicks < minTicks)
{
minTicks = sw.ElapsedTicks;
}
}
var usPerTick = 1_000_000.0 / Stopwatch.Frequency;
var minUs = minTicks * usPerTick;
var avgUs = totalTicks * usPerTick / TimingIterations;
log.WriteLine($"timing over {TimingIterations} warm Finds: min={minUs:F1}us avg={avgUs:F1}us");
return (minUs, avgUs);
}
}