ModernUO/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheLifecycleTests.cs
Kamron Batman b852bca41e
perf(pathing): pool the StepCache strata buffer, then clean up the pathing engine around it (#2523)
Started as an allocation pass over `StepCache` and grew into a cleanup of the surrounding pathing engine. Four commits, each independently reviewable; net **−560 lines**.

Build clean (0 warnings). All 122 `Server.Tests.Pathfinding` tests pass.

---

## 1. `perf`: pool the strata buffer, cut a hot-path dictionary lookup

**The headline is that `TryGetMask` — the actual hot path — was already allocation-free.** `StepMask` is a readonly struct, `StaticTileEnumerable` is a `ref struct`, `ChunkMissState` is a struct in a `Dictionary`. So most of this is a bake-throughput and GC-churn win, with one exception noted below.

`BuildChunk` accumulated packed multi-Z strata into a `List<byte>` that grew by doubling (256 → 512 → 1024 → …) and then paid a final `ToArray()`. A full map bake runs it ~114k times. It now writes into a `byte[]` rented from `STArrayPool<byte>.Shared` through a span writer, and hands the chunk one exact-size copy.

**This required fixing a latent out-of-bounds guard.** The record-fit check reserved headroom for **8** strata (`StratumByteLength * 8`) while `ComputeStandableSurfaceZs` can return up to **16** — so a cell could write 305 bytes starting from a 65,383-byte offset. Against a `List` that was benign (it just grew past 64 KB, and emitted offsets stayed under the `NoStrata` sentinel). Against a fixed-size rented buffer it is an out-of-bounds write, so tightening it was a *prerequisite* for the pooling, not a drive-by. The guard is now exact, which additionally proves no emitted offset can collide with `NoStrata == ushort.MaxValue`.

**One genuine query-path win:** `ShouldPromoteAfterMiss` did *two* dictionary lookups per miss — a `TryGetValue`, then an indexer assignment that re-hashes and re-probes. It now mutates in place via `CollectionsMarshal.GetValueRefOrNullRef`. This runs on every uncached chunk touch during A* expansion. The window-expiry branch keeps its explicit early return, so `MissPromotionThreshold == 1` still resets rather than promoting.

Also dropped `StepProbe.ComputeStrataAt` / `ComputedStratum` (dead code, zero callers) and collapsed six 18-argument `new StepMask(0, 0, …, kind)` blocks into `Fallthrough(kind)`.

**Considered and rejected:** pooling the `Direction[]` that `Find` returns. It *escapes* the call — `MovementPath` holds it across ticks while `PathFollower` walks `m_Index` through it — so it cannot be rented-and-returned, and it cannot be borrowed from the shared `BitmapAStarAlgorithm.Instance` without one creature clobbering another's in-flight path. `CheckPath` rate-limits repaths to one per 2s per creature, putting this at roughly 60 KB/sec at 1,000 pathing creatures. Not worth a public API break plus a use-after-return footgun.

## 2. `docs`: rewrite the comments for publication

The comments had accumulated as development notes: internal phase jargon (`Tier 4`, `the Phase-2 synthesizer`), change narration aimed at a reviewer (`which the old ComputeStandingZ anchor missed`, `legacy behavior`), benchmark anecdotes (`benchmarked as near-optimal`, `a ~20 ns lookup`), and paragraphs restating the code.

Rewritten to keep the rationale you cannot recover by reading the code — why the source-Z guard cannot be widened, why multis fall through with a halo, why the promotion gate counts Finds rather than calls, why `ComputeFingerprint` must hash the *files* and not the live tile tables — and drop the history that got us there.

Three comments were **factually wrong**, not just wordy:

- `CacheEvictionTimer` and `CacheStats` documented a class called `StaticWalkabilityCache`. No such class exists — it is `StepCache`.
- `StepCacheFile` declared `File layout v8` while `FormatVersion` is 9, and called the current record layout "the v6 layout" in four places. The layout descriptions are now unversioned so they cannot drift again.
- `StepProbe.ComputeStandingZ` claimed `StepCache` uses it to bake `SourceZ`. It has not since the baker moved to the clearance-aware `ComputeStandableSurfaceZs`; only a parity test calls it.

## 3. `refactor`: simplify `StepCacheFile.Write`, consolidate the format tests

`SaveToFile` walked `_keysList` **twice** — once to count the map's chunks, then again through a `ChunkEnumerator` closure to emit them — because `Write` needed the count up front to size its index array. Both loops had the same root cause. Passing a **span** collapses them: the count is just `span.Length`.

That deletes the `ChunkEnumerator` delegate, the closure over the list enumerator, and **both `InvalidOperationException` throws**, which existed only to police the delegate's "yield exactly `chunkCount` chunks" contract — a contract a span makes unrepresentable.

`Write` now patches the header's `IndexOffset` by seeking back to it rather than reaching into the writer's live buffer with `BinaryPrimitives`. That also retires `IndexOffsetFieldPosition`, a hand-maintained byte offset that had to track the header layout, and sidesteps the stale-array hazard that motivated the manual patch (`BufferWriter` reallocates on growth).

**Tests:** `StepCacheFileV6/V7/V8Tests` were named for the format version that introduced each transform — and the format is now **v9**, so all three names described formats the loader rejects outright. Beyond triplicated builders and plumbing, two things were actually broken:

- The three near-identical rejection tests each cited a `MinSupportedVersion` that had since moved (`"version 5 < MinSupportedVersion 6"`, `"6 < 7"`, `"7 < 8"`). They passed for the wrong reason.
- `AssertBaseEqual` (used by V7 and V8) **silently skipped the swim and strata trailers**. A regression dropping either would not have failed those tests.

Now one `StepCacheFileFormatTests`, named for behavior — predictive-Z elision, compression, compact index — with a single `AssertIdentical` that does check both trailers, the three rejection tests folded into one theory that also covers a future version, and a zero-chunk case the delegate-based writer never had coverage for.

## 4. `test`: consolidate the parity and lifecycle tests

Three files tested "parity" and none of the names said *which*. They were three different layers, and the seams are the useful part, so they are now one `StepCacheParityTests` that names them:

| Test | Compares | Answers |
|---|---|---|
| `ProbeMatchesSlowPath` | StepProbe vs MovementImpl | Is the bake right? |
| `CacheMatchesProbe` | StepCache vs StepProbe | Is it stored and returned intact? |
| `CacheServesReachableWalkStates` | StepCache vs MovementImpl | End to end, over the states A* visits |

Merging removed a duplicated stub `Mobile`, duplicated region seeds, and a filename/class mismatch (`StepProbeParityTests.cs` declared `StaticWalkabilityParityTests`). `SwimBake_ProducesWetCells` moved with it — it lived in the cache parity file but never touched the cache.

Tests reached into `StepCache._chunks` via `GetField` in **9 places**, each rebuilding the key encoding and cell-index arithmetic by hand. `StepCache` now exposes `GetResidentChunk` and `ResidentIndexInSync` alongside the internal test hooks it already had (`LazyReaderHasChunk`, `CurrentFindGeneration`), and the shared arithmetic moved to `PathingTestSupport`. All 9 reflection blocks are gone.

`StepCacheLifecycleTests` is regrouped by what it covers — promotion gate, fallthrough routes, strata, swim layer, eviction — with the `Tier4*` names dropped. Removed `Singleton_IsAvailable`, which asserted an inline-initialized static property was not null; that is the entire 123 → 122 test-count delta.

---

## Verification

Tests were mutation-checked rather than just run, since round-trip and parity tests can pass while a transform silently no-ops:

- Injecting an off-by-one into the `IndexOffset` patch fails **15 of 123** — the format tests are load-bearing.
- Offsetting the cache's cell index by one fails **7 of 10** parity cases, and the 3 that stay green are exactly the ones that do not touch the cache. The layering localizes a fault rather than just reporting one.
2026-07-12 20:02:29 -07:00

424 lines
15 KiB
C#

using System.Collections.Generic;
using System.Reflection;
using System.Threading;
using Server.Engines.Pathing.Cache;
using Server.Items;
using Xunit;
using static Server.Tests.Pathfinding.PathingTestSupport;
namespace Server.Tests.Pathfinding;
/// <summary>
/// How the cache decides what to build, what to serve, and what to throw away: the promotion gate,
/// the four fallthrough routes out of <see cref="StepCache.TryGetMask"/>, the strata and swim
/// layers, and LRU eviction.
/// </summary>
[Collection("Sequential Pathfinding Tests")]
public class StepCacheLifecycleTests
{
/// <summary>Resets to a known state and returns the singleton.</summary>
private static StepCache FreshCache(int promotionThreshold)
{
var cache = StepCache.Instance;
cache.Clear();
cache.MissPromotionThreshold = promotionThreshold;
return cache;
}
/// <summary>Builds the plain chunk and hands it back for a test to inject state into.</summary>
private static StepChunk BuiltPlainChunk(StepCache cache, Map map)
{
cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10);
var chunk = cache.GetResidentChunk(map.MapID, PlainX >> 4, PlainY >> 4);
Assert.NotNull(chunk);
return chunk;
}
[Fact]
public void Clear_OnEmptyCache_LeavesStatsZero()
{
var stats = FreshCache(2).GetStats();
Assert.Equal(0, stats.ResidentChunks);
Assert.Equal(0L, stats.Hits);
Assert.Equal(0L, stats.BuildsTotal);
}
// ---- promotion gate ----
/// <summary>
/// A chunk nothing has shown sustained interest in must not be built. The caller reads
/// IsHit=false as "use the slow path", which is the cheaper trade for a pet crossing a chunk
/// once: BuildChunk costs far more than the handful of slow-path steps it would save.
/// </summary>
[Fact]
public void FirstTouch_DefersBuild_AndFallsThrough()
{
var cache = FreshCache(promotionThreshold: 2);
var map = TestMap;
var lookup = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10);
Assert.False(lookup.IsHit);
Assert.Equal(CacheHitKind.Fallthrough_NotBuilt, lookup.HitKind);
var stats = cache.GetStats();
Assert.Equal(0, stats.ResidentChunks);
Assert.Equal(0L, stats.BuildsTotal);
Assert.Equal(0L, stats.MissesNotBuilt);
Assert.Equal(1L, stats.FallthroughNotBuilt);
}
[SkippableFact]
public void SecondTouchInsideWindow_PromotesAndServes()
{
TileDataRequirement.SkipIfMissing();
var cache = FreshCache(promotionThreshold: 2);
var map = TestMap;
Assert.False(cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10).IsHit);
var promoted = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10);
Assert.True(promoted.IsHit);
Assert.Equal(CacheHitKind.Miss_NotBuilt, promoted.HitKind);
Assert.Equal((byte)0xC1, promoted.WalkMask); // pinned: open plain, walkable N/NE/... per the bake
Assert.Equal((sbyte)10, promoted.WalkZ_N);
var stats = cache.GetStats();
Assert.Equal(1, stats.ResidentChunks);
Assert.Equal(1L, stats.MissesNotBuilt);
Assert.Equal(1L, stats.BuildsTotal);
Assert.Equal(1L, stats.FallthroughNotBuilt);
// Now resident: a third query is a clean hit, not another miss.
var hit = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10);
Assert.Equal(CacheHitKind.Hit, hit.HitKind);
Assert.Equal((byte)0xC1, hit.WalkMask);
}
/// <summary>
/// Two touches spread wider than the window are not interest, they're coincidence — a chunk
/// someone glanced through, then an unrelated creature wandering past minutes later. The count
/// restarts rather than accumulating toward a build.
/// </summary>
[Fact]
public void SecondTouchAfterWindow_RestartsTheCount()
{
var cache = FreshCache(promotionThreshold: 2);
cache.MissPromotionWindowMs = 1;
var map = TestMap;
Assert.False(cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10).IsHit);
Thread.Sleep(20); // outrun the window
var second = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10);
Assert.False(second.IsHit);
Assert.Equal(CacheHitKind.Fallthrough_NotBuilt, second.HitKind);
Assert.Equal(0, cache.GetStats().ResidentChunks);
Assert.Equal(2L, cache.GetStats().FallthroughNotBuilt);
}
/// <summary>
/// The gate counts Finds, not probes. A single pathfind hits a chunk once per cell it expands
/// there, so counting probes would cross any threshold on the second cell and gate nothing at
/// all — the deferral would be dead code.
/// </summary>
[SkippableFact]
public void ManyProbesInOneFind_CountAsOneTouch()
{
TileDataRequirement.SkipIfMissing();
var cache = FreshCache(promotionThreshold: 2);
var map = TestMap;
cache.BeginFindGeneration();
for (var i = 0; i < 8; i++)
{
// Eight different cells, all inside the same chunk.
var lookup = cache.TryGetMask(map, PlainX + i, PlainY, sourceZ: 10);
Assert.False(lookup.IsHit);
Assert.Equal(CacheHitKind.Fallthrough_NotBuilt, lookup.HitKind);
}
Assert.Equal(0, cache.GetStats().ResidentChunks);
Assert.Equal(0L, cache.GetStats().BuildsTotal);
Assert.Equal(8L, cache.GetStats().FallthroughNotBuilt);
// A second Find is the second distinct touch, and crosses the threshold.
cache.BeginFindGeneration();
var promoted = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10);
Assert.Equal(CacheHitKind.Miss_NotBuilt, promoted.HitKind);
Assert.Equal(1, cache.GetStats().ResidentChunks);
Assert.Equal(1L, cache.GetStats().BuildsTotal);
}
/// <summary>Distinct Finds still don't promote if they straddle the window.</summary>
[Fact]
public void TwoFindsAcrossTheWindow_DoNotPromote()
{
var cache = FreshCache(promotionThreshold: 2);
cache.MissPromotionWindowMs = 1;
var map = TestMap;
cache.BeginFindGeneration();
Assert.False(cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10).IsHit);
Thread.Sleep(20);
cache.BeginFindGeneration();
var second = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10);
Assert.Equal(CacheHitKind.Fallthrough_NotBuilt, second.HitKind);
Assert.Equal(0, cache.GetStats().ResidentChunks);
}
[Fact]
public void EachChunkIsTrackedSeparately()
{
var cache = FreshCache(promotionThreshold: 2);
var map = TestMap;
// One touch each, in two different chunks: neither reaches the threshold on its own.
Assert.False(cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10).IsHit);
Assert.False(cache.TryGetMask(map, 1600, 1700, sourceZ: 10).IsHit);
Assert.Equal(0, cache.GetStats().ResidentChunks);
Assert.Equal(2L, cache.GetStats().FallthroughNotBuilt);
}
// ---- fallthrough routes ----
[Fact]
public void OffMapCell_FallsThrough()
{
var lookup = FreshCache(2).TryGetMask(TestMap, -1, -1, sourceZ: 0);
Assert.False(lookup.IsHit);
Assert.Equal(CacheHitKind.Fallthrough_OffMap, lookup.HitKind);
Assert.Equal((byte)0, lookup.WalkMask);
}
/// <summary>
/// A multi's cells fall through, and so does the 1-cell halo around it: a cell's mask encodes
/// the edges TO its neighbours, so a wall one cell over has to block them.
/// </summary>
[SkippableFact]
public void MultiCoveredCell_AndItsHalo_FallThrough()
{
TileDataRequirement.SkipIfMissing();
var cache = FreshCache(promotionThreshold: 1);
var map = TestMap;
// A cell nowhere near a multi still serves from the static cache.
Assert.True(cache.TryGetMask(map, PlainX, PlainY, 10).IsHit);
// Mark an isolated sector as multi-bearing. Sector.HasMultis only tests Count > 0 and the
// fallthrough never dereferences the multi, so a single null entry is enough — no real
// BaseMulti needed.
const int mx = 2000;
const int my = 2000;
var sx = mx >> 4;
var sy = my >> 4;
var sector = map.GetRealSector(sx, sy);
var multisField = typeof(Map.Sector).GetField("_multis", BindingFlags.NonPublic | BindingFlags.Instance);
Assert.NotNull(multisField);
var original = multisField.GetValue(sector);
try
{
multisField.SetValue(sector, new List<BaseMulti> { null });
// Inside the multi's sector.
Assert.Equal(CacheHitKind.Fallthrough_Multi, cache.TryGetMask(map, mx, my, 0).HitKind);
// Last cell of the neighbouring sector: its halo reaches across the boundary.
Assert.Equal(CacheHitKind.Fallthrough_Multi, cache.TryGetMask(map, sx * 16 - 1, my, 0).HitKind);
// One cell further out: halo no longer reaches, so the static cache handles it.
Assert.NotEqual(CacheHitKind.Fallthrough_Multi, cache.TryGetMask(map, sx * 16 - 2, my, 0).HitKind);
Assert.True(cache.GetStats().FallthroughMulti >= 2);
}
finally
{
multisField.SetValue(sector, original);
}
}
/// <summary>A query too far from the cell's baked Z gets no answer, rather than a wrong one.</summary>
[Fact]
public void SourceZFarFromBake_FallsThrough()
{
var cache = FreshCache(promotionThreshold: 1);
var map = TestMap;
cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10);
var before = cache.GetStats().FallthroughSourceZMismatch;
var lookup = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 100);
Assert.False(lookup.IsHit);
Assert.Equal(CacheHitKind.Fallthrough_SourceZMismatch, lookup.HitKind);
Assert.Equal(before + 1L, cache.GetStats().FallthroughSourceZMismatch);
}
// ---- strata ----
[Fact]
public void Stratum_MatchingQueryZ_IsServed()
{
var cache = FreshCache(promotionThreshold: 1);
var map = TestMap;
var chunk = BuiltPlainChunk(cache, map);
var offsets = NoStrataOffsets();
offsets[CellIndex(PlainX, PlainY)] = 0;
chunk.SetStrata(offsets, OneStratum(zCenter: 42, walkMask: 0b0000_0011, walkZs: [42, 42]));
var lookup = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 42);
Assert.True(lookup.IsHit);
Assert.Equal((byte)0b0000_0011, lookup.WalkMask);
Assert.Equal((sbyte)42, lookup.WalkZ_N);
Assert.Equal((sbyte)42, lookup.WalkZ_NE);
}
[Fact]
public void Stratum_QueryZOutOfReach_FallsThrough()
{
var cache = FreshCache(promotionThreshold: 1);
var map = TestMap;
var chunk = BuiltPlainChunk(cache, map);
var offsets = NoStrataOffsets();
offsets[CellIndex(PlainX, PlainY)] = 0;
chunk.SetStrata(offsets, OneStratum(zCenter: 42));
// 10 is more than StepHeight from the only stratum, so nothing can answer.
var lookup = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10);
Assert.False(lookup.IsHit);
Assert.Equal(CacheHitKind.Fallthrough_MultiZ, lookup.HitKind);
}
/// <summary>
/// A cell flagged multi-Z is served only from its strata. If it has none that match — here, a
/// zero-count record — it must fall through rather than quietly fall back to the main mask,
/// which was baked for a different surface.
/// </summary>
[Fact]
public void MultiZCell_WithNoUsableStratum_FallsThrough()
{
var cache = FreshCache(promotionThreshold: 1);
var map = TestMap;
var chunk = BuiltPlainChunk(cache, map);
var before = cache.GetStats().FallthroughMultiZ;
var offsets = NoStrataOffsets();
offsets[CellIndex(PlainX, PlainY)] = 0;
chunk.SetStrata(offsets, [0]); // a record declaring zero strata
var lookup = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10);
Assert.False(lookup.IsHit);
Assert.Equal(CacheHitKind.Fallthrough_MultiZ, lookup.HitKind);
Assert.Equal(before + 1L, cache.GetStats().FallthroughMultiZ);
}
// ---- swim layer ----
[Fact]
public void SwimLayer_QueryAtWaterZ_IsServedFromTheLayer()
{
var cache = FreshCache(promotionThreshold: 1);
var map = TestMap;
var chunk = BuiltPlainChunk(cache, map);
var cell = CellIndex(PlainX, PlainY);
var bakedZ = chunk.SourceZ[cell];
// Place the water surface well clear of the walk surface, so the primary source-Z guard is
// guaranteed to reject the swim query and hand it to the layer.
var swimZ = (sbyte)(bakedZ - 20);
chunk.AllocateSwimLayer();
chunk.SwimSourceZ[cell] = swimZ;
chunk.SwimMask[cell] = 0b0000_0011;
chunk.SwimZN_Layer[cell] = swimZ;
chunk.SwimZNE_Layer[cell] = swimZ;
// At the walk surface, the layer is not consulted at all.
Assert.Equal(CacheHitKind.Hit, cache.TryGetMask(map, PlainX, PlainY, bakedZ).HitKind);
var swim = cache.TryGetMask(map, PlainX, PlainY, swimZ);
Assert.True(swim.IsHit);
Assert.Equal((byte)0, swim.WalkMask); // a swimmer can't walk
Assert.Equal((byte)0b0000_0011, swim.WetMask);
Assert.Equal(swimZ, swim.SwimZ_N);
Assert.Equal(swimZ, swim.SwimZ_NE);
}
/// <summary>
/// An inland cell in a chunk that has a swim layer carries the NoSwimLayerCell sentinel. That
/// sentinel is sbyte.MinValue, so a query at sbyte.MinValue would match it exactly on a naive
/// distance check — the guard has to reject the sentinel before measuring anything.
/// </summary>
[Fact]
public void SwimLayer_SentinelCell_IsNeverMatched()
{
var cache = FreshCache(promotionThreshold: 1);
var map = TestMap;
var chunk = BuiltPlainChunk(cache, map);
chunk.AllocateSwimLayer(); // allocated for some other cell; this one stays at the sentinel
var cell = CellIndex(PlainX, PlainY);
Assert.Equal(StepChunk.NoSwimLayerCell, chunk.SwimSourceZ[cell]);
var before = cache.GetStats().FallthroughSourceZMismatch;
var lookup = cache.TryGetMask(map, PlainX, PlainY, sourceZ: sbyte.MinValue);
Assert.False(lookup.IsHit);
Assert.Equal(CacheHitKind.Fallthrough_SourceZMismatch, lookup.HitKind);
Assert.Equal(before + 1L, cache.GetStats().FallthroughSourceZMismatch);
}
// ---- eviction ----
[Fact]
public void LruCap_EvictsDownToTheCap()
{
var cache = FreshCache(promotionThreshold: 1);
cache.MaxResidentChunks = 4;
try
{
var map = TestMap;
// Five chunks into a cache that holds four.
for (var i = 0; i < 5; i++)
{
cache.TryGetMask(map, PlainX + i * 16, PlainY, sourceZ: 10);
Thread.Sleep(2); // separate their LastTouchedTicks so LRU has something to order by
}
cache.EnforceLruCap();
Assert.Equal(4, cache.GetStats().ResidentChunks);
Assert.True(cache.GetStats().EvictionsByLruCap >= 1L);
Assert.True(cache.ResidentIndexInSync(), "eviction desynced the key list from the resident set");
}
finally
{
cache.MaxResidentChunks = 8192;
}
}
}