From b852bca41e63fc5cb8fe7679b1ff83018ab82224 Mon Sep 17 00:00:00 2001 From: Kamron Batman <3953314+kamronbatman@users.noreply.github.com> Date: Sun, 12 Jul 2026 20:02:29 -0700 Subject: [PATCH] perf(pathing): pool the StepCache strata buffer, then clean up the pathing engine around it (#2523) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Started as an allocation pass over `StepCache` and grew into a cleanup of the surrounding pathing engine. Four commits, each independently reviewable; net **−560 lines**. Build clean (0 warnings). All 122 `Server.Tests.Pathfinding` tests pass. --- ## 1. `perf`: pool the strata buffer, cut a hot-path dictionary lookup **The headline is that `TryGetMask` — the actual hot path — was already allocation-free.** `StepMask` is a readonly struct, `StaticTileEnumerable` is a `ref struct`, `ChunkMissState` is a struct in a `Dictionary`. So most of this is a bake-throughput and GC-churn win, with one exception noted below. `BuildChunk` accumulated packed multi-Z strata into a `List` that grew by doubling (256 → 512 → 1024 → …) and then paid a final `ToArray()`. A full map bake runs it ~114k times. It now writes into a `byte[]` rented from `STArrayPool.Shared` through a span writer, and hands the chunk one exact-size copy. **This required fixing a latent out-of-bounds guard.** The record-fit check reserved headroom for **8** strata (`StratumByteLength * 8`) while `ComputeStandableSurfaceZs` can return up to **16** — so a cell could write 305 bytes starting from a 65,383-byte offset. Against a `List` that was benign (it just grew past 64 KB, and emitted offsets stayed under the `NoStrata` sentinel). Against a fixed-size rented buffer it is an out-of-bounds write, so tightening it was a *prerequisite* for the pooling, not a drive-by. The guard is now exact, which additionally proves no emitted offset can collide with `NoStrata == ushort.MaxValue`. **One genuine query-path win:** `ShouldPromoteAfterMiss` did *two* dictionary lookups per miss — a `TryGetValue`, then an indexer assignment that re-hashes and re-probes. It now mutates in place via `CollectionsMarshal.GetValueRefOrNullRef`. This runs on every uncached chunk touch during A* expansion. The window-expiry branch keeps its explicit early return, so `MissPromotionThreshold == 1` still resets rather than promoting. Also dropped `StepProbe.ComputeStrataAt` / `ComputedStratum` (dead code, zero callers) and collapsed six 18-argument `new StepMask(0, 0, …, kind)` blocks into `Fallthrough(kind)`. **Considered and rejected:** pooling the `Direction[]` that `Find` returns. It *escapes* the call — `MovementPath` holds it across ticks while `PathFollower` walks `m_Index` through it — so it cannot be rented-and-returned, and it cannot be borrowed from the shared `BitmapAStarAlgorithm.Instance` without one creature clobbering another's in-flight path. `CheckPath` rate-limits repaths to one per 2s per creature, putting this at roughly 60 KB/sec at 1,000 pathing creatures. Not worth a public API break plus a use-after-return footgun. ## 2. `docs`: rewrite the comments for publication The comments had accumulated as development notes: internal phase jargon (`Tier 4`, `the Phase-2 synthesizer`), change narration aimed at a reviewer (`which the old ComputeStandingZ anchor missed`, `legacy behavior`), benchmark anecdotes (`benchmarked as near-optimal`, `a ~20 ns lookup`), and paragraphs restating the code. Rewritten to keep the rationale you cannot recover by reading the code — why the source-Z guard cannot be widened, why multis fall through with a halo, why the promotion gate counts Finds rather than calls, why `ComputeFingerprint` must hash the *files* and not the live tile tables — and drop the history that got us there. Three comments were **factually wrong**, not just wordy: - `CacheEvictionTimer` and `CacheStats` documented a class called `StaticWalkabilityCache`. No such class exists — it is `StepCache`. - `StepCacheFile` declared `File layout v8` while `FormatVersion` is 9, and called the current record layout "the v6 layout" in four places. The layout descriptions are now unversioned so they cannot drift again. - `StepProbe.ComputeStandingZ` claimed `StepCache` uses it to bake `SourceZ`. It has not since the baker moved to the clearance-aware `ComputeStandableSurfaceZs`; only a parity test calls it. ## 3. `refactor`: simplify `StepCacheFile.Write`, consolidate the format tests `SaveToFile` walked `_keysList` **twice** — once to count the map's chunks, then again through a `ChunkEnumerator` closure to emit them — because `Write` needed the count up front to size its index array. Both loops had the same root cause. Passing a **span** collapses them: the count is just `span.Length`. That deletes the `ChunkEnumerator` delegate, the closure over the list enumerator, and **both `InvalidOperationException` throws**, which existed only to police the delegate's "yield exactly `chunkCount` chunks" contract — a contract a span makes unrepresentable. `Write` now patches the header's `IndexOffset` by seeking back to it rather than reaching into the writer's live buffer with `BinaryPrimitives`. That also retires `IndexOffsetFieldPosition`, a hand-maintained byte offset that had to track the header layout, and sidesteps the stale-array hazard that motivated the manual patch (`BufferWriter` reallocates on growth). **Tests:** `StepCacheFileV6/V7/V8Tests` were named for the format version that introduced each transform — and the format is now **v9**, so all three names described formats the loader rejects outright. Beyond triplicated builders and plumbing, two things were actually broken: - The three near-identical rejection tests each cited a `MinSupportedVersion` that had since moved (`"version 5 < MinSupportedVersion 6"`, `"6 < 7"`, `"7 < 8"`). They passed for the wrong reason. - `AssertBaseEqual` (used by V7 and V8) **silently skipped the swim and strata trailers**. A regression dropping either would not have failed those tests. Now one `StepCacheFileFormatTests`, named for behavior — predictive-Z elision, compression, compact index — with a single `AssertIdentical` that does check both trailers, the three rejection tests folded into one theory that also covers a future version, and a zero-chunk case the delegate-based writer never had coverage for. ## 4. `test`: consolidate the parity and lifecycle tests Three files tested "parity" and none of the names said *which*. They were three different layers, and the seams are the useful part, so they are now one `StepCacheParityTests` that names them: | Test | Compares | Answers | |---|---|---| | `ProbeMatchesSlowPath` | StepProbe vs MovementImpl | Is the bake right? | | `CacheMatchesProbe` | StepCache vs StepProbe | Is it stored and returned intact? | | `CacheServesReachableWalkStates` | StepCache vs MovementImpl | End to end, over the states A* visits | Merging removed a duplicated stub `Mobile`, duplicated region seeds, and a filename/class mismatch (`StepProbeParityTests.cs` declared `StaticWalkabilityParityTests`). `SwimBake_ProducesWetCells` moved with it — it lived in the cache parity file but never touched the cache. Tests reached into `StepCache._chunks` via `GetField` in **9 places**, each rebuilding the key encoding and cell-index arithmetic by hand. `StepCache` now exposes `GetResidentChunk` and `ResidentIndexInSync` alongside the internal test hooks it already had (`LazyReaderHasChunk`, `CurrentFindGeneration`), and the shared arithmetic moved to `PathingTestSupport`. All 9 reflection blocks are gone. `StepCacheLifecycleTests` is regrouped by what it covers — promotion gate, fallthrough routes, strata, swim layer, eviction — with the `Tier4*` names dropped. Removed `Singleton_IsAvailable`, which asserted an inline-initialized static property was not null; that is the entire 123 → 122 test-count delta. --- ## Verification Tests were mutation-checked rather than just run, since round-trip and parity tests can pass while a transform silently no-ops: - Injecting an off-by-one into the `IndexOffset` patch fails **15 of 123** — the format tests are load-bearing. - Offsetting the cache's cell index by one fails **7 of 10** parity cases, and the 3 that stay green are exactly the ones that do not touch the cache. The layering localizes a fault rather than just reporting one. --- .../Engines/Pathing/PathingTestSupport.cs | 69 ++ .../Pathing/StepCacheFileFormatTests.cs | 406 ++++++++++++ .../Engines/Pathing/StepCacheFileTests.cs | 31 +- .../Engines/Pathing/StepCacheFileV6Tests.cs | 280 -------- .../Engines/Pathing/StepCacheFileV7Tests.cs | 122 ---- .../Engines/Pathing/StepCacheFileV8Tests.cs | 121 ---- .../Pathing/StepCacheLifecycleTests.cs | 584 +++++++--------- .../Engines/Pathing/StepCacheParityTests.cs | 378 ++++++++--- .../StepCacheStaticSurfaceParityTests.cs | 183 ----- .../Engines/Pathing/StepProbeParityTests.cs | 127 ---- .../Engines/Pathing/BitmapAStarAlgorithm.cs | 140 ++-- .../Pathing/Cache/CacheEvictionTimer.cs | 5 +- .../Engines/Pathing/Cache/CacheHitKind.cs | 18 +- .../Engines/Pathing/Cache/CacheStats.cs | 5 +- .../Engines/Pathing/Cache/MultiMaskCache.cs | 84 ++- .../Engines/Pathing/Cache/StepCache.cs | 625 ++++++++---------- .../Engines/Pathing/Cache/StepCacheFile.cs | 297 ++++----- .../Engines/Pathing/Cache/StepChunk.cs | 74 +-- .../Engines/Pathing/Cache/StepMask.cs | 12 +- .../Engines/Pathing/Cache/StepProbe.cs | 158 +---- .../Engines/Pathing/PathCacheCommands.cs | 78 ++- .../UOContent/Engines/Pathing/PathDiag.cs | 44 +- .../Engines/Pathing/PathfindRecorder.cs | 45 +- 23 files changed, 1663 insertions(+), 2223 deletions(-) create mode 100644 Projects/UOContent.Tests/Tests/Engines/Pathing/PathingTestSupport.cs create mode 100644 Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileFormatTests.cs delete mode 100644 Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileV6Tests.cs delete mode 100644 Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileV7Tests.cs delete mode 100644 Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileV8Tests.cs delete mode 100644 Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheStaticSurfaceParityTests.cs delete mode 100644 Projects/UOContent.Tests/Tests/Engines/Pathing/StepProbeParityTests.cs diff --git a/Projects/UOContent.Tests/Tests/Engines/Pathing/PathingTestSupport.cs b/Projects/UOContent.Tests/Tests/Engines/Pathing/PathingTestSupport.cs new file mode 100644 index 000000000..9027332c7 --- /dev/null +++ b/Projects/UOContent.Tests/Tests/Engines/Pathing/PathingTestSupport.cs @@ -0,0 +1,69 @@ +using System; +using Server.Engines.Pathing.Cache; + +namespace Server.Tests.Pathfinding; + +/// +/// Shared fixtures for the step-cache tests: the walker the parity tests measure against, the +/// cell-index arithmetic, and builders for the chunk state several tests inject by hand. +/// +internal static class PathingTestSupport +{ + /// + /// Trammel. Every seed coordinate below is a real location on it, so these tests need the + /// client's map files; they skip when those are absent. + /// + public static Map TestMap => Map.Maps[1]; + + /// + /// A cell in open Britain countryside — flat, walkable in all directions, no statics. The + /// default subject when a test needs a chunk to exist and doesn't care what's in it. + /// + public const int PlainX = 1500; + public const int PlainY = 1600; + + /// Index of world cell (x, y) within its own chunk. + public static int CellIndex(int x, int y) => ((y & 15) << 4) | (x & 15); + + /// A strata offset table with every cell marked single-Z. + public static ushort[] NoStrataOffsets() + { + var offsets = new ushort[StepChunk.CellsPerChunk]; + Array.Fill(offsets, StepChunk.NoStrata); + return offsets; + } + + /// + /// Packs a one-stratum record: a count byte, then the stratum itself. Directions not named in + /// stay at 0. Mirrors the layout StepCache.WriteStratum produces. + /// + public static byte[] OneStratum(sbyte zCenter, byte walkMask = 0, byte wetMask = 0, params sbyte[] walkZs) + { + var data = new byte[1 + StepChunk.StratumByteLength]; + data[0] = 1; // stratum count + data[1] = (byte)zCenter; + data[2] = walkMask; + data[3] = wetMask; + + // walkZ_N..NW occupy bytes 4..11; swimZ_N..NW follow at 12..19. + for (var i = 0; i < walkZs.Length && i < 8; i++) + { + data[4 + i] = (byte)walkZs[i]; + } + + return data; + } + + /// + /// The default static walker. Deriving straight from rather than + /// BaseCreature is the point: MovementImpl then sees no creature capabilities (no swim, no fly, + /// no door-opening), which is exactly the walker the cache bakes for. + /// + public sealed class StaticWalker : Mobile + { + public StaticWalker() + { + Body = 0xC9; + } + } +} diff --git a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileFormatTests.cs b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileFormatTests.cs new file mode 100644 index 000000000..3fe72b74a --- /dev/null +++ b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileFormatTests.cs @@ -0,0 +1,406 @@ +using System; +using System.Collections.Generic; +using System.IO; +using Server.Engines.Pathing.Cache; +using Xunit; + +namespace Server.Tests.Pathfinding; + +/// +/// The .swb encoding, exercised through Write → OpenForLazy → TryReadChunk. Three transforms stack +/// in a record and each can silently corrupt the ones under it, so every chunk shape here is +/// asserted byte-identical after a round trip: +/// +/// predictive-Z — a directional-Z array that matches its prediction is omitted entirely, +/// compression — each record deflates independently, or stores raw when that doesn't shrink it, +/// compact index — the trailer carries no file offsets; the reader sums record lengths instead. +/// +/// Several tests assert on file size, because a round trip alone cannot tell you a transform ran: +/// an encoder that elided nothing and compressed nothing would still round-trip perfectly. +/// +[Collection("Sequential Pathfinding Tests")] +public class StepCacheFileFormatTests +{ + // ---- chunk builders ---- + + /// + /// Per-cell varying masks and Zs. Nothing about it is uniform or predictable, so it exercises + /// the Full record with residual arrays present. + /// + private static StepChunk VariedChunk(int seed = 0) + { + var c = new StepChunk(); + for (var i = 0; i < StepChunk.CellsPerChunk; i++) + { + c.WalkMask[i] = (byte)((i + seed) & 0xFF); + c.WetMask[i] = (byte)((i * 7 + seed) & 0xFF); + c.SourceZ[i] = (sbyte)((i + seed) % 40 - 20); + c.WalkZN[i] = (sbyte)(c.SourceZ[i] + i % 3); + c.SwimZS[i] = (sbyte)(c.SourceZ[i] - i % 2); + } + + return c; + } + + /// Every cell identical — the Uniform record, ~28 bytes on disk. + private static StepChunk UniformChunk(sbyte z = 10) + { + var c = new StepChunk(); + Array.Fill(c.WalkMask, (byte)0xC1); + Array.Fill(c.SourceZ, z); + foreach (var arr in AllBaseZArrays(c)) + { + Array.Fill(arr, z); + } + + return c; + } + + /// + /// Flat terrain, but NOT uniform: masks and SourceZ vary per cell while every directional Z + /// equals its masked prediction. That is the exact shape predictive-Z is built for, so all 16 + /// arrays must elide. It doubles as the coastline case — partial walkability, non-zero SourceZ, + /// and 0 in every blocked direction, which is where a naive (unmasked) predictor would emit a + /// -SourceZ residual on every blocked direction and elide nothing. + /// + private static StepChunk FlatFullChunk(sbyte baseZ = 10) + { + var c = new StepChunk(); + for (var i = 0; i < StepChunk.CellsPerChunk; i++) + { + c.WalkMask[i] = (byte)(i & 0xFF); + c.WetMask[i] = (byte)(~i & 0xFF); + c.SourceZ[i] = (sbyte)(baseZ + i % 7 - 3); + } + + var walk = new[] { c.WalkZN, c.WalkZNE, c.WalkZE, c.WalkZSE, c.WalkZS, c.WalkZSW, c.WalkZW, c.WalkZNW }; + var swim = new[] { c.SwimZN, c.SwimZNE, c.SwimZE, c.SwimZSE, c.SwimZS, c.SwimZSW, c.SwimZW, c.SwimZNW }; + for (var i = 0; i < StepChunk.CellsPerChunk; i++) + { + for (var b = 0; b < 8; b++) + { + walk[b][i] = (sbyte)((c.WalkMask[i] >> b & 1) != 0 ? c.SourceZ[i] : 0); + swim[b][i] = (sbyte)((c.WetMask[i] >> b & 1) != 0 ? c.SourceZ[i] : 0); + } + } + + return c; + } + + private static StepChunk WithSwimLayer(StepChunk c) + { + c.AllocateSwimLayer(); + for (var i = 0; i < StepChunk.CellsPerChunk; i++) + { + c.SwimSourceZ[i] = (sbyte)(i % 30 - 15); + c.SwimMask[i] = (byte)(i * 5 & 0xFF); + c.SwimZN_Layer[i] = (sbyte)(i % 7); + c.SwimZNW_Layer[i] = (sbyte)-(i % 4); + } + + return c; + } + + private static StepChunk WithStrataAt(StepChunk c, int cell) + { + var offsets = new ushort[StepChunk.CellsPerChunk]; + Array.Fill(offsets, StepChunk.NoStrata); + offsets[cell] = 0; + + var data = new byte[1 + StepChunk.StratumByteLength]; + data[0] = 1; + c.SetStrata(offsets, data); + + return c; + } + + private static sbyte[][] AllBaseZArrays(StepChunk c) => + [ + c.WalkZN, c.WalkZNE, c.WalkZE, c.WalkZSE, c.WalkZS, c.WalkZSW, c.WalkZW, c.WalkZNW, + c.SwimZN, c.SwimZNE, c.SwimZE, c.SwimZSE, c.SwimZS, c.SwimZSW, c.SwimZW, c.SwimZNW + ]; + + private static sbyte[][] AllSwimLayerArrays(StepChunk c) => + [ + c.SwimZN_Layer, c.SwimZNE_Layer, c.SwimZE_Layer, c.SwimZSE_Layer, + c.SwimZS_Layer, c.SwimZSW_Layer, c.SwimZW_Layer, c.SwimZNW_Layer + ]; + + // ---- round-trip plumbing ---- + + private static string Write(params (int cx, int cy, StepChunk c)[] chunks) + { + var path = Path.Combine(Path.GetTempPath(), $"swb_{Guid.NewGuid():N}.swb"); + StepCacheFile.Write(path, 1u, chunks); + return path; + } + + private static StepChunk RoundTrip(StepChunk src, out long fileLength, int cx = 3, int cy = 4) + { + var path = Write((cx, cy, src)); + try + { + fileLength = new FileInfo(path).Length; + + using var reader = StepCacheFile.OpenForLazy(path); + Assert.NotNull(reader); + + var rt = reader!.TryReadChunk(cx, cy); + Assert.NotNull(rt); + return rt!; + } + finally + { + File.Delete(path); + } + } + + private static StepChunk RoundTrip(StepChunk src) => RoundTrip(src, out _); + + private static void AssertIdentical(StepChunk expected, StepChunk actual) + { + Assert.True(expected.WalkMask.AsSpan().SequenceEqual(actual.WalkMask), "WalkMask differs"); + Assert.True(expected.WetMask.AsSpan().SequenceEqual(actual.WetMask), "WetMask differs"); + Assert.True(expected.SourceZ.AsSpan().SequenceEqual(actual.SourceZ), "SourceZ differs"); + + var ez = AllBaseZArrays(expected); + var az = AllBaseZArrays(actual); + for (var i = 0; i < ez.Length; i++) + { + Assert.True(ez[i].AsSpan().SequenceEqual(az[i]), $"base Z array {i} differs"); + } + + Assert.Equal(expected.HasSwimLayer, actual.HasSwimLayer); + if (expected.HasSwimLayer) + { + Assert.True(expected.SwimSourceZ.AsSpan().SequenceEqual(actual.SwimSourceZ), "SwimSourceZ differs"); + Assert.True(expected.SwimMask.AsSpan().SequenceEqual(actual.SwimMask), "SwimMask differs"); + + var el = AllSwimLayerArrays(expected); + var al = AllSwimLayerArrays(actual); + for (var i = 0; i < el.Length; i++) + { + Assert.True(el[i].AsSpan().SequenceEqual(al[i]), $"swim-layer Z array {i} differs"); + } + } + + Assert.True(expected.StrataData.SequenceEqual(actual.StrataData), "StrataData differs"); + } + + // ---- predictive-Z transform ---- + + [Theory] + [InlineData((sbyte)0, (sbyte)0)] + [InlineData((sbyte)10, (sbyte)10)] + [InlineData((sbyte)0, (sbyte)10)] + [InlineData((sbyte)10, (sbyte)0)] + [InlineData((sbyte)-20, (sbyte)15)] + [InlineData(sbyte.MinValue, sbyte.MaxValue)] + [InlineData(sbyte.MaxValue, sbyte.MinValue)] + [InlineData(sbyte.MinValue, (sbyte)1)] + [InlineData((sbyte)127, (sbyte)-1)] + public void Residual_RoundTripsLosslessly_AcrossTheFullSByteRange(sbyte z, sbyte predict) + { + var residual = StepCacheFile.EncodeResidual(z, predict); + Assert.Equal(z, StepCacheFile.DecodeZ(predict, residual)); + } + + [Theory] + [InlineData((byte)0b0000_0001, 0, (sbyte)42, (sbyte)42)] // passable -> predict SourceZ + [InlineData((byte)0b0000_0000, 0, (sbyte)42, (sbyte)0)] // blocked -> predict 0 + [InlineData((byte)0b1000_0000, 7, (sbyte)-13, (sbyte)-13)] + [InlineData((byte)0b0111_1111, 7, (sbyte)-13, (sbyte)0)] + public void Predict_IsSourceZWherePassable_ZeroWhereBlocked(byte maskByte, int bit, sbyte sourceZ, sbyte expected) + { + Assert.Equal(expected, StepCacheFile.Predict(maskByte, bit, sourceZ)); + } + + [Fact] + public void FlatChunk_ElidesEveryZArray() + { + var src = FlatFullChunk(); + var rt = RoundTrip(src, out var fileLength); + + AssertIdentical(src, rt); + + // A Full record carrying all 16 Z arrays runs past 5 KB. Landing under 1100 bytes is only + // possible if every one of them elided. + Assert.True(fileLength < 1100, $"expected every base Z array to elide; file was {fileLength} bytes"); + } + + [Fact] + public void CoastlineChunk_ElidesEveryZArray() + { + // Partial walkability with a non-zero SourceZ: the shape that defeats an unmasked predictor. + var src = FlatFullChunk(baseZ: 25); + var rt = RoundTrip(src, out var fileLength); + + AssertIdentical(src, rt); + Assert.True(fileLength < 1100, $"masked predictor should elide every array; file was {fileLength} bytes"); + } + + [Fact] + public void SlopeInOneDirection_StoresOnlyThatZArray() + { + var flat = FlatFullChunk(); + RoundTrip(flat, out var flatLength); + + // Raise WalkZN on cells walkable to the north. Exactly one array (WalkZN) now disagrees + // with its prediction; the other 15 must still elide. + var sloped = FlatFullChunk(); + for (var i = 0; i < StepChunk.CellsPerChunk; i++) + { + if ((sloped.WalkMask[i] & 1) != 0) + { + sloped.WalkZN[i]++; + } + } + + var rt = RoundTrip(sloped, out var slopedLength); + + AssertIdentical(sloped, rt); + Assert.True(slopedLength > flatLength, "a present Z array should grow the record"); + Assert.True( + slopedLength <= flatLength + StepChunk.CellsPerChunk, + $"only one 256-byte residual array should have been added; grew by {slopedLength - flatLength}" + ); + } + + // ---- compression ---- + + [Fact] + public void VariedChunk_Compresses_AndRoundTrips() + { + var src = VariedChunk(seed: 4); + var rt = RoundTrip(src, out var fileLength); + + AssertIdentical(src, rt); + + // The uncompressed Full record for a varied chunk exceeds 5 KB. + Assert.True(fileLength < 4000, $"expected compression to shrink the record; file was {fileLength} bytes"); + } + + [Fact] + public void UniformChunk_StoredRaw_RoundTrips() + { + // A Uniform body is ~28 bytes and deflate cannot shrink it, so the writer stores it raw and + // the reader has to notice that from the payload length alone. + var src = UniformChunk(z: 12); + var rt = RoundTrip(src, out var fileLength); + + AssertIdentical(src, rt); + Assert.True(fileLength < 200, $"uniform record should stay tiny; file was {fileLength} bytes"); + } + + // ---- optional trailers ---- + + [Fact] + public void SwimLayer_RoundTrips() => AssertIdentical( + WithSwimLayer(VariedChunk()), + RoundTrip(WithSwimLayer(VariedChunk())) + ); + + [Fact] + public void Strata_RoundTrips() + { + var src = WithStrataAt(VariedChunk(), cell: 10); + var rt = RoundTrip(src); + + AssertIdentical(src, rt); + Assert.True(rt.IsCellMultiZ(10)); + } + + [Fact] + public void SwimLayerAndStrata_RoundTripTogether() + { + // Both trailers present at once, which is the only case that pins their relative order. + var src = WithStrataAt(WithSwimLayer(VariedChunk()), cell: 20); + var rt = RoundTrip(src); + + AssertIdentical(src, rt); + Assert.True(rt.HasSwimLayer); + Assert.True(rt.IsCellMultiZ(20)); + } + + // ---- compact index ---- + + [Fact] + public void MultipleChunks_ResolveIndividually_FromDerivedOffsets() + { + // The index stores no offsets, so a reader that mis-sums record lengths would hand back a + // neighbouring chunk's bytes. Distinct content per coordinate is what catches that. The mix + // of record sizes matters: a raw-stored Uniform sits between two compressed Full records, + // and one coordinate is large enough to exercise the packed key's high 16 bits. + var chunks = new List<(int cx, int cy, StepChunk c)> + { + (1, 1, VariedChunk(seed: 3)), + (2, 5, UniformChunk(z: 14)), + (10, 3, VariedChunk(seed: 99)), + (300, 200, VariedChunk(seed: 17)) + }; + + var path = Write(chunks.ToArray()); + try + { + using var reader = StepCacheFile.OpenForLazy(path); + Assert.NotNull(reader); + Assert.Equal((uint)chunks.Count, reader!.ChunkCount); + + foreach (var (cx, cy, src) in chunks) + { + Assert.True(reader.Has(cx, cy), $"missing chunk ({cx},{cy})"); + + var rt = reader.TryReadChunk(cx, cy); + Assert.NotNull(rt); + AssertIdentical(src, rt!); + } + + Assert.Null(reader.TryReadChunk(7, 7)); // never written + } + finally + { + File.Delete(path); + } + } + + [Fact] + public void EmptyChunkSet_WritesAReadableFile() + { + var path = Write(); + try + { + using var reader = StepCacheFile.OpenForLazy(path); + Assert.NotNull(reader); + Assert.Equal(0u, reader!.ChunkCount); + Assert.Null(reader.TryReadChunk(0, 0)); + } + finally + { + File.Delete(path); + } + } + + // ---- version gate ---- + + [Theory] + [InlineData(0u)] + [InlineData(5u)] + [InlineData(8u)] + [InlineData(StepCacheFile.FormatVersion + 1)] + public void UnsupportedVersion_IsRejected(uint version) + { + var path = Write((0, 0, UniformChunk())); + try + { + var bytes = File.ReadAllBytes(path); + BitConverter.GetBytes(version).CopyTo(bytes, 4); // Version sits right after Magic + File.WriteAllBytes(path, bytes); + + Assert.Null(StepCacheFile.OpenForLazy(path)); + } + finally + { + File.Delete(path); + } + } +} diff --git a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileTests.cs b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileTests.cs index 1ef3aa8ab..d29b79ef9 100644 --- a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileTests.cs +++ b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileTests.cs @@ -272,15 +272,9 @@ public class StepCacheFileTests } /// - /// First-touch on a chunk that the lazy reader can satisfy must NOT route through the - /// miss tracker — file-loaded chunks represent an explicit prior decision to keep - /// them warm. This guards the deployment shape where an admin ships .swb files and - /// expects the very first NPC pathfind in any region to use cache (not slow path). - /// - /// - /// A chunk with an injected swim layer must serialize and deserialize via the lazy - /// reader without losing the layer. Validates v3 file format end-to-end: swim layer - /// fields survive Save → Clear → LazyOpen → first-touch query. + /// A chunk's swim layer must survive Save → Clear → LazyOpen → first-touch query. The layer is + /// an optional trailer, so a chunk that has one is the only thing that proves it is written and + /// read back rather than silently dropped. /// [Fact] public void SwimLayer_RoundTrips_ThroughLazyReader() @@ -292,19 +286,14 @@ public class StepCacheFileTests var map = Map.Maps[1]; Assert.NotNull(map); - // Build a chunk and inject a synthetic swim layer onto cell (1500, 1600). + // Build a chunk and inject a synthetic swim layer onto one cell. cache.TryGetMask(map, 1500, 1600, sourceZ: 10); - var chunksField = typeof(StepCache).GetField( - "_chunks", - System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance - ); - var chunks = (System.Collections.Generic.Dictionary)chunksField!.GetValue(cache)!; - var key = StepCache.EncodeKey(map.MapID, 1500 >> 4, 1600 >> 4); - var chunk = chunks[key]; + var chunk = cache.GetResidentChunk(map.MapID, 1500 >> 4, 1600 >> 4); + Assert.NotNull(chunk); chunk.AllocateSwimLayer(); - var cellIndex = ((1600 - ((1600 >> 4) << 4)) << 4) | (1500 - ((1500 >> 4) << 4)); + var cellIndex = PathingTestSupport.CellIndex(1500, 1600); chunk.SwimSourceZ[cellIndex] = -7; chunk.SwimMask[cellIndex] = 0b0000_1111; chunk.SwimZN_Layer[cellIndex] = -7; @@ -402,6 +391,12 @@ public class StepCacheFileTests } } + /// + /// A chunk the .swb can satisfy must be served on first touch, without consulting the promotion + /// gate. This is the deployment shape where an admin ships baked files and expects the very + /// first pathfind through a region to use the cache rather than the slow path — the gate would + /// otherwise defer that first touch and defeat the whole point of shipping the bake. + /// [SkippableFact] public void LazyReaderHit_BypassesMissTrackerOnFirstTouch() { diff --git a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileV6Tests.cs b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileV6Tests.cs deleted file mode 100644 index 24764c272..000000000 --- a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileV6Tests.cs +++ /dev/null @@ -1,280 +0,0 @@ -using System; -using System.IO; -using Server.Engines.Pathing.Cache; -using Xunit; - -namespace Server.Tests.Pathfinding; - -// v6 = predictive-Z residuals on top of the v5 uniform-elision format. Each base directional -// Z array is stored as a masked residual against the cell's own SourceZ; arrays that match -// their prediction are omitted entirely (ZArrayMask bit clear) and synthesized at read. -[Collection("Sequential Pathfinding Tests")] -public class StepCacheFileV6Tests -{ - [Theory] - [InlineData((sbyte)0, (sbyte)0)] - [InlineData((sbyte)10, (sbyte)10)] - [InlineData((sbyte)0, (sbyte)10)] - [InlineData((sbyte)10, (sbyte)0)] - [InlineData((sbyte)-20, (sbyte)15)] - [InlineData(sbyte.MinValue, sbyte.MaxValue)] - [InlineData(sbyte.MaxValue, sbyte.MinValue)] - [InlineData(sbyte.MinValue, (sbyte)1)] - [InlineData((sbyte)127, (sbyte)-1)] - public void Residual_RoundTrips_Losslessly_ForAllInputs(sbyte z, sbyte predict) - { - var residual = StepCacheFile.EncodeResidual(z, predict); - Assert.Equal(z, StepCacheFile.DecodeZ(predict, residual)); - } - - [Theory] - [InlineData((byte)0b0000_0001, 0, (sbyte)42, (sbyte)42)] // bit set -> sourceZ - [InlineData((byte)0b0000_0000, 0, (sbyte)42, (sbyte)0)] // bit clear -> 0 - [InlineData((byte)0b1000_0000, 7, (sbyte)-13, (sbyte)-13)] - [InlineData((byte)0b0111_1111, 7, (sbyte)-13, (sbyte)0)] - public void Predict_UsesSourceZWhenBitSet_ZeroOtherwise(byte maskByte, int bit, sbyte sourceZ, sbyte expected) - { - Assert.Equal(expected, StepCacheFile.Predict(maskByte, bit, sourceZ)); - } - - // ---- builders ---- - - // A FULL chunk (not uniform: masks/SourceZ vary per cell) whose every directional-Z equals - // its masked prediction => all 16 base Z arrays must elide. Doubles as the coastline case: - // per-cell partial walkability with SourceZ != 0, flat where walkable, 0 where not. - private static StepChunk FlatFullChunk(int multis = 3, sbyte baseZ = 10) - { - var c = new StepChunk { BuiltMultisVersion = multis }; - for (var i = 0; i < StepChunk.CellsPerChunk; i++) - { - c.WalkMask[i] = (byte)(i & 0xFF); - c.WetMask[i] = (byte)(~i & 0xFF); - c.SourceZ[i] = (sbyte)(baseZ + i % 7 - 3); // varies, mostly != 0 - } - SetFlatDirectional(c); - return c; - } - - // Sets every directional-Z to its masked prediction (walkable/wet -> SourceZ, else 0), - // i.e. perfectly flat terrain. Such arrays all elide under v6. - private static void SetFlatDirectional(StepChunk c) - { - var walk = new[] { c.WalkZN, c.WalkZNE, c.WalkZE, c.WalkZSE, c.WalkZS, c.WalkZSW, c.WalkZW, c.WalkZNW }; - var swim = new[] { c.SwimZN, c.SwimZNE, c.SwimZE, c.SwimZSE, c.SwimZS, c.SwimZSW, c.SwimZW, c.SwimZNW }; - for (var i = 0; i < StepChunk.CellsPerChunk; i++) - { - for (var b = 0; b < 8; b++) - { - walk[b][i] = (sbyte)((c.WalkMask[i] >> b & 1) != 0 ? c.SourceZ[i] : 0); - swim[b][i] = (sbyte)((c.WetMask[i] >> b & 1) != 0 ? c.SourceZ[i] : 0); - } - } - } - - private static StepChunk VariedChunk(int multis = 3) - { - var c = new StepChunk { BuiltMultisVersion = multis }; - for (var i = 0; i < StepChunk.CellsPerChunk; i++) - { - c.WalkMask[i] = (byte)(i & 0xFF); - c.WetMask[i] = (byte)((i * 7) & 0xFF); - c.SourceZ[i] = (sbyte)(i % 40 - 20); - c.WalkZN[i] = (sbyte)(c.SourceZ[i] + i % 3); - c.SwimZS[i] = (sbyte)(c.SourceZ[i] - i % 2); - } - return c; - } - - private static StepChunk SwimChunk(int multis = 6) - { - var c = VariedChunk(multis); - c.AllocateSwimLayer(); - for (var i = 0; i < StepChunk.CellsPerChunk; i++) - { - c.SwimSourceZ[i] = (sbyte)(i % 30 - 15); - c.SwimMask[i] = (byte)((i * 5) & 0xFF); - c.SwimZN_Layer[i] = (sbyte)(i % 7); - c.SwimZNW_Layer[i] = (sbyte)-(i % 4); - } - return c; - } - - // ---- round-trip plumbing ---- - - private static string Write1(StepChunk c, int cx, int cy) - { - var path = Path.Combine(Path.GetTempPath(), $"swbv6_{Guid.NewGuid():N}.swb"); - var emitted = false; - StepCacheFile.Write(path, 1u, 1u, (out int ox, out int oy, out StepChunk oc) => - { - if (emitted) { ox = oy = 0; oc = null!; return false; } - emitted = true; ox = cx; oy = cy; oc = c; return true; - }); - return path; - } - - private static StepChunk RoundTrip(StepChunk src, int cx, int cy, out long fileLen) - { - var path = Write1(src, cx, cy); - try - { - fileLen = new FileInfo(path).Length; - using var reader = StepCacheFile.OpenForLazy(path); - Assert.NotNull(reader); - var rt = reader!.TryReadChunk(cx, cy); - Assert.NotNull(rt); - return rt!; - } - finally { File.Delete(path); } - } - - private static void AssertChunksEqual(StepChunk a, StepChunk b) - { - Assert.Equal(a.BuiltMultisVersion, b.BuiltMultisVersion); - Assert.True(a.WalkMask.AsSpan().SequenceEqual(b.WalkMask)); - Assert.True(a.WetMask.AsSpan().SequenceEqual(b.WetMask)); - Assert.True(a.SourceZ.AsSpan().SequenceEqual(b.SourceZ)); - - var az = new[] { a.WalkZN, a.WalkZNE, a.WalkZE, a.WalkZSE, a.WalkZS, a.WalkZSW, a.WalkZW, a.WalkZNW, - a.SwimZN, a.SwimZNE, a.SwimZE, a.SwimZSE, a.SwimZS, a.SwimZSW, a.SwimZW, a.SwimZNW }; - var bz = new[] { b.WalkZN, b.WalkZNE, b.WalkZE, b.WalkZSE, b.WalkZS, b.WalkZSW, b.WalkZW, b.WalkZNW, - b.SwimZN, b.SwimZNE, b.SwimZE, b.SwimZSE, b.SwimZS, b.SwimZSW, b.SwimZW, b.SwimZNW }; - for (var i = 0; i < az.Length; i++) - { - Assert.True(az[i].AsSpan().SequenceEqual(bz[i]), $"base Z array {i} differs"); - } - - Assert.Equal(a.HasSwimLayer, b.HasSwimLayer); - if (a.HasSwimLayer) - { - Assert.True(a.SwimSourceZ.AsSpan().SequenceEqual(b.SwimSourceZ)); - Assert.True(a.SwimMask.AsSpan().SequenceEqual(b.SwimMask)); - var al = new[] { a.SwimZN_Layer, a.SwimZNE_Layer, a.SwimZE_Layer, a.SwimZSE_Layer, - a.SwimZS_Layer, a.SwimZSW_Layer, a.SwimZW_Layer, a.SwimZNW_Layer }; - var bl = new[] { b.SwimZN_Layer, b.SwimZNE_Layer, b.SwimZE_Layer, b.SwimZSE_Layer, - b.SwimZS_Layer, b.SwimZSW_Layer, b.SwimZW_Layer, b.SwimZNW_Layer }; - for (var i = 0; i < al.Length; i++) - { - Assert.True(al[i].AsSpan().SequenceEqual(bl[i]), $"swim-layer Z array {i} differs"); - } - } - } - - // ---- transform tests (Task 2) ---- - - [Fact] - public void FlatFull_AllArraysElide_RoundTripsAndIsCompact() - { - var src = FlatFullChunk(); - var rt = RoundTrip(src, 5, 6, out var fileLen); - AssertChunksEqual(src, rt); - // Full record with all 16 Z arrays elided: header(48) + ~783-byte record + index(20). - // A v5 full record alone is > 5 KB, so a sub-1100-byte file proves elision fired. - Assert.True(fileLen < 1100, $"expected all base Z arrays to elide; file was {fileLen} bytes"); - } - - [Fact] - public void SlopedSubset_OnlyVaryingArraysPresent_RoundTrips() - { - var flat = FlatFullChunk(); - var flatPath = Write1(flat, 1, 1); - long flatLen; - try { flatLen = new FileInfo(flatPath).Length; } finally { File.Delete(flatPath); } - - // Bump WalkZN by +1 on cells walkable to the N (slope in one direction only) -> exactly - // one base Z array (WalkZN, bit 0) becomes present; the other 15 still elide. - var sloped = FlatFullChunk(); - for (var i = 0; i < StepChunk.CellsPerChunk; i++) - { - if ((sloped.WalkMask[i] & 1) != 0) - { - sloped.WalkZN[i] = (sbyte)(sloped.WalkZN[i] + 1); - } - } - - var rt = RoundTrip(sloped, 2, 3, out var slopedLen); - AssertChunksEqual(sloped, rt); - Assert.True(slopedLen > flatLen, "one present array should grow the record vs all-flat"); - Assert.True(slopedLen <= flatLen + StepChunk.CellsPerChunk, "only one 256-byte residual array should be added"); - } - - [Fact] - public void Varied_Full_RoundTrips_Identically() - { - var src = VariedChunk(multis: 4); - AssertChunksEqual(src, RoundTrip(src, 1, 2, out _)); - } - - // ---- shape coverage (Task 3) ---- - - [Fact] - public void Coastline_NonzeroSourceZ_PartialWalkability_AllElide() - { - // FlatFullChunk already models a coastline: per-cell partial walk/wet masks, SourceZ != 0, - // flat where walkable and 0 (baker default) where not. A plain SourceZ residual would emit - // -SourceZ on every unwalkable direction; the masked predictor must drive ALL arrays to elide. - var src = FlatFullChunk(multis: 2, baseZ: 25); - var rt = RoundTrip(src, 7, 7, out var fileLen); - AssertChunksEqual(src, rt); - Assert.True(fileLen < 1100, $"masked predictor should elide every array on flat coastline; file was {fileLen} bytes"); - } - - [Fact] - public void SwimLayer_Full_RoundTrips_Identically() - { - var src = SwimChunk(multis: 8); - var rt = RoundTrip(src, 7, 8, out _); - Assert.True(rt.HasSwimLayer); - AssertChunksEqual(src, rt); - } - - [Fact] - public void Strata_Full_RoundTrips_Identically() - { - var src = VariedChunk(multis: 5); - var offsets = new ushort[StepChunk.CellsPerChunk]; - Array.Fill(offsets, StepChunk.NoStrata); - offsets[10] = 0; - var data = new byte[1 + StepChunk.StratumByteLength]; - data[0] = 1; - src.SetStrata(offsets, data); - - var rt = RoundTrip(src, 3, 4, out _); - AssertChunksEqual(src, rt); - Assert.True(rt.IsCellMultiZ(10)); - Assert.True(rt.StrataData.SequenceEqual(src.StrataData)); - } - - [Fact] - public void SwimAndStrata_Full_RoundTrips_Identically() - { - // Combined trailer ordering: swim-layer trailer THEN strata trailer, after the residual blocks. - var src = SwimChunk(multis: 11); - var offsets = new ushort[StepChunk.CellsPerChunk]; - Array.Fill(offsets, StepChunk.NoStrata); - offsets[20] = 0; - var data = new byte[1 + StepChunk.StratumByteLength]; - data[0] = 1; - src.SetStrata(offsets, data); - - var rt = RoundTrip(src, 9, 9, out _); - Assert.True(rt.HasSwimLayer); - Assert.True(rt.IsCellMultiZ(20)); - AssertChunksEqual(src, rt); - Assert.True(rt.StrataData.SequenceEqual(src.StrataData)); - } - - [Fact] - public void OlderVersion_IsRejected() - { - var path = Write1(FlatFullChunk(), 0, 0); - try - { - var bytes = File.ReadAllBytes(path); - bytes[4] = 5; bytes[5] = 0; bytes[6] = 0; bytes[7] = 0; // version 5 < MinSupportedVersion 6 - File.WriteAllBytes(path, bytes); - Assert.Null(StepCacheFile.OpenForLazy(path)); - } - finally { File.Delete(path); } - } -} diff --git a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileV7Tests.cs b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileV7Tests.cs deleted file mode 100644 index f5e517972..000000000 --- a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileV7Tests.cs +++ /dev/null @@ -1,122 +0,0 @@ -using System; -using System.IO; -using Server.Engines.Pathing.Cache; -using Xunit; - -namespace Server.Tests.Pathfinding; - -// v7 = per-chunk libdeflate compression on top of the v6 predictive-Z format. Each record is -// compressed independently (random access preserved) behind a u32 uncompressed-length prefix; -// records that do not shrink (tiny Uniform records) are stored raw. -[Collection("Sequential Pathfinding Tests")] -public class StepCacheFileV7Tests -{ - private static StepChunk VariedChunk(int multis = 3) - { - var c = new StepChunk { BuiltMultisVersion = multis }; - for (var i = 0; i < StepChunk.CellsPerChunk; i++) - { - c.WalkMask[i] = (byte)(i & 0xFF); - c.WetMask[i] = (byte)((i * 7) & 0xFF); - c.SourceZ[i] = (sbyte)(i % 40 - 20); - c.WalkZN[i] = (sbyte)(c.SourceZ[i] + i % 3); - c.SwimZS[i] = (sbyte)(c.SourceZ[i] - i % 2); - } - return c; - } - - private static StepChunk UniformChunk(byte walk = 0xC1, sbyte z = 10, int multis = 7) - { - var c = new StepChunk { BuiltMultisVersion = multis }; - Array.Fill(c.WalkMask, walk); - Array.Fill(c.SourceZ, z); - foreach (var arr in new[] - { - c.WalkZN, c.WalkZNE, c.WalkZE, c.WalkZSE, c.WalkZS, c.WalkZSW, c.WalkZW, c.WalkZNW, - c.SwimZN, c.SwimZNE, c.SwimZE, c.SwimZSE, c.SwimZS, c.SwimZSW, c.SwimZW, c.SwimZNW - }) - { - Array.Fill(arr, z); - } - return c; - } - - private static string Write1(StepChunk c, int cx, int cy) - { - var path = Path.Combine(Path.GetTempPath(), $"swbv7_{Guid.NewGuid():N}.swb"); - var emitted = false; - StepCacheFile.Write(path, 1u, 1u, (out int ox, out int oy, out StepChunk oc) => - { - if (emitted) { ox = oy = 0; oc = null!; return false; } - emitted = true; ox = cx; oy = cy; oc = c; return true; - }); - return path; - } - - private static StepChunk RoundTrip(StepChunk src, int cx, int cy, out long fileLen) - { - var path = Write1(src, cx, cy); - try - { - fileLen = new FileInfo(path).Length; - using var reader = StepCacheFile.OpenForLazy(path); - Assert.NotNull(reader); - var rt = reader!.TryReadChunk(cx, cy); - Assert.NotNull(rt); - return rt!; - } - finally { File.Delete(path); } - } - - private static void AssertBaseEqual(StepChunk a, StepChunk b) - { - Assert.Equal(a.BuiltMultisVersion, b.BuiltMultisVersion); - Assert.True(a.WalkMask.AsSpan().SequenceEqual(b.WalkMask)); - Assert.True(a.WetMask.AsSpan().SequenceEqual(b.WetMask)); - Assert.True(a.SourceZ.AsSpan().SequenceEqual(b.SourceZ)); - var az = new[] { a.WalkZN, a.WalkZNE, a.WalkZE, a.WalkZSE, a.WalkZS, a.WalkZSW, a.WalkZW, a.WalkZNW, - a.SwimZN, a.SwimZNE, a.SwimZE, a.SwimZSE, a.SwimZS, a.SwimZSW, a.SwimZW, a.SwimZNW }; - var bz = new[] { b.WalkZN, b.WalkZNE, b.WalkZE, b.WalkZSE, b.WalkZS, b.WalkZSW, b.WalkZW, b.WalkZNW, - b.SwimZN, b.SwimZNE, b.SwimZE, b.SwimZSE, b.SwimZS, b.SwimZSW, b.SwimZW, b.SwimZNW }; - for (var i = 0; i < az.Length; i++) - { - Assert.True(az[i].AsSpan().SequenceEqual(bz[i]), $"base Z array {i} differs"); - } - } - - [Fact] - public void Varied_Compresses_AndRoundTrips() - { - var src = VariedChunk(multis: 4); - var rt = RoundTrip(src, 1, 2, out var fileLen); - AssertBaseEqual(src, rt); - // The uncompressed v6 Full record for a varied chunk is > 5 KB. Compressed + header(48) - // + index(20), the whole file must be well under that — proving compression engaged. - Assert.True(fileLen < 4000, $"expected compression to shrink the record; file was {fileLen} bytes"); - } - - [Fact] - public void Uniform_StoredRaw_RoundTrips() - { - // A Uniform record body is ~24 bytes; libdeflate cannot shrink it, so WriteChunk stores it - // raw (payload length == uncompressed length). The reader must take the raw path and rebuild. - var src = UniformChunk(walk: 0xC1, z: 12, multis: 9); - var rt = RoundTrip(src, 5, 6, out var fileLen); - AssertBaseEqual(src, rt); - Assert.True(fileLen < 200, $"uniform record should stay tiny; file was {fileLen} bytes"); - } - - [Fact] - public void V6_IsRejected() - { - var path = Write1(UniformChunk(), 0, 0); - try - { - var bytes = File.ReadAllBytes(path); - bytes[4] = 6; bytes[5] = 0; bytes[6] = 0; bytes[7] = 0; // version 6 < MinSupportedVersion 7 - File.WriteAllBytes(path, bytes); - Assert.Null(StepCacheFile.OpenForLazy(path)); - } - finally { File.Delete(path); } - } -} diff --git a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileV8Tests.cs b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileV8Tests.cs deleted file mode 100644 index 8d3cd495b..000000000 --- a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheFileV8Tests.cs +++ /dev/null @@ -1,121 +0,0 @@ -using System; -using System.Collections.Generic; -using System.IO; -using Server.Engines.Pathing.Cache; -using Xunit; - -namespace Server.Tests.Pathfinding; - -// v8 = compact index on top of the v7 compression format. The trailer drops the per-chunk file -// offset (reconstructed by cumulative record length in write order) and packs the key to u32. -[Collection("Sequential Pathfinding Tests")] -public class StepCacheFileV8Tests -{ - private static StepChunk VariedChunk(int seed, int multis) - { - var c = new StepChunk { BuiltMultisVersion = multis }; - for (var i = 0; i < StepChunk.CellsPerChunk; i++) - { - c.WalkMask[i] = (byte)((i + seed) & 0xFF); - c.WetMask[i] = (byte)((i * 7 + seed) & 0xFF); - c.SourceZ[i] = (sbyte)((i + seed) % 40 - 20); - c.WalkZN[i] = (sbyte)(c.SourceZ[i] + i % 3); - c.SwimZS[i] = (sbyte)(c.SourceZ[i] - i % 2); - } - return c; - } - - private static StepChunk UniformChunk(sbyte z, int multis) - { - var c = new StepChunk { BuiltMultisVersion = multis }; - Array.Fill(c.WalkMask, (byte)0xC1); - Array.Fill(c.SourceZ, z); - foreach (var arr in new[] - { - c.WalkZN, c.WalkZNE, c.WalkZE, c.WalkZSE, c.WalkZS, c.WalkZSW, c.WalkZW, c.WalkZNW, - c.SwimZN, c.SwimZNE, c.SwimZE, c.SwimZSE, c.SwimZS, c.SwimZSW, c.SwimZW, c.SwimZNW - }) - { - Array.Fill(arr, z); - } - return c; - } - - private static void AssertBaseEqual(StepChunk a, StepChunk b) - { - Assert.Equal(a.BuiltMultisVersion, b.BuiltMultisVersion); - Assert.True(a.WalkMask.AsSpan().SequenceEqual(b.WalkMask)); - Assert.True(a.WetMask.AsSpan().SequenceEqual(b.WetMask)); - Assert.True(a.SourceZ.AsSpan().SequenceEqual(b.SourceZ)); - var az = new[] { a.WalkZN, a.WalkZNE, a.WalkZE, a.WalkZSE, a.WalkZS, a.WalkZSW, a.WalkZW, a.WalkZNW, - a.SwimZN, a.SwimZNE, a.SwimZE, a.SwimZSE, a.SwimZS, a.SwimZSW, a.SwimZW, a.SwimZNW }; - var bz = new[] { b.WalkZN, b.WalkZNE, b.WalkZE, b.WalkZSE, b.WalkZS, b.WalkZSW, b.WalkZW, b.WalkZNW, - b.SwimZN, b.SwimZNE, b.SwimZE, b.SwimZSE, b.SwimZS, b.SwimZSW, b.SwimZW, b.SwimZNW }; - for (var i = 0; i < az.Length; i++) - { - Assert.True(az[i].AsSpan().SequenceEqual(bz[i]), $"base Z array {i} differs"); - } - } - - private static string WriteMany(IReadOnlyList<(int cx, int cy, StepChunk c)> chunks) - { - var path = Path.Combine(Path.GetTempPath(), $"swbv8_{Guid.NewGuid():N}.swb"); - var idx = 0; - StepCacheFile.Write(path, 1u, (uint)chunks.Count, (out int ox, out int oy, out StepChunk oc) => - { - if (idx >= chunks.Count) { ox = oy = 0; oc = null!; return false; } - var e = chunks[idx++]; - ox = e.cx; oy = e.cy; oc = e.c; - return true; - }); - return path; - } - - [Fact] - public void MultiChunk_RoundTrips_WithDerivedOffsets() - { - // Distinct chunks at distinct coords. A wrong derived offset would read another chunk's - // bytes, so per-chunk identity verifies cumulative offset reconstruction across records. - var chunks = new List<(int, int, StepChunk)> - { - (1, 1, VariedChunk(seed: 3, multis: 2)), - (2, 5, UniformChunk(z: 14, multis: 9)), // tiny record (stored-raw path) in the middle - (10, 3, VariedChunk(seed: 99, multis: 4)), - (300, 200, VariedChunk(seed: 17, multis: 5)), // large packed-key coords (high 16 bits) - }; - - var path = WriteMany(chunks); - try - { - using var reader = StepCacheFile.OpenForLazy(path); - Assert.NotNull(reader); - Assert.Equal((uint)chunks.Count, reader!.ChunkCount); - - foreach (var (cx, cy, src) in chunks) - { - Assert.True(reader.Has(cx, cy), $"missing chunk ({cx},{cy})"); - var rt = reader.TryReadChunk(cx, cy); - Assert.NotNull(rt); - AssertBaseEqual(src, rt!); - } - - // A coordinate that was never written must not resolve. - Assert.Null(reader.TryReadChunk(7, 7)); - } - finally { File.Delete(path); } - } - - [Fact] - public void V7_IsRejected() - { - var path = WriteMany(new List<(int, int, StepChunk)> { (0, 0, UniformChunk(z: 10, multis: 1)) }); - try - { - var bytes = File.ReadAllBytes(path); - bytes[4] = 7; bytes[5] = 0; bytes[6] = 0; bytes[7] = 0; // version 7 < MinSupportedVersion 8 - File.WriteAllBytes(path, bytes); - Assert.Null(StepCacheFile.OpenForLazy(path)); - } - finally { File.Delete(path); } - } -} diff --git a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheLifecycleTests.cs b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheLifecycleTests.cs index 436f78c7e..7b630173c 100644 --- a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheLifecycleTests.cs +++ b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheLifecycleTests.cs @@ -1,48 +1,64 @@ using System.Collections.Generic; using System.Reflection; +using System.Threading; using Server.Engines.Pathing.Cache; using Server.Items; using Xunit; +using static Server.Tests.Pathfinding.PathingTestSupport; namespace Server.Tests.Pathfinding; +/// +/// How the cache decides what to build, what to serve, and what to throw away: the promotion gate, +/// the four fallthrough routes out of , the strata and swim +/// layers, and LRU eviction. +/// [Collection("Sequential Pathfinding Tests")] public class StepCacheLifecycleTests { - [Fact] - public void Singleton_IsAvailable() + /// Resets to a known state and returns the singleton. + private static StepCache FreshCache(int promotionThreshold) { var cache = StepCache.Instance; - Assert.NotNull(cache); + cache.Clear(); + cache.MissPromotionThreshold = promotionThreshold; + return cache; + } + + /// Builds the plain chunk and hands it back for a test to inject state into. + private static StepChunk BuiltPlainChunk(StepCache cache, Map map) + { + cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10); + + var chunk = cache.GetResidentChunk(map.MapID, PlainX >> 4, PlainY >> 4); + Assert.NotNull(chunk); + return chunk; } [Fact] public void Clear_OnEmptyCache_LeavesStatsZero() { - var cache = StepCache.Instance; - cache.Clear(); + var stats = FreshCache(2).GetStats(); - var stats = cache.GetStats(); Assert.Equal(0, stats.ResidentChunks); Assert.Equal(0L, stats.Hits); Assert.Equal(0L, stats.BuildsTotal); } + // ---- promotion gate ---- + + /// + /// A chunk nothing has shown sustained interest in must not be built. The caller reads + /// IsHit=false as "use the slow path", which is the cheaper trade for a pet crossing a chunk + /// once: BuildChunk costs far more than the handful of slow-path steps it would save. + /// [Fact] - public void TryGetMask_FirstTouchOnUnbuiltChunk_DefersBuildAndReturnsFallthrough() + public void FirstTouch_DefersBuild_AndFallsThrough() { - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 2; + var cache = FreshCache(promotionThreshold: 2); + var map = TestMap; - var map = Map.Maps[1]; - Assert.NotNull(map); - - // First touch on a chunk that has no resident copy and no lazy reader behind it - // must NOT eagerly build. Caller (BitmapAStarAlgorithm) interprets IsHit=false as - // "use slow path" — pets/hireables passing briefly through a chunk avoid the - // ~700µs BuildChunk cost they'd never amortize. - var lookup = cache.TryGetMask(map, 1500, 1600, sourceZ: 10); + var lookup = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10); Assert.False(lookup.IsHit); Assert.Equal(CacheHitKind.Fallthrough_NotBuilt, lookup.HitKind); @@ -55,25 +71,20 @@ public class StepCacheLifecycleTests } [SkippableFact] - public void TryGetMask_SecondTouchWithinWindow_PromotesAndBuilds() + public void SecondTouchInsideWindow_PromotesAndServes() { TileDataRequirement.SkipIfMissing(); - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 2; - var map = Map.Maps[1]; + var cache = FreshCache(promotionThreshold: 2); + var map = TestMap; - // First touch defers; second touch inside the promotion window builds + serves. - // Pinned cell (1500, 1600, z=10): mask=0xC1 - var first = cache.TryGetMask(map, 1500, 1600, sourceZ: 10); - Assert.False(first.IsHit); + Assert.False(cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10).IsHit); - var second = cache.TryGetMask(map, 1500, 1600, sourceZ: 10); - Assert.True(second.IsHit); - Assert.Equal(CacheHitKind.Miss_NotBuilt, second.HitKind); - Assert.Equal((byte)0xC1, second.WalkMask); - Assert.Equal((sbyte)10, second.WalkZ_N); + var promoted = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10); + Assert.True(promoted.IsHit); + Assert.Equal(CacheHitKind.Miss_NotBuilt, promoted.HitKind); + Assert.Equal((byte)0xC1, promoted.WalkMask); // pinned: open plain, walkable N/NE/... per the bake + Assert.Equal((sbyte)10, promoted.WalkZ_N); var stats = cache.GetStats(); Assert.Equal(1, stats.ResidentChunks); @@ -81,58 +92,53 @@ public class StepCacheLifecycleTests Assert.Equal(1L, stats.BuildsTotal); Assert.Equal(1L, stats.FallthroughNotBuilt); - // Third query of same cell → Hit (chunk now resident). - var third = cache.TryGetMask(map, 1500, 1600, sourceZ: 10); - Assert.True(third.IsHit); - Assert.Equal(CacheHitKind.Hit, third.HitKind); - Assert.Equal((byte)0xC1, third.WalkMask); + // Now resident: a third query is a clean hit, not another miss. + var hit = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10); + Assert.Equal(CacheHitKind.Hit, hit.HitKind); + Assert.Equal((byte)0xC1, hit.WalkMask); } + /// + /// Two touches spread wider than the window are not interest, they're coincidence — a chunk + /// someone glanced through, then an unrelated creature wandering past minutes later. The count + /// restarts rather than accumulating toward a build. + /// [Fact] - public void TryGetMask_SecondTouchAfterWindow_RestartsCounterAndDefers() + public void SecondTouchAfterWindow_RestartsTheCount() { - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 2; - cache.MissPromotionWindowMs = 1; // 1ms window for testability + var cache = FreshCache(promotionThreshold: 2); + cache.MissPromotionWindowMs = 1; - var map = Map.Maps[1]; + var map = TestMap; - var first = cache.TryGetMask(map, 1500, 1600, sourceZ: 10); - Assert.False(first.IsHit); + Assert.False(cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10).IsHit); + Thread.Sleep(20); // outrun the window - System.Threading.Thread.Sleep(20); // exceed the window - - // Second touch lands outside the window: tracker resets the count to 1, returns - // Fallthrough_NotBuilt again — chunks the player just glanced through don't get - // promoted just because they get re-touched minutes later by an unrelated NPC. - var second = cache.TryGetMask(map, 1500, 1600, sourceZ: 10); + var second = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10); Assert.False(second.IsHit); Assert.Equal(CacheHitKind.Fallthrough_NotBuilt, second.HitKind); Assert.Equal(0, cache.GetStats().ResidentChunks); Assert.Equal(2L, cache.GetStats().FallthroughNotBuilt); } + /// + /// The gate counts Finds, not probes. A single pathfind hits a chunk once per cell it expands + /// there, so counting probes would cross any threshold on the second cell and gate nothing at + /// all — the deferral would be dead code. + /// [SkippableFact] - public void TryGetMask_MultipleCallsInSameFindGeneration_StayInFallthrough() + public void ManyProbesInOneFind_CountAsOneTouch() { TileDataRequirement.SkipIfMissing(); - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 2; - var map = Map.Maps[1]; + var cache = FreshCache(promotionThreshold: 2); + var map = TestMap; - // Open a pathfind. Multiple TryGetMask calls inside this Find target the same chunk - // (different cells). The promotion gate counts distinct Finds, not raw probes — these - // calls must NOT increment the per-chunk counter, even though there are many of them. - // Without this, A* expansion would trip the gate on the second cell expansion in any - // visited chunk, defeating the whole point of deferred promotion. cache.BeginFindGeneration(); for (var i = 0; i < 8; i++) { - // All cells are inside chunk (1500>>4, 1600>>4) = (93, 100). - var lookup = cache.TryGetMask(map, 1500 + i, 1600, sourceZ: 10); + // Eight different cells, all inside the same chunk. + var lookup = cache.TryGetMask(map, PlainX + i, PlainY, sourceZ: 10); Assert.False(lookup.IsHit); Assert.Equal(CacheHitKind.Fallthrough_NotBuilt, lookup.HitKind); } @@ -141,111 +147,101 @@ public class StepCacheLifecycleTests Assert.Equal(0L, cache.GetStats().BuildsTotal); Assert.Equal(8L, cache.GetStats().FallthroughNotBuilt); - // Begin a NEW Find — this is the second distinct touch under the per-Find gate. - // The chunk now crosses the threshold and promotes. + // A second Find is the second distinct touch, and crosses the threshold. cache.BeginFindGeneration(); - var promoted = cache.TryGetMask(map, 1500, 1600, sourceZ: 10); - Assert.True(promoted.IsHit); + var promoted = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10); + Assert.Equal(CacheHitKind.Miss_NotBuilt, promoted.HitKind); Assert.Equal(1, cache.GetStats().ResidentChunks); Assert.Equal(1L, cache.GetStats().BuildsTotal); } + /// Distinct Finds still don't promote if they straddle the window. [Fact] - public void TryGetMask_TwoFindGenerationsAcrossWindow_RestartsCounter() + public void TwoFindsAcrossTheWindow_DoNotPromote() { - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 2; - cache.MissPromotionWindowMs = 1; // 1ms window for testability + var cache = FreshCache(promotionThreshold: 2); + cache.MissPromotionWindowMs = 1; - var map = Map.Maps[1]; + var map = TestMap; cache.BeginFindGeneration(); - Assert.False(cache.TryGetMask(map, 1500, 1600, sourceZ: 10).IsHit); + Assert.False(cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10).IsHit); - System.Threading.Thread.Sleep(20); // exceed window + Thread.Sleep(20); - // Second Find lands outside the window. Even though it's a distinct generation, - // the elapsed-time check resets the counter to 1, so no promotion. cache.BeginFindGeneration(); - var second = cache.TryGetMask(map, 1500, 1600, sourceZ: 10); - Assert.False(second.IsHit); + var second = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10); + Assert.Equal(CacheHitKind.Fallthrough_NotBuilt, second.HitKind); Assert.Equal(0, cache.GetStats().ResidentChunks); } [Fact] - public void TryGetMask_DistinctChunks_TrackedIndependently() + public void EachChunkIsTrackedSeparately() { - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 2; + var cache = FreshCache(promotionThreshold: 2); + var map = TestMap; - var map = Map.Maps[1]; + // One touch each, in two different chunks: neither reaches the threshold on its own. + Assert.False(cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10).IsHit); + Assert.False(cache.TryGetMask(map, 1600, 1700, sourceZ: 10).IsHit); - // Two different chunks, one touch each — both must defer (each has its own counter). - var chunkA = cache.TryGetMask(map, 1500, 1600, sourceZ: 10); - var chunkB = cache.TryGetMask(map, 1600, 1700, sourceZ: 10); // different chunk - - Assert.False(chunkA.IsHit); - Assert.False(chunkB.IsHit); Assert.Equal(0, cache.GetStats().ResidentChunks); Assert.Equal(2L, cache.GetStats().FallthroughNotBuilt); } + // ---- fallthrough routes ---- + [Fact] - public void TryGetMask_OffMap_ReturnsFalseFallthrough() + public void OffMapCell_FallsThrough() { - var cache = StepCache.Instance; - cache.Clear(); - - var map = Map.Maps[1]; - - var lookup = cache.TryGetMask(map, -1, -1, sourceZ: 0); + var lookup = FreshCache(2).TryGetMask(TestMap, -1, -1, sourceZ: 0); Assert.False(lookup.IsHit); Assert.Equal(CacheHitKind.Fallthrough_OffMap, lookup.HitKind); Assert.Equal((byte)0, lookup.WalkMask); } + /// + /// A multi's cells fall through, and so does the 1-cell halo around it: a cell's mask encodes + /// the edges TO its neighbours, so a wall one cell over has to block them. + /// [SkippableFact] - public void MultiCoveredCell_AndHalo_RouteToFallthrough() + public void MultiCoveredCell_AndItsHalo_FallThrough() { TileDataRequirement.SkipIfMissing(); - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 1; // eager build so a multi-free cell serves immediately - var map = Map.Maps[1]; + var cache = FreshCache(promotionThreshold: 1); + var map = TestMap; - // A cell far from any multi serves from the static cache. - Assert.True(cache.TryGetMask(map, 1500, 1600, 10).IsHit); + // A cell nowhere near a multi still serves from the static cache. + Assert.True(cache.TryGetMask(map, PlainX, PlainY, 10).IsHit); - // Inject a multi into an isolated sector. Sector.HasMultis only checks Count > 0, so a - // single-entry list is enough to mark the sector as multi-bearing — the fallthrough - // decision never dereferences the multi, so no real BaseMulti instance is needed. + // Mark an isolated sector as multi-bearing. Sector.HasMultis only tests Count > 0 and the + // fallthrough never dereferences the multi, so a single null entry is enough — no real + // BaseMulti needed. const int mx = 2000; const int my = 2000; var sx = mx >> 4; var sy = my >> 4; + var sector = map.GetRealSector(sx, sy); var multisField = typeof(Map.Sector).GetField("_multis", BindingFlags.NonPublic | BindingFlags.Instance); Assert.NotNull(multisField); + var original = multisField.GetValue(sector); try { multisField.SetValue(sector, new List { null }); - // Cell inside the multi sector → routed to the live path. + // Inside the multi's sector. Assert.Equal(CacheHitKind.Fallthrough_Multi, cache.TryGetMask(map, mx, my, 0).HitKind); - // Cell in the adjacent sector but on the shared boundary → caught by the 1-cell halo - // (its mask would otherwise propose an edge into the multi sector). - var boundaryX = sx * 16 - 1; // last tile of sector sx-1; halo (x+1) reaches into sx - Assert.Equal(CacheHitKind.Fallthrough_Multi, cache.TryGetMask(map, boundaryX, my, 0).HitKind); + // Last cell of the neighbouring sector: its halo reaches across the boundary. + Assert.Equal(CacheHitKind.Fallthrough_Multi, cache.TryGetMask(map, sx * 16 - 1, my, 0).HitKind); - // Two tiles out → interior of the multi-free sector, unaffected. + // One cell further out: halo no longer reaches, so the static cache handles it. Assert.NotEqual(CacheHitKind.Fallthrough_Multi, cache.TryGetMask(map, sx * 16 - 2, my, 0).HitKind); Assert.True(cache.GetStats().FallthroughMulti >= 2); @@ -256,95 +252,38 @@ public class StepCacheLifecycleTests } } + /// A query too far from the cell's baked Z gets no answer, rather than a wrong one. [Fact] - public void MultiZCell_RoutesToFallthrough() + public void SourceZFarFromBake_FallsThrough() { - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 1; // eager build for prime-then-inspect tests + var cache = FreshCache(promotionThreshold: 1); + var map = TestMap; - var map = Map.Maps[1]; + cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10); + var before = cache.GetStats().FallthroughSourceZMismatch; - // Build a chunk first so it exists. - cache.TryGetMask(map, 1500, 1600, 10); - - // Snapshot current FallthroughMultiZ in case (1500, 1600) is naturally multi-Z - // in real tile data; we only assert the synthetic injection produces a delta of 1. - var preInjectionFallthroughMultiZ = cache.GetStats().FallthroughMultiZ; - - // Inject a multi-Z bit via reflection on the resident chunk. - var chunksField = typeof(StepCache).GetField( - "_chunks", - System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance - ); - Assert.NotNull(chunksField); - var chunks = (System.Collections.Generic.Dictionary)chunksField.GetValue(cache); - - var key = StepCache.EncodeKey(map.MapID, 1500 >> 4, 1600 >> 4); - Assert.True(chunks.ContainsKey(key)); - var chunk = chunks[key]; - - // Inject "this cell has strata but none match the query Z" — proves the cache - // still falls through to slow path when no stratum can answer. - var cellIndex = ((1600 - ((1600 >> 4) << 4)) << 4) | (1500 - ((1500 >> 4) << 4)); - var offsets = new ushort[StepChunk.CellsPerChunk]; - for (var i = 0; i < offsets.Length; i++) - { - offsets[i] = StepChunk.NoStrata; - } - offsets[cellIndex] = 0; // points to a 0-stratum-count entry → no match - var data = new byte[] { 0 }; - chunk.SetStrata(offsets, data); - - var lookup = cache.TryGetMask(map, 1500, 1600, 10); + var lookup = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 100); Assert.False(lookup.IsHit); - Assert.Equal(CacheHitKind.Fallthrough_MultiZ, lookup.HitKind); - - var stats = cache.GetStats(); - Assert.Equal(preInjectionFallthroughMultiZ + 1L, stats.FallthroughMultiZ); + Assert.Equal(CacheHitKind.Fallthrough_SourceZMismatch, lookup.HitKind); + Assert.Equal(before + 1L, cache.GetStats().FallthroughSourceZMismatch); } + // ---- strata ---- + [Fact] - public void Tier4Strata_MatchingZ_ReturnsHitFromStratum() + public void Stratum_MatchingQueryZ_IsServed() { - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 1; + var cache = FreshCache(promotionThreshold: 1); + var map = TestMap; + var chunk = BuiltPlainChunk(cache, map); - var map = Map.Maps[1]; - cache.TryGetMask(map, 1500, 1600, 10); + var offsets = NoStrataOffsets(); + offsets[CellIndex(PlainX, PlainY)] = 0; + chunk.SetStrata(offsets, OneStratum(zCenter: 42, walkMask: 0b0000_0011, walkZs: [42, 42])); - var chunksField = typeof(StepCache).GetField( - "_chunks", - System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance - ); - var chunks = (System.Collections.Generic.Dictionary)chunksField.GetValue(cache); - var key = StepCache.EncodeKey(map.MapID, 1500 >> 4, 1600 >> 4); - var chunk = chunks[key]; + var lookup = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 42); - // Inject one stratum at zCenter=42, walkMask=0b00000011 (N + NE). - // Query at sourceZ=42 must hit and return that stratum's data. - var cellIndex = ((1600 - ((1600 >> 4) << 4)) << 4) | (1500 - ((1500 >> 4) << 4)); - var offsets = new ushort[StepChunk.CellsPerChunk]; - for (var i = 0; i < offsets.Length; i++) - { - offsets[i] = StepChunk.NoStrata; - } - offsets[cellIndex] = 0; - - var data = new byte[1 + StepChunk.StratumByteLength]; - data[0] = 1; // count - data[1] = 42; // zCenter - data[2] = 0b0000_0011; // walkMask (N | NE) - data[3] = 0; // wetMask - data[4] = 42; data[5] = 42; data[6] = 0; data[7] = 0; - data[8] = 0; data[9] = 0; data[10] = 0; data[11] = 0; - data[12] = 0; data[13] = 0; data[14] = 0; data[15] = 0; - data[16] = 0; data[17] = 0; data[18] = 0; data[19] = 0; - chunk.SetStrata(offsets, data); - - var lookup = cache.TryGetMask(map, 1500, 1600, 42); Assert.True(lookup.IsHit); Assert.Equal((byte)0b0000_0011, lookup.WalkMask); Assert.Equal((sbyte)42, lookup.WalkZ_N); @@ -352,195 +291,130 @@ public class StepCacheLifecycleTests } [Fact] - public void SwimLayer_NotInjected_StaysFallthroughOnSourceZMismatch() + public void Stratum_QueryZOutOfReach_FallsThrough() { - // Sanity check: a chunk WITHOUT a swim layer falls through on source-Z mismatch - // exactly like before. Validates we didn't accidentally serve garbage when the - // chunk has no shore cells. - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 1; + var cache = FreshCache(promotionThreshold: 1); + var map = TestMap; + var chunk = BuiltPlainChunk(cache, map); - var map = Map.Maps[1]; + var offsets = NoStrataOffsets(); + offsets[CellIndex(PlainX, PlainY)] = 0; + chunk.SetStrata(offsets, OneStratum(zCenter: 42)); - cache.TryGetMask(map, 1500, 1600, sourceZ: 10); // build chunk - var beforeMismatch = cache.GetStats().FallthroughSourceZMismatch; + // 10 is more than StepHeight from the only stratum, so nothing can answer. + var lookup = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10); - // Same cell but query Z far from baked Z → source-Z guard fires. - var lookup = cache.TryGetMask(map, 1500, 1600, sourceZ: 100); - Assert.False(lookup.IsHit); - Assert.Equal(CacheHitKind.Fallthrough_SourceZMismatch, lookup.HitKind); - Assert.Equal(beforeMismatch + 1L, cache.GetStats().FallthroughSourceZMismatch); - } - - [Fact] - public void SwimLayer_InjectedMatchingZ_ReturnsHitFromSwimLayer() - { - // Inject a synthetic swim layer onto a resident chunk and verify a query at the - // swim source Z routes through the swim-layer fallback, returning the swim mask. - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 1; - - var map = Map.Maps[1]; - cache.TryGetMask(map, 1500, 1600, sourceZ: 10); - - var chunksField = typeof(StepCache).GetField( - "_chunks", - System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance - ); - var chunks = (System.Collections.Generic.Dictionary)chunksField!.GetValue(cache)!; - var key = StepCache.EncodeKey(map.MapID, 1500 >> 4, 1600 >> 4); - var chunk = chunks[key]; - - chunk.AllocateSwimLayer(); - var cellIndex = ((1600 - ((1600 >> 4) << 4)) << 4) | (1500 - ((1500 >> 4) << 4)); - chunk.SwimSourceZ[cellIndex] = -5; - chunk.SwimMask[cellIndex] = 0b0000_0011; - chunk.SwimZN_Layer[cellIndex] = -5; - chunk.SwimZNE_Layer[cellIndex] = -5; - // Other directions stay 0 — Mask bits 0 and 1 cover N and NE. - - // Query at the chunk's primary SourceZ — primary path serves walk-layer data, - // swim layer not consulted. - var bakedSourceZ = chunk.SourceZ[cellIndex]; - var walkLookup = cache.TryGetMask(map, 1500, 1600, bakedSourceZ); - Assert.True(walkLookup.IsHit); - Assert.Equal(CacheHitKind.Hit, walkLookup.HitKind); - // Walk-layer query produces walk-layer walkMask (whatever the bake found), NOT - // the synthetic swim mask we injected. - - // Query at the swim source Z — primary source-Z guard fails (|−5 − bakedZ| > 2 - // assuming baked Z is land surface), swim-layer fallback serves with our mask. - if (System.Math.Abs(-5 - bakedSourceZ) <= 2) - { - // Bake landed near water Z — adjust the test to a clearer swim Z. - chunk.SwimSourceZ[cellIndex] = (sbyte)(bakedSourceZ - 20); - } - var swimLookup = cache.TryGetMask(map, 1500, 1600, chunk.SwimSourceZ[cellIndex]); - Assert.True(swimLookup.IsHit); - Assert.Equal(CacheHitKind.Hit, swimLookup.HitKind); - Assert.Equal((byte)0, swimLookup.WalkMask); // walk = 0 at swim Z - Assert.Equal(chunk.SwimMask[cellIndex], swimLookup.WetMask); - Assert.Equal(chunk.SwimZN_Layer[cellIndex], swimLookup.SwimZ_N); - Assert.Equal(chunk.SwimZNE_Layer[cellIndex], swimLookup.SwimZ_NE); - } - - [Fact] - public void SwimLayer_InjectedButCellHasNoSentinel_FallsThrough() - { - // Chunk has the swim layer (some other cell is shore), but THIS cell is inland - // (SwimSourceZ = NoSwimLayerCell). Query at non-matching walk Z must fall through, - // not erroneously match -128 against the query. - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 1; - - var map = Map.Maps[1]; - cache.TryGetMask(map, 1500, 1600, sourceZ: 10); - - var chunksField = typeof(StepCache).GetField( - "_chunks", - System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance - ); - var chunks = (System.Collections.Generic.Dictionary)chunksField!.GetValue(cache)!; - var key = StepCache.EncodeKey(map.MapID, 1500 >> 4, 1600 >> 4); - var chunk = chunks[key]; - - // Allocate layer but leave THIS cell at the sentinel. - chunk.AllocateSwimLayer(); - var cellIndex = ((1600 - ((1600 >> 4) << 4)) << 4) | (1500 - ((1500 >> 4) << 4)); - Assert.Equal(StepChunk.NoSwimLayerCell, chunk.SwimSourceZ[cellIndex]); - - var beforeMismatch = cache.GetStats().FallthroughSourceZMismatch; - // Query at -128 (the sentinel value) — must NOT match. The guard short-circuits - // on the sentinel before computing |sourceZ - SwimSourceZ|. - var lookup = cache.TryGetMask(map, 1500, 1600, sbyte.MinValue); - Assert.False(lookup.IsHit); - Assert.Equal(CacheHitKind.Fallthrough_SourceZMismatch, lookup.HitKind); - Assert.Equal(beforeMismatch + 1L, cache.GetStats().FallthroughSourceZMismatch); - } - - [Fact] - public void Tier4Strata_NonMatchingZ_FallsThrough() - { - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 1; - - var map = Map.Maps[1]; - cache.TryGetMask(map, 1500, 1600, 10); - - var chunksField = typeof(StepCache).GetField( - "_chunks", - System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance - ); - var chunks = (System.Collections.Generic.Dictionary)chunksField.GetValue(cache); - var key = StepCache.EncodeKey(map.MapID, 1500 >> 4, 1600 >> 4); - var chunk = chunks[key]; - - // Stratum at zCenter=42; query at sourceZ=10 (delta > StepHeight=2). Must fallthrough. - var cellIndex = ((1600 - ((1600 >> 4) << 4)) << 4) | (1500 - ((1500 >> 4) << 4)); - var offsets = new ushort[StepChunk.CellsPerChunk]; - for (var i = 0; i < offsets.Length; i++) - { - offsets[i] = StepChunk.NoStrata; - } - offsets[cellIndex] = 0; - - var data = new byte[1 + StepChunk.StratumByteLength]; - data[0] = 1; data[1] = 42; // zCenter=42, all other bytes 0 - chunk.SetStrata(offsets, data); - - var lookup = cache.TryGetMask(map, 1500, 1600, 10); Assert.False(lookup.IsHit); Assert.Equal(CacheHitKind.Fallthrough_MultiZ, lookup.HitKind); } + /// + /// A cell flagged multi-Z is served only from its strata. If it has none that match — here, a + /// zero-count record — it must fall through rather than quietly fall back to the main mask, + /// which was baked for a different surface. + /// [Fact] - public void LruCap_OverflowEvictsToCap() + public void MultiZCell_WithNoUsableStratum_FallsThrough() { - var cache = StepCache.Instance; - cache.Clear(); + var cache = FreshCache(promotionThreshold: 1); + var map = TestMap; + var chunk = BuiltPlainChunk(cache, map); + + var before = cache.GetStats().FallthroughMultiZ; + + var offsets = NoStrataOffsets(); + offsets[CellIndex(PlainX, PlainY)] = 0; + chunk.SetStrata(offsets, [0]); // a record declaring zero strata + + var lookup = cache.TryGetMask(map, PlainX, PlainY, sourceZ: 10); + + Assert.False(lookup.IsHit); + Assert.Equal(CacheHitKind.Fallthrough_MultiZ, lookup.HitKind); + Assert.Equal(before + 1L, cache.GetStats().FallthroughMultiZ); + } + + // ---- swim layer ---- + + [Fact] + public void SwimLayer_QueryAtWaterZ_IsServedFromTheLayer() + { + var cache = FreshCache(promotionThreshold: 1); + var map = TestMap; + var chunk = BuiltPlainChunk(cache, map); + + var cell = CellIndex(PlainX, PlainY); + var bakedZ = chunk.SourceZ[cell]; + + // Place the water surface well clear of the walk surface, so the primary source-Z guard is + // guaranteed to reject the swim query and hand it to the layer. + var swimZ = (sbyte)(bakedZ - 20); + + chunk.AllocateSwimLayer(); + chunk.SwimSourceZ[cell] = swimZ; + chunk.SwimMask[cell] = 0b0000_0011; + chunk.SwimZN_Layer[cell] = swimZ; + chunk.SwimZNE_Layer[cell] = swimZ; + + // At the walk surface, the layer is not consulted at all. + Assert.Equal(CacheHitKind.Hit, cache.TryGetMask(map, PlainX, PlainY, bakedZ).HitKind); + + var swim = cache.TryGetMask(map, PlainX, PlainY, swimZ); + Assert.True(swim.IsHit); + Assert.Equal((byte)0, swim.WalkMask); // a swimmer can't walk + Assert.Equal((byte)0b0000_0011, swim.WetMask); + Assert.Equal(swimZ, swim.SwimZ_N); + Assert.Equal(swimZ, swim.SwimZ_NE); + } + + /// + /// An inland cell in a chunk that has a swim layer carries the NoSwimLayerCell sentinel. That + /// sentinel is sbyte.MinValue, so a query at sbyte.MinValue would match it exactly on a naive + /// distance check — the guard has to reject the sentinel before measuring anything. + /// + [Fact] + public void SwimLayer_SentinelCell_IsNeverMatched() + { + var cache = FreshCache(promotionThreshold: 1); + var map = TestMap; + var chunk = BuiltPlainChunk(cache, map); + + chunk.AllocateSwimLayer(); // allocated for some other cell; this one stays at the sentinel + + var cell = CellIndex(PlainX, PlainY); + Assert.Equal(StepChunk.NoSwimLayerCell, chunk.SwimSourceZ[cell]); + + var before = cache.GetStats().FallthroughSourceZMismatch; + var lookup = cache.TryGetMask(map, PlainX, PlainY, sourceZ: sbyte.MinValue); + + Assert.False(lookup.IsHit); + Assert.Equal(CacheHitKind.Fallthrough_SourceZMismatch, lookup.HitKind); + Assert.Equal(before + 1L, cache.GetStats().FallthroughSourceZMismatch); + } + + // ---- eviction ---- + + [Fact] + public void LruCap_EvictsDownToTheCap() + { + var cache = FreshCache(promotionThreshold: 1); cache.MaxResidentChunks = 4; - cache.MissPromotionThreshold = 1; try { - var map = Map.Maps[1]; + var map = TestMap; - // Build 5 distinct chunks by querying different sectors. + // Five chunks into a cache that holds four. for (var i = 0; i < 5; i++) { - var x = 1500 + i * 16; - var y = 1600; - cache.TryGetMask(map, x, y, 10); - System.Threading.Thread.Sleep(2); // ensure LastTouchedTicks differs + cache.TryGetMask(map, PlainX + i * 16, PlainY, sourceZ: 10); + Thread.Sleep(2); // separate their LastTouchedTicks so LRU has something to order by } cache.EnforceLruCap(); Assert.Equal(4, cache.GetStats().ResidentChunks); Assert.True(cache.GetStats().EvictionsByLruCap >= 1L); - - // _keysList must stay in lockstep with _chunks. A desync would silently - // break sampled eviction (KeyNotFoundException on stale keys, or a stuck - // resident set on missing keys). - var chunksField = typeof(StepCache).GetField( - "_chunks", - System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance - ); - var keysListField = typeof(StepCache).GetField( - "_keysList", - System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance - ); - var chunks = (System.Collections.Generic.Dictionary)chunksField.GetValue(cache); - var keysList = (System.Collections.Generic.List)keysListField.GetValue(cache); - Assert.Equal(chunks.Count, keysList.Count); - foreach (var k in keysList) - { - Assert.True(chunks.ContainsKey(k), $"keysList holds key {k} not in _chunks"); - } + Assert.True(cache.ResidentIndexInSync(), "eviction desynced the key list from the resident set"); } finally { diff --git a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheParityTests.cs b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheParityTests.cs index 0dc1eb12f..b38c15277 100644 --- a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheParityTests.cs +++ b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheParityTests.cs @@ -1,125 +1,126 @@ using System; +using System.Collections.Generic; using Server.Engines.Pathing.Cache; using Xunit; using Xunit.Abstractions; +using static Server.Tests.Pathfinding.PathingTestSupport; namespace Server.Tests.Pathfinding; +/// +/// The cache is only worth having if it answers exactly as MovementImpl would. These tests pin +/// that down at each layer, so a failure says which one broke: +/// +/// StepProbe vs MovementImpl — does the bake compute the right answer? +/// StepCache vs StepProbe — does the chunk store and return it intact? +/// StepCache vs MovementImpl — end to end, over the states A* actually visits. +/// +/// The end-to-end test is the one that matters, but it can only tell you something is wrong; the +/// two layer tests tell you where. It also measures coverage, not just correctness — a cache that +/// falls through on everything agrees with the slow path perfectly and is worthless. +/// [Collection("Sequential Pathfinding Tests")] public class StepCacheParityTests { private readonly ITestOutputHelper _output; - public StepCacheParityTests(ITestOutputHelper output) - { - _output = output; - } + public StepCacheParityTests(ITestOutputHelper output) => _output = output; - [Theory] + // ---- layer 1: the bake agrees with MovementImpl ---- + + /// + /// Sweeps a region and compares StepProbe's mask against MovementImpl for all 8 directions. + /// The probe stores raw masks and leaves the diagonal corner-cut to the caller, so the rule has + /// to be applied here before the two are comparable. + /// + [SkippableTheory] [InlineData("britain_inn_dense", 1480, 1610, 32)] [InlineData("trammel_open_plain", 1500, 1600, 32)] - [InlineData("britain_causeway", 1475, 1641, 32)] - public void CacheMatchesBaker(string label, int xStart, int yStart, int size) + public void ProbeMatchesSlowPath(string label, int xStart, int yStart, int size) { - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 1; // sweep cells expecting cache to answer immediately + TileDataRequirement.SkipIfMissing(); - var map = Map.Maps[1]; + var map = TestMap; Assert.NotNull(map); + var walker = new StaticWalker(); + walker.MoveToWorld(new Point3D(xStart, yStart, 0), map); + var disagreements = 0; var samples = 0; - var multiZ = 0; - var wetCells = 0; - - // The cache anchors each cell at the surface a creature actually STANDS on - // (clearance-aware), not the land average. Query at that same standable Z so the - // source-Z guard doesn't false-positive (e.g. on a raised causeway or sewer walkway - // whose surface sits well above the land). Cells with no standable walk surface are - // skipped — there's nothing for a walker to compare against. - Span surfZ = stackalloc sbyte[16]; + var walkable = 0; for (var x = xStart; x < xStart + size; x++) { for (var y = yStart; y < yStart + size; y++) { - if (StepProbe.ComputeStandableSurfaceZs(map, x, y, surfZ) == 0) + map.GetAverageZ(x, y, out _, out var avgZ, out _); + var sourceZ = (sbyte)avgZ; + var loc = new Point3D(x, y, sourceZ); + + var probe = StepProbe.ComputeMaskAt(map, x, y, sourceZ); + + for (var d = 0; d < 8; d++) { - continue; - } - var sourceZ = surfZ[0]; + var dir = (Direction)d; + samples++; - var baker = StepProbe.ComputeMaskAt(map, x, y, sourceZ); + var slowOk = Movement.Movement.CheckMovement(walker, map, loc, dir, out var slowZ); - var lookup = cache.TryGetMask(map, x, y, sourceZ); + // Creature corner-cut: a diagonal needs at least one flanking cardinal. + var probeOk = probe.IsWalkable(dir); + if (probeOk && (d & 1) == 1) + { + probeOk = probe.IsWalkable((Direction)((d - 1) & 7)) || probe.IsWalkable((Direction)((d + 1) & 7)); + } - samples++; + if (slowOk) + { + walkable++; + } - if (lookup.HitKind == CacheHitKind.Fallthrough_MultiZ) - { - multiZ++; - continue; - } - - Assert.True(lookup.IsHit, $"Cache returned !ok at ({x},{y}) hitKind={lookup.HitKind}"); - - if (lookup.WalkMask != baker.WalkMask) - { - disagreements++; - _output.WriteLine($"WALK MASK DIFF @ ({x},{y}) cache=0x{lookup.WalkMask:X2} baker=0x{baker.WalkMask:X2}"); - continue; - } - - if (lookup.WetMask != baker.WetMask) - { - disagreements++; - _output.WriteLine($"WET MASK DIFF @ ({x},{y}) cache=0x{lookup.WetMask:X2} baker=0x{baker.WetMask:X2}"); - continue; - } - - if (lookup.WetMask != 0) - { - wetCells++; - } - - if (lookup.WalkZ_N != baker.WalkZ_N - || lookup.WalkZ_NE != baker.WalkZ_NE || lookup.WalkZ_E != baker.WalkZ_E - || lookup.WalkZ_SE != baker.WalkZ_SE || lookup.WalkZ_S != baker.WalkZ_S - || lookup.WalkZ_SW != baker.WalkZ_SW || lookup.WalkZ_W != baker.WalkZ_W - || lookup.WalkZ_NW != baker.WalkZ_NW) - { - disagreements++; - _output.WriteLine($"Z DIFF @ ({x},{y}) cache=({lookup.WalkZ_N},{lookup.WalkZ_NE},{lookup.WalkZ_E},{lookup.WalkZ_SE},{lookup.WalkZ_S},{lookup.WalkZ_SW},{lookup.WalkZ_W},{lookup.WalkZ_NW}) baker=({baker.WalkZ_N},{baker.WalkZ_NE},{baker.WalkZ_E},{baker.WalkZ_SE},{baker.WalkZ_S},{baker.WalkZ_SW},{baker.WalkZ_W},{baker.WalkZ_NW})"); + if (slowOk != probeOk) + { + disagreements++; + _output.WriteLine($"WALKABLE DIFF @ ({x},{y},{sourceZ}) dir={dir} slow={slowOk} probe={probeOk}"); + } + else if (slowOk && slowZ != probe.GetWalkZ(dir)) + { + disagreements++; + _output.WriteLine($"Z DIFF @ ({x},{y},{sourceZ}) dir={dir} slow={slowZ} probe={probe.GetWalkZ(dir)}"); + } } } } - _output.WriteLine($"[{label}] samples={samples} disagreements={disagreements} multiZ={multiZ} wetCells={wetCells}"); + walker.Delete(); + _output.WriteLine($"[{label}] samples={samples} walkable={walkable} disagreements={disagreements}"); - // Non-vacuity: at least the inn region must have at least one cell that produced a real cache answer. + // The dense region must contain a mix. All-walkable or all-blocked would mean the sweep + // agreed about nothing interesting. if (label == "britain_inn_dense") { - Assert.True(samples - multiZ > 0, "expected real cache answers in dense region"); + Assert.NotEqual(0, walkable); + Assert.NotEqual(samples, walkable); } Assert.Equal(0, disagreements); } /// - /// Non-vacuity guard for the swim bake: scans a wide swath of the south-Britain bay - /// (Atlantic coast) and asserts at least one cell has a non-zero WetMask. Catches the - /// failure mode where StepProbe silently bakes zero swim output everywhere. + /// The swim bake must actually produce swim output. A probe that silently returned an empty + /// WetMask everywhere would pass every parity test above — walkers would still agree — while + /// leaving every swimming creature unable to move. /// [SkippableFact] - public void SwimBake_ProducesWetCells_OnKnownWaterRegion() + public void ProbeBakesWetCells_OnAKnownCoastline() { TileDataRequirement.SkipIfMissing(); - var map = Map.Maps[1]; + + var map = TestMap; Assert.NotNull(map); - // South Britain → Britain bay, includes Atlantic shoreline. 64×64 = 4096 cells; - // even a partial coastline straddle should yield dozens of wet cells. + // South Britain into Britain bay: 64x64 straddling the Atlantic shoreline. const int xStart = 1430; const int yStart = 1740; const int size = 64; @@ -131,15 +132,240 @@ public class StepCacheParityTests { map.GetAverageZ(x, y, out _, out var avgZ, out _); var sourceZ = (sbyte)StepProbe.ComputeStandingZ(map, x, y, avgZ); - var baker = StepProbe.ComputeMaskAt(map, x, y, sourceZ); - if (baker.WetMask != 0) + if (StepProbe.ComputeMaskAt(map, x, y, sourceZ).WetMask != 0) { wetCells++; } } } - _output.WriteLine($"south-britain swim probe: wetCells={wetCells} of 4096"); - Assert.True(wetCells > 0, "swim bake produced zero wet cells across a 64×64 coastal region"); + _output.WriteLine($"south-britain coastline: wetCells={wetCells} of {size * size}"); + Assert.True(wetCells > 0, $"swim bake produced zero wet cells across a {size}x{size} coastal region"); + } + + // ---- layer 2: the chunk returns what was baked ---- + + /// + /// Sweeps a region and compares what the cache serves against what StepProbe computes for the + /// same cell. The chunk is built from the probe, so any disagreement is a storage fault — a + /// bad cell index, a Z array crossed with another, a guard firing when it shouldn't. + /// + /// Queries run at the cell's standable surface Z, which is where the cache anchors. Querying at + /// the land average instead would trip the source-Z guard on raised terrain (a causeway, a + /// walkway) and report a fallthrough that is correct behaviour rather than a fault. + /// + [Theory] + [InlineData("britain_inn_dense", 1480, 1610, 32)] + [InlineData("trammel_open_plain", 1500, 1600, 32)] + [InlineData("britain_causeway", 1475, 1641, 32)] + public void CacheMatchesProbe(string label, int xStart, int yStart, int size) + { + var cache = StepCache.Instance; + cache.Clear(); + cache.MissPromotionThreshold = 1; // build on first touch: every cell should get a real answer + + var map = TestMap; + Assert.NotNull(map); + + var disagreements = 0; + var samples = 0; + var multiZ = 0; + + Span surfaces = stackalloc sbyte[16]; + + for (var x = xStart; x < xStart + size; x++) + { + for (var y = yStart; y < yStart + size; y++) + { + if (StepProbe.ComputeStandableSurfaceZs(map, x, y, surfaces) == 0) + { + continue; // nothing for a walker to stand on here + } + + var sourceZ = surfaces[0]; + var probe = StepProbe.ComputeMaskAt(map, x, y, sourceZ); + var cached = cache.TryGetMask(map, x, y, sourceZ); + samples++; + + if (cached.HitKind == CacheHitKind.Fallthrough_MultiZ) + { + multiZ++; + continue; + } + + Assert.True(cached.IsHit, $"cache returned {cached.HitKind} at ({x},{y})"); + + if (cached.WalkMask != probe.WalkMask) + { + disagreements++; + _output.WriteLine($"WALK MASK DIFF @ ({x},{y}) cache=0x{cached.WalkMask:X2} probe=0x{probe.WalkMask:X2}"); + continue; + } + + if (cached.WetMask != probe.WetMask) + { + disagreements++; + _output.WriteLine($"WET MASK DIFF @ ({x},{y}) cache=0x{cached.WetMask:X2} probe=0x{probe.WetMask:X2}"); + continue; + } + + for (var d = 0; d < 8; d++) + { + var dir = (Direction)d; + if (cached.GetWalkZ(dir) != probe.GetWalkZ(dir)) + { + disagreements++; + _output.WriteLine( + $"Z DIFF @ ({x},{y}) dir={dir} cache={cached.GetWalkZ(dir)} probe={probe.GetWalkZ(dir)}" + ); + break; + } + } + } + } + + _output.WriteLine($"[{label}] samples={samples} disagreements={disagreements} multiZ={multiZ}"); + + // Guard the sweep itself: if every cell fell through as multi-Z, the comparison above never + // actually ran and a zero disagreement count would mean nothing. + if (label == "britain_inn_dense") + { + Assert.True(samples - multiZ > 0, "no cell produced a real cache answer — the sweep proved nothing"); + } + + Assert.Equal(0, disagreements); + } + + // ---- layer 3: end to end, over the states A* actually visits ---- + + /// + /// Flood-fills outward from a known-walkable tile using MovementImpl itself, and demands the + /// cache serve — and agree on — every state it reaches. + /// + /// The fill is what makes this meaningful. MovementImpl returns the Z a step lands on, so each + /// reached (x, y, z) is a genuine standing state at its true Z: exactly the set A* would query, + /// discovered rather than assumed. It follows stair treads up at their own Zs and climbs onto + /// upper floors, so a single seed covers a whole connected structure with no fixed-Z guess to + /// get wrong. That matters because the failure this test exists to catch — anchoring a cell at + /// the land beneath a walkway instead of the walkway itself — is invisible to any test that + /// queries at the land Z, and turned the Britain sewer into a ~98% cache miss. + /// + /// Cardinals only: the cache stores raw masks and applies the corner-cut at query time, so a + /// raw diagonal bit legitimately differs from MovementImpl's diagonal answer. + /// + [Theory] + // Seeds chosen for the terrain classes the standable-surface bake has to get right. Each one + // floods across a wide local area, so a handful covers thousands of states without a map walk. + [InlineData("brit_sewer_walkway", 6034, 1476, 5, 2500)] // static walkway over impassable land + [InlineData("brit_inn_stairs_to_floors", 1495, 1628, 10, 2500)] // stairs up to multi-Z upper floors + [InlineData("brit_town_cobblestones", 1494, 1626, 10, 2500)] // mixed buildings, stairs, raised floors + [InlineData("trammel_open_plain", 1500, 1600, 10, 2500)] // flat ground: catches clearance false-positives + public void CacheServesReachableWalkStates(string label, int sx, int sy, int sz, int maxStates) + { + var cache = StepCache.Instance; + cache.Clear(); + cache.MissPromotionThreshold = 1; // build on first touch: every reached state should be answered + + var map = TestMap; + Assert.NotNull(map); + + var walker = new StaticWalker(); + walker.MoveToWorld(new Point3D(sx, sy, sz), map); + + var startIsWalkable = false; + for (var d = 0; d < 8 && !startIsWalkable; d++) + { + startIsWalkable = Movement.Movement.CheckMovement(walker, map, new Point3D(sx, sy, sz), (Direction)d, out _); + } + + Assert.True(startIsWalkable, $"[{label}] seed ({sx},{sy},{sz}) is not walkable — bad waypoint"); + + var visited = new HashSet<(int x, int y, int z)> { (sx, sy, sz) }; + var frontier = new Queue<(int x, int y, int z)>(); + frontier.Enqueue((sx, sy, sz)); + + var states = 0; + var fellThrough = 0; + var disagreements = 0; + const int maxLog = 12; + + while (frontier.Count > 0) + { + var (x, y, z) = frontier.Dequeue(); + var loc = new Point3D(x, y, z); + var cached = cache.TryGetMask(map, x, y, (sbyte)z); + states++; + + if (!cached.IsHit) + { + if (fellThrough < maxLog) + { + _output.WriteLine($"FELL THROUGH @ ({x},{y},{z}) hitKind={cached.HitKind}"); + } + + fellThrough++; + } + + for (var d = 0; d < 8; d++) + { + var dir = (Direction)d; + var slowOk = Movement.Movement.CheckMovement(walker, map, loc, dir, out var slowZ); + + if (slowOk) + { + var nx = x; + var ny = y; + Movement.Movement.Offset(dir, ref nx, ref ny); + + if (visited.Count < maxStates && visited.Add((nx, ny, slowZ))) + { + frontier.Enqueue((nx, ny, slowZ)); + } + } + + if ((d & 1) != 0 || !cached.IsHit) + { + continue; + } + + if (cached.IsWalkable(dir) != slowOk) + { + if (disagreements < maxLog) + { + _output.WriteLine($"WALK DIFF @ ({x},{y},{z}) dir={dir} slow={slowOk} cache={cached.IsWalkable(dir)}"); + } + + disagreements++; + } + else if (slowOk && slowZ != cached.GetWalkZ(dir)) + { + if (disagreements < maxLog) + { + _output.WriteLine($"Z DIFF @ ({x},{y},{z}) dir={dir} slow={slowZ} cache={cached.GetWalkZ(dir)}"); + } + + disagreements++; + } + } + } + + walker.Delete(); + + var fallthroughPct = states == 0 ? 0 : 100.0 * fellThrough / states; + _output.WriteLine($"[{label}] states={states} fellThrough={fellThrough} ({fallthroughPct:F2}%) disagreements={disagreements}"); + + Assert.True(states > 50, $"[{label}] flood-fill stalled at {states} states — bad waypoint"); + + // Where the cache answers at all, it must be right. + Assert.Equal(0, disagreements); + + // And it must answer nearly everywhere. A small residual is legitimate: a walkable surface + // directly beneath a bridge or stair ramp falls through because the bake's clearance check + // is deliberately conservative there. An anchor regression is not small — the pre-fix sewer + // fell through on ~98% — so a 1% ceiling separates the two comfortably. + Assert.True( + fallthroughPct < 1.0, + $"[{label}] cache fell through on {fallthroughPct:F2}% ({fellThrough}/{states}) of reachable states" + ); } } diff --git a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheStaticSurfaceParityTests.cs b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheStaticSurfaceParityTests.cs deleted file mode 100644 index 8f05291ae..000000000 --- a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepCacheStaticSurfaceParityTests.cs +++ /dev/null @@ -1,183 +0,0 @@ -using System.Collections.Generic; -using Server.Engines.Pathing.Cache; -using Xunit; -using Xunit.Abstractions; - -namespace Server.Tests.Pathfinding; - -/// -/// Parity coverage for "walkable static surface above a land tile" terrain — sewers, -/// dungeon walkways, bridges, raised foundations, and stacked building floors. -/// -/// The original parity tests only queried at the LAND-anchored standing Z and skipped -/// multi-Z fallthroughs, so they never noticed that a query at the REAL walk Z — the static -/// surface a creature actually stands on — returns -/// , because the baker anchored -/// SourceZ at the land average instead of the walkway. In the Britain sewer that's a ~98% -/// cache miss on a known walk-path (confirmed via [PathDiag). -/// -/// Method: flood-fill outward from a known-walkable start using -/// — the slow path the cache mirrors. Each -/// reached (x, y, z) is a genuine standing state at its TRUE Z (CheckMovement returns the -/// destination Z it lands on), exactly the set of states A* would query. For every reached -/// state the cache must serve a Hit and agree with the slow path. This naturally follows -/// ramped stairs (each tread at its own Z) and climbs to upper floors, so one start covers -/// the whole connected structure — no fragile fixed-Z assumption. -/// -/// A bare test world has no spawned items/mobiles, so CheckMovement reduces to static -/// walkability (no door/dynamic interference). Parity restricted to cardinal directions: -/// the cache stores raw masks and applies the diagonal corner-cut at query time, so a raw -/// diagonal bit legitimately differs from CheckMovement's diagonal result. -/// -/// EXPECTED: RED before the standable-surface bake (reached states fall through at their -/// true Z); GREEN after. -/// -[Collection("Sequential Pathfinding Tests")] -public class StepCacheStaticSurfaceParityTests -{ - private readonly ITestOutputHelper _output; - - public StepCacheStaticSurfaceParityTests(ITestOutputHelper output) - { - _output = output; - } - - [Theory] - // label, start X, Y, Z (a real in-game walkable tile), max states to explore. Seeds are - // chosen to span the terrain classes the standable-surface bake must get right; the - // flood-fill spreads from each across a wide local area, so a handful of seeds exercises - // thousands of distinct (cell, Z) states without an exhaustive whole-map walk. - // sewer — static walkway @ z=5 over impassable land; covers dungeon walkways + bridges. - // inn — stair foot @ z=10; climbs the stairs onto the 1st & 2nd floors (multi-Z). - // plain — open Britain ground; guards against clearance false-positives on flat land. - // town — Britain cobblestones near the inn; mixed buildings, stairs, raised floors. - [InlineData("brit_sewer_walkway", 6034, 1476, 5, 2500)] - [InlineData("brit_inn_stairs_to_floors", 1495, 1628, 10, 2500)] - [InlineData("trammel_open_plain", 1500, 1600, 10, 2500)] - [InlineData("brit_town_cobblestones", 1494, 1626, 10, 2500)] // plain ground: guards against clearance false-positives - public void CacheServesReachableWalkStates(string label, int sx, int sy, int sz, int maxStates) - { - var cache = StepCache.Instance; - cache.Clear(); - cache.MissPromotionThreshold = 1; // eager build — expect the cache to answer every state - - var map = Map.Maps[1]; - Assert.NotNull(map); - - var stub = new ParityStubMobile(); - stub.MoveToWorld(new Point3D(sx, sy, sz), map); - - // Sanity: the start must itself be a walkable standing state via the slow path. - var startWalkable = false; - for (var d = 0; d < 8; d++) - { - if (Movement.Movement.CheckMovement(stub, map, new Point3D(sx, sy, sz), (Direction)d, out _)) - { - startWalkable = true; - break; - } - } - Assert.True(startWalkable, $"[{label}] start ({sx},{sy},{sz}) is not walkable per the slow path — bad waypoint"); - - var visited = new HashSet<(int x, int y, int z)>(); - var queue = new Queue<(int x, int y, int z)>(); - visited.Add((sx, sy, sz)); - queue.Enqueue((sx, sy, sz)); - - var states = 0; - var fellThrough = 0; - var disagreements = 0; - const int maxLog = 12; - - while (queue.Count > 0) - { - var (x, y, z) = queue.Dequeue(); - states++; - - var loc = new Point3D(x, y, z); - var lookup = cache.TryGetMask(map, x, y, (sbyte)z); - - if (!lookup.IsHit) - { - if (fellThrough < maxLog) - { - _output.WriteLine($"FELL THROUGH @ ({x},{y},{z}) hitKind={lookup.HitKind}"); - } - fellThrough++; - } - - for (var d = 0; d < 8; d++) - { - var dir = (Direction)d; - var slowOk = Movement.Movement.CheckMovement(stub, map, loc, dir, out var nz); - - // Expand the frontier through every legal move (incl. diagonals). - if (slowOk) - { - var nx = x; - var ny = y; - Movement.Movement.Offset(dir, ref nx, ref ny); - var next = (nx, ny, (int)nz); - if (visited.Count < maxStates && visited.Add(next)) - { - queue.Enqueue(next); - } - } - - // Parity on cardinals only (diagonals carry the query-time corner-cut rule). - if ((d & 1) == 0 && lookup.IsHit) - { - var cacheOk = lookup.IsWalkable(dir); - if (cacheOk != slowOk) - { - if (disagreements < maxLog) - { - _output.WriteLine($"WALK DIFF @ ({x},{y},{z}) dir={dir} slow={slowOk} cache={cacheOk}"); - } - disagreements++; - } - else if (slowOk && nz != lookup.GetWalkZ(dir)) - { - if (disagreements < maxLog) - { - _output.WriteLine($"Z DIFF @ ({x},{y},{z}) dir={dir} slow={nz} cache={lookup.GetWalkZ(dir)}"); - } - disagreements++; - } - } - } - } - - stub.Delete(); - - var fallthroughPct = states == 0 ? 0 : 100.0 * fellThrough / states; - _output.WriteLine($"[{label}] states={states} fellThrough={fellThrough} ({fallthroughPct:F2}%) disagreements={disagreements}"); - - Assert.True(states > 50, $"[{label}] only explored {states} states — flood-fill stalled, bad waypoint"); - - // Correctness is strict: where the cache DOES answer, it must agree with the slow path. - Assert.Equal(0, disagreements); - - // Coverage: nearly every reachable state should be cache-served. A small residual is - // expected and acceptable — a walkable surface sitting directly under a bridge/stair - // ramp falls through to the slow path (correct, just uncached) because the bake's - // clearance check is intentionally conservative there. A real anchor regression shows - // up as a large fraction (the pre-fix sewer was ~98%), which this still catches. - Assert.True( - fallthroughPct < 1.0, - $"[{label}] cache fell through on {fallthroughPct:F2}% ({fellThrough}/{states}) of reachable states — coverage regression" - ); - } - - /// - /// Default static walker: inherits straight from Mobile so MovementImpl sees no - /// BaseCreature flags (CanSwim/CanFly false, bc==null). Mirrors the existing parity stub. - /// - private class ParityStubMobile : Mobile - { - public ParityStubMobile() - { - Body = 0xC9; - } - } -} diff --git a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepProbeParityTests.cs b/Projects/UOContent.Tests/Tests/Engines/Pathing/StepProbeParityTests.cs deleted file mode 100644 index 429d60ca7..000000000 --- a/Projects/UOContent.Tests/Tests/Engines/Pathing/StepProbeParityTests.cs +++ /dev/null @@ -1,127 +0,0 @@ -using Server.Engines.Pathing.Cache; -using Xunit; -using Xunit.Abstractions; - -namespace Server.Tests.Pathfinding; - -[Collection("Sequential Pathfinding Tests")] -public class StaticWalkabilityParityTests -{ - private readonly ITestOutputHelper _output; - - public StaticWalkabilityParityTests(ITestOutputHelper output) - { - _output = output; - } - - [SkippableTheory] - [InlineData("britain_inn_dense", 1480, 1610, 32)] - [InlineData("trammel_open_plain", 1500, 1600, 32)] - public void BakerMatchesCheckMovement(string label, int xStart, int yStart, int size) - { - TileDataRequirement.SkipIfMissing(); - var map = Map.Maps[1]; - Assert.NotNull(map); - - var stub = new ParityStubMobile(); - stub.MoveToWorld(new Point3D(xStart, yStart, 0), map); - - var disagreements = 0; - var samples = 0; - var oldWalkable = 0; - var newWalkable = 0; - - for (var x = xStart; x < xStart + size; x++) - { - for (var y = yStart; y < yStart + size; y++) - { - map.GetAverageZ(x, y, out _, out var avgZ, out _); - var sourceZ = (sbyte)avgZ; - - var loc = new Point3D(x, y, sourceZ); - - var bakerResult = StepProbe.ComputeMaskAt(map, x, y, sourceZ); - - for (var d = 0; d < 8; d++) - { - var dir = (Direction)d; - samples++; - - var oldOk = Movement.Movement.CheckMovement(stub, map, loc, dir, out var oldZ); - - // Apply creature diagonal corner-cut rule at query time: - // diagonal walkable iff raw-diagonal AND (left-partner OR right-partner). - // (Raw masks are correct per spec; baker omits diagonal logic per design.) - var newOk = bakerResult.IsWalkable(dir); - if (newOk && (d & 1) == 1) - { - var leftPartner = (Direction)((d - 1) & 7); - var rightPartner = (Direction)((d + 1) & 7); - if (!bakerResult.IsWalkable(leftPartner) && !bakerResult.IsWalkable(rightPartner)) - { - newOk = false; - } - } - - var newZ = bakerResult.GetWalkZ(dir); - - if (oldOk) - { - oldWalkable++; - } - if (newOk) - { - newWalkable++; - } - - if (oldOk != newOk) - { - disagreements++; - _output.WriteLine( - $"DISAGREE walkable @ ({x},{y},{sourceZ}) dir={dir}: " + - $"old={oldOk} new={newOk}" - ); - } - else if (oldOk && oldZ != newZ) - { - disagreements++; - _output.WriteLine( - $"DISAGREE destZ @ ({x},{y},{sourceZ}) dir={dir}: " + - $"old={oldZ} new={newZ}" - ); - } - } - } - } - - stub.Delete(); - - _output.WriteLine( - $"[{label}] Samples: {samples}, Disagreements: {disagreements}, " + - $"OldWalkable: {oldWalkable}, NewWalkable: {newWalkable}" - ); - - // Non-vacuity guard for the variety case: at least one region must show some - // blocked directions. The open_plain region is allowed to be all-walkable. - if (label == "britain_inn_dense") - { - Assert.NotEqual(0, oldWalkable); - Assert.NotEqual(samples, oldWalkable); - } - - Assert.Equal(0, disagreements); - } - - /// - /// Minimal Mobile stub for parity testing. Inherits directly from Mobile so that - /// MovementImpl sees no BaseCreature-specific flags (CanSwim=false, CanFly=false, - /// bc==null → BaseCreature branches skipped) giving us the default static walker baseline. - /// - private class ParityStubMobile : Mobile - { - public ParityStubMobile() - { - Body = 0xC9; // arbitrary horse body - } - } -} diff --git a/Projects/UOContent/Engines/Pathing/BitmapAStarAlgorithm.cs b/Projects/UOContent/Engines/Pathing/BitmapAStarAlgorithm.cs index 9008c18bd..2c830db20 100644 --- a/Projects/UOContent/Engines/Pathing/BitmapAStarAlgorithm.cs +++ b/Projects/UOContent/Engines/Pathing/BitmapAStarAlgorithm.cs @@ -26,12 +26,11 @@ using MoveImpl = Server.Movement.MovementImpl; namespace Server.PathAlgorithms; /// -/// A* pathfinder with a single bitmap-cache lookup per cell expansion. Default walkers -/// take one call returning the 8-direction -/// mask + per-direction Z. Non-default walkers (non-GM players, creatures with swim/fly/ -/// door/clip capabilities) and per-cell cache fallthroughs route through -/// , which runs the per-direction -/// loop for that one cell. +/// A* pathfinder that expands a cell with a single lookup, +/// which returns all 8 directions' walkability and destination Zs at once. Where the cache can't +/// answer — a fallthrough on that cell, or a flying creature the static cache can't model — +/// runs the per-direction +/// loop for that one cell instead, so a partial cache miss costs only the cells it affects. /// public class BitmapAStarAlgorithm : PathAlgorithm { @@ -50,61 +49,50 @@ public class BitmapAStarAlgorithm : PathAlgorithm private const int PlaneOffset = 128; private const int PlaneCount = 13; private const int PlaneHeight = 20; - // Default shared singleton (MaxSearchNodes = 1000, set from config in Configure). Typed - // as the concrete class so Configure can set its instance config; assignable anywhere a - // PathAlgorithm is expected. Specialized variants are just additional instances. + // The shared default. A differently-configured variant is just another instance. public static readonly BitmapAStarAlgorithm Instance = new(); - // Scratch buffers — reused across every Find on THIS instance. Per-instance (not static) - // so independently-configured algorithms don't share state. ~320 KB per instance; create - // specialized instances once (static readonly), never per-call. Safe to reuse per Find - // because the game loop is single-threaded and Find is never re-entered. + // Scratch reused across every Find on this instance — roughly 320 KB of it, so create + // instances once and hold them, never per call. Per-instance rather than static so two + // differently-configured algorithms don't share state. Reuse is safe because the game loop is + // single-threaded and Find never re-enters. private readonly Direction[] _path = new Direction[AreaSize * AreaSize]; private readonly PathNode[] _nodes = new PathNode[NodeCount]; private readonly byte[] _nodeStates = new byte[NodeCount]; private readonly int[] _successors = new int[8]; private readonly PriorityQueue _openQueue = new(); - // A* node-expansion budget: the search bails (returning null) after this many node - // expansions. Benchmarked as near-optimal: above the ~500 needed to solve walled-off - // indoor routes, below the ~1500 window-exhaustion cost ceiling where a failed - // (unreachable) search's worst-case cost spikes for no solving benefit. Successful - // searches terminate on goal-found, so this never touches the common open-terrain case. - // Per-instance so specialized algorithms (e.g. a wider-budget variant for special NPCs) - // can coexist; the shared default lives on Instance and is set from config in Configure. + // Expansion budget: the search gives up and returns null past this many nodes. It bounds the + // cost of an unreachable goal, which would otherwise exhaust the whole search window. A + // successful search stops when it finds the goal, so the budget only binds on hard or hopeless + // routes — it needs to stay high enough to solve walled-off indoor ones. public int MaxSearchNodes { get; set; } = 1000; private int _xOffset; private int _yOffset; - // When set, GetSuccessors delegates to the per-cell slow path on every expansion - // (creature has CanFly — Z-jumping is beyond the cache's static-only scope). + // Every expansion goes to the slow path: the creature can fly, and arbitrary Z-jumping is + // outside what a static cache can model. private bool _currentMobileNeedsSlowPath; - // When set, diagonal corner-cut uses the strict AND-rule (BOTH cardinal partners - // must be walkable) instead of the lenient creature OR-rule. Cache still applies — - // partner bits live in the same source-cell mask byte. Non-GM players only. + // Diagonal corner-cut uses the strict rule — both cardinal partners walkable, not just one. + // Non-GM players only. The cache still applies; the partner bits are in the same mask byte. private bool _currentMobilePlayerStrict; - // Capability overlay applied to cache results. Layered each cell: + // Capability overlay on the cache's two rule sets, applied per cell as // effective = (walkMask & !cantWalk) | (wetMask & canSwim) - // Reset at end of Find. private bool _currentMobileCanSwim; private bool _currentMobileCantWalk; - // Dynamic-obstacle pass capability flags (per-mobile, captured in Find). - // Mirrors MovementImpl.Check's per-mobile derivations so per-cell items/mobiles - // checks can be evaluated without re-deriving. + // Per-mobile flags for the dynamic-obstacle pass, derived once in Find rather than per cell. private bool _currentMobileIgnoreDoors; private bool _currentMobileIgnoreSpellFields; private bool _currentMobileIgnoreMovableImpassables; public static void Configure() { - // A* node-expansion budget. Default 1000 is benchmarked near-optimal (see - // MaxSearchNodes). Applied to the shared singleton; specialized instances pass their - // own value. Written back to server.cfg on first boot. Auto-invoked at startup via - // AssemblyHandler.Invoke("Configure"). + // Shard-tunable expansion budget for the shared instance; see MaxSearchNodes. Written back + // to server.cfg on first boot so it's discoverable. Instance.MaxSearchNodes = ServerConfiguration.GetOrUpdateSetting( "pathfinding.maxSearchNodes", 1000 @@ -136,10 +124,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm return null; } - // Mark a new Find generation so the StepCache promotion gate counts THIS pathfind - // as one touch per chunk regardless of how many times the expansion frontier - // probes a given chunk. Without this, A* hits each visited chunk dozens of times - // and trips the threshold immediately. + // The frontier probes a given chunk dozens of times over one search; opening a generation + // is what makes the cache's promotion gate count all of that as a single touch. StepCache.Instance.BeginFindGeneration(); PathfindRecorder.RecordIfEnabled(m, map, start, goal); @@ -161,7 +147,7 @@ public class BitmapAStarAlgorithm : PathAlgorithm _currentMobileIgnoreMovableImpassables = false; } - // Mirrors MovementImpl: dead/spectral mobiles also ignore doors. + // Dead and spectral mobiles pass through doors too. Mirrors MovementImpl. _currentMobileIgnoreDoors |= !m.Alive || m.Body.BodyID == 0x3DB || m.IsDeadBondedPet; _currentMobileIgnoreSpellFields = m is PlayerMobile && map != Map.Felucca; @@ -209,7 +195,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm _nodeStates[bestNode] = 2; - // Set MovementImpl globals so per-cell slow-path fallthroughs see the right state. + // MovementImpl reads these statics, so a slow-path fallthrough on any cell below needs + // them set for this mobile. if (bc != null) { MoveImpl.AlwaysIgnoreDoors = bc.CanOpenDoors; @@ -326,11 +313,10 @@ public class BitmapAStarAlgorithm : PathAlgorithm } /// - /// One call returns the 8-direction - /// walkable mask + destination Zs. Diagonal corner-cut applies the lenient creature - /// OR-rule using partner bits in the same mask byte — no neighbor-chunk lookup needed. - /// On cache fallthrough or for non-default walkers, defers to - /// for THIS cell only. + /// Expands one cell into its walkable neighbours. A single cache lookup covers all 8 + /// directions, including the partner bits the diagonal corner-cut needs, so no neighbouring + /// cell has to be consulted. Falls back to for this cell + /// alone when the cache can't answer. /// private int GetSuccessors(int p, Mobile m, Map map) { @@ -353,16 +339,12 @@ public class BitmapAStarAlgorithm : PathAlgorithm if (!lookup.IsHit) { - // Multi-covered cells: synthesize a multi-aware mask in ONE pass (over land + statics + - // house/boat component tiles) instead of the slow path's 8x per-cell CheckMovement. - // Fliers and cache-off already returned at the top of GetSuccessors, so this only runs - // for cacheable walkers/swimmers. The synthesized mask flows through the SAME - // capability-overlay + diagonal corner-cut + dynamic-obstacle loop below as a static hit. + // A multi-covered cell still gets a whole-cell mask, synthesized over the house or boat + // components rather than looked up. That keeps it out of the slow path's 8 separate + // CheckMovement calls, and the result flows through the same overlay, corner-cut and + // dynamic-obstacle logic below as a cache hit would. if (lookup.HitKind == CacheHitKind.Fallthrough_Multi) { - // Multi-covered cell: the per-multiID interior cache serves a ~20 ns lookup for - // interior cells (and records the right counter); it falls back internally to the - // Phase-2 live synthesizer for perimeter / terrain-dirty / foundation cells. lookup = MultiMaskCache.Instance.GetMask(map, p3D.X, p3D.Y, (sbyte)p3D.Z); } else @@ -371,8 +353,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm } } - // Capability overlay: walking allowed unless cantWalk; swimming allowed if canSwim. - // Partner bits used for diagonal corner-cut also use the effective mask. + // Overlay the mobile's capabilities onto the cache's two rule sets. The corner-cut below + // reads its partner bits from this effective mask, not the raw one. var walkBits = _currentMobileCantWalk ? (byte)0 : lookup.WalkMask; var swimBits = _currentMobileCanSwim ? lookup.WetMask : (byte)0; var mask = (byte)(walkBits | swimBits); @@ -393,9 +375,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm continue; } - // Diagonal corner-cut. Creatures (default): OR-rule — at least one cardinal - // partner walkable. Non-GM players: AND-rule — BOTH partners must be walkable. - // Partner bits live in the same source-cell mask byte either way. + // Diagonal corner-cut: a creature needs at least one of the two flanking cardinals to + // be walkable, a non-GM player needs both. if ((i & 1) == 1) { var leftBit = 1 << ((i - 1) & 0x7); @@ -408,9 +389,9 @@ public class BitmapAStarAlgorithm : PathAlgorithm } } - // Walking takes precedence over swimming when both apply (matches MovementImpl's - // surface-selection: closest-to-startZ wins, and walk surface is always closer - // when the creature is currently standing on land). + // Walking wins over swimming where both are possible. MovementImpl picks the surface + // closest to the start Z, and for a creature standing on land that is always the walk + // surface. var useWalkZ = (walkBits & (1 << i)) != 0; var z = useWalkZ ? i switch @@ -441,8 +422,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm var absX = x + _xOffset; var absY = y + _yOffset; - // Dynamic-obstacle pass: items + mobiles at the target cell. Cache only - // covers static walkability; dynamic state has to be checked at query time. + // The cache only knows static terrain, so items and mobiles at the target cell have to + // be checked live. if (IsBlockedByDynamic(m, map, absX, absY, z)) { continue; @@ -464,11 +445,9 @@ public class BitmapAStarAlgorithm : PathAlgorithm private const int MobileHeight = 15; /// - /// Mirrors MovementImpl's dynamic-item / mobile collision phase for a target cell. - /// Items: ImpassableSurface that overlap (z, z+PersonHeight), respecting capability - /// overrides (CanOpenDoors → ignore door items; CanMoveOverObstacles → ignore movables; - /// non-Felucca players → ignore spell fields). Mobiles: any other mobile whose Z range - /// overlaps and which we can't move over. + /// MovementImpl's item and mobile collision phase for one target cell: impassable items + /// overlapping the mobile's vertical envelope block it, subject to the capability overrides, + /// as does any other mobile it can't move over. /// private bool IsBlockedByDynamic(Mobile m, Map map, int x, int y, int z) { @@ -509,9 +488,9 @@ public class BitmapAStarAlgorithm : PathAlgorithm } } - // A* must be able to plan a path to the goal cell even when the target mobile is - // standing on it (the follower stops within range short of it). Skip the mob-block - // check at the goal cell ONLY; everywhere else dynamic mobiles still block. + // The goal cell is usually occupied by whatever the mobile is chasing, so blocking on it + // would fail every pursuit. The follower stops short of the goal anyway. Every other cell + // still blocks on mobiles. var skipMobCheck = x == MoveImpl.Goal.X && y == MoveImpl.Goal.Y; if (!skipMobCheck) @@ -534,18 +513,16 @@ public class BitmapAStarAlgorithm : PathAlgorithm } /// - /// Mirrors MovementImpl.CanMoveOver — true when m can step onto t's cell (dead bodies, - /// hidden staff, etc.). + /// True when m can step onto t's cell — a corpse, hidden staff, and so on. Mirrors + /// MovementImpl.CanMoveOver. /// private static bool CanMoveOver(Mobile m, Mobile t) => !t.Alive || !m.Alive || t.IsDeadBondedPet || m.IsDeadBondedPet || t.Hidden && t.AccessLevel > AccessLevel.Player; /// - /// Per-direction loop for a single source cell. - /// Runs on cache fallthrough or when is set. - /// CheckMovement validates land/statics/items via MovementImpl; dynamic mobile blocking - /// is layered on top because MovementImpl doesn't iterate same-cell mobiles. + /// Expands one cell the long way, with a CheckMovement call per direction. The same-cell + /// mobile check is layered on top because MovementImpl doesn't iterate those. /// private int GetSuccessorsSlowPath(Mobile m, Map map, int px, int py, Point3D p3D, int[] vals) { @@ -586,11 +563,10 @@ public class BitmapAStarAlgorithm : PathAlgorithm } /// - /// True for creatures whose movement rules the static cache can't model. Currently - /// only CanFly — flying creatures Z-jump arbitrarily and the cache's source-Z guard - /// would over-fire. CanSwim / CantWalk are handled via the capability overlay (walkMask - /// + wetMask). CanOpenDoors / CanMoveOverObstacles only affect dynamic items and don't - /// disqualify the cache. + /// True for creatures the static cache can't model at all. Only flying ones qualify: they + /// Z-jump freely, so the cache's source-Z guard would reject nearly every cell anyway. Swim + /// and cant-walk are handled by the capability overlay, and the door / obstacle capabilities + /// only affect dynamic items, so none of those disqualify the cache. /// private static bool RequiresSlowPath(Mobile m) => m is BaseCreature bc && bc.CanFly; } diff --git a/Projects/UOContent/Engines/Pathing/Cache/CacheEvictionTimer.cs b/Projects/UOContent/Engines/Pathing/Cache/CacheEvictionTimer.cs index 031d3ed21..e1cdd2261 100644 --- a/Projects/UOContent/Engines/Pathing/Cache/CacheEvictionTimer.cs +++ b/Projects/UOContent/Engines/Pathing/Cache/CacheEvictionTimer.cs @@ -3,9 +3,8 @@ using System; namespace Server.Engines.Pathing.Cache; /// -/// Periodic backstop that enforces the StaticWalkabilityCache resident-chunk cap. -/// Steady-state cost is a single early-return; only fires real work when the cache -/// has overflowed MaxResidentChunks. Runs on the game thread; no locking required. +/// Periodic backstop that enforces 's resident-chunk cap. Costs a +/// single early-return unless the cache has overflowed MaxResidentChunks. /// public class CacheEvictionTimer : Timer { diff --git a/Projects/UOContent/Engines/Pathing/Cache/CacheHitKind.cs b/Projects/UOContent/Engines/Pathing/Cache/CacheHitKind.cs index ad4d8b3e7..f851559b1 100644 --- a/Projects/UOContent/Engines/Pathing/Cache/CacheHitKind.cs +++ b/Projects/UOContent/Engines/Pathing/Cache/CacheHitKind.cs @@ -1,18 +1,18 @@ namespace Server.Engines.Pathing.Cache; /// -/// Outcome categories for StepCache.TryGetMask. Used for telemetry and to drive -/// the slow-path fallthrough decision in callers. Ordering is load-bearing: -/// values 0-2 are hits, values 3+ are fallthroughs (see StepMask.IsHit). +/// Outcome of . Drives both telemetry and the caller's +/// decision to fall back to the slow path. Ordering is load-bearing: 0-2 are usable answers, +/// 3+ are fallthroughs, and tests that boundary. /// public enum CacheHitKind : byte { - Hit = 0, // clean default-walker answer from the resident chunk + Hit = 0, // served from the resident chunk Miss_NotBuilt = 1, // chunk wasn't resident; built and returned - Miss_DirtyRebuild = 2, // version mismatch; rebuilt and returned - Fallthrough_MultiZ = 3, // cell has multiple walkable surfaces; caller must use slow path + Miss_DirtyRebuild = 2, // chunk was stale; rebuilt and returned + Fallthrough_MultiZ = 3, // stacked walkable surfaces, none matching the query Z Fallthrough_OffMap = 4, // out of bounds - Fallthrough_SourceZMismatch = 5, // |loc.Z - BakedSourceZ| > StepHeight; cache answer would diverge - Fallthrough_NotBuilt = 6, // first-touch miss without lazy file hit; build deferred until second touch - Fallthrough_Multi = 7, // a multi (house/boat) covers this cell or its halo; use the live path + Fallthrough_SourceZMismatch = 5, // |query Z - baked SourceZ| > StepHeight; a cached answer would diverge + Fallthrough_NotBuilt = 6, // first touch of an unbuilt chunk; the promotion gate defers the build + Fallthrough_Multi = 7, // a multi (house/boat) covers this cell or its halo } diff --git a/Projects/UOContent/Engines/Pathing/Cache/CacheStats.cs b/Projects/UOContent/Engines/Pathing/Cache/CacheStats.cs index 72c225423..419481e87 100644 --- a/Projects/UOContent/Engines/Pathing/Cache/CacheStats.cs +++ b/Projects/UOContent/Engines/Pathing/Cache/CacheStats.cs @@ -1,8 +1,9 @@ namespace Server.Engines.Pathing.Cache; /// -/// Snapshot of StaticWalkabilityCache counters. Returned by GetStats() and consumed -/// by the [PathCacheStats admin command. All counters are monotonic except ResidentChunks. +/// Snapshot of 's counters, as returned by GetStats() and reported by +/// the [PathCacheStats command. Every counter is monotonic except ResidentChunks, and all of +/// them reset on Clear() — they count since the last clear, not since startup. /// public readonly struct CacheStats( int residentChunks, diff --git a/Projects/UOContent/Engines/Pathing/Cache/MultiMaskCache.cs b/Projects/UOContent/Engines/Pathing/Cache/MultiMaskCache.cs index efb047d23..9cfd32f8c 100644 --- a/Projects/UOContent/Engines/Pathing/Cache/MultiMaskCache.cs +++ b/Projects/UOContent/Engines/Pathing/Cache/MultiMaskCache.cs @@ -6,13 +6,19 @@ using Server.Multis; namespace Server.Engines.Pathing.Cache; /// -/// Warm, in-memory cache of per-multiID local-frame walkability masks for INTERIOR multi cells -/// (cell + all 8 neighbours covered by the multi → terrain-neighbour-free → position-invariant). -/// Wraps the Phase-2 synthesizer (StepProbe.ComputeMultiMaskAt). Cleanliness is decided ONCE per -/// instance (BaseMulti.PathInteriorCacheState, via ComputeFootprintClean): a clean instance — whole -/// footprint terrain below the floor — serves interior cells from the shared per-multiID cache; -/// dirty instances, boats (movers), and HouseFoundation (runtime-mutable) fall back to live-synth. -/// Keyed by multiID & 0x3FFF. +/// Caches walkability masks for the interior cells of a multi, shared across every instance of the +/// same multiID. +/// +/// An interior cell — one whose 8 neighbours are all covered by the multi — has no terrain +/// neighbour, so its mask depends only on the multi's own component tiles and is identical at every +/// position the design is placed. That makes it cacheable in the multi's local frame and reusable +/// across instances; perimeter cells are not, and fall back to . +/// +/// The catch is terrain intruding into the multi's floor envelope, which would make a cell's mask +/// position-dependent after all. rules that out per instance, +/// once: if the whole footprint's terrain sits below the lowest floor, no interior cell can see it. +/// A dirty instance, or a (whose design mutates at runtime), always +/// synthesizes live rather than risk serving a wrong mask. /// public sealed class MultiMaskCache { @@ -25,26 +31,20 @@ public sealed class MultiMaskCache public void Clear() => _byMultiId.Clear(); /// - /// Returns the multi-aware StepMask for a covered cell (x,y,sourceZ). Serves a cached interior - /// mask when available and the guards pass (counted as a MultiMaskCacheHit); otherwise falls - /// back to the Phase-2 live synthesizer ComputeMultiMaskAt (counted as a MultiLocalHit), caching - /// the result if the cell is interior and clean. Always returns a usable mask (HitKind == Hit). + /// The multi-aware mask for a covered cell, always usable (HitKind is always Hit). Served from + /// the shared cache when the cell is a clean interior one, synthesized live otherwise. /// public StepMask GetMask(Map map, int x, int y, sbyte sourceZ) { if (!TryResolveCoveringMulti(map, x, y, out var multi, out var lx, out var ly) - || multi is HouseFoundation) // runtime-mutable per-instance DesignState MCL + || multi is HouseFoundation) // its DesignState MCL changes at runtime { return LiveSynth(map, x, y, sourceZ); } - // Boats are cached too: their per-multiID deck masks are movement-invariant (built once per - // heading), and the per-instance clean gate below + the ItemID/location/map resets keep a - // moving/turning boat correct. Narrow boats have little interior; wide galleons gain a lot. + // Boats are cached despite moving: a deck mask is built in the local frame and is invariant + // under translation, and the clean gate plus the ItemID/location/map resets cover turning. - // Per-instance footprint cleanliness (computed once, stored on the multi; reset on move). - // Clean ⇒ no terrain intrusion anywhere in the footprint ⇒ interior cells are exact from the - // shared per-multiID cache. Dirty ⇒ degrade to the live synthesizer (never serve a wrong mask). if (multi.PathInteriorCacheState == MultiInteriorCacheState.Unknown) { multi.PathInteriorCacheState = @@ -62,7 +62,7 @@ public sealed class MultiMaskCache if (state == MultiLocalMask.CellState.Cached) { - // Footprint is clean, so only the source-Z match matters (terrain can't intrude). + // A clean footprint rules terrain out, so the source-Z match is the only guard left. var worldFloorZ = local.FloorZAt(lx, ly) + multi.Z; if (Math.Abs(sourceZ - worldFloorZ) <= StepHeight) { @@ -78,8 +78,7 @@ public sealed class MultiMaskCache return LiveSynth(map, x, y, sourceZ); } - // Unknown → classify + (if interior) build & cache. No per-cell terrain guard needed: the - // instance is clean, so every interior cell's 3x3 terrain is below the floor. + // First touch of this cell: synthesize, then classify and cache if it's interior. var mask = LiveSynth(map, x, y, sourceZ); if (IsInteriorLocalCell(mcl, lx, ly) && TryToLocalZ(mask, multi.Z, out var localMask) @@ -113,8 +112,7 @@ public sealed class MultiMaskCache } /// - /// Finds the multi covering (x,y) and the local cell indices into its MCL. Mirrors - /// Map.StaticTileEnumerator / BaseMulti.Contains. Returns false if no multi covers the cell. + /// Finds the multi covering (x,y) and the cell's indices into its MCL, or false if none does. /// public static bool TryResolveCoveringMulti(Map map, int x, int y, out BaseMulti multi, out int lx, out int ly) { @@ -138,9 +136,9 @@ public sealed class MultiMaskCache } /// - /// True iff local cell (lx,ly) and all 8 neighbours are covered by the multi (have MCL tiles). - /// Such a cell's 8-direction transition is fully determined by the multi (no terrain neighbour), - /// so its mask is position-invariant. A pure function of the MCL. + /// True when (lx,ly) and all 8 of its neighbours carry MCL tiles. Such a cell has no terrain + /// neighbour, so the multi alone determines its transitions and its mask is position-invariant. + /// A pure function of the MCL. /// public static bool IsInteriorLocalCell(MultiComponentList mcl, int lx, int ly) { @@ -161,9 +159,9 @@ public sealed class MultiMaskCache } /// - /// Converts a world-frame mask's per-direction Zs to local Z (subtract multiZ). Returns false - /// if any local Z doesn't fit sbyte (caller must then NOT cache the cell — rare; only when - /// |multiZ| is large enough to push a world Z out of range). Mask (walk/wet) bits are copied. + /// Rebases a world-frame mask's per-direction Zs into the multi's local frame. Returns false + /// when a local Z overflows sbyte — only reachable at extreme |multiZ| — and the caller must + /// then leave the cell uncached. /// public static bool TryToLocalZ(StepMask world, int multiZ, out StepMask local) { @@ -191,9 +189,8 @@ public sealed class MultiMaskCache } /// - /// True iff all terrain (land + statics) at (x,y) sits strictly below , - /// so a creature standing on the multi floor never sees terrain in its envelope and the cached - /// (terrain-free) mask is exact. Cheap: one land-top read + the cell's static-tile array scan. + /// True when all terrain (land + statics) at (x,y) sits strictly below , + /// so a creature standing on the multi's floor never sees terrain in its envelope. /// public static bool TerrainTopBelow(Map map, int x, int y, sbyte floorZ) { @@ -216,7 +213,7 @@ public sealed class MultiMaskCache return true; } - /// Highest terrain (land + statics) top at (x,y). Building block for the cleanliness check. + /// Highest terrain (land + statics) top at (x,y). public static int TerrainTop(Map map, int x, int y) { map.GetAverageZ(x, y, out _, out _, out var top); @@ -234,10 +231,10 @@ public sealed class MultiMaskCache } /// - /// True iff the multi's WHOLE footprint terrain sits below its lowest standable floor — i.e. - /// maxTerrain < minFloor over all covered cells. When true, no covered cell's terrain (nor any - /// neighbour's) can intrude into a creature's floor envelope, so interior cells of this design are - /// safe to serve from the shared per-multiID cache for THIS instance. One-time per instance. + /// True when the multi's entire footprint terrain sits below its lowest standable floor. No + /// covered cell's terrain — nor any neighbour's — can then intrude into a creature's floor + /// envelope, which is what makes this instance's interior cells safe to serve from the shared + /// per-multiID cache. Evaluated once per instance and cached on the multi. /// public static bool ComputeFootprintClean(Map map, BaseMulti multi) { @@ -252,7 +249,7 @@ public sealed class MultiMaskCache var col = mcl.Tiles[lx][ly]; if (col.Length == 0) { - continue; // uncovered local cell + continue; } foreach (var tile in col) @@ -278,7 +275,7 @@ public sealed class MultiMaskCache if (minFloorLocal == int.MaxValue) { - return false; // no standable floor anywhere → don't cache (defensive) + return false; // no standable floor anywhere; refuse to cache rather than guess } return maxTerrain < minFloorLocal + multi.Z; @@ -304,8 +301,9 @@ public sealed class MultiMaskCache } /// -/// Per-multiID lazily-filled grid of interior-cell masks. Cell state: Unknown (not yet classified), -/// Cached (interior + clean → mask valid), NonInterior (perimeter/edge/terrain-dirty → live-synth). +/// One multiID's grid of interior-cell masks, filled in as cells are first touched. A cell is +/// Unknown until classified, then either Cached (interior — the mask is valid) or NonInterior +/// (perimeter — synthesize live). /// internal sealed class MultiLocalMask { @@ -314,8 +312,8 @@ internal sealed class MultiLocalMask private readonly int _width; private readonly int _height; private readonly CellState[] _state; - private readonly StepMask[] _mask; // local-Z mask, valid when state == Cached - private readonly sbyte[] _floorZ; // local floor Z, valid when state == Cached + private readonly StepMask[] _mask; // local-frame mask, valid only when state == Cached + private readonly sbyte[] _floorZ; // local floor Z, valid only when state == Cached public MultiLocalMask(int width, int height) { diff --git a/Projects/UOContent/Engines/Pathing/Cache/StepCache.cs b/Projects/UOContent/Engines/Pathing/Cache/StepCache.cs index 5955ddd35..a1b368cb0 100644 --- a/Projects/UOContent/Engines/Pathing/Cache/StepCache.cs +++ b/Projects/UOContent/Engines/Pathing/Cache/StepCache.cs @@ -1,17 +1,24 @@ using System; using System.Collections.Generic; +using System.Diagnostics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using Server.Buffers; +using Server.Collections; using Server.Logging; namespace Server.Engines.Pathing.Cache; /// -/// Singleton store of per-chunk static walkability data. Chunks correspond to -/// Map.SectorSize = 16; key encoding packs (mapId, chunkX, chunkY) into a long. -/// Lazily built on first query; invalidated by version-check vs Sector.MultisVersion; -/// memory bounded by MaxResidentChunks via probabilistic LRU eviction. +/// Singleton store of static walkability, keyed by 16x16 chunk (one per map sector). Chunks build +/// on demand and memory stays bounded by MaxResidentChunks through probabilistic LRU eviction, so +/// the cache is usable with no on-disk bake at all; a baked .swb file only removes the first-touch +/// build cost. /// -/// Default-walker scope only. Cells with multi-Z surfaces and queries for non-default -/// walkers route to the MovementImpl slow path via the Fallthrough_* hit kinds. +/// The cache answers for a default walker on static terrain. Anything outside that — a multi +/// covering the cell, a query Z that doesn't match what the cell was baked at, stacked surfaces +/// with no matching stratum — returns a Fallthrough_* kind, and the caller resolves that cell +/// through MovementImpl instead. Callers must check . /// public sealed class StepCache { @@ -19,26 +26,23 @@ public sealed class StepCache public static StepCache Instance { get; } = new(); - private readonly Dictionary _chunks = new(); - // Parallel list of keys for O(1) random sampling during eviction. Kept in lockstep - // with _chunks: append on Miss_NotBuilt, swap-and-pop on eviction. + private readonly Dictionary _chunks = []; + // Keys of _chunks, kept in lockstep with it, so eviction can sample a random resident chunk + // in O(1). Appended on insert, swap-and-popped on eviction. private readonly List _keysList = []; - // Second-touch promotion tracker. A chunk's first miss within the window returns - // Fallthrough_NotBuilt; the caller takes the slow path. The Nth DISTINCT-FIND miss - // within the same window (where N = MissPromotionThreshold) promotes to BuildChunk + - // serve. We count distinct Find generations, not raw TryGetMask calls — A* expansion - // hits each visited chunk many times in one Find, so per-call counting hits threshold - // immediately and defeats the gate. Per-Find counting filters single-Find pass-throughs - // (pet following a moving player) while still promoting chunks revisited by multiple - // Finds (NPC patrolling fixed territory). - private readonly Dictionary _chunkMissTracker = new(); + // Promotion gate. A chunk's first miss returns Fallthrough_NotBuilt and the caller takes the + // slow path; only once misses reach MissPromotionThreshold within MissPromotionWindowMs does + // the chunk get built and served. This keeps one-off traffic — a pet trailing a player across + // the map — from building chunks nothing will query again, while a creature working a fixed + // territory still warms the chunks it revisits. + // + // The gate counts distinct Finds, not TryGetMask calls: A* probes each chunk it visits dozens + // of times within a single pathfind, so per-call counting would cross any threshold instantly + // and gate nothing. + private readonly Dictionary _chunkMissTracker = []; private const int MaxMissTrackerEntries = 4096; - // Generation counter incremented by BeginFindGeneration(). Sentinel 0 = "no Find started - // yet"; treated as a distinct generation per call so callers that bypass BeginFindGeneration - // (single-call tests, BakeMap with threshold=1) get sensible behavior. - private struct ChunkMissState { public byte MissCount; @@ -78,24 +82,21 @@ public sealed class StepCache public bool PreloadOnLazyOpen { get; set; } /// - /// Number of misses on the same chunk within - /// required to trigger a build. 1 = eager (legacy behavior). 2 = second-touch (default, - /// filters single-touch pass-throughs). + /// Misses on the same chunk, within , needed to build it. + /// 1 builds eagerly on first touch; the default 2 waits for a second Find to show interest. /// public int MissPromotionThreshold { get; set; } = 2; /// - /// Window over which misses against the same chunk accumulate toward promotion. - /// Misses spaced wider than this restart the count. Default 30s. + /// How long misses on a chunk accumulate toward promotion. A gap wider than this restarts + /// the count. /// public uint MissPromotionWindowMs { get; set; } = 30_000; /// - /// Marks the start of a new pathfind. The promotion gate counts distinct Find - /// generations per chunk, not raw TryGetMask calls — call this once at the top of - /// each pathfind invocation so multiple cell expansions within one Find don't trip - /// the threshold. Wraps at uint.MaxValue back to 1 (0 is reserved as the - /// "no Find started yet" sentinel). + /// Opens a new pathfind for the promotion gate. Call once per pathfind: the gate counts + /// distinct Finds, so without this every cell expansion would count separately and the + /// threshold would be met immediately. Wraps back to 1, since 0 means "no Find open". /// public void BeginFindGeneration() { @@ -107,12 +108,11 @@ public sealed class StepCache } } - /// Test-only: read the current Find generation. + /// The open pathfind's generation, or 0 if none. See . internal uint CurrentFindGeneration { get; private set; } /// - /// Pack (mapId, chunkX, chunkY) into a single long key. - /// Layout: [reserved 16][mapId 16][chunkX 16][chunkY 16]. + /// Packs (mapId, chunkX, chunkY) into one key: [reserved 16][mapId 16][chunkX 16][chunkY 16]. /// internal static long EncodeKey(int mapId, int chunkX, int chunkY) => ((long)(mapId & 0xFFFF) << 32) | ((long)(chunkX & 0xFFFF) << 16) | (long)(chunkY & 0xFFFF); @@ -134,9 +134,8 @@ public sealed class StepCache ); /// - /// Drop all cached chunks AND zero every telemetry counter. Used by tests and - /// benchmarks that need a known cold-start state. Counter reset is intentional — - /// counters are since-last-clear, not since-startup. + /// Returns the cache to a cold-start state: drops every chunk, closes the .swb readers, and + /// zeroes the counters. /// public void Clear() { @@ -146,10 +145,8 @@ public sealed class StepCache } /// - /// Drop all resident chunks AND zero counters, but keep lazy readers open. - /// Useful in benchmark loops that want to measure "first query after boot" cost - /// without paying the lazy-reader reopen overhead each iteration. Same intent as - /// minus the file-handle teardown. + /// without the file-handle teardown: drops the resident chunks and zeroes + /// the counters, but leaves the .swb readers open so the next query can refill from them. /// public void ClearResidentChunks() { @@ -171,16 +168,14 @@ public sealed class StepCache _buildsTotal = 0; } - // Per-map open .swb readers, populated by TryOpenLazyReader at startup. Chunks are - // fetched on demand from the file when ResolveMissingChunk fires; resident memory - // stays bounded by MaxResidentChunks regardless of file size. - private readonly Dictionary _lazyReaders = new(); + // Open .swb readers, one per map. Chunks are pulled from them on demand, so resident memory + // stays bounded by MaxResidentChunks no matter how large the file is. + private readonly Dictionary _lazyReaders = []; /// - /// Walk every chunk in , populate the resident set, then - /// save to . Returns the number of chunks written. - /// Designed for offline / fixture use; blocks the calling thread for many seconds - /// on a full Trammel walk. + /// Builds every chunk in the map and writes them to , returning the + /// number written. Blocks the caller for many seconds on a full-size map — run it offline or + /// during maintenance, not on a live shard at peak. /// public int BakeMap(int mapId, string path) { @@ -190,12 +185,11 @@ public sealed class StepCache return 0; } - // BakeMap is an explicit decision to populate every chunk; the promotion gate - // would otherwise return Fallthrough_NotBuilt for every chunk (each touched once) - // and the bake would write an empty file. Force eager build for the duration. + // A bake touches each chunk exactly once, so the promotion gate would defer every one of + // them and write an empty file. Baking is an explicit decision to populate everything, so + // build eagerly for the duration. var prevThreshold = MissPromotionThreshold; MissPromotionThreshold = 1; - var startTick = Core.TickCount; try { var chunkCols = (map.Width + ChunkSize - 1) / ChunkSize; @@ -207,12 +201,13 @@ public sealed class StepCache mapId, chunkCols, chunkRows, chunkCols * chunkRows ); + var stopWatch = Stopwatch.StartNew(); for (var cy = 0; cy < chunkRows; cy++) { for (var cx = 0; cx < chunkCols; cx++) { - // Any sourceZ works — the chunk is built on first access regardless of - // whether the query returns Hit or Fallthrough_SourceZMismatch. + // The sourceZ is irrelevant here: the chunk gets built on first access whether + // the query ends up a Hit or a Fallthrough_SourceZMismatch. TryGetMask(map, cx * ChunkSize, cy * ChunkSize, sourceZ: 0); } @@ -221,14 +216,14 @@ public sealed class StepCache logger.Information( "PathBake map {MapId}: row {Row}/{Rows} ({Pct}%), {Resident} chunks resident, {Elapsed:F1}s, {HeapMB} MB heap", mapId, cy + 1, chunkRows, (cy + 1) * 100 / chunkRows, - _chunks.Count, (Core.TickCount - startTick) / 1000.0, GC.GetTotalMemory(false) >> 20 + _chunks.Count, stopWatch.ElapsedMilliseconds / 1000.0, GC.GetTotalMemory(false) >> 20 ); } } logger.Information( "PathBake map {MapId}: walk complete in {Elapsed:F1}s, writing {Resident} chunks to disk...", - mapId, (Core.TickCount - startTick) / 1000.0, _chunks.Count + mapId, stopWatch.ElapsedMilliseconds / 1000.0, _chunks.Count ); } finally @@ -240,50 +235,31 @@ public sealed class StepCache } /// - /// Persist all resident chunks for to a .swb file. Returns - /// the number of chunks written. The file embeds a TileData fingerprint so a stale - /// file (built before a client patch) can be detected and rejected at open time. + /// Writes the map's resident chunks to a .swb file and returns the count. The file carries a + /// fingerprint of the tile and map data, so a bake made before a client patch is detected and + /// rejected when it is next opened. /// public int SaveToFile(string path, int mapId) { - var matching = 0; + using var chunks = PooledRefList<(int chunkX, int chunkY, StepChunk chunk)>.Create(); + foreach (var key in _keysList) { - DecodeKey(key, out var keyMapId, out _, out _); + DecodeKey(key, out var keyMapId, out var chunkX, out var chunkY); if (keyMapId == mapId) { - matching++; + chunks.Add((chunkX, chunkY, _chunks[key])); } } - var enumerator = _keysList.GetEnumerator(); - StepCacheFile.Write(path, (uint)mapId, (uint)matching, EmitChunk); - enumerator.Dispose(); - return matching; - - bool EmitChunk(out int chunkX, out int chunkY, out StepChunk chunk) - { - while (enumerator.MoveNext()) - { - var key = enumerator.Current; - DecodeKey(key, out var emittedMapId, out chunkX, out chunkY); - if (emittedMapId == mapId) - { - chunk = _chunks[key]; - return true; - } - } - chunkX = chunkY = 0; - chunk = null!; - return false; - } + StepCacheFile.Write(path, (uint)mapId, chunks.AsSpan()); + return chunks.Count; } /// - /// Open a .swb file as a lazy backing store for . Reads only - /// header + chunk-offset index (~16 bytes per chunk); individual records are fetched - /// on demand by . Returns false on missing file, - /// magic / version mismatch, or TileData hash mismatch (stale bake). + /// Opens a .swb file as a backing store for the map, reading only the header and chunk index + /// up front; records are pulled as queries ask for them. Returns false if the file is missing, + /// unreadable, or a stale bake whose fingerprint no longer matches the live tile data. /// public bool TryOpenLazyReader(string path, int mapId) { @@ -323,10 +299,7 @@ public sealed class StepCache } /// - /// Materializes every chunk in into the resident set. - /// Called from when - /// is set. Skips chunks whose live doesn't - /// match the file's snapshot — those will rebake on first query. + /// Loads every chunk in the file into the resident set, for . /// private void PreloadFromLazyReader(int mapId, StepCacheFile.LazyReader reader) { @@ -361,29 +334,52 @@ public sealed class StepCache ); } - /// - /// Number of .swb readers currently open. Mostly for tests / telemetry. - /// + /// Number of .swb readers currently open. public int OpenLazyReaderCount => _lazyReaders.Count; /// - /// True if a valid .swb reader is open for . A reader only opens via - /// after validates the - /// file's fingerprint against the live tile data, so "has reader" already means "present and - /// up-to-date" — the boot prebake uses this to skip baking maps that don't need it, instead of - /// recomputing the fingerprint a second time. + /// True when a .swb reader is open for the map. A reader only opens after its fingerprint + /// validates against the live tile data, so this already answers "is there an up-to-date bake + /// for this map?" — the boot prebake leans on that to skip maps rather than fingerprint them + /// a second time. /// public bool HasLazyReader(int mapId) => _lazyReaders.ContainsKey(mapId); - /// Test-only diagnostic: does the lazy reader for hold an offset for (chunkX, chunkY)? + /// Diagnostic: does the map's .swb hold a record for (chunkX, chunkY)? internal bool LazyReaderHasChunk(int mapId, int chunkX, int chunkY) => _lazyReaders.TryGetValue(mapId, out var r) && r.Has(chunkX, chunkY); /// - /// Closes all open lazy readers, releasing their underlying file streams. Called from - /// so test cleanup can delete .swb files (they're held with - /// FileShare.Read | FileShare.Delete, so this is mostly belt-and-suspenders). + /// Diagnostic: the resident chunk covering (chunkX, chunkY), or null if it isn't resident. + /// Exposed so tests can inspect and inject chunk state without reflecting into the internals. /// + internal StepChunk GetResidentChunk(int mapId, int chunkX, int chunkY) => + _chunks.GetValueOrDefault(EncodeKey(mapId, chunkX, chunkY)); + + /// + /// Diagnostic: whether the eviction key list still mirrors the resident set exactly. A desync + /// breaks sampled eviction — a stale key throws on lookup, a missing one pins a chunk resident + /// forever — and it is invisible from the outside, so tests assert on it directly. + /// + internal bool ResidentIndexInSync() + { + if (_keysList.Count != _chunks.Count) + { + return false; + } + + foreach (var key in _keysList) + { + if (!_chunks.ContainsKey(key)) + { + return false; + } + } + + return true; + } + + /// Closes every open .swb reader, releasing the underlying file streams. public void CloseLazyReaders() { foreach (var reader in _lazyReaders.Values) @@ -394,18 +390,16 @@ public sealed class StepCache } /// - /// Probabilistic LRU sample size — picks SampleSize random resident chunks per - /// eviction and evicts the oldest of that sample. Approximates true LRU at a tiny - /// fraction of the cost (no full sort). Redis uses the same approach (`maxmemory-samples`). - /// 5 yields ~quality-of-true-LRU for cache eviction; higher values trade speed for accuracy. + /// How many random resident chunks each eviction samples before dropping the oldest of them. + /// Sampling approximates true LRU closely enough at a fraction of the cost, since it needs no + /// sort and no access-ordered structure. Raising it trades speed for accuracy. /// private const int LruSampleSize = 5; /// - /// If resident chunk count exceeds MaxResidentChunks, evict via probabilistic LRU - /// until the count is at or below the cap. Per-eviction cost is O(LruSampleSize), - /// independent of resident count — sustained cap pressure has no perpetual perf hit. - /// Called from CacheEvictionTimer; also callable directly from tests. + /// Evicts chunks until the resident count is back within MaxResidentChunks. Each eviction costs + /// O() regardless of how many chunks are resident, so sustained cap + /// pressure doesn't degrade. Driven by . /// public void EnforceLruCap() { @@ -421,8 +415,8 @@ public sealed class StepCache long oldestTouched = long.MaxValue; long oldestKey = 0; - // Sample LruSampleSize random keys; track the oldest by LastTouchedTicks. - // With replacement is fine — collisions are rare and don't break correctness. + // Sampling with replacement: a repeated key just wastes one sample, it can't pick a + // wrong victim. var samples = Math.Min(LruSampleSize, _keysList.Count); for (var s = 0; s < samples; s++) { @@ -459,11 +453,28 @@ public sealed class StepCache private const int ChunkSize = 16; /// - /// True if a multi (house / boat) covers (x, y) or any of its 8 neighbours. Multi-covered - /// cells — plus the 1-cell halo, because a cell's mask encodes the edges TO its neighbours, so - /// a neighbouring wall must block those edges — are served by the live movement path, not the - /// static chunk cache. Cheap: an interior cell checks only its own sector (chunk == sector); - /// only edge/corner cells additionally check the adjacent sector(s) the halo reaches. + /// All-zero mask carrying a Fallthrough_* kind. is false for + /// these, so the caller ignores the payload and takes the slow path. + /// + private static StepMask Fallthrough(CacheHitKind kind) => + new(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, kind); + + /// Bumps the telemetry counter matching a served (non-fallthrough) hit kind. + private void RecordServed(CacheHitKind kind) + { + switch (kind) + { + case CacheHitKind.Miss_NotBuilt: { _missesNotBuilt++; break; } + case CacheHitKind.Miss_DirtyRebuild: { _missesDirtyRebuild++; break; } + case CacheHitKind.Hit: { _hits++; break; } + } + } + + /// + /// True when a multi covers (x, y) or any of its 8 neighbours. The halo matters because a + /// cell's mask encodes the edges TO its neighbours, so a wall one cell over has to block those + /// edges. Since a chunk is a sector, an interior cell only inspects its own sector's HasMultis + /// flag; edge and corner cells additionally check whichever adjacent sectors the halo reaches. /// private static bool MultiInfluence(Map map, int x, int y) { @@ -480,7 +491,7 @@ public sealed class StepCache var south = (y & 15) == 15; if (!(west || east || north || south)) { - return false; // interior cell — its whole halo is inside the (multi-free) own sector + return false; // interior cell: its whole halo lies in this sector, which has no multis } return west && map.GetRealSector(sx - 1, sy).HasMultis @@ -494,49 +505,25 @@ public sealed class StepCache } /// - /// Hot-path query. Returns the cached mask + 8 destination Z values + hit kind. - /// Inspect to decide whether to use the result or fall - /// back to the slow path. + /// The hot-path query: one lookup yields the cell's 8-direction mask, its 8 destination Zs, + /// and the hit kind. Check before trusting the payload — on any + /// fallthrough it is all zeroes and the caller must resolve the cell through MovementImpl. /// public StepMask TryGetMask(Map map, int x, int y, sbyte sourceZ) { if (map == null || map == Map.Internal || x < 0 || y < 0 || x >= map.Width || y >= map.Height) { _fallthroughOffMap++; - return new StepMask( - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - CacheHitKind.Fallthrough_OffMap - ); + return Fallthrough(CacheHitKind.Fallthrough_OffMap); } - // Multis (houses, boats) are not baked into the static chunk cache (they're dynamic - // content). If a multi covers this cell or its 1-cell halo, route to the live movement - // path, which is fully multi-aware. Gated on Sector.HasMultis, so the multi-free majority - // of the map pays a single (interior) sector lookup. + // Multis are dynamic, so they are never baked into a chunk. Cells they touch go to the + // multi-aware path instead. The check is gated on Sector.HasMultis, so the multi-free + // majority of the map pays one sector lookup for it. if (MultiInfluence(map, x, y)) { _fallthroughMulti++; - return new StepMask( - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - CacheHitKind.Fallthrough_Multi - ); + return Fallthrough(CacheHitKind.Fallthrough_Multi); } var chunkX = x >> 4; @@ -546,8 +533,8 @@ public sealed class StepCache var hitKindResult = CacheHitKind.Hit; if (!_chunks.TryGetValue(key, out var chunk)) { - // Try lazy file first — file-loaded chunks bypass the miss tracker because - // the .swb represents an explicit prior decision to keep this chunk warm. + // The .swb is consulted before the promotion gate: a baked chunk is already an explicit + // decision to keep this area warm, and loading it is far cheaper than building it. chunk = TryLoadFromLazyReader(map, chunkX, chunkY); if (chunk != null) { @@ -565,94 +552,44 @@ public sealed class StepCache else { _fallthroughNotBuilt++; - return new StepMask( - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - CacheHitKind.Fallthrough_NotBuilt - ); + return Fallthrough(CacheHitKind.Fallthrough_NotBuilt); } } - // A resident chunk is static-only — it never goes stale from multis (multi-covered cells - // fall through to the live path above). + // No staleness check: a resident chunk holds only static terrain, and every cell a multi + // could have changed already fell through above. chunk.LastTouchedTicks = Core.TickCount; var cellIndex = ((y - (chunkY << 4)) << 4) | (x - (chunkX << 4)); if (chunk.IsCellMultiZ(cellIndex)) { - // Tier 4: try the per-cell strata. Each stratum is keyed by its bake-time - // standing-Z; a query matches when |sourceZ - stratum.zCenter| <= StepHeight. + // Stacked surfaces: pick the stratum baked nearest the query Z. Multi-Z cells are + // served only from strata, never from the main mask. if (TryStratumHit(chunk, cellIndex, sourceZ, hitKindResult, out var stratumResult)) { - switch (hitKindResult) - { - case CacheHitKind.Miss_NotBuilt: { _missesNotBuilt++; break; } - case CacheHitKind.Miss_DirtyRebuild: { _missesDirtyRebuild++; break; } - case CacheHitKind.Hit: { _hits++; break; } - } + RecordServed(hitKindResult); return stratumResult; } + _fallthroughMultiZ++; - return new StepMask( - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - CacheHitKind.Fallthrough_MultiZ - ); + return Fallthrough(CacheHitKind.Fallthrough_MultiZ); } - // Source-Z guard: the cache stores one answer per cell baked at SourceZ. - // StepHeight tolerance accepts incremental Z jitter; loosening it breaks parity - // because tile reachability shifts at step-height boundaries. + // Source-Z guard. A cell holds one answer, baked at one standing Z, so a query from too far + // above or below it would get an answer that doesn't apply. The StepHeight tolerance + // absorbs ordinary Z jitter and cannot be widened: reachability flips at exactly that + // boundary, so a looser guard would serve answers that disagree with MovementImpl. if (Math.Abs(sourceZ - chunk.SourceZ[cellIndex]) > StepHeight) { - // Swim-layer fallback for shore cells: if the chunk has the layer and this - // cell's water-surface Z is within StepHeight of the query, serve from the - // swim layer (computed at swim-perspective Z). Walker queries on shore cells - // fall through this branch via their Z mismatch with SwimSourceZ. + // Unless this is a shore cell and the query is coming from the water, in which case the + // swim layer holds the answer baked from the water surface. if (chunk.HasSwimLayer) { var swimSrc = chunk.SwimSourceZ[cellIndex]; if (swimSrc != StepChunk.NoSwimLayerCell && Math.Abs(sourceZ - swimSrc) <= StepHeight) { - switch (hitKindResult) - { - case CacheHitKind.Miss_NotBuilt: { _missesNotBuilt++; break; } - case CacheHitKind.Miss_DirtyRebuild: { _missesDirtyRebuild++; break; } - case CacheHitKind.Hit: { _hits++; break; } - } + RecordServed(hitKindResult); return new StepMask( 0, chunk.SwimMask[cellIndex], 0, 0, 0, 0, 0, 0, 0, 0, @@ -670,35 +607,10 @@ public sealed class StepCache } _fallthroughSourceZMismatch++; - return new StepMask( - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - 0, - CacheHitKind.Fallthrough_SourceZMismatch - ); + return Fallthrough(CacheHitKind.Fallthrough_SourceZMismatch); } - switch (hitKindResult) - { - case CacheHitKind.Miss_NotBuilt: { _missesNotBuilt++; break; } - case CacheHitKind.Miss_DirtyRebuild: { _missesDirtyRebuild++; break; } - case CacheHitKind.Hit: { _hits++; break; } - } + RecordServed(hitKindResult); return new StepMask( chunk.WalkMask[cellIndex], @@ -724,41 +636,32 @@ public sealed class StepCache } /// - /// Returns a fresh StepChunk loaded from the lazy file reader, or null if there's no - /// open reader for the map / no record at (chunkX, chunkY) / the loaded snapshot is - /// stale relative to the live sector's . A null - /// return means the caller should consult the miss tracker; a stale return means - /// "rebuild, the .swb is out of date and a future SaveToFile will overwrite it." + /// Loads a chunk from the map's .swb, or null if no reader is open or the file has no record at + /// (chunkX, chunkY). No staleness check is needed here — the fingerprint was validated when the + /// file was opened, and the chunks are static-only. /// - private StepChunk TryLoadFromLazyReader(Map map, int chunkX, int chunkY) - { - if (!_lazyReaders.TryGetValue(map.MapID, out var reader)) - { - return null; - } - // Static-only chunks are valid once the file fingerprint matched at open time; multi-covered - // cells fall through before reaching here. Returns null when the file lacks this chunk. - return reader.TryReadChunk(chunkX, chunkY); - } + private StepChunk TryLoadFromLazyReader(Map map, int chunkX, int chunkY) => + _lazyReaders.TryGetValue(map.MapID, out var reader) ? reader.TryReadChunk(chunkX, chunkY) : null; /// - /// Records a miss for and decides whether to build now - /// or defer to slow path. Counts distinct Find generations, not raw calls — multiple - /// TryGetMask calls within one Find (BeginFindGeneration scope) count as one touch. - /// Returns true when DISTINCT-FIND misses within the window cross - /// ; caller should run BuildChunk and serve. - /// Returns false otherwise; caller should return Fallthrough_NotBuilt so the algorithm - /// uses the slow path. Generation 0 ("no Find active") treats every call as distinct, - /// preserving legacy semantics for callers that don't call BeginFindGeneration. + /// Records a miss and answers whether the chunk has now earned a build. True means build and + /// serve; false means return Fallthrough_NotBuilt and let the caller take the slow path. + /// + /// A miss only counts once per Find (see ). With no Find open + /// — a direct caller, or a bake — every call counts separately. /// private bool ShouldPromoteAfterMiss(long chunkKey) { - // Environment.TickCount, not Core.TickCount: tests/bench fixtures may not advance - // the game-loop tick. The promotion window is wall-clock anyway. + // Environment.TickCount rather than Core.TickCount: the window is wall-clock, and test and + // benchmark fixtures don't necessarily advance the game loop's tick. var now = (uint)Environment.TickCount; var gen = CurrentFindGeneration; - if (_chunkMissTracker.TryGetValue(chunkKey, out var state)) + // One hash lookup for the whole update — the entry is mutated through the ref instead + // of being re-hashed and re-probed by an indexer assignment. Safe to hold across the + // Remove below only because nothing reads it afterwards. + ref var state = ref CollectionsMarshal.GetValueRefOrNullRef(_chunkMissTracker, chunkKey); + if (!Unsafe.IsNullRef(ref state)) { // Same Find generation as the last touch — A* expansion is probing this chunk // multiple times in one pathfind. Don't increment; the gate counts distinct @@ -772,12 +675,11 @@ public sealed class StepCache var elapsed = now - state.LastMissTickStamp; if (elapsed > MissPromotionWindowMs) { - _chunkMissTracker[chunkKey] = new ChunkMissState - { - MissCount = 1, - LastMissTickStamp = now, - LastFindGeneration = gen - }; + // Outside the window — restart the count. Never promotes on this call, even at + // threshold 1, matching the pre-existing gate semantics. + state.MissCount = 1; + state.LastMissTickStamp = now; + state.LastFindGeneration = gen; return false; } @@ -788,12 +690,9 @@ public sealed class StepCache return true; } - _chunkMissTracker[chunkKey] = new ChunkMissState - { - MissCount = newCount, - LastMissTickStamp = now, - LastFindGeneration = gen - }; + state.MissCount = newCount; + state.LastMissTickStamp = now; + state.LastFindGeneration = gen; return false; } @@ -817,27 +716,30 @@ public sealed class StepCache } /// - /// Drop tracker entries older than the promotion window. Called when the tracker hits - /// its capacity ceiling. If the prune doesn't reclaim anything (every entry is in - /// window), the cap is enforced by clearing — the worst case is a few extra - /// Fallthrough_NotBuilt returns until traffic re-establishes hot chunks. + /// Drops tracker entries that have aged out of the promotion window, once the tracker hits its + /// capacity ceiling. When nothing has aged out, the whole tracker is cleared to enforce the cap + /// — that costs a few extra Fallthrough_NotBuilt returns while traffic re-establishes the hot + /// chunks, which is cheaper than letting the tracker grow without bound. /// private void PruneMissTracker(uint now) { var window = MissPromotionWindowMs; var beforeCount = _chunkMissTracker.Count; - var toRemove = new List(); + + using var toRemove = PooledRefQueue.Create(); foreach (var kvp in _chunkMissTracker) { if (now - kvp.Value.LastMissTickStamp > window) { - toRemove.Add(kvp.Key); + toRemove.Enqueue(kvp.Key); } } - foreach (var k in toRemove) + + while (toRemove.Count > 0) { - _chunkMissTracker.Remove(k); + _chunkMissTracker.Remove(toRemove.Dequeue()); } + if (_chunkMissTracker.Count == beforeCount) { _chunkMissTracker.Clear(); @@ -851,14 +753,15 @@ public sealed class StepCache var baseX = chunkX << 4; var baseY = chunkY << 4; - // Tier 4 strata accumulator. Lazily allocated when the first multi-Z cell - // appears; otherwise the chunk has zero strata overhead. + // Strata accumulator, created on the first multi-Z cell so single-Z chunks pay nothing. + // strataData is rented scratch; the chunk receives an exact-size copy, so the pooled array + // never escapes this method. ushort[] strataOffsetByCell = null; - List strataData = null; + byte[] strataData = null; + var strataLen = 0; - // Reused per cell: the standable surface Zs (walkway / bridge / floor levels). 16 is - // generous — clearance forces standable surfaces >= PersonHeight apart, so a 256-tall - // Z range admits at most ~16 anyway. + // Standable surface Zs for the current cell. 16 slots is generous: clearance forces + // surfaces at least PersonHeight apart, so an sbyte Z range can't hold more than ~16. Span surfaceZs = stackalloc sbyte[16]; for (var dy = 0; dy < ChunkSize; dy++) @@ -871,16 +774,13 @@ public sealed class StepCache map.GetAverageZ(x, y, out _, out var avgZ, out _); - // Anchor the cell at the surface a creature actually STANDS on, not the land - // average. For plain overworld that's the land; for static-over-land terrain - // (sewer/dungeon walkways, bridges, stair treads, raised foundations, upper - // building floors) it's the walkable static surface — which the old - // ComputeStandingZ(avgZ) anchor missed, producing source-Z fallthroughs (or, - // within the StepHeight tolerance band on stairs, a wrong vertical-neighbor - // answer baked at the adjacent tread). ComputeStandableSurfaceZs returns the - // standable surfaces ascending; the lowest is the primary anchor and A* tracks - // newZ to match it. Cells with no standable walk surface (deep water, solid - // rock) fall back to the land avg so the swim layer / wetMask still bake. + // Anchor the cell at the surface a creature stands on, not the land average. On + // open terrain those coincide, but on static-over-land geometry — walkways, + // bridges, stair treads, upper floors — the walkable surface is the static, and + // anchoring at the land below it would make every query fall through the source-Z + // guard. Surfaces come back ascending and the lowest is the anchor; A* tracks its + // per-cell Z to match. A cell with no standable surface at all (deep water, solid + // rock) falls back to the land average so its swim data still bakes. var surfaceCount = StepProbe.ComputeStandableSurfaceZs(map, x, y, surfaceZs); var standingZ = surfaceCount > 0 ? surfaceZs[0] : (sbyte)Math.Clamp(avgZ, sbyte.MinValue, sbyte.MaxValue); @@ -906,15 +806,12 @@ public sealed class StepCache chunk.SwimZW[cell] = result.SwimZ_W; chunk.SwimZNW[cell] = result.SwimZ_NW; - // Shore-cell handling: if the cell has BOTH a walk surface (standing Z) - // AND a water surface (Wet land tile or wet static) at a Z separated by - // > StepHeight, populate the swim layer at swim-perspective Z. Only when - // ComputeMaskAt produces a non-zero swim mask — bridges/docks/piers with - // insufficient vertical clearance for a swim creature's body envelope - // produce wetMask=0 (StaticsBlockAt rejects them), and we skip those cells - // rather than baking a stratum that always answers "no movement." The - // sentinel NoSwimLayerCell stays in SwimSourceZ for skipped cells; the - // chunk only sets HasSwimLayer when at least one cell got a usable entry. + // Shore cell: a walkable surface and a water surface more than StepHeight apart. + // The main mask is baked at the walk surface, so a swimmer querying from the water + // would fail the source-Z guard; bake it a second answer from the water surface. + // An empty swim mask means the water is unreachable anyway — a dock or pier with + // too little clearance for a swimmer's body — so leave those cells at the + // NoSwimLayerCell sentinel rather than store an answer that always says "blocked". var swimZRaw = StepProbe.ComputeSwimStandingZ(map, x, y); if (swimZRaw != int.MinValue && Math.Abs(swimZRaw - standingZ) > StepHeight) { @@ -939,36 +836,33 @@ public sealed class StepCache } } - // Stacked walkable surfaces at one cell (ground + 1st + 2nd building floors, - // a bridge over a walkable path, etc.): bake a stratum per standable surface - // so a query at any floor's Z hits. The primary (lowest) surface is also in - // the main mask above, but multi-Z cells are served exclusively from strata, - // so every standable surface — including the primary — must appear here. - // Single-surface cells (the common case, incl. stair treads and sewer - // walkways) skip this entirely and stay on the fast single-mask path. + // Stacked walkable surfaces — a bridge over a path, the floors of a building — + // need one stratum each so a query at any of their Zs finds an answer. Every + // surface goes in, including the lowest, because a multi-Z cell is served only + // from its strata and never from the main mask baked above. if (surfaceCount >= 2) { if (strataOffsetByCell == null) { strataOffsetByCell = new ushort[StepChunk.CellsPerChunk]; - for (var i = 0; i < strataOffsetByCell.Length; i++) - { - strataOffsetByCell[i] = StepChunk.NoStrata; - } - strataData = new List(256); + strataOffsetByCell.AsSpan().Fill(StepChunk.NoStrata); + // NoStrata bounds the packed data to NoStrata bytes (see StepChunk), so + // renting that much up front leaves the record guard below as the only + // bound the writes need. + strataData = STArrayPool.Shared.Rent(StepChunk.NoStrata); } - // Cap at 65,535 byte offsets — well above realistic per-chunk strata - // volume. If we ever blow past this we silently leave the cell single-Z - // (it keeps the land-anchored main mask and falls through off-surface). - if (strataData.Count <= ushort.MaxValue - StepChunk.StratumByteLength * 8) + // One count byte plus a record per surface. A cell whose record won't fit stays + // single-Z: it keeps the main mask and falls through off its anchor surface. + var recordLength = 1 + surfaceCount * StepChunk.StratumByteLength; + if (strataLen + recordLength <= StepChunk.NoStrata) { - strataOffsetByCell[cell] = (ushort)strataData.Count; - strataData.Add((byte)surfaceCount); + strataOffsetByCell[cell] = (ushort)strataLen; + strataData[strataLen++] = (byte)surfaceCount; for (var i = 0; i < surfaceCount; i++) { var sz = surfaceZs[i]; - AppendStratumBytes(strataData, new StepProbe.ComputedStratum(sz, StepProbe.ComputeMaskAt(map, x, y, sz))); + WriteStratum(strataData, ref strataLen, sz, StepProbe.ComputeMaskAt(map, x, y, sz)); } } } @@ -977,7 +871,8 @@ public sealed class StepCache if (strataOffsetByCell != null) { - chunk.SetStrata(strataOffsetByCell, strataData.ToArray()); + chunk.SetStrata(strataOffsetByCell, strataData.AsSpan(0, strataLen).ToArray()); + STArrayPool.Shared.Return(strataData); } _buildsTotal++; @@ -985,10 +880,10 @@ public sealed class StepCache } /// - /// Tier 4 strata lookup. Walks the cell's stratum list, returns true with the first - /// stratum whose zCenter is within StepHeight of . - /// Layout matches : u8 count, then count × 19-byte - /// stratum (sbyte zCenter, byte walkMask, byte wetMask, 8 sbyte walkZ, 8 sbyte swimZ). + /// Finds the cell's stratum matching — the first whose zCenter is + /// within StepHeight — and builds its mask. False when the cell has no strata or none of them + /// sit near enough, in which case the caller falls through. Reads the layout + /// writes. /// private static bool TryStratumHit( StepChunk chunk, int cellIndex, sbyte sourceZ, CacheHitKind hitKind, out StepMask result @@ -1050,29 +945,33 @@ public sealed class StepCache return false; } - private static void AppendStratumBytes(List dst, in StepProbe.ComputedStratum s) + /// + /// Packs one stratum into at , advancing it by + /// . Layout must stay in lockstep with + /// and . + /// + private static void WriteStratum(Span dst, ref int pos, sbyte zCenter, in StepMask mask) { - dst.Add((byte)s.ZCenter); - dst.Add(s.Mask.WalkMask); - dst.Add(s.Mask.WetMask); - dst.Add((byte)s.Mask.WalkZ_N); - dst.Add((byte)s.Mask.WalkZ_NE); - dst.Add((byte)s.Mask.WalkZ_E); - dst.Add((byte)s.Mask.WalkZ_SE); - dst.Add((byte)s.Mask.WalkZ_S); - dst.Add((byte)s.Mask.WalkZ_SW); - dst.Add((byte)s.Mask.WalkZ_W); - dst.Add((byte)s.Mask.WalkZ_NW); - dst.Add((byte)s.Mask.SwimZ_N); - dst.Add((byte)s.Mask.SwimZ_NE); - dst.Add((byte)s.Mask.SwimZ_E); - dst.Add((byte)s.Mask.SwimZ_SE); - dst.Add((byte)s.Mask.SwimZ_S); - dst.Add((byte)s.Mask.SwimZ_SW); - dst.Add((byte)s.Mask.SwimZ_W); - dst.Add((byte)s.Mask.SwimZ_NW); + dst[pos++] = (byte)zCenter; + dst[pos++] = mask.WalkMask; + dst[pos++] = mask.WetMask; + dst[pos++] = (byte)mask.WalkZ_N; + dst[pos++] = (byte)mask.WalkZ_NE; + dst[pos++] = (byte)mask.WalkZ_E; + dst[pos++] = (byte)mask.WalkZ_SE; + dst[pos++] = (byte)mask.WalkZ_S; + dst[pos++] = (byte)mask.WalkZ_SW; + dst[pos++] = (byte)mask.WalkZ_W; + dst[pos++] = (byte)mask.WalkZ_NW; + dst[pos++] = (byte)mask.SwimZ_N; + dst[pos++] = (byte)mask.SwimZ_NE; + dst[pos++] = (byte)mask.SwimZ_E; + dst[pos++] = (byte)mask.SwimZ_SE; + dst[pos++] = (byte)mask.SwimZ_S; + dst[pos++] = (byte)mask.SwimZ_SW; + dst[pos++] = (byte)mask.SwimZ_W; + dst[pos++] = (byte)mask.SwimZ_NW; } - private const int PersonHeight = 16; private const int StepHeight = 2; } diff --git a/Projects/UOContent/Engines/Pathing/Cache/StepCacheFile.cs b/Projects/UOContent/Engines/Pathing/Cache/StepCacheFile.cs index 8cbc5c51f..538807448 100644 --- a/Projects/UOContent/Engines/Pathing/Cache/StepCacheFile.cs +++ b/Projects/UOContent/Engines/Pathing/Cache/StepCacheFile.cs @@ -9,40 +9,36 @@ using Server.Compression; namespace Server.Engines.Pathing.Cache; /// -/// Binary serializer + lazy reader for the step cache. Persists chunk records to disk -/// so a server warm-starts without paying chunk-build cost on the first pathfind through -/// a region. Lazy: opening a file reads only the header + chunk-offset index (~few KB -/// for tens of thousands of chunks), then individual chunks are seeked + deserialized -/// only when the cache asks for them. RAM stays bounded by MaxResidentChunks regardless -/// of file size. +/// Binary serializer and reader for the step cache, so a shard can warm-start instead of building +/// chunks on the first pathfind through each region. Opening a file reads only the header and chunk +/// index; a chunk record is seeked and inflated when the cache actually asks for it, which keeps +/// resident memory bounded by MaxResidentChunks no matter how large the file is. /// -/// File layout v8 (little-endian, BufferWriter / BufferReader convention): +/// File layout (little-endian, BufferWriter / BufferReader convention): /// /// Header (40 bytes): /// u32 Magic = 0x42575300 ('SWB\0') -/// u32 Version = current FormatVersion (9) +/// u32 Version = FormatVersion /// u32 MapId -/// u64 Fingerprint XxHash3 over (1) LandTable + ItemTable flags AND (2) the -/// on-disk bytes of mapX.mul / .uop, staidxX.mul, staticsX.mul. -/// Rejects a load when EITHER tile flags shifted (client patch) -/// OR the map data was rewritten (CentredSharp / UOFiddler edit). -/// The .mul format has no built-in CRC; this is the only way -/// to detect those mutations. -/// u64 BakeTimestamp DateTime.UtcNow.Ticks at write time (informational). +/// u64 Fingerprint XxHash3 over tiledata.mul and the map's own .mul / .uop files. +/// Detects both a client patch that shifts tile flags and a map edit +/// that rewrites the terrain; see ComputeFingerprint. The .mul format +/// carries no CRC of its own, so hashing is the only way to catch either. +/// u64 BakeTimestamp DateTime.UtcNow.Ticks at write time. Informational. /// u32 ChunkCount -/// u64 IndexOffset File position where the chunk index begins. +/// u64 IndexOffset Where the index trailer begins. /// /// Per chunk (ChunkCount times, variable size): /// u32 UncompressedLen Size of the inflated record body below. -/// byte[] Payload The record body (the v6 layout that follows), libdeflate- -/// compressed. If the on-disk payload length (index recordLength − -/// 4) equals UncompressedLen, the body was stored raw because -/// compression did not shrink it (tiny Uniform records). +/// byte[] Payload The record body, libdeflate-compressed — or stored raw when +/// compression didn't shrink it, as happens with tiny Uniform +/// records. The reader tells the two apart by comparing the payload +/// length against UncompressedLen. /// -/// Record body (after inflate — the v6 layout): +/// Record body (after inflate): /// u16 ChunkX /// u16 ChunkY -/// u32 BuiltMultisVersion (reserved since v9 — always 0; chunks are static-only) +/// u32 BuiltMultisVersion Reserved, always 0 — chunks are static-only. /// u8 Kind 0 = Full; 2 = Uniform /// // Uniform (Kind == 2): ~28-byte record — all 256 cells share these single values: /// byte walkMask, wetMask; sbyte sourceZ; sbyte walkZ_N..NW (8); sbyte swimZ_N..NW (8) @@ -78,34 +74,29 @@ namespace Server.Engines.Pathing.Cache; /// The file offset is not stored — reconstructed as a cumulative sum of recordLength /// starting at HeaderSize (the first record sits immediately after the header). /// -/// Per-chunk fixed portion (Kind + flags + ZArrayMask + WalkMask + WetMask + SourceZ): -/// ~783 bytes; each present base Z array adds 256 bytes (0..16 present, so up to ~4 KB). -/// A fully-flat Full chunk stores no residual blocks. Strata trailer: 516 + N × ~30 bytes -/// for a chunk with N multi-Z cells averaging ~2 strata each. LRU bookkeeping -/// (LastTouchedTicks) is intentionally not persisted. +/// A chunk's fixed portion runs ~783 bytes, and each directional-Z array that survives prediction +/// adds 256 more, so a Full record lands between ~783 bytes and ~4 KB. The strata trailer adds +/// 516 bytes plus roughly 30 per multi-Z cell. LastTouchedTicks is deliberately not persisted — +/// LRU state means nothing across a restart. /// -/// Files with version < are silently rejected -/// at open time (treated as missing) and overwritten on the next save. +/// Files below are treated as missing and overwritten on the +/// next save. The cache regenerates from the map data, so a format bump only costs a one-time +/// re-bake. /// internal static class StepCacheFile { public const uint Magic = 0x42575300; // 'SWB\0' - // v9: chunks are STATIC-ONLY (land + statics.mul, no multis). v8 and earlier baked multis - // (houses/boats) into chunks, which is unsafe to persist — multis are dynamic, and the - // BuiltMultisVersion they were tagged with is a non-persisted session counter. Bumping the - // version rejects those old files so they re-bake static-only. The BuiltMultisVersion record - // field is retained as a reserved (always-0) u32 to avoid a layout change. public const uint FormatVersion = 9; /// - /// Lowest format version this binary can load. Files below it are treated as missing - /// (silently rejected) and overwritten by the next SaveToFile / BakeMap. The cache is - /// fully regenerable, so a format bump just forces a one-time re-bake of stale files. + /// Oldest format this binary will load. Anything older is treated as missing rather than + /// migrated: the cache is fully regenerable from the map data, so a re-bake is always + /// available and always correct. /// public const uint MinSupportedVersion = 9; - // Per-chunk record discriminator (first byte after BuiltMultisVersion). 1 is reserved. + // Record discriminator. 1 is reserved. private const byte KindFull = 0; private const byte KindUniform = 2; @@ -118,12 +109,12 @@ internal static class StepCacheFile + sizeof(uint) // ChunkCount + sizeof(ulong); // IndexOffset - // Index entry (v8 compact): u32 packedKey ((chunkX << 16) | chunkY) + u32 recordLength. - // The file offset is NOT stored — entries are in record write order, so the reader - // reconstructs each offset by cumulative sum of record lengths starting at HeaderSize. + // One index entry: u32 packedKey ((chunkX << 16) | chunkY) + u32 recordLength. The file offset + // isn't stored — entries sit in record write order, so the reader rebuilds each offset as a + // running sum of the lengths before it, starting at HeaderSize. private const int IndexEntryBytes = sizeof(uint) + sizeof(uint); - /// Fixed-size portion of a chunk record (everything except the optional strata + swim trailers). + /// A chunk record minus its optional strata and swim trailers. private const int BytesPerChunkBase = sizeof(ushort) + sizeof(ushort) + sizeof(uint) + sizeof(byte) + sizeof(byte) + sizeof(byte) // Kind + HasStrata + HasSwimLayer @@ -135,17 +126,8 @@ internal static class StepCacheFile + 8 * StepChunk.CellsPerChunk; // SwimZ[8] /// - /// Byte offset of the IndexOffset u64 within the header - /// (Magic+Version+MapId+Fingerprint+BakeTimestamp+ChunkCount = 32). Patched after chunks land. - /// - private const int IndexOffsetFieldPosition = 32; - - public delegate bool ChunkEnumerator(out int chunkX, out int chunkY, out StepChunk chunk); - - /// - /// Peek at a .swb file's Fingerprint field (header byte offset 12) without - /// reading any chunk data. Returns false on missing file, bad magic, or wrong - /// version. Cheap — reads 20 bytes total. + /// Reads just a .swb file's fingerprint — 20 bytes, no chunk data. False if the file is + /// missing, isn't a .swb, or is a version this binary can't load. /// public static bool TryReadFingerprint(string path, out ulong fingerprint) { @@ -183,31 +165,28 @@ internal static class StepCacheFile } /// - /// Combined XxHash3 fingerprint over (1) the on-disk tiledata.mul file and (2) the - /// per-map .mul / .uop file contents (via ). - /// Bake files carry this hash so a load can refuse to populate the cache when EITHER the - /// tile data shifted (client patch) OR the map data was rewritten (CentredSharp / UOFiddler - /// edit). The .mul format has no built-in CRC; this is the only way to detect those mutations. + /// Hashes the inputs a bake depends on: tiledata.mul and the map's own .mul / .uop + /// files. A file carrying a stale hash is refused at open time, which is what catches a client + /// patch that shifts tile flags or a map editor that rewrites the terrain. Neither format has a + /// CRC of its own, so hashing is the only signal available. /// - /// IMPORTANT: hash the FILES, never the in-memory / + /// This must hash the FILES, never the in-memory / /// . The server patches those tables at runtime (ItemFixes, - /// LOSBlocker, PotionKeg, CTF, ...) at nondeterministic lifecycle points, so a fingerprint over - /// the live tables varies with WHEN it is taken; the file hash is the only lifecycle-stable - /// "did the client's tile data change?" signal. Server-side tile patches are applied identically - /// every boot and intentionally do NOT invalidate the cache — change one and you must - /// [PathCacheClear or bump the format. + /// LOSBlocker, PotionKeg, CTF), so a hash of the live tables changes depending on when it is + /// taken — useless as a fingerprint. Those server-side patches apply identically every boot and + /// deliberately do NOT invalidate the cache; if you change one, run [PathCacheClear or bump + /// yourself. /// public static ulong ComputeFingerprint(int mapId) { var hasher = HashUtility.CreateXxHash3(); - // (1) tiledata.mul — hashed once, cached. The authoritative source for tile flags/heights. Span tileDataBytes = stackalloc byte[sizeof(ulong)]; BinaryPrimitives.WriteUInt64LittleEndian(tileDataBytes, TileDataFileFingerprint()); hasher.Append(tileDataBytes); - // (2) Map files (mapX.mul / .uop, staidxX.mul, staticsX.mul). TileMatrix already - // streamed them through XxHash3 once at construction; mix the result in. + // TileMatrix already streamed the map files through XxHash3 when it was built; reuse that + // rather than re-reading them. var map = Map.Maps[mapId]; if (map != null && map != Map.Internal && map.Tiles != null) { @@ -223,10 +202,9 @@ internal static class StepCacheFile private static bool _tileDataFileFingerprintComputed; /// - /// XxHash3 over the raw tiledata.mul bytes, computed once and cached — the file never - /// changes during a run. Mirrors for the map - /// files. Returns 0 if the file can't be found (the server can't run without it anyway, so - /// this only matters in stripped test hosts, where 0 is a fine deterministic constant). + /// XxHash3 of the raw tiledata.mul bytes, computed once — the file can't change while + /// the server runs. Returns 0 when the file is absent, which only happens in stripped test + /// hosts; a real server can't boot without it, and 0 is a fine deterministic stand-in. /// private static ulong TileDataFileFingerprint() { @@ -249,84 +227,68 @@ internal static class StepCacheFile } /// - /// Writes the file: header (with placeholder IndexOffset) → chunks (offsets recorded) - /// → index trailer → patches the header IndexOffset. must - /// equal the actual number of chunks will yield. + /// Writes the map's chunks to : header, then one record per chunk, then + /// the index trailer. IndexOffset isn't known until the records are down, so it goes in as a + /// placeholder and gets patched by seeking back to it. /// - public static void Write(string path, uint mapId, uint chunkCount, ChunkEnumerator next) + public static void Write(string path, uint mapId, ReadOnlySpan<(int chunkX, int chunkY, StepChunk chunk)> chunks) { Directory.CreateDirectory(Path.GetDirectoryName(path) ?? "."); - // Initial estimate: base record + a modest strata budget per chunk. Coastline - // chunks add another ~2.5 KB (swim layer) but they're a small fraction of any - // map; the writer grows on overflow so under-estimating just causes a few - // realloc/copy cycles during the bake — not a correctness issue. - var capacity = HeaderSize + (BytesPerChunkBase + 256) * (int)chunkCount + IndexEntryBytes * (int)chunkCount; - var buffer = new byte[capacity]; - var w = new BufferWriter(buffer, prefixStr: false); + // A rough estimate: the base record plus a small strata budget per chunk. Coastline chunks + // run ~2.5 KB over it for their swim layer, but they're a small share of any map, and the + // writer grows on overflow — under-estimating costs a few reallocs during a bake, nothing more. + var capacity = HeaderSize + (BytesPerChunkBase + 256 + IndexEntryBytes) * chunks.Length; + var w = new BufferWriter(new byte[capacity], prefixStr: false); w.Write(Magic); w.Write(FormatVersion); w.Write(mapId); w.Write(ComputeFingerprint((int)mapId)); w.Write((ulong)DateTime.UtcNow.Ticks); - w.Write(chunkCount); - w.Write(0UL); // IndexOffset placeholder, patched after chunks + w.Write((uint)chunks.Length); + var indexOffsetPosition = w.Position; + w.Write(0UL); // patched below, once the records are written and the index position is known - // Each record is built uncompressed into recordScratch, then libdeflate-compressed into - // compScratch and framed as [u32 uncompressedLen][payload]. + // Each record is built into recordScratch, compressed into compScratch, then framed as + // [u32 uncompressedLen][payload]. var packer = Deflate.Maximum; var recordScratch = new byte[BytesPerChunkBase + 1024]; var compScratch = new byte[packer.MaxPackSize(recordScratch.Length)]; - var indexEntries = new (ulong key, ulong offset, uint length)[chunkCount]; - var written = 0u; - while (next(out var chunkX, out var chunkY, out var chunk)) + // Record lengths only — the index stores no offsets, so the reader rebuilds them by + // summing these in order. + var lengths = new uint[chunks.Length]; + for (var i = 0; i < chunks.Length; i++) { - if (written >= chunkCount) - { - throw new InvalidOperationException( - $"StepCacheFile.Write: enumerator yielded more than the declared {chunkCount} chunks" - ); - } - var chunkOffset = (ulong)w.Position; + var (chunkX, chunkY, chunk) = chunks[i]; + var start = w.Position; WriteChunk(w, chunkX, chunkY, chunk, packer, ref recordScratch, ref compScratch); - var chunkLength = (uint)((ulong)w.Position - chunkOffset); - indexEntries[written] = (PackChunkKey(chunkX, chunkY), chunkOffset, chunkLength); - written++; - } - - if (written != chunkCount) - { - throw new InvalidOperationException( - $"StepCacheFile.Write: declared {chunkCount} chunks but enumerator yielded {written}" - ); + lengths[i] = (uint)(w.Position - start); } var indexOffset = (ulong)w.Position; - for (var i = 0u; i < chunkCount; i++) + for (var i = 0; i < chunks.Length; i++) { - // v8 compact entry: u32 packedKey ((chunkX << 16) | chunkY) + u32 recordLength. - // Offset is omitted; entries are in record write order so the reader derives it. - var key = indexEntries[i].key; - var packedKey = ((uint)(key >> 32) << 16) | (uint)(key & 0xFFFF); - w.Write(packedKey); - w.Write(indexEntries[i].length); + var (chunkX, chunkY, _) = chunks[i]; + w.Write((uint)((chunkX & 0xFFFF) << 16 | chunkY & 0xFFFF)); + w.Write(lengths[i]); } - // Patch IndexOffset on the writer's current backing buffer (BufferWriter may - // have grown during chunk writes; the original `buffer` ref is stale after grow). - var liveBuffer = w.Buffer; - BinaryPrimitives.WriteUInt64LittleEndian(liveBuffer.AsSpan(IndexOffsetFieldPosition, 8), indexOffset); - var totalBytes = (int)w.Position; - File.WriteAllBytes(path, liveBuffer.AsSpan(0, totalBytes).ToArray()); + + w.Seek(indexOffsetPosition, SeekOrigin.Begin); + w.Write(indexOffset); + + // w.Buffer, not the array handed to the constructor: BufferWriter reallocates on growth, + // which leaves that original reference pointing at a stale array. + File.WriteAllBytes(path, w.Buffer.AsSpan(0, totalBytes).ToArray()); } /// - /// Opens a .swb file and reads only its header + chunk-offset index. Returns null on - /// missing file, magic / version mismatch, or Fingerprint mismatch (a stale bake - /// against a freshly patched client). Callers own disposal of the returned reader. + /// Opens a .swb file, reading only its header and chunk index. Null if the file is missing, + /// isn't a loadable .swb, or is a stale bake whose fingerprint no longer matches the live tile + /// and map data. The caller owns the returned reader. /// public static LazyReader OpenForLazy(string path) { @@ -361,8 +323,6 @@ internal static class StepCacheFile var version = BinaryPrimitives.ReadUInt32LittleEndian(headerBuf[4..]); if (version < MinSupportedVersion || version > FormatVersion) { - // Below the minimum supported version: treat as missing. Older files - // get silently overwritten on the next SaveToFile / BakeMap. stream.Dispose(); return null; } @@ -379,7 +339,7 @@ internal static class StepCacheFile return null; } - // Read the chunk-offset index in one shot. + // Pull the whole index in one read. var indexBytes = (int)chunkCount * IndexEntryBytes; var indexBuf = new byte[indexBytes]; stream.Position = (long)indexOffset; @@ -389,9 +349,8 @@ internal static class StepCacheFile return null; } - // v8 compact index: { u32 packedKey, u32 length } per chunk, in record write order. - // The file offset is not stored — reconstruct it by cumulative record length starting - // at the first record (immediately after the header). + // Entries are in record write order and carry no offset, so rebuild each one as a + // running sum of the record lengths, starting just past the header. var offsets = new Dictionary((int)chunkCount); var runningOffset = (ulong)HeaderSize; for (var i = 0; i < chunkCount; i++) @@ -416,27 +375,27 @@ internal static class StepCacheFile private static ulong PackChunkKey(int chunkX, int chunkY) => ((ulong)(uint)chunkX << 32) | (uint)chunkY; /// - /// Predicted directional-Z for one cell/direction: the cell's own SourceZ when the - /// direction is walkable/wet (mask bit set), else 0 — matching the baker, which leaves - /// non-walkable directional slots at their zero-initialized default - /// (StepProbe.ComputeMaskAt clears walkZs/swimZs and writes only on a successful step). + /// Guesses a cell's destination Z for one direction: on flat ground a step lands at the Z you + /// left from, so predict SourceZ where the direction is passable and 0 where it isn't. The + /// zero matches the baker, which only writes a slot on a successful step and leaves the rest + /// cleared. Most terrain is flat, so most predictions are exact and most residuals are 0 — + /// which is what makes the residual arrays compress away to nothing. /// internal static sbyte Predict(byte dirMaskByte, int bit, sbyte sourceZ) => (dirMaskByte >> bit & 1) != 0 ? sourceZ : (sbyte)0; /// - /// Residual of an absolute directional-Z against its prediction. Unchecked two's-complement - /// so the transform is byte-exact for ALL sbyte inputs (no value-range constraint). + /// A destination Z's difference from its prediction. Wraps deliberately: two's-complement + /// round-trips exactly for every sbyte input, so no value range is off-limits. /// internal static sbyte EncodeResidual(sbyte z, sbyte predict) => unchecked((sbyte)(z - predict)); - /// Inverse of : absolute directional-Z = predict + residual. + /// Inverse of . internal static sbyte DecodeZ(sbyte predict, sbyte residual) => unchecked((sbyte)(predict + residual)); /// - /// The base directional-Z array for direction index d in canonical order: walk N..NW (0-7), - /// then swim N..NW (8-15). Index d uses WalkMask (d < 8) or WetMask (d >= 8) with - /// direction bit (d & 7). + /// The destination-Z array for direction index d, in the canonical order the format stores them: + /// walk N..NW as 0-7, then swim N..NW as 8-15. /// private static sbyte[] GetBaseZArray(StepChunk c, int d) => d switch { @@ -448,10 +407,9 @@ internal static class StepCacheFile }; /// - /// Builds the uncompressed v6 record for one chunk into , libdeflate- - /// compresses it, and writes it framed as [u32 uncompressedLen][payload]. The payload is the - /// compressed bytes, or — when compression does not shrink the record (tiny Uniform records) — - /// the raw record itself; the reader distinguishes the two by payload length vs uncompressedLen. + /// Builds one chunk's record, compresses it, and frames it as [u32 uncompressedLen][payload]. + /// When compression fails to shrink the record — as it does on the tiny Uniform ones — the raw + /// record is stored instead, and the reader tells the two apart by payload length. /// private static void WriteChunk( BufferWriter w, int chunkX, int chunkY, StepChunk chunk, @@ -460,7 +418,7 @@ internal static class StepCacheFile { var rw = new BufferWriter(recordScratch, prefixStr: false); BuildRecord(rw, chunkX, chunkY, chunk); - recordScratch = rw.Buffer; // may have grown; keep the larger buffer for reuse + recordScratch = rw.Buffer; // may have grown; hold onto the larger buffer for the next chunk var recordLen = (int)rw.Position; var bound = packer.MaxPackSize(recordLen); @@ -478,8 +436,8 @@ internal static class StepCacheFile } else { - // Incompressible (or expanded): store the record raw. The reader detects this when - // the on-disk payload length equals the uncompressed length. + // Compression didn't help, so store the record raw. Payload length == uncompressedLen + // is how the reader recognizes that. w.Write(recordScratch.AsSpan(0, recordLen)); } } @@ -490,8 +448,8 @@ internal static class StepCacheFile w.Write((ushort)chunkY); w.Write((uint)chunk.BuiltMultisVersion); - // Kind: 0 = Full, 2 = Uniform. A uniform chunk (no strata, no swim layer, all 19 base - // arrays constant) stores one cell's worth of data (~28-byte record total). + // A uniform chunk — every cell identical — collapses to one cell's worth of data, ~28 bytes. + // Open water and solid rock make up a lot of a map, so this is worth the branch. if (chunk.IsUniform()) { w.Write(KindUniform); @@ -517,7 +475,7 @@ internal static class StepCacheFile return; } - w.Write(KindFull); // Full + w.Write(KindFull); var strataOffsetByCell = chunk.GetStrataOffsetByCellForSerialization(); var strataData = chunk.GetStrataDataForSerialization(); @@ -526,9 +484,9 @@ internal static class StepCacheFile w.Write((byte)(hasStrata ? 1 : 0)); w.Write((byte)(hasSwimLayer ? 1 : 0)); - // Predictive-Z: each base directional Z array is stored as a masked residual against - // SourceZ. Bit d of ZArrayMask is set only when array d differs from its prediction - // somewhere; cleared arrays are omitted and rebuilt from mask+SourceZ at read. + // Each destination-Z array is stored as residuals against its prediction (see Predict). An + // array that matches its prediction everywhere — the common case on flat terrain — is + // omitted entirely, and its ZArrayMask bit stays clear so the reader synthesizes it. ushort zArrayMask = 0; for (var d = 0; d < 16; d++) { @@ -583,7 +541,6 @@ internal static class StepCacheFile if (hasStrata) { - // 256 × u16 offsets, then u32 length-prefixed strata byte array. for (var i = 0; i < StepChunk.CellsPerChunk; i++) { w.Write(strataOffsetByCell[i]); @@ -600,7 +557,7 @@ internal static class StepCacheFile private static StepChunk ReadChunk(byte[] buffer) { var r = new BufferReader(buffer); - // Skip ChunkX + ChunkY (already known via the index lookup). + // ChunkX + ChunkY — already known from the index lookup that got us here. r.ReadUShort(); r.ReadUShort(); var multisVersion = (int)r.ReadUInt(); @@ -608,7 +565,7 @@ internal static class StepCacheFile var chunk = new StepChunk { BuiltMultisVersion = multisVersion }; - if (kind == KindUniform) // Uniform — one cell's worth of the 19 base arrays, fill all 256 cells. + if (kind == KindUniform) // one cell's values, broadcast to all 256 { Array.Fill(chunk.WalkMask, r.ReadByte()); Array.Fill(chunk.WetMask, r.ReadByte()); @@ -640,8 +597,8 @@ internal static class StepCacheFile r.Read(chunk.WetMask); ReadSBytes(r, chunk.SourceZ); - // Predictive-Z reconstruction: present arrays carry residuals (z = predict + residual); - // absent arrays are synthesized from mask+SourceZ (z = predict, residual implicitly 0). + // Inverse of the write path: a stored array carries residuals to add back to the + // prediction, an omitted one IS the prediction. Span residual = stackalloc sbyte[StepChunk.CellsPerChunk]; for (var d = 0; d < 16; d++) { @@ -706,16 +663,15 @@ internal static class StepCacheFile r.Read(MemoryMarshal.Cast(arr.AsSpan())); /// - /// Open handle on a .swb file. Holds the FileStream + chunk-offset index. Chunks are - /// fetched on demand via ; only the records actually queried - /// are ever materialized. Dispose releases the underlying stream. + /// An open .swb file: the stream plus the chunk index. Only the records actually asked for are + /// ever read or inflated. Dispose releases the stream. /// internal sealed class LazyReader : IDisposable { private FileStream _stream; private readonly Dictionary _offsets; - private byte[] _buffer; // raw on-disk record: [u32 uncompressedLen][payload] - private byte[] _bodyBuffer; // decompressed v6 record, parsed by ReadChunk + private byte[] _buffer; // the raw framed record as it sits on disk + private byte[] _bodyBuffer; // that record, inflated, ready for ReadChunk public uint MapId { get; } public ulong Fingerprint { get; } @@ -725,11 +681,7 @@ internal static class StepCacheFile public bool Has(int chunkX, int chunkY) => _offsets.ContainsKey(PackChunkKey(chunkX, chunkY)); - /// - /// Enumerates every (chunkX, chunkY) coordinate the file holds. Used by - /// when preload is enabled to materialize all chunks - /// upfront instead of on first query. - /// + /// Every (chunkX, chunkY) the file holds. Used to preload the whole file. public IEnumerable<(int chunkX, int chunkY)> EnumerateChunkCoords() { foreach (var key in _offsets.Keys) @@ -754,9 +706,8 @@ internal static class StepCacheFile } /// - /// Returns the chunk record at (, ) - /// from the file, or null if the file doesn't contain it. Single seek + bulk read, - /// sized exactly to the chunk's recorded length (which varies with strata size). + /// Reads one chunk from the file, or null if the file has no record for it. One seek and + /// one read, sized to the record's indexed length. /// public StepChunk TryReadChunk(int chunkX, int chunkY) { @@ -771,7 +722,6 @@ internal static class StepCacheFile return null; } - // Grow the on-disk scratch buffer if this chunk's record is larger than what we have. if (entry.length > _buffer.Length) { _buffer = new byte[entry.length]; @@ -784,8 +734,8 @@ internal static class StepCacheFile return null; } - // Frame: [u32 uncompressedLen][payload]. payload is libdeflate-compressed, unless its - // length equals uncompressedLen, in which case it was stored raw (incompressible). + // [u32 uncompressedLen][payload], where the payload is compressed unless its length + // already equals uncompressedLen — then it was stored raw. var uncompressedLen = (int)BinaryPrimitives.ReadUInt32LittleEndian(_buffer); var payloadLen = (int)entry.length - sizeof(uint); if (_bodyBuffer.Length < uncompressedLen) @@ -799,7 +749,8 @@ internal static class StepCacheFile } else { - // Decompression is level-independent, so reuse the shared per-thread binding. + // Deflate.Standard, not .Maximum: the level only affects packing, and inflate has + // to accept whatever the writer produced regardless. var result = Deflate.Standard.Unpack( _bodyBuffer.AsSpan(0, uncompressedLen), _buffer.AsSpan(sizeof(uint), payloadLen), diff --git a/Projects/UOContent/Engines/Pathing/Cache/StepChunk.cs b/Projects/UOContent/Engines/Pathing/Cache/StepChunk.cs index 3d26aee5d..80d42f8e0 100644 --- a/Projects/UOContent/Engines/Pathing/Cache/StepChunk.cs +++ b/Projects/UOContent/Engines/Pathing/Cache/StepChunk.cs @@ -3,20 +3,19 @@ using System; namespace Server.Engines.Pathing.Cache; /// -/// Per-chunk storage backing StepCache. Holds raw walk + swim masks and destination Z -/// values for each of 256 cells in a 16x16 chunk, plus build-time metadata (multis version, multi-Z strata) -/// and LRU bookkeeping. +/// Per-chunk storage backing : walk + swim masks and destination Zs for +/// each of the 256 cells in a 16x16 chunk, plus the optional multi-Z strata and swim layers and +/// the LRU timestamp. /// internal sealed class StepChunk { public const int CellsPerChunk = 256; // 16 x 16 - /// Bit i of WalkMask[c] = "default walker can step from cell c to neighbor (Direction)i". - /// Raw — no diagonal corner-cut applied here. + /// Bit i of WalkMask[c]: a default walker can step from cell c to neighbour (Direction)i. + /// Raw — no diagonal corner-cut applied, so callers must AND the partner bits themselves. public readonly byte[] WalkMask = new byte[CellsPerChunk]; - /// Bit i of WetMask[c] = "swim-only mob can step from cell c to neighbor (Direction)i". - /// Layered with WalkMask via canSwim/cantWalk capability flags. + /// Bit i of WetMask[c]: a swim-only mob can step from cell c to neighbour (Direction)i. public readonly byte[] WetMask = new byte[CellsPerChunk]; public readonly sbyte[] SourceZ = new sbyte[CellsPerChunk]; @@ -40,14 +39,13 @@ internal sealed class StepChunk public readonly sbyte[] SwimZNW = new sbyte[CellsPerChunk]; /// - /// Swim layer — populated only for chunks containing at least one shore cell (a cell - /// with both a walkable land surface and a water surface separated by > StepHeight). - /// On shore cells, queries from the swim source Z miss the primary source-Z guard; - /// the swim layer carries the correct wetMask + per-direction destination Zs computed - /// from the water surface's perspective. For non-shore cells in a chunk that has the - /// layer, [cell] = sentinel. - /// All swim-layer arrays are null on chunks with no shore cells (~90% of map chunks - /// on Trammel) — zero memory cost on the common case. + /// Marks a cell with no swim-layer entry, in a chunk that has the layer. + /// + /// The swim layer exists only on chunks holding at least one shore cell — a cell with both a + /// walkable surface and a water surface more than StepHeight apart. A swim query there sits + /// too far from the primary SourceZ to pass the source-Z guard, so the layer carries a second + /// mask and destination-Z set computed from the water surface instead. Chunks with no shore + /// cells leave every swim-layer array null. /// public const sbyte NoSwimLayerCell = sbyte.MinValue; @@ -79,9 +77,8 @@ internal sealed class StepChunk public sbyte[] SwimZNW_Layer => _swimZNW_extra; /// - /// Lazily allocates the swim-layer arrays and seeds with - /// the sentinel. Called at bake time the first time a - /// shore cell is detected in this chunk. + /// Allocates the swim-layer arrays and seeds with + /// . Called on the first shore cell found in this chunk. /// internal void AllocateSwimLayer() { @@ -105,28 +102,30 @@ internal sealed class StepChunk } } - /// Sentinel: cell has no strata — single-Z, use the main Walk/Wet arrays. + /// + /// Marks a single-Z cell: no strata, read the main Walk/Wet arrays instead. Because this + /// takes ushort.MaxValue, a real strata offset is at most NoStrata - 1, which bounds + /// to NoStrata bytes. + /// public const ushort NoStrata = ushort.MaxValue; /// - /// Length-256 offset table: StrataOffsetByCell[cell] = byte offset into - /// where this cell's strata begin, or - /// for cells without multi-Z. Null when the chunk has zero multi-Z cells. + /// Length-256 table mapping a cell to the byte offset in where its + /// strata begin, or . Null when no cell in the chunk is multi-Z. /// private ushort[] _strataOffsetByCell; /// - /// Packed per-cell strata. For each cell with strata: - /// u8 stratumCount, then stratumCount × Stratum (19 bytes each): - /// sbyte zCenter, byte walkMask, byte wetMask, - /// sbyte walkZ_N..NW (8), sbyte swimZ_N..NW (8) + /// Packed strata for the multi-Z cells. Per cell: u8 stratumCount, then stratumCount records + /// of bytes — sbyte zCenter, byte walkMask, byte wetMask, + /// sbyte walkZ_N..NW (8), sbyte swimZ_N..NW (8). /// private byte[] _strataData; - /// Snapshot of Sector.MultisVersion at the time this chunk was built. + /// Reserved. Chunks are static-only, so this is always 0. public int BuiltMultisVersion; - /// Updated on every cache hit/miss. Used by LRU fallback eviction. + /// Refreshed on every query that reaches this chunk. Drives LRU eviction. public long LastTouchedTicks; /// Size in bytes of one Stratum record in StrataData. @@ -140,10 +139,8 @@ internal sealed class StepChunk _strataData == null ? ReadOnlySpan.Empty : _strataData.AsSpan(); /// - /// Single-shot setter for the chunk's strata. Pass null/null to clear (chunk becomes - /// "no multi-Z"). Otherwise must be length 256 with - /// for cells without strata, and the - /// packed strata records. + /// Sets the chunk's strata in one shot. must be length 256, + /// carrying for single-Z cells. Pass null/null to clear. /// internal void SetStrata(ushort[] offsetByCell, byte[] data) { @@ -151,18 +148,17 @@ internal sealed class StepChunk _strataData = data; } - /// Serialization hook: returns the raw offset array (or null if no strata). + /// Serialization hook: the raw offset array, or null if the chunk has no strata. internal ushort[] GetStrataOffsetByCellForSerialization() => _strataOffsetByCell; - /// Serialization hook: returns the raw data array (or null if no strata). + /// Serialization hook: the raw data array, or null if the chunk has no strata. internal byte[] GetStrataDataForSerialization() => _strataData; /// - /// True when every cell shares one value across WalkMask, WetMask, SourceZ, and all 16 - /// directional-Z arrays, and the chunk has neither multi-Z strata nor a swim layer. Such a - /// chunk serializes to a ~28-byte uniform record (StepCacheFile v5) instead of the full - /// record. Chunks with a swim layer (shore cells) are never uniform — their per-cell swim - /// data must be preserved via the Full record. + /// True when all 256 cells share one value across WalkMask, WetMask, SourceZ and every + /// directional-Z array, with no strata and no swim layer — open water or solid rock, mostly. + /// collapses such a chunk to a ~28-byte record. A swim layer + /// disqualifies a chunk outright: its per-cell shore data would not survive the collapse. /// internal bool IsUniform() => _strataOffsetByCell == null && !HasSwimLayer diff --git a/Projects/UOContent/Engines/Pathing/Cache/StepMask.cs b/Projects/UOContent/Engines/Pathing/Cache/StepMask.cs index 832f3d853..6817f9ae0 100644 --- a/Projects/UOContent/Engines/Pathing/Cache/StepMask.cs +++ b/Projects/UOContent/Engines/Pathing/Cache/StepMask.cs @@ -1,10 +1,10 @@ namespace Server.Engines.Pathing.Cache; /// -/// Per-cell, per-direction static walkability data baked by -/// and stored by . WalkMask + WalkZ_* applies under default-walker -/// rules (cantWalk=false, canSwim=false). WetMask + SwimZ_* applies under swim-only rules -/// (cantWalk=true, canSwim=true). Algorithms layer the right rules per mobile. +/// Per-cell, per-direction walkability baked by and stored by +/// . Two rule sets travel together: WalkMask + WalkZ_* for a default +/// walker (cantWalk=false, canSwim=false), WetMask + SwimZ_* for a swim-only mob +/// (cantWalk=true, canSwim=true). Callers overlay whichever applies to the mobile. /// public readonly struct StepMask( byte walkMask, @@ -49,8 +49,8 @@ public readonly struct StepMask( public readonly CacheHitKind HitKind = hitKind; /// - /// True when the cache produced a usable answer (Hit / Miss_NotBuilt / Miss_DirtyRebuild). - /// False on Fallthrough_*, in which case the caller must use the slow path for this cell. + /// True when the cache produced a usable answer. False on any Fallthrough_*, where the + /// payload is all zeroes and the caller must resolve this cell via the slow path. /// public bool IsHit => HitKind <= CacheHitKind.Miss_DirtyRebuild; diff --git a/Projects/UOContent/Engines/Pathing/Cache/StepProbe.cs b/Projects/UOContent/Engines/Pathing/Cache/StepProbe.cs index a8c89991b..54a880b51 100644 --- a/Projects/UOContent/Engines/Pathing/Cache/StepProbe.cs +++ b/Projects/UOContent/Engines/Pathing/Cache/StepProbe.cs @@ -4,123 +4,37 @@ using CalcMoves = Server.Movement.Movement; namespace Server.Engines.Pathing.Cache; /// -/// Computes static-only walkability for a single cell — the per-cell, per-direction -/// "can step" mask and destination Z, based purely on land + statics.mul tiles (NOT -/// multis). Mirrors .Check minus the item and mobile collision -/// phases. Multis (houses, boats) are intentionally excluded: they're dynamic content, so -/// cells they cover route to the live movement path via 's -/// multi-halo fallthrough rather than being baked into the static chunk cache. +/// Computes the 8-direction "can step" mask and destination Zs for a single cell from land and +/// statics alone. Mirrors .Check minus the item and mobile collision +/// phases, which belong to the caller's dynamic-obstacle pass. +/// +/// Multis (houses, boats) are excluded from the static bake because they are dynamic content; +/// cells they cover route to the live movement path via 's multi halo. +/// is the opt-in exception for those cells. +/// +/// Each call bakes both rule sets: walker (canSwim=false, cantWalk=false) and swim-only +/// (canSwim=true, cantWalk=true). Diagonal corner-cut is not applied — callers hold the partner +/// bits in the same mask byte and combine them at query time. /// -/// -/// Bakes two rule sets per cell: walker (canSwim=false, cantWalk=false) and swim-only -/// (canSwim=true, cantWalk=true). Item / mobile collision phases are omitted (they're -/// the dynamic-obstacle pass's job). Diagonal corner-cut is NOT applied here; callers -/// must AND the partner-cell results at query time. -/// public static class StepProbe { private const int PersonHeight = 16; private const int StepHeight = 2; - public readonly struct ComputedStratum(sbyte zCenter, StepMask mask) - { - public readonly sbyte ZCenter = zCenter; - public readonly StepMask Mask = mask; - } - /// - /// Tier 4 strata builder: enumerates the distinct walkable standing-Zs at (x, y) - /// — one per land surface plus one per walkable static — and runs - /// at each, producing a per-stratum walkability snapshot. - /// Returns null when the cell has 0 or 1 strata (single-Z; the caller should use - /// the chunk's main mask). - /// - public static ComputedStratum[] ComputeStrataAt(Map map, int x, int y) - { - if (map == null || map == Map.Internal) - { - return null; - } - if (x < 0 || y < 0 || x >= map.Width || y >= map.Height) - { - return null; - } - - // Collect candidate Zs. 16 slots is generous — multi-Z cells in practice rarely - // exceed 3-4 surfaces (bridge over land, paver-over-ground, multi-floor stairs). - Span zs = stackalloc int[16]; - var count = 0; - - var landTile = map.Tiles.GetLandTile(x, y); - var landFlags = TileData.LandTable[landTile.ID & TileData.MaxLandValue].Flags; - if (!landTile.Ignored && (landFlags & TileFlag.Impassable) == 0) - { - map.GetAverageZ(x, y, out _, out var landCenter, out _); - zs[count++] = landCenter; - } - - foreach (var tile in map.Tiles.GetStaticTiles(x, y)) - { - if (count >= zs.Length) - { - break; - } - var data = TileData.ItemTable[tile.ID & TileData.MaxItemValue]; - if (!data.Surface || data.Impassable) - { - continue; - } - zs[count++] = tile.Z + data.CalcHeight; - } - - if (count <= 1) - { - return null; - } - - // Sort and merge near-equal Zs. Two Zs separated by less than 2*StepHeight collapse - // into a single stratum — the slow path's tolerance treats them as the same surface. - zs[..count].Sort(); - Span distinct = stackalloc int[16]; - var distinctCount = 0; - for (var i = 0; i < count; i++) - { - if (distinctCount == 0 || zs[i] - distinct[distinctCount - 1] > 2 * StepHeight) - { - distinct[distinctCount++] = zs[i]; - } - } - - if (distinctCount <= 1) - { - return null; - } - - var strata = new ComputedStratum[distinctCount]; - for (var i = 0; i < distinctCount; i++) - { - var z = (sbyte)Math.Clamp(distinct[i], sbyte.MinValue, sbyte.MaxValue); - strata[i] = new ComputedStratum(z, ComputeMaskAt(map, x, y, z)); - } - return strata; - } - - /// - /// Writes the distinct surface Zs at (x, y) that a default walker (PersonHeight envelope) - /// can actually STAND on — each candidate surface (walkable land center + every walkable - /// static top) that has PersonHeight of vertical clearance free of impassable statics — - /// into , ascending, and returns the count. + /// Writes the surface Zs at (x, y) a default walker can actually stand on into + /// , ascending, and returns the count. A candidate surface — the + /// walkable land centre, or any walkable static's top — qualifies only if a PersonHeight + /// envelope above it is clear of impassable statics. /// - /// This is the clearance-aware counterpart to 's candidate - /// gather: it drops surfaces a creature cannot occupy (land under a sewer walkway, ground - /// under a low bridge), so the result is exactly the set of standing Zs the slow path can - /// resolve to. Two standable surfaces are inherently >= PersonHeight apart (an upper - /// surface within PersonHeight of a lower one removes the lower one's clearance), so a - /// single ascending pass with an exact-duplicate skip is sufficient. + /// The clearance test is what makes this the exact set of standing Zs the slow path can + /// resolve to: it drops surfaces a creature cannot occupy, like the land beneath a sewer + /// walkway or a low bridge. That in turn means two surviving surfaces are always at least + /// PersonHeight apart (an upper surface any closer would have taken the lower one's + /// clearance away), so one ascending pass with a duplicate skip suffices. /// - /// Used by the baker to capture walkable static-over-land surfaces (sewer/dungeon - /// walkways, bridges, raised foundations, upper building floors) that the land-anchored - /// main mask would otherwise miss. + /// The baker anchors each cell here so static-over-land geometry — walkways, bridges, raised + /// foundations, upper floors — bakes at the Z a creature stands on rather than the land average. /// public static int ComputeStandableSurfaceZs(Map map, int x, int y, Span zs) { @@ -190,21 +104,17 @@ public static class StepProbe ComputeMaskCore(map, x, y, sourceZ, includeMultis: false); /// - /// Multi-aware counterpart to : synthesizes the full 8-direction - /// walkability mask for a cell covered by (or adjacent to) a multi, folding house/boat component - /// tiles into the surface/step logic via GetStaticAndMultiTiles. Replaces the slow path's 8x - /// per-cell CheckMovement for Fallthrough_Multi cells. Item/mobile collision is still handled by - /// the caller's dynamic-obstacle pass. + /// Multi-aware counterpart to , for cells a multi covers or + /// neighbours: folds house/boat component tiles into the same surface/step logic. Builds the + /// whole 8-direction mask in one pass, where the slow path would run CheckMovement eight times. /// public static StepMask ComputeMultiMaskAt(Map map, int x, int y, sbyte sourceZ) => ComputeMaskCore(map, x, y, sourceZ, includeMultis: true); /// - /// Shared per-cell 8-direction mask builder. With includeMultis=false this reproduces the - /// static-only bake (land + statics.mul). With includeMultis=true it also folds in multi - /// (house/boat) component tiles via GetStaticAndMultiTiles — the multi-aware synthesizer used - /// for Fallthrough_Multi cells. Item/mobile collision phases are still omitted (the dynamic pass - /// owns them). + /// Shared 8-direction mask builder behind and + /// . is the only difference: + /// it swaps the tile source to GetStaticAndMultiTiles so house and boat components participate. /// private static StepMask ComputeMaskCore(Map map, int x, int y, sbyte sourceZ, bool includeMultis) { @@ -224,8 +134,8 @@ public static class StepProbe byte wetMask = 0; Span walkZs = stackalloc sbyte[8]; Span swimZs = stackalloc sbyte[8]; - // stackalloc is NOT zero-initialized — unwritten slots hold whatever was on the - // stack. Clear before use; the loop only writes slots where the step succeeds. + // stackalloc is not zero-initialized, and the loop below writes a slot only where the + // step succeeds, so blocked directions would otherwise carry stack garbage. walkZs.Clear(); swimZs.Clear(); @@ -262,10 +172,10 @@ public static class StepProbe } /// - /// Returns the slow path's standing-Z for a default walker at (x, y). Mirrors - /// MovementImpl.Check's surface-selection — paver Z+1 for paver-over-ground, - /// landCenter for bare land. Used by to bake SourceZ so - /// A*'s tracked-per-cell Z matches the cache's bake-time assumption. + /// The standing-Z a default walker at (x, y) resolves to under the slow path's + /// surface-selection rules: paver Z+1 over paver-on-ground, land centre on bare land. + /// The baker anchors cells with instead, which is + /// clearance-aware; this remains the direct MovementImpl equivalent for parity checks. /// public static int ComputeStandingZ(Map map, int x, int y, int locZ) { diff --git a/Projects/UOContent/Engines/Pathing/PathCacheCommands.cs b/Projects/UOContent/Engines/Pathing/PathCacheCommands.cs index 14a41b6c0..f14886564 100644 --- a/Projects/UOContent/Engines/Pathing/PathCacheCommands.cs +++ b/Projects/UOContent/Engines/Pathing/PathCacheCommands.cs @@ -8,23 +8,23 @@ namespace Server.Engines.Pathing; /// /// Admin commands for inspecting and operating the pathfinding step cache. -/// [PathCacheStats — current resident-chunk count + hit/miss/eviction telemetry. -/// [PathCacheClear — drop all cached chunks, close lazy readers, zero counters. -/// [PathBake — walk a whole map building the full static cache, then save it. -/// [PathCacheSave — persist resident chunks per map to Data/Pathfinding/<mapId>.swb. -/// [PathCacheLoad — open those files as lazy backing stores. Also runs at startup. -/// [PathRecord — toggle JSONL telemetry capture for replay / benchmark corpora. +/// [PathCacheStats — resident-chunk count and hit/miss/eviction telemetry. +/// [PathCacheClear — drop all cached chunks, close the .swb readers, zero the counters. +/// [PathBake — build a map's full static cache and save it. +/// [PathCacheSave — persist the resident chunks to Data/Pathfinding/<mapId>.swb. +/// [PathCacheLoad — open those files as backing stores. Also runs at startup. +/// [PathRecord — toggle capture of pathfind telemetry. /// -/// The step cache works WITHOUT any .swb file — chunks build on demand as creatures path. -/// A baked .swb is an optional optimization that removes first-pathfind-after-boot latency -/// for shard owners who want it; is how you produce one. +/// None of this is required: the cache builds chunks on demand as creatures path, with or without +/// a .swb on disk. Baking one is purely an optimization that trades disk and a few minutes of bake +/// time for the removal of first-pathfind-after-boot latency. /// public static class PathCacheCommands { private static readonly ILogger logger = LogFactory.GetLogger(typeof(PathCacheCommands)); - // modernuo.json flag: when true, Initialize() bakes any missing/stale .swb at startup. - // The first-boot ConfigurePrompts() prompt writes it. + // When set, Initialize() bakes any missing or stale .swb at startup. ConfigurePrompts() asks + // for it on first boot. private const string PrebakeSetting = "pathfinding.prebakeMaps"; private static string PathFor(int mapId) => @@ -32,9 +32,8 @@ public static class PathCacheCommands public static void Configure() { - // Resident-chunk cap is shard-tunable. Default 8192 ≈ 40 MB; small shards may - // want lower, large shards (or full-map bakes) may want higher. Setting is - // written back to server.cfg on first boot for discoverability. + // Resident-chunk cap, shard-tunable — the default works out to roughly 40 MB. Written back + // to server.cfg on first boot so it's discoverable. StepCache.Instance.MaxResidentChunks = ServerConfiguration.GetOrUpdateSetting( "pathfinding.maxResidentChunks", 8192 @@ -52,13 +51,13 @@ public static class PathCacheCommands } /// - /// First-boot prompt, auto-invoked by AssemblyHandler.Invoke("ConfigurePrompts") in - /// the startup sequence — after assemblies load (so content can prompt) but before Serilog - /// starts, so the console prompt isn't interleaved with async log output. Offers to pre-bake - /// the pathfinding .swb cache for the selected maps; the answer persists in - /// modernuo.json (), so it's asked exactly once. Skipped when the - /// setting already exists or when input is redirected (headless/CI) — operators can set the - /// flag directly. The bake itself happens later in . + /// Asks, once, whether to pre-bake the .swb cache; does the work later. + /// The answer persists, so the question is never repeated, and it's skipped entirely when input + /// is redirected — a headless or CI boot sets directly instead. + /// + /// Runs in the ConfigurePrompts phase because that's the one window where content can prompt: + /// assemblies are loaded, but Serilog hasn't started, so console output won't interleave with + /// async log writes. /// public static void ConfigurePrompts() { @@ -81,16 +80,15 @@ public static class PathCacheCommands } /// - /// Auto-invoked by AssemblyHandler.Invoke("Initialize") after the tile matrix and - /// world are loaded. When is set, bakes any map whose - /// .swb is missing or stale, so the first pathfind on each region is already warm. A - /// fresh cache makes this a no-op, so only first boot — or a client/map update that changes - /// the fingerprint — pays the cost. + /// Bakes any map whose .swb is missing or stale, when is + /// set. Runs in the Initialize phase, once the tile matrix and world are loaded. An up-to-date + /// cache makes it a no-op, so the cost lands only on a first boot or after a client or map + /// update moves the fingerprint. /// - /// Validity is decided by : runs - /// in the earlier Configure phase, opening (and fingerprint- - /// validating) a reader for every up-to-date .swb. So a map with an open reader is - /// already good and we skip it — no need to recompute the fingerprint a second time here. + /// A map is judged up-to-date by whether it has an open reader. + /// already ran in the earlier Configure phase and only opens a reader for a .swb whose + /// fingerprint validates, so an open reader is proof of a good bake — no need to fingerprint + /// the map a second time here. /// public static void Initialize() { @@ -110,7 +108,7 @@ public static class PathCacheCommands if (StepCache.Instance.HasLazyReader(map.MapID)) { - continue; // AutoLoadAtStartup already opened a fingerprint-valid .swb for this map + continue; // already has a fingerprint-valid .swb open } var path = PathFor(map.MapID); @@ -127,15 +125,14 @@ public static class PathCacheCommands if (baked > 0) { logger.Information("PathBake: pre-bake complete ({Count} map(s) written).", baked); - AutoLoadAtStartup(); // (re)open the freshly written files as lazy backing stores + AutoLoadAtStartup(); // reopen what we just wrote } } /// - /// Open Data/Pathfinding/<mapId>.swb as a lazy backing store for every map. - /// Reads only the header + chunk-offset index up front (~16 bytes per chunk); - /// individual chunk records are fetched on demand when the cache asks for them. - /// RAM stays bounded by MaxResidentChunks regardless of file size. + /// Opens Data/Pathfinding/<mapId>.swb as a backing store for every map. Only the header and + /// index are read up front; chunk records are fetched as the cache asks for them, so resident + /// memory stays bounded by the LRU cap however large the files are. /// private static void AutoLoadAtStartup() { @@ -196,9 +193,9 @@ public static class PathCacheCommands continue; } - // BakeMap walks the whole map (building every chunk) and writes the .swb. The - // chunks are left resident afterward; drop them so peak memory is bounded to one - // map at a time and the post-command footprint returns to the LRU cap. + // BakeMap leaves every chunk it built resident. Drop them between maps so peak memory + // is one map's worth rather than all of them, and the footprint afterwards is back + // under the LRU cap. var written = StepCache.Instance.BakeMap(map.MapID, PathFor(map.MapID)); StepCache.Instance.ClearResidentChunks(); @@ -218,8 +215,7 @@ public static class PathCacheCommands return; } - // Reopen the freshly written files as lazy backing stores so they're usable now - // without a restart (resident memory stays bounded by the LRU cap). + // Reopen what we just wrote, so the bake is usable immediately without a restart. AutoLoadAtStartup(); from.SendMessage($"PathBake: {totalChunks} chunks across {totalMaps} map(s) in {sw.Elapsed.TotalSeconds:F1}s; lazy readers reopened."); } diff --git a/Projects/UOContent/Engines/Pathing/PathDiag.cs b/Projects/UOContent/Engines/Pathing/PathDiag.cs index d115c3c6c..38bd36bc4 100644 --- a/Projects/UOContent/Engines/Pathing/PathDiag.cs +++ b/Projects/UOContent/Engines/Pathing/PathDiag.cs @@ -8,26 +8,19 @@ using Server.Targeting; namespace Server.Engines.Pathing; /// -/// Developer diagnostic for the bitmap A* step cache. Stand where a creature would start, -/// run [PathDiag, and target the goal. The detailed report is appended to -/// Logs/pathdiag.log; a short summary is sent to the invoking client. For the route -/// it records: -/// 1. the raw tile makeup of the start and goal cells (land + statics) and the -/// clearance-aware standable surfaces the baker anchors to — the ground truth for -/// "why does the cache (not) serve this cell"; -/// 2. one warm -served Find with the per-pathfind cache -/// hit/fallthrough breakdown and fallthrough fraction — a high fallthrough fraction -/// means the cache isn't helping the route (it pays the lookup then uses the slow path); -/// 3. warm timing over many iterations. +/// Diagnoses why the step cache does or doesn't serve a given route. Stand where the creature +/// would start, run [PathDiag, target the goal; the full report lands in +/// Logs/pathdiag.log and a summary goes to the client. It reports the tile makeup of the +/// start and goal cells alongside the standable surfaces the baker anchors to, the cache +/// hit/fallthrough breakdown for one warm Find, and warm timings. /// -/// Primarily useful when bringing up custom maps / facets: it shows whether static-over-land -/// geometry (dungeon walkways, bridges, stairs, raised foundations, stacked floors) is being -/// baked at the right Z. +/// The fallthrough fraction is the number to read: a high one means the cache is paying for a +/// lookup on every cell and then taking the slow path anyway. That usually points at +/// static-over-land geometry — dungeon walkways, bridges, stairs, stacked floors — baking at the +/// wrong Z, which is why this is most useful when bringing up a custom map or facet. /// -/// Output goes to a log file rather than the console because the live server uses Serilog and -/// raw Console writes interleave badly with it. The promotion gate is forced to eager -/// (threshold 1) for the duration so the cache builds on first touch and the numbers reflect -/// its best case; the previous threshold is restored afterward. +/// The promotion gate is forced eager for the duration, so the numbers reflect the cache's best +/// case rather than an artifact of chunks not having been built yet. /// public static class PathDiag { @@ -67,7 +60,7 @@ public static class PathDiag var cache = StepCache.Instance; var previousThreshold = cache.MissPromotionThreshold; - cache.MissPromotionThreshold = 1; // eager build — measure the cache's best case + cache.MissPromotionThreshold = 1; // build eagerly, so we measure the cache's best case StreamWriter log = null; try @@ -99,9 +92,8 @@ public static class PathDiag } /// - /// Writes the raw tile makeup of one cell plus the surfaces the baker anchors to. A large - /// gap between the query Z and the standable surfaces is the signature of a route the - /// cache can't serve (the creature stands on a static surface far from the land average). + /// Dumps one cell's tiles and the surfaces the baker anchors to. A wide gap between the query Z + /// and every standable surface is the signature of a cell the cache can't serve. /// private static void DumpCell(TextWriter log, Map map, int x, int y, int queryZ, string label) { @@ -133,9 +125,8 @@ public static class PathDiag } /// - /// Runs one warm Find and records the StepCache counter delta for it — the per-pathfind - /// cache hit/fallthrough mix and the fallthrough fraction. Returns a summary for the - /// caller to relay to the player. + /// Runs one Find against a warm cache and reports the counter delta it produced — the + /// hit/fallthrough mix for that single pathfind. /// private static (string result, double fallthroughPct, long total) RunInstrumentedFind( TextWriter log, Mobile from, Map map, Point3D start, Point3D goal @@ -143,7 +134,8 @@ public static class PathDiag { var cache = StepCache.Instance; - // Warm every chunk the route touches before measuring. + // Build every chunk the route touches first, so the measured Find below reports steady-state + // behaviour rather than first-touch misses. for (var i = 0; i < 3; i++) { BitmapAStarAlgorithm.Instance.Find(from, map, start, goal); diff --git a/Projects/UOContent/Engines/Pathing/PathfindRecorder.cs b/Projects/UOContent/Engines/Pathing/PathfindRecorder.cs index f21637dc4..c3490d235 100644 --- a/Projects/UOContent/Engines/Pathing/PathfindRecorder.cs +++ b/Projects/UOContent/Engines/Pathing/PathfindRecorder.cs @@ -7,25 +7,14 @@ using Server.Text; namespace Server.Engines.Pathing; /// -/// Admin-toggled telemetry: appends a JSONL line per pathfind request to a file. -/// One record per BitmapAStarAlgorithm.Find call, capturing the inputs (start, goal, -/// map, capability flags) needed to replay the scenario in benchmarks. Output format -/// matches the corpus the BDN harness consumes. +/// Appends one JSONL record per pathfind, capturing the inputs — start, goal, map, capability +/// flags — needed to replay it later in a benchmark. Toggled at runtime with [PathRecord; +/// only seeds the initial state from server.cfg. /// -/// Hot-toggleable at runtime via the [PathRecord admin command — no restart needed. -/// only seeds the initial state from server.cfg -/// (pathfinding.recorder.enable, default false). -/// -/// Holds a single StreamWriter open while recording; its internal buffer absorbs -/// per-record writes without per-call File.Open / File.Append. Each record is built -/// in a stack-allocated ValueStringBuilder (zero per-int allocation for the field -/// formatting), then handed to the writer as a ReadOnlySpan<char>. -/// -/// Workload note: intended for short bursts of capture (turn on, walk a region -/// or trigger a scenario, turn off). On a busy server with hundreds of pathfinds -/// per second, sustained recording can saturate the StreamWriter's 4 KB buffer and -/// block the game thread on disk writes. A backpressure-aware async sink is a -/// future enhancement if 24/7 capture becomes a use case. +/// Meant for short bursts: turn it on, walk the region or trigger the scenario, turn it off. The +/// writes go through a StreamWriter's buffer on the game thread, so a busy shard doing hundreds of +/// pathfinds a second can saturate that buffer and stall the loop on disk I/O. Sustained capture +/// would need an async sink with backpressure. /// public static class PathfindRecorder { @@ -52,9 +41,8 @@ public static class PathfindRecorder } /// - /// Toggle recording. When enabling, opens an append-mode StreamWriter; when - /// disabling, flushes + disposes it. Idempotent — calling twice with the same - /// state is a no-op. + /// Toggles recording, opening the file on enable and flushing and closing it on disable. + /// Idempotent. /// public static void SetEnabled(bool enabled) { @@ -98,9 +86,8 @@ public static class PathfindRecorder } /// - /// Force a flush of the writer's internal buffer to disk. Safe to call when - /// disabled (no-op). Useful after a burst of recording when an admin wants to - /// inspect the file without waiting for buffer fill or disable. + /// Pushes the writer's buffer to disk, so a capture can be inspected without disabling first. + /// No-op when disabled. /// public static void Flush() { @@ -115,9 +102,8 @@ public static class PathfindRecorder } /// - /// Capture one Find call. Hot path: cheap when disabled (single bool check). - /// When enabled, formats one JSONL line and writes it through the StreamWriter's - /// internal buffer — flush is amortized across many calls. + /// Records one Find. Sits on the pathfinding hot path, so it costs a single bool check when + /// disabled. /// public static void RecordIfEnabled(Mobile m, Map map, Point3D start, Point3D goal) { @@ -140,9 +126,8 @@ public static class PathfindRecorder try { - // One interpolation handles every numeric field with no per-int ToString - // allocation; bool fields use explicit literal spans because JSON wants - // lowercase "true"/"false" and bool.ToString() yields "True"/"False". + // One interpolation covers every numeric field without a per-field ToString. The bools + // are appended as literals because JSON wants lowercase and bool.ToString() capitalizes. using var vsb = ValueStringBuilder.Create(192); vsb.Append( $"{{\"Name\":\"recorded\",\"MapId\":{map.MapID},\"StartX\":{start.X},\"StartY\":{start.Y},\"StartZ\":{start.Z},\"GoalX\":{goal.X},\"GoalY\":{goal.Y},\"GoalZ\":{goal.Z},\"CanSwim\":"