warrentyexpired/ModernUO
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

perf(pathing): pool the StepCache strata buffer, then clean up the pathing engine around it (#2523)

Browse source 
Started as an allocation pass over `StepCache` and grew into a cleanup of the surrounding pathing engine. Four commits, each independently reviewable; net **−560 lines**.

Build clean (0 warnings). All 122 `Server.Tests.Pathfinding` tests pass.

---

## 1. `perf`: pool the strata buffer, cut a hot-path dictionary lookup

**The headline is that `TryGetMask` — the actual hot path — was already allocation-free.** `StepMask` is a readonly struct, `StaticTileEnumerable` is a `ref struct`, `ChunkMissState` is a struct in a `Dictionary`. So most of this is a bake-throughput and GC-churn win, with one exception noted below.

`BuildChunk` accumulated packed multi-Z strata into a `List<byte>` that grew by doubling (256 → 512 → 1024 → …) and then paid a final `ToArray()`. A full map bake runs it ~114k times. It now writes into a `byte[]` rented from `STArrayPool<byte>.Shared` through a span writer, and hands the chunk one exact-size copy.

**This required fixing a latent out-of-bounds guard.** The record-fit check reserved headroom for **8** strata (`StratumByteLength * 8`) while `ComputeStandableSurfaceZs` can return up to **16** — so a cell could write 305 bytes starting from a 65,383-byte offset. Against a `List` that was benign (it just grew past 64 KB, and emitted offsets stayed under the `NoStrata` sentinel). Against a fixed-size rented buffer it is an out-of-bounds write, so tightening it was a *prerequisite* for the pooling, not a drive-by. The guard is now exact, which additionally proves no emitted offset can collide with `NoStrata == ushort.MaxValue`.

**One genuine query-path win:** `ShouldPromoteAfterMiss` did *two* dictionary lookups per miss — a `TryGetValue`, then an indexer assignment that re-hashes and re-probes. It now mutates in place via `CollectionsMarshal.GetValueRefOrNullRef`. This runs on every uncached chunk touch during A* expansion. The window-expiry branch keeps its explicit early return, so `MissPromotionThreshold == 1` still resets rather than promoting.

Also dropped `StepProbe.ComputeStrataAt` / `ComputedStratum` (dead code, zero callers) and collapsed six 18-argument `new StepMask(0, 0, …, kind)` blocks into `Fallthrough(kind)`.

**Considered and rejected:** pooling the `Direction[]` that `Find` returns. It *escapes* the call — `MovementPath` holds it across ticks while `PathFollower` walks `m_Index` through it — so it cannot be rented-and-returned, and it cannot be borrowed from the shared `BitmapAStarAlgorithm.Instance` without one creature clobbering another's in-flight path. `CheckPath` rate-limits repaths to one per 2s per creature, putting this at roughly 60 KB/sec at 1,000 pathing creatures. Not worth a public API break plus a use-after-return footgun.

## 2. `docs`: rewrite the comments for publication

The comments had accumulated as development notes: internal phase jargon (`Tier 4`, `the Phase-2 synthesizer`), change narration aimed at a reviewer (`which the old ComputeStandingZ anchor missed`, `legacy behavior`), benchmark anecdotes (`benchmarked as near-optimal`, `a ~20 ns lookup`), and paragraphs restating the code.

Rewritten to keep the rationale you cannot recover by reading the code — why the source-Z guard cannot be widened, why multis fall through with a halo, why the promotion gate counts Finds rather than calls, why `ComputeFingerprint` must hash the *files* and not the live tile tables — and drop the history that got us there.

Three comments were **factually wrong**, not just wordy:

- `CacheEvictionTimer` and `CacheStats` documented a class called `StaticWalkabilityCache`. No such class exists — it is `StepCache`.
- `StepCacheFile` declared `File layout v8` while `FormatVersion` is 9, and called the current record layout "the v6 layout" in four places. The layout descriptions are now unversioned so they cannot drift again.
- `StepProbe.ComputeStandingZ` claimed `StepCache` uses it to bake `SourceZ`. It has not since the baker moved to the clearance-aware `ComputeStandableSurfaceZs`; only a parity test calls it.

## 3. `refactor`: simplify `StepCacheFile.Write`, consolidate the format tests

`SaveToFile` walked `_keysList` **twice** — once to count the map's chunks, then again through a `ChunkEnumerator` closure to emit them — because `Write` needed the count up front to size its index array. Both loops had the same root cause. Passing a **span** collapses them: the count is just `span.Length`.

That deletes the `ChunkEnumerator` delegate, the closure over the list enumerator, and **both `InvalidOperationException` throws**, which existed only to police the delegate's "yield exactly `chunkCount` chunks" contract — a contract a span makes unrepresentable.

`Write` now patches the header's `IndexOffset` by seeking back to it rather than reaching into the writer's live buffer with `BinaryPrimitives`. That also retires `IndexOffsetFieldPosition`, a hand-maintained byte offset that had to track the header layout, and sidesteps the stale-array hazard that motivated the manual patch (`BufferWriter` reallocates on growth).

**Tests:** `StepCacheFileV6/V7/V8Tests` were named for the format version that introduced each transform — and the format is now **v9**, so all three names described formats the loader rejects outright. Beyond triplicated builders and plumbing, two things were actually broken:

- The three near-identical rejection tests each cited a `MinSupportedVersion` that had since moved (`"version 5 < MinSupportedVersion 6"`, `"6 < 7"`, `"7 < 8"`). They passed for the wrong reason.
- `AssertBaseEqual` (used by V7 and V8) **silently skipped the swim and strata trailers**. A regression dropping either would not have failed those tests.

Now one `StepCacheFileFormatTests`, named for behavior — predictive-Z elision, compression, compact index — with a single `AssertIdentical` that does check both trailers, the three rejection tests folded into one theory that also covers a future version, and a zero-chunk case the delegate-based writer never had coverage for.

## 4. `test`: consolidate the parity and lifecycle tests

Three files tested "parity" and none of the names said *which*. They were three different layers, and the seams are the useful part, so they are now one `StepCacheParityTests` that names them:

| Test | Compares | Answers |
|---|---|---|
| `ProbeMatchesSlowPath` | StepProbe vs MovementImpl | Is the bake right? |
| `CacheMatchesProbe` | StepCache vs StepProbe | Is it stored and returned intact? |
| `CacheServesReachableWalkStates` | StepCache vs MovementImpl | End to end, over the states A* visits |

Merging removed a duplicated stub `Mobile`, duplicated region seeds, and a filename/class mismatch (`StepProbeParityTests.cs` declared `StaticWalkabilityParityTests`). `SwimBake_ProducesWetCells` moved with it — it lived in the cache parity file but never touched the cache.

Tests reached into `StepCache._chunks` via `GetField` in **9 places**, each rebuilding the key encoding and cell-index arithmetic by hand. `StepCache` now exposes `GetResidentChunk` and `ResidentIndexInSync` alongside the internal test hooks it already had (`LazyReaderHasChunk`, `CurrentFindGeneration`), and the shared arithmetic moved to `PathingTestSupport`. All 9 reflection blocks are gone.

`StepCacheLifecycleTests` is regrouped by what it covers — promotion gate, fallthrough routes, strata, swim layer, eviction — with the `Tier4*` names dropped. Removed `Singleton_IsAvailable`, which asserted an inline-initialized static property was not null; that is the entire 123 → 122 test-count delta.

---

## Verification

Tests were mutation-checked rather than just run, since round-trip and parity tests can pass while a transform silently no-ops:

- Injecting an off-by-one into the `IndexOffset` patch fails **15 of 123** — the format tests are load-bearing.
- Offsetting the cache's cell index by one fails **7 of 10** parity cases, and the 3 that stay green are exactly the ones that do not touch the cache. The layering localizes a fault rather than just reporting one.
This commit is contained in:
Kamron Batman 2026-07-12 20:02:29 -07:00 • committed by GitHub
parent e035768ef8
commit b852bca41e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
23 changed files with 1663 additions and 2223 deletions
Download patch file Download diff file Expand all files Collapse all files

140
Projects/UOContent/Engines/Pathing/BitmapAStarAlgorithm.cs
View file

@ -26,12 +26,11 @@ using MoveImpl = Server.Movement.MovementImpl;
namespace Server.PathAlgorithms;
/// <summary>
/// A* pathfinder with a single bitmap-cache lookup per cell expansion. Default walkers
/// take one <see cref="StepCache.TryGetMask"/> call returning the 8-direction
/// mask + per-direction Z. Non-default walkers (non-GM players, creatures with swim/fly/
/// door/clip capabilities) and per-cell cache fallthroughs route through
/// <see cref="GetSuccessorsSlowPath"/>, which runs the per-direction
/// <see cref="CalcMoves.CheckMovement"/> loop for that one cell.
/// A* pathfinder that expands a cell with a single <see cref="StepCache.TryGetMask"/> lookup,
/// which returns all 8 directions' walkability and destination Zs at once. Where the cache can't
/// answer — a fallthrough on that cell, or a flying creature the static cache can't model —
/// <see cref="GetSuccessorsSlowPath"/> runs the per-direction <see cref="CalcMoves.CheckMovement"/>
/// loop for that one cell instead, so a partial cache miss costs only the cells it affects.
/// </summary>
public class BitmapAStarAlgorithm : PathAlgorithm
{
@ -50,61 +49,50 @@ public class BitmapAStarAlgorithm : PathAlgorithm
private const int PlaneOffset = 128;
private const int PlaneCount = 13;
private const int PlaneHeight = 20;
// Default shared singleton (MaxSearchNodes = 1000, set from config in Configure). Typed
// as the concrete class so Configure can set its instance config; assignable anywhere a
// PathAlgorithm is expected. Specialized variants are just additional instances.
// The shared default. A differently-configured variant is just another instance.
public static readonly BitmapAStarAlgorithm Instance = new();
// Scratch buffers — reused across every Find on THIS instance. Per-instance (not static)
// so independently-configured algorithms don't share state. ~320 KB per instance; create
// specialized instances once (static readonly), never per-call. Safe to reuse per Find
// because the game loop is single-threaded and Find is never re-entered.
// Scratch reused across every Find on this instance — roughly 320 KB of it, so create
// instances once and hold them, never per call. Per-instance rather than static so two
// differently-configured algorithms don't share state. Reuse is safe because the game loop is
// single-threaded and Find never re-enters.
private readonly Direction[] _path = new Direction[AreaSize * AreaSize];
private readonly PathNode[] _nodes = new PathNode[NodeCount];
private readonly byte[] _nodeStates = new byte[NodeCount];
private readonly int[] _successors = new int[8];
private readonly PriorityQueue<int, int> _openQueue = new();
// A* node-expansion budget: the search bails (returning null) after this many node
// expansions. Benchmarked as near-optimal: above the ~500 needed to solve walled-off
// indoor routes, below the ~1500 window-exhaustion cost ceiling where a failed
// (unreachable) search's worst-case cost spikes for no solving benefit. Successful
// searches terminate on goal-found, so this never touches the common open-terrain case.
// Per-instance so specialized algorithms (e.g. a wider-budget variant for special NPCs)
// can coexist; the shared default lives on Instance and is set from config in Configure.
// Expansion budget: the search gives up and returns null past this many nodes. It bounds the
// cost of an unreachable goal, which would otherwise exhaust the whole search window. A
// successful search stops when it finds the goal, so the budget only binds on hard or hopeless
// routes — it needs to stay high enough to solve walled-off indoor ones.
public int MaxSearchNodes { get; set; } = 1000;
private int _xOffset;
private int _yOffset;
// When set, GetSuccessors delegates to the per-cell slow path on every expansion
// (creature has CanFly — Z-jumping is beyond the cache's static-only scope).
// Every expansion goes to the slow path: the creature can fly, and arbitrary Z-jumping is
// outside what a static cache can model.
private bool _currentMobileNeedsSlowPath;
// When set, diagonal corner-cut uses the strict AND-rule (BOTH cardinal partners
// must be walkable) instead of the lenient creature OR-rule. Cache still applies —
// partner bits live in the same source-cell mask byte. Non-GM players only.
// Diagonal corner-cut uses the strict rule — both cardinal partners walkable, not just one.
// Non-GM players only. The cache still applies; the partner bits are in the same mask byte.
private bool _currentMobilePlayerStrict;
// Capability overlay applied to cache results. Layered each cell:
// Capability overlay on the cache's two rule sets, applied per cell as
// effective = (walkMask & !cantWalk) | (wetMask & canSwim)
// Reset at end of Find.
private bool _currentMobileCanSwim;
private bool _currentMobileCantWalk;
// Dynamic-obstacle pass capability flags (per-mobile, captured in Find).
// Mirrors MovementImpl.Check's per-mobile derivations so per-cell items/mobiles
// checks can be evaluated without re-deriving.
// Per-mobile flags for the dynamic-obstacle pass, derived once in Find rather than per cell.
private bool _currentMobileIgnoreDoors;
private bool _currentMobileIgnoreSpellFields;
private bool _currentMobileIgnoreMovableImpassables;
public static void Configure()
{
// A* node-expansion budget. Default 1000 is benchmarked near-optimal (see
// MaxSearchNodes). Applied to the shared singleton; specialized instances pass their
// own value. Written back to server.cfg on first boot. Auto-invoked at startup via
// AssemblyHandler.Invoke("Configure").
// Shard-tunable expansion budget for the shared instance; see MaxSearchNodes. Written back
// to server.cfg on first boot so it's discoverable.
Instance.MaxSearchNodes = ServerConfiguration.GetOrUpdateSetting(
"pathfinding.maxSearchNodes",
1000
@ -136,10 +124,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm
return null;
}
// Mark a new Find generation so the StepCache promotion gate counts THIS pathfind
// as one touch per chunk regardless of how many times the expansion frontier
// probes a given chunk. Without this, A* hits each visited chunk dozens of times
// and trips the threshold immediately.
// The frontier probes a given chunk dozens of times over one search; opening a generation
// is what makes the cache's promotion gate count all of that as a single touch.
StepCache.Instance.BeginFindGeneration();
PathfindRecorder.RecordIfEnabled(m, map, start, goal);
@ -161,7 +147,7 @@ public class BitmapAStarAlgorithm : PathAlgorithm
_currentMobileIgnoreMovableImpassables = false;
}
// Mirrors MovementImpl: dead/spectral mobiles also ignore doors.
// Dead and spectral mobiles pass through doors too. Mirrors MovementImpl.
_currentMobileIgnoreDoors |= !m.Alive || m.Body.BodyID == 0x3DB || m.IsDeadBondedPet;
_currentMobileIgnoreSpellFields = m is PlayerMobile && map != Map.Felucca;
@ -209,7 +195,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm
_nodeStates[bestNode] = 2;
// Set MovementImpl globals so per-cell slow-path fallthroughs see the right state.
// MovementImpl reads these statics, so a slow-path fallthrough on any cell below needs
// them set for this mobile.
if (bc != null)
{
MoveImpl.AlwaysIgnoreDoors = bc.CanOpenDoors;
@ -326,11 +313,10 @@ public class BitmapAStarAlgorithm : PathAlgorithm
}
/// <summary>
/// One <see cref="StepCache.TryGetMask"/> call returns the 8-direction
/// walkable mask + destination Zs. Diagonal corner-cut applies the lenient creature
/// OR-rule using partner bits in the same mask byte — no neighbor-chunk lookup needed.
/// On cache fallthrough or for non-default walkers, defers to
/// <see cref="GetSuccessorsSlowPath"/> for THIS cell only.
/// Expands one cell into its walkable neighbours. A single cache lookup covers all 8
/// directions, including the partner bits the diagonal corner-cut needs, so no neighbouring
/// cell has to be consulted. Falls back to <see cref="GetSuccessorsSlowPath"/> for this cell
/// alone when the cache can't answer.
/// </summary>
private int GetSuccessors(int p, Mobile m, Map map)
{
@ -353,16 +339,12 @@ public class BitmapAStarAlgorithm : PathAlgorithm
if (!lookup.IsHit)
{
// Multi-covered cells: synthesize a multi-aware mask in ONE pass (over land + statics +
// house/boat component tiles) instead of the slow path's 8x per-cell CheckMovement.
// Fliers and cache-off already returned at the top of GetSuccessors, so this only runs
// for cacheable walkers/swimmers. The synthesized mask flows through the SAME
// capability-overlay + diagonal corner-cut + dynamic-obstacle loop below as a static hit.
// A multi-covered cell still gets a whole-cell mask, synthesized over the house or boat
// components rather than looked up. That keeps it out of the slow path's 8 separate
// CheckMovement calls, and the result flows through the same overlay, corner-cut and
// dynamic-obstacle logic below as a cache hit would.
if (lookup.HitKind == CacheHitKind.Fallthrough_Multi)
{
// Multi-covered cell: the per-multiID interior cache serves a ~20 ns lookup for
// interior cells (and records the right counter); it falls back internally to the
// Phase-2 live synthesizer for perimeter / terrain-dirty / foundation cells.
lookup = MultiMaskCache.Instance.GetMask(map, p3D.X, p3D.Y, (sbyte)p3D.Z);
}
else
@ -371,8 +353,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm
}
}
// Capability overlay: walking allowed unless cantWalk; swimming allowed if canSwim.
// Partner bits used for diagonal corner-cut also use the effective mask.
// Overlay the mobile's capabilities onto the cache's two rule sets. The corner-cut below
// reads its partner bits from this effective mask, not the raw one.
var walkBits = _currentMobileCantWalk ? (byte)0 : lookup.WalkMask;
var swimBits = _currentMobileCanSwim ? lookup.WetMask : (byte)0;
var mask = (byte)(walkBits | swimBits);
@ -393,9 +375,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm
continue;
}
// Diagonal corner-cut. Creatures (default): OR-rule — at least one cardinal
// partner walkable. Non-GM players: AND-rule — BOTH partners must be walkable.
// Partner bits live in the same source-cell mask byte either way.
// Diagonal corner-cut: a creature needs at least one of the two flanking cardinals to
// be walkable, a non-GM player needs both.
if ((i & 1) == 1)
{
var leftBit = 1 << ((i - 1) & 0x7);
@ -408,9 +389,9 @@ public class BitmapAStarAlgorithm : PathAlgorithm
}
}
// Walking takes precedence over swimming when both apply (matches MovementImpl's
// surface-selection: closest-to-startZ wins, and walk surface is always closer
// when the creature is currently standing on land).
// Walking wins over swimming where both are possible. MovementImpl picks the surface
// closest to the start Z, and for a creature standing on land that is always the walk
// surface.
var useWalkZ = (walkBits & (1 << i)) != 0;
var z = useWalkZ
? i switch
@ -441,8 +422,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm
var absX = x + _xOffset;
var absY = y + _yOffset;
// Dynamic-obstacle pass: items + mobiles at the target cell. Cache only
// covers static walkability; dynamic state has to be checked at query time.
// The cache only knows static terrain, so items and mobiles at the target cell have to
// be checked live.
if (IsBlockedByDynamic(m, map, absX, absY, z))
{
continue;
@ -464,11 +445,9 @@ public class BitmapAStarAlgorithm : PathAlgorithm
private const int MobileHeight = 15;
/// <summary>
/// Mirrors MovementImpl's dynamic-item / mobile collision phase for a target cell.
/// Items: ImpassableSurface that overlap (z, z+PersonHeight), respecting capability
/// overrides (CanOpenDoors → ignore door items; CanMoveOverObstacles → ignore movables;
/// non-Felucca players → ignore spell fields). Mobiles: any other mobile whose Z range
/// overlaps and which we can't move over.
/// MovementImpl's item and mobile collision phase for one target cell: impassable items
/// overlapping the mobile's vertical envelope block it, subject to the capability overrides,
/// as does any other mobile it can't move over.
/// </summary>
private bool IsBlockedByDynamic(Mobile m, Map map, int x, int y, int z)
{
@ -509,9 +488,9 @@ public class BitmapAStarAlgorithm : PathAlgorithm
}
}
// A* must be able to plan a path to the goal cell even when the target mobile is
// standing on it (the follower stops within range short of it). Skip the mob-block
// check at the goal cell ONLY; everywhere else dynamic mobiles still block.
// The goal cell is usually occupied by whatever the mobile is chasing, so blocking on it
// would fail every pursuit. The follower stops short of the goal anyway. Every other cell
// still blocks on mobiles.
var skipMobCheck = x == MoveImpl.Goal.X && y == MoveImpl.Goal.Y;
if (!skipMobCheck)
@ -534,18 +513,16 @@ public class BitmapAStarAlgorithm : PathAlgorithm
}
/// <summary>
/// Mirrors MovementImpl.CanMoveOver — true when m can step onto t's cell (dead bodies,
/// hidden staff, etc.).
/// True when m can step onto t's cell — a corpse, hidden staff, and so on. Mirrors
/// MovementImpl.CanMoveOver.
/// </summary>
private static bool CanMoveOver(Mobile m, Mobile t) =>
!t.Alive || !m.Alive || t.IsDeadBondedPet || m.IsDeadBondedPet
|| t.Hidden && t.AccessLevel > AccessLevel.Player;
/// <summary>
/// Per-direction <see cref="CalcMoves.CheckMovement"/> loop for a single source cell.
/// Runs on cache fallthrough or when <see cref="_currentMobileNeedsSlowPath"/> is set.
/// CheckMovement validates land/statics/items via MovementImpl; dynamic mobile blocking
/// is layered on top because MovementImpl doesn't iterate same-cell mobiles.
/// Expands one cell the long way, with a CheckMovement call per direction. The same-cell
/// mobile check is layered on top because MovementImpl doesn't iterate those.
/// </summary>
private int GetSuccessorsSlowPath(Mobile m, Map map, int px, int py, Point3D p3D, int[] vals)
{
@ -586,11 +563,10 @@ public class BitmapAStarAlgorithm : PathAlgorithm
}
/// <summary>
/// True for creatures whose movement rules the static cache can't model. Currently
/// only CanFly — flying creatures Z-jump arbitrarily and the cache's source-Z guard
/// would over-fire. CanSwim / CantWalk are handled via the capability overlay (walkMask
/// + wetMask). CanOpenDoors / CanMoveOverObstacles only affect dynamic items and don't
/// disqualify the cache.
/// True for creatures the static cache can't model at all. Only flying ones qualify: they
/// Z-jump freely, so the cache's source-Z guard would reject nearly every cell anyway. Swim
/// and cant-walk are handled by the capability overlay, and the door / obstacle capabilities
/// only affect dynamic items, so none of those disqualify the cache.
/// </summary>
private static bool RequiresSlowPath(Mobile m) => m is BaseCreature bc && bc.CanFly;
}

5
Projects/UOContent/Engines/Pathing/Cache/CacheEvictionTimer.cs
View file

@ -3,9 +3,8 @@ using System;
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Periodic backstop that enforces the StaticWalkabilityCache resident-chunk cap.
/// Steady-state cost is a single early-return; only fires real work when the cache
/// has overflowed MaxResidentChunks. Runs on the game thread; no locking required.
/// Periodic backstop that enforces <see cref="StepCache"/>'s resident-chunk cap. Costs a
/// single early-return unless the cache has overflowed MaxResidentChunks.
/// </summary>
public class CacheEvictionTimer : Timer
{

18
Projects/UOContent/Engines/Pathing/Cache/CacheHitKind.cs
View file

@ -1,18 +1,18 @@
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Outcome categories for StepCache.TryGetMask. Used for telemetry and to drive
/// the slow-path fallthrough decision in callers. Ordering is load-bearing:
/// values 0-2 are hits, values 3+ are fallthroughs (see StepMask.IsHit).
/// Outcome of <see cref="StepCache.TryGetMask"/>. Drives both telemetry and the caller's
/// decision to fall back to the slow path. Ordering is load-bearing: 0-2 are usable answers,
/// 3+ are fallthroughs, and <see cref="StepMask.IsHit"/> tests that boundary.
/// </summary>
public enum CacheHitKind : byte
{
Hit = 0, // clean default-walker answer from the resident chunk
Hit = 0, // served from the resident chunk
Miss_NotBuilt = 1, // chunk wasn't resident; built and returned
Miss_DirtyRebuild = 2, // version mismatch; rebuilt and returned
Fallthrough_MultiZ = 3, // cell has multiple walkable surfaces; caller must use slow path
Miss_DirtyRebuild = 2, // chunk was stale; rebuilt and returned
Fallthrough_MultiZ = 3, // stacked walkable surfaces, none matching the query Z
Fallthrough_OffMap = 4, // out of bounds
Fallthrough_SourceZMismatch = 5, // |loc.Z - BakedSourceZ| > StepHeight; cache answer would diverge
Fallthrough_NotBuilt = 6, // first-touch miss without lazy file hit; build deferred until second touch
Fallthrough_Multi = 7, // a multi (house/boat) covers this cell or its halo; use the live path
Fallthrough_SourceZMismatch = 5, // |query Z - baked SourceZ| > StepHeight; a cached answer would diverge
Fallthrough_NotBuilt = 6, // first touch of an unbuilt chunk; the promotion gate defers the build
Fallthrough_Multi = 7, // a multi (house/boat) covers this cell or its halo
}

5
Projects/UOContent/Engines/Pathing/Cache/CacheStats.cs
View file

@ -1,8 +1,9 @@
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Snapshot of StaticWalkabilityCache counters. Returned by GetStats() and consumed
/// by the [PathCacheStats admin command. All counters are monotonic except ResidentChunks.
/// Snapshot of <see cref="StepCache"/>'s counters, as returned by GetStats() and reported by
/// the [PathCacheStats command. Every counter is monotonic except ResidentChunks, and all of
/// them reset on Clear() — they count since the last clear, not since startup.
/// </summary>
public readonly struct CacheStats(
int residentChunks,

84
Projects/UOContent/Engines/Pathing/Cache/MultiMaskCache.cs
View file

@ -6,13 +6,19 @@ using Server.Multis;
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Warm, in-memory cache of per-multiID local-frame walkability masks for INTERIOR multi cells
/// (cell + all 8 neighbours covered by the multi → terrain-neighbour-free → position-invariant).
/// Wraps the Phase-2 synthesizer (StepProbe.ComputeMultiMaskAt). Cleanliness is decided ONCE per
/// instance (BaseMulti.PathInteriorCacheState, via ComputeFootprintClean): a clean instance — whole
/// footprint terrain below the floor — serves interior cells from the shared per-multiID cache;
/// dirty instances, boats (movers), and HouseFoundation (runtime-mutable) fall back to live-synth.
/// Keyed by multiID &amp; 0x3FFF.
/// Caches walkability masks for the interior cells of a multi, shared across every instance of the
/// same multiID.
///
/// An interior cell — one whose 8 neighbours are all covered by the multi — has no terrain
/// neighbour, so its mask depends only on the multi's own component tiles and is identical at every
/// position the design is placed. That makes it cacheable in the multi's local frame and reusable
/// across instances; perimeter cells are not, and fall back to <see cref="StepProbe.ComputeMultiMaskAt"/>.
///
/// The catch is terrain intruding into the multi's floor envelope, which would make a cell's mask
/// position-dependent after all. <see cref="ComputeFootprintClean"/> rules that out per instance,
/// once: if the whole footprint's terrain sits below the lowest floor, no interior cell can see it.
/// A dirty instance, or a <see cref="HouseFoundation"/> (whose design mutates at runtime), always
/// synthesizes live rather than risk serving a wrong mask.
/// </summary>
public sealed class MultiMaskCache
{
@ -25,26 +31,20 @@ public sealed class MultiMaskCache
public void Clear() => _byMultiId.Clear();
/// <summary>
/// Returns the multi-aware StepMask for a covered cell (x,y,sourceZ). Serves a cached interior
/// mask when available and the guards pass (counted as a MultiMaskCacheHit); otherwise falls
/// back to the Phase-2 live synthesizer ComputeMultiMaskAt (counted as a MultiLocalHit), caching
/// the result if the cell is interior and clean. Always returns a usable mask (HitKind == Hit).
/// The multi-aware mask for a covered cell, always usable (HitKind is always Hit). Served from
/// the shared cache when the cell is a clean interior one, synthesized live otherwise.
/// </summary>
public StepMask GetMask(Map map, int x, int y, sbyte sourceZ)
{
if (!TryResolveCoveringMulti(map, x, y, out var multi, out var lx, out var ly)
|| multi is HouseFoundation) // runtime-mutable per-instance DesignState MCL
|| multi is HouseFoundation) // its DesignState MCL changes at runtime
{
return LiveSynth(map, x, y, sourceZ);
}
// Boats are cached too: their per-multiID deck masks are movement-invariant (built once per
// heading), and the per-instance clean gate below + the ItemID/location/map resets keep a
// moving/turning boat correct. Narrow boats have little interior; wide galleons gain a lot.
// Boats are cached despite moving: a deck mask is built in the local frame and is invariant
// under translation, and the clean gate plus the ItemID/location/map resets cover turning.
// Per-instance footprint cleanliness (computed once, stored on the multi; reset on move).
// Clean ⇒ no terrain intrusion anywhere in the footprint ⇒ interior cells are exact from the
// shared per-multiID cache. Dirty ⇒ degrade to the live synthesizer (never serve a wrong mask).
if (multi.PathInteriorCacheState == MultiInteriorCacheState.Unknown)
{
multi.PathInteriorCacheState =
@ -62,7 +62,7 @@ public sealed class MultiMaskCache
if (state == MultiLocalMask.CellState.Cached)
{
// Footprint is clean, so only the source-Z match matters (terrain can't intrude).
// A clean footprint rules terrain out, so the source-Z match is the only guard left.
var worldFloorZ = local.FloorZAt(lx, ly) + multi.Z;
if (Math.Abs(sourceZ - worldFloorZ) <= StepHeight)
{
@ -78,8 +78,7 @@ public sealed class MultiMaskCache
return LiveSynth(map, x, y, sourceZ);
}
// Unknown → classify + (if interior) build & cache. No per-cell terrain guard needed: the
// instance is clean, so every interior cell's 3x3 terrain is below the floor.
// First touch of this cell: synthesize, then classify and cache if it's interior.
var mask = LiveSynth(map, x, y, sourceZ);
if (IsInteriorLocalCell(mcl, lx, ly)
&& TryToLocalZ(mask, multi.Z, out var localMask)
@ -113,8 +112,7 @@ public sealed class MultiMaskCache
}
/// <summary>
/// Finds the multi covering (x,y) and the local cell indices into its MCL. Mirrors
/// Map.StaticTileEnumerator / BaseMulti.Contains. Returns false if no multi covers the cell.
/// Finds the multi covering (x,y) and the cell's indices into its MCL, or false if none does.
/// </summary>
public static bool TryResolveCoveringMulti(Map map, int x, int y, out BaseMulti multi, out int lx, out int ly)
{
@ -138,9 +136,9 @@ public sealed class MultiMaskCache
}
/// <summary>
/// True iff local cell (lx,ly) and all 8 neighbours are covered by the multi (have MCL tiles).
/// Such a cell's 8-direction transition is fully determined by the multi (no terrain neighbour),
/// so its mask is position-invariant. A pure function of the MCL.
/// True when (lx,ly) and all 8 of its neighbours carry MCL tiles. Such a cell has no terrain
/// neighbour, so the multi alone determines its transitions and its mask is position-invariant.
/// A pure function of the MCL.
/// </summary>
public static bool IsInteriorLocalCell(MultiComponentList mcl, int lx, int ly)
{
@ -161,9 +159,9 @@ public sealed class MultiMaskCache
}
/// <summary>
/// Converts a world-frame mask's per-direction Zs to local Z (subtract multiZ). Returns false
/// if any local Z doesn't fit sbyte (caller must then NOT cache the cell — rare; only when
/// |multiZ| is large enough to push a world Z out of range). Mask (walk/wet) bits are copied.
/// Rebases a world-frame mask's per-direction Zs into the multi's local frame. Returns false
/// when a local Z overflows sbyte — only reachable at extreme |multiZ| — and the caller must
/// then leave the cell uncached.
/// </summary>
public static bool TryToLocalZ(StepMask world, int multiZ, out StepMask local)
{
@ -191,9 +189,8 @@ public sealed class MultiMaskCache
}
/// <summary>
/// True iff all terrain (land + statics) at (x,y) sits strictly below <paramref name="floorZ"/>,
/// so a creature standing on the multi floor never sees terrain in its envelope and the cached
/// (terrain-free) mask is exact. Cheap: one land-top read + the cell's static-tile array scan.
/// True when all terrain (land + statics) at (x,y) sits strictly below <paramref name="floorZ"/>,
/// so a creature standing on the multi's floor never sees terrain in its envelope.
/// </summary>
public static bool TerrainTopBelow(Map map, int x, int y, sbyte floorZ)
{
@ -216,7 +213,7 @@ public sealed class MultiMaskCache
return true;
}
/// <summary>Highest terrain (land + statics) top at (x,y). Building block for the cleanliness check.</summary>
/// <summary>Highest terrain (land + statics) top at (x,y).</summary>
public static int TerrainTop(Map map, int x, int y)
{
map.GetAverageZ(x, y, out _, out _, out var top);
@ -234,10 +231,10 @@ public sealed class MultiMaskCache
}
/// <summary>
/// True iff the multi's WHOLE footprint terrain sits below its lowest standable floor — i.e.
/// maxTerrain &lt; minFloor over all covered cells. When true, no covered cell's terrain (nor any
/// neighbour's) can intrude into a creature's floor envelope, so interior cells of this design are
/// safe to serve from the shared per-multiID cache for THIS instance. One-time per instance.
/// True when the multi's entire footprint terrain sits below its lowest standable floor. No
/// covered cell's terrain — nor any neighbour's — can then intrude into a creature's floor
/// envelope, which is what makes this instance's interior cells safe to serve from the shared
/// per-multiID cache. Evaluated once per instance and cached on the multi.
/// </summary>
public static bool ComputeFootprintClean(Map map, BaseMulti multi)
{
@ -252,7 +249,7 @@ public sealed class MultiMaskCache
var col = mcl.Tiles[lx][ly];
if (col.Length == 0)
{
continue; // uncovered local cell
continue;
}
foreach (var tile in col)
@ -278,7 +275,7 @@ public sealed class MultiMaskCache
if (minFloorLocal == int.MaxValue)
{
return false; // no standable floor anywhere → don't cache (defensive)
return false; // no standable floor anywhere; refuse to cache rather than guess
}
return maxTerrain < minFloorLocal + multi.Z;
@ -304,8 +301,9 @@ public sealed class MultiMaskCache
}
/// <summary>
/// Per-multiID lazily-filled grid of interior-cell masks. Cell state: Unknown (not yet classified),
/// Cached (interior + clean → mask valid), NonInterior (perimeter/edge/terrain-dirty → live-synth).
/// One multiID's grid of interior-cell masks, filled in as cells are first touched. A cell is
/// Unknown until classified, then either Cached (interior — the mask is valid) or NonInterior
/// (perimeter — synthesize live).
/// </summary>
internal sealed class MultiLocalMask
{
@ -314,8 +312,8 @@ internal sealed class MultiLocalMask
private readonly int _width;
private readonly int _height;
private readonly CellState[] _state;
private readonly StepMask[] _mask; // local-Z mask, valid when state == Cached
private readonly sbyte[] _floorZ; // local floor Z, valid when state == Cached
private readonly StepMask[] _mask; // local-frame mask, valid only when state == Cached
private readonly sbyte[] _floorZ; // local floor Z, valid only when state == Cached
public MultiLocalMask(int width, int height)
{

625
Projects/UOContent/Engines/Pathing/Cache/StepCache.cs
View file

@ -1,17 +1,24 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using Server.Buffers;
using Server.Collections;
using Server.Logging;
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Singleton store of per-chunk static walkability data. Chunks correspond to
/// Map.SectorSize = 16; key encoding packs (mapId, chunkX, chunkY) into a long.
/// Lazily built on first query; invalidated by version-check vs Sector.MultisVersion;
/// memory bounded by MaxResidentChunks via probabilistic LRU eviction.
/// Singleton store of static walkability, keyed by 16x16 chunk (one per map sector). Chunks build
/// on demand and memory stays bounded by MaxResidentChunks through probabilistic LRU eviction, so
/// the cache is usable with no on-disk bake at all; a baked .swb file only removes the first-touch
/// build cost.
///
/// Default-walker scope only. Cells with multi-Z surfaces and queries for non-default
/// walkers route to the MovementImpl slow path via the Fallthrough_* hit kinds.
/// The cache answers for a default walker on static terrain. Anything outside that — a multi
/// covering the cell, a query Z that doesn't match what the cell was baked at, stacked surfaces
/// with no matching stratum — returns a Fallthrough_* kind, and the caller resolves that cell
/// through MovementImpl instead. Callers must check <see cref="StepMask.IsHit"/>.
/// </summary>
public sealed class StepCache
{
@ -19,26 +26,23 @@ public sealed class StepCache
public static StepCache Instance { get; } = new();
private readonly Dictionary<long, StepChunk> _chunks = new();
// Parallel list of keys for O(1) random sampling during eviction. Kept in lockstep
// with _chunks: append on Miss_NotBuilt, swap-and-pop on eviction.
private readonly Dictionary<long, StepChunk> _chunks = [];
// Keys of _chunks, kept in lockstep with it, so eviction can sample a random resident chunk
// in O(1). Appended on insert, swap-and-popped on eviction.
private readonly List<long> _keysList = [];
// Second-touch promotion tracker. A chunk's first miss within the window returns
// Fallthrough_NotBuilt; the caller takes the slow path. The Nth DISTINCT-FIND miss
// within the same window (where N = MissPromotionThreshold) promotes to BuildChunk +
// serve. We count distinct Find generations, not raw TryGetMask calls — A* expansion
// hits each visited chunk many times in one Find, so per-call counting hits threshold
// immediately and defeats the gate. Per-Find counting filters single-Find pass-throughs
// (pet following a moving player) while still promoting chunks revisited by multiple
// Finds (NPC patrolling fixed territory).
private readonly Dictionary<long, ChunkMissState> _chunkMissTracker = new();
// Promotion gate. A chunk's first miss returns Fallthrough_NotBuilt and the caller takes the
// slow path; only once misses reach MissPromotionThreshold within MissPromotionWindowMs does
// the chunk get built and served. This keeps one-off traffic — a pet trailing a player across
// the map — from building chunks nothing will query again, while a creature working a fixed
// territory still warms the chunks it revisits.
//
// The gate counts distinct Finds, not TryGetMask calls: A* probes each chunk it visits dozens
// of times within a single pathfind, so per-call counting would cross any threshold instantly
// and gate nothing.
private readonly Dictionary<long, ChunkMissState> _chunkMissTracker = [];
private const int MaxMissTrackerEntries = 4096;
// Generation counter incremented by BeginFindGeneration(). Sentinel 0 = "no Find started
// yet"; treated as a distinct generation per call so callers that bypass BeginFindGeneration
// (single-call tests, BakeMap with threshold=1) get sensible behavior.
private struct ChunkMissState
{
public byte MissCount;
@ -78,24 +82,21 @@ public sealed class StepCache
public bool PreloadOnLazyOpen { get; set; }
/// <summary>
/// Number of misses on the same chunk within <see cref="MissPromotionWindowMs"/>
/// required to trigger a build. 1 = eager (legacy behavior). 2 = second-touch (default,
/// filters single-touch pass-throughs).
/// Misses on the same chunk, within <see cref="MissPromotionWindowMs"/>, needed to build it.
/// 1 builds eagerly on first touch; the default 2 waits for a second Find to show interest.
/// </summary>
public int MissPromotionThreshold { get; set; } = 2;
/// <summary>
/// Window over which misses against the same chunk accumulate toward promotion.
/// Misses spaced wider than this restart the count. Default 30s.
/// How long misses on a chunk accumulate toward promotion. A gap wider than this restarts
/// the count.
/// </summary>
public uint MissPromotionWindowMs { get; set; } = 30_000;
/// <summary>
/// Marks the start of a new pathfind. The promotion gate counts distinct Find
/// generations per chunk, not raw TryGetMask calls — call this once at the top of
/// each pathfind invocation so multiple cell expansions within one Find don't trip
/// the threshold. Wraps at uint.MaxValue back to 1 (0 is reserved as the
/// "no Find started yet" sentinel).
/// Opens a new pathfind for the promotion gate. Call once per pathfind: the gate counts
/// distinct Finds, so without this every cell expansion would count separately and the
/// threshold would be met immediately. Wraps back to 1, since 0 means "no Find open".
/// </summary>
public void BeginFindGeneration()
{
@ -107,12 +108,11 @@ public sealed class StepCache
}
}
/// <summary>Test-only: read the current Find generation.</summary>
/// <summary>The open pathfind's generation, or 0 if none. See <see cref="BeginFindGeneration"/>.</summary>
internal uint CurrentFindGeneration { get; private set; }
/// <summary>
/// Pack (mapId, chunkX, chunkY) into a single long key.
/// Layout: [reserved 16][mapId 16][chunkX 16][chunkY 16].
/// Packs (mapId, chunkX, chunkY) into one key: [reserved 16][mapId 16][chunkX 16][chunkY 16].
/// </summary>
internal static long EncodeKey(int mapId, int chunkX, int chunkY) =>
((long)(mapId & 0xFFFF) << 32) | ((long)(chunkX & 0xFFFF) << 16) | (long)(chunkY & 0xFFFF);
@ -134,9 +134,8 @@ public sealed class StepCache
);
/// <summary>
/// Drop all cached chunks AND zero every telemetry counter. Used by tests and
/// benchmarks that need a known cold-start state. Counter reset is intentional —
/// counters are since-last-clear, not since-startup.
/// Returns the cache to a cold-start state: drops every chunk, closes the .swb readers, and
/// zeroes the counters.
/// </summary>
public void Clear()
{
@ -146,10 +145,8 @@ public sealed class StepCache
}
/// <summary>
/// Drop all resident chunks AND zero counters, but keep lazy readers open.
/// Useful in benchmark loops that want to measure "first query after boot" cost
/// without paying the lazy-reader reopen overhead each iteration. Same intent as
/// <see cref="Clear"/> minus the file-handle teardown.
/// <see cref="Clear"/> without the file-handle teardown: drops the resident chunks and zeroes
/// the counters, but leaves the .swb readers open so the next query can refill from them.
/// </summary>
public void ClearResidentChunks()
{
@ -171,16 +168,14 @@ public sealed class StepCache
_buildsTotal = 0;
}
// Per-map open .swb readers, populated by TryOpenLazyReader at startup. Chunks are
// fetched on demand from the file when ResolveMissingChunk fires; resident memory
// stays bounded by MaxResidentChunks regardless of file size.
private readonly Dictionary<int, StepCacheFile.LazyReader> _lazyReaders = new();
// Open .swb readers, one per map. Chunks are pulled from them on demand, so resident memory
// stays bounded by MaxResidentChunks no matter how large the file is.
private readonly Dictionary<int, StepCacheFile.LazyReader> _lazyReaders = [];
/// <summary>
/// Walk every chunk in <paramref name="mapId"/>, populate the resident set, then
/// save to <paramref name="path"/>. Returns the number of chunks written.
/// Designed for offline / fixture use; blocks the calling thread for many seconds
/// on a full Trammel walk.
/// Builds every chunk in the map and writes them to <paramref name="path"/>, returning the
/// number written. Blocks the caller for many seconds on a full-size map — run it offline or
/// during maintenance, not on a live shard at peak.
/// </summary>
public int BakeMap(int mapId, string path)
{
@ -190,12 +185,11 @@ public sealed class StepCache
return 0;
}
// BakeMap is an explicit decision to populate every chunk; the promotion gate
// would otherwise return Fallthrough_NotBuilt for every chunk (each touched once)
// and the bake would write an empty file. Force eager build for the duration.
// A bake touches each chunk exactly once, so the promotion gate would defer every one of
// them and write an empty file. Baking is an explicit decision to populate everything, so
// build eagerly for the duration.
var prevThreshold = MissPromotionThreshold;
MissPromotionThreshold = 1;
var startTick = Core.TickCount;
try
{
var chunkCols = (map.Width + ChunkSize - 1) / ChunkSize;
@ -207,12 +201,13 @@ public sealed class StepCache
mapId, chunkCols, chunkRows, chunkCols * chunkRows
);
var stopWatch = Stopwatch.StartNew();
for (var cy = 0; cy < chunkRows; cy++)
{
for (var cx = 0; cx < chunkCols; cx++)
{
// Any sourceZ works — the chunk is built on first access regardless of
// whether the query returns Hit or Fallthrough_SourceZMismatch.
// The sourceZ is irrelevant here: the chunk gets built on first access whether
// the query ends up a Hit or a Fallthrough_SourceZMismatch.
TryGetMask(map, cx * ChunkSize, cy * ChunkSize, sourceZ: 0);
}
@ -221,14 +216,14 @@ public sealed class StepCache
logger.Information(
"PathBake map {MapId}: row {Row}/{Rows} ({Pct}%), {Resident} chunks resident, {Elapsed:F1}s, {HeapMB} MB heap",
mapId, cy + 1, chunkRows, (cy + 1) * 100 / chunkRows,
_chunks.Count, (Core.TickCount - startTick) / 1000.0, GC.GetTotalMemory(false) >> 20
_chunks.Count, stopWatch.ElapsedMilliseconds / 1000.0, GC.GetTotalMemory(false) >> 20
);
}
}
logger.Information(
"PathBake map {MapId}: walk complete in {Elapsed:F1}s, writing {Resident} chunks to disk...",
mapId, (Core.TickCount - startTick) / 1000.0, _chunks.Count
mapId, stopWatch.ElapsedMilliseconds / 1000.0, _chunks.Count
);
}
finally
@ -240,50 +235,31 @@ public sealed class StepCache
}
/// <summary>
/// Persist all resident chunks for <paramref name="mapId"/> to a .swb file. Returns
/// the number of chunks written. The file embeds a TileData fingerprint so a stale
/// file (built before a client patch) can be detected and rejected at open time.
/// Writes the map's resident chunks to a .swb file and returns the count. The file carries a
/// fingerprint of the tile and map data, so a bake made before a client patch is detected and
/// rejected when it is next opened.
/// </summary>
public int SaveToFile(string path, int mapId)
{
var matching = 0;
using var chunks = PooledRefList<(int chunkX, int chunkY, StepChunk chunk)>.Create();
foreach (var key in _keysList)
{
DecodeKey(key, out var keyMapId, out _, out _);
DecodeKey(key, out var keyMapId, out var chunkX, out var chunkY);
if (keyMapId == mapId)
{
matching++;
chunks.Add((chunkX, chunkY, _chunks[key]));
}
}
var enumerator = _keysList.GetEnumerator();
StepCacheFile.Write(path, (uint)mapId, (uint)matching, EmitChunk);
enumerator.Dispose();
return matching;
bool EmitChunk(out int chunkX, out int chunkY, out StepChunk chunk)
{
while (enumerator.MoveNext())
{
var key = enumerator.Current;
DecodeKey(key, out var emittedMapId, out chunkX, out chunkY);
if (emittedMapId == mapId)
{
chunk = _chunks[key];
return true;
}
}
chunkX = chunkY = 0;
chunk = null!;
return false;
}
StepCacheFile.Write(path, (uint)mapId, chunks.AsSpan());
return chunks.Count;
}
/// <summary>
/// Open a .swb file as a lazy backing store for <paramref name="mapId"/>. Reads only
/// header + chunk-offset index (~16 bytes per chunk); individual records are fetched
/// on demand by <see cref="ResolveMissingChunk"/>. Returns false on missing file,
/// magic / version mismatch, or TileData hash mismatch (stale bake).
/// Opens a .swb file as a backing store for the map, reading only the header and chunk index
/// up front; records are pulled as queries ask for them. Returns false if the file is missing,
/// unreadable, or a stale bake whose fingerprint no longer matches the live tile data.
/// </summary>
public bool TryOpenLazyReader(string path, int mapId)
{
@ -323,10 +299,7 @@ public sealed class StepCache
}
/// <summary>
/// Materializes every chunk in <paramref name="reader"/> into the resident set.
/// Called from <see cref="TryOpenLazyReader"/> when <see cref="PreloadOnLazyOpen"/>
/// is set. Skips chunks whose live <see cref="Map.Sector.MultisVersion"/> doesn't
/// match the file's snapshot — those will rebake on first query.
/// Loads every chunk in the file into the resident set, for <see cref="PreloadOnLazyOpen"/>.
/// </summary>
private void PreloadFromLazyReader(int mapId, StepCacheFile.LazyReader reader)
{
@ -361,29 +334,52 @@ public sealed class StepCache
);
}
/// <summary>
/// Number of .swb readers currently open. Mostly for tests / telemetry.
/// </summary>
/// <summary>Number of .swb readers currently open.</summary>
public int OpenLazyReaderCount => _lazyReaders.Count;
/// <summary>
/// True if a valid .swb reader is open for <paramref name="mapId"/>. A reader only opens via
/// <see cref="TryOpenLazyReader"/> after <see cref="StepCacheFile.OpenForLazy"/> validates the
/// file's fingerprint against the live tile data, so "has reader" already means "present and
/// up-to-date" — the boot prebake uses this to skip baking maps that don't need it, instead of
/// recomputing the fingerprint a second time.
/// True when a .swb reader is open for the map. A reader only opens after its fingerprint
/// validates against the live tile data, so this already answers "is there an up-to-date bake
/// for this map?" — the boot prebake leans on that to skip maps rather than fingerprint them
/// a second time.
/// </summary>
public bool HasLazyReader(int mapId) => _lazyReaders.ContainsKey(mapId);
/// <summary>Test-only diagnostic: does the lazy reader for <paramref name="mapId"/> hold an offset for (chunkX, chunkY)?</summary>
/// <summary>Diagnostic: does the map's .swb hold a record for (chunkX, chunkY)?</summary>
internal bool LazyReaderHasChunk(int mapId, int chunkX, int chunkY) =>
_lazyReaders.TryGetValue(mapId, out var r) && r.Has(chunkX, chunkY);
/// <summary>
/// Closes all open lazy readers, releasing their underlying file streams. Called from
/// <see cref="Clear"/> so test cleanup can delete .swb files (they're held with
/// FileShare.Read | FileShare.Delete, so this is mostly belt-and-suspenders).
/// Diagnostic: the resident chunk covering (chunkX, chunkY), or null if it isn't resident.
/// Exposed so tests can inspect and inject chunk state without reflecting into the internals.
/// </summary>
internal StepChunk GetResidentChunk(int mapId, int chunkX, int chunkY) =>
_chunks.GetValueOrDefault(EncodeKey(mapId, chunkX, chunkY));
/// <summary>
/// Diagnostic: whether the eviction key list still mirrors the resident set exactly. A desync
/// breaks sampled eviction — a stale key throws on lookup, a missing one pins a chunk resident
/// forever — and it is invisible from the outside, so tests assert on it directly.
/// </summary>
internal bool ResidentIndexInSync()
{
if (_keysList.Count != _chunks.Count)
{
return false;
}
foreach (var key in _keysList)
{
if (!_chunks.ContainsKey(key))
{
return false;
}
}
return true;
}
/// <summary>Closes every open .swb reader, releasing the underlying file streams.</summary>
public void CloseLazyReaders()
{
foreach (var reader in _lazyReaders.Values)
@ -394,18 +390,16 @@ public sealed class StepCache
}
/// <summary>
/// Probabilistic LRU sample size — picks SampleSize random resident chunks per
/// eviction and evicts the oldest of that sample. Approximates true LRU at a tiny
/// fraction of the cost (no full sort). Redis uses the same approach (`maxmemory-samples`).
/// 5 yields ~quality-of-true-LRU for cache eviction; higher values trade speed for accuracy.
/// How many random resident chunks each eviction samples before dropping the oldest of them.
/// Sampling approximates true LRU closely enough at a fraction of the cost, since it needs no
/// sort and no access-ordered structure. Raising it trades speed for accuracy.
/// </summary>
private const int LruSampleSize = 5;
/// <summary>
/// If resident chunk count exceeds MaxResidentChunks, evict via probabilistic LRU
/// until the count is at or below the cap. Per-eviction cost is O(LruSampleSize),
/// independent of resident count — sustained cap pressure has no perpetual perf hit.
/// Called from CacheEvictionTimer; also callable directly from tests.
/// Evicts chunks until the resident count is back within MaxResidentChunks. Each eviction costs
/// O(<see cref="LruSampleSize"/>) regardless of how many chunks are resident, so sustained cap
/// pressure doesn't degrade. Driven by <see cref="CacheEvictionTimer"/>.
/// </summary>
public void EnforceLruCap()
{
@ -421,8 +415,8 @@ public sealed class StepCache
long oldestTouched = long.MaxValue;
long oldestKey = 0;
// Sample LruSampleSize random keys; track the oldest by LastTouchedTicks.
// With replacement is fine — collisions are rare and don't break correctness.
// Sampling with replacement: a repeated key just wastes one sample, it can't pick a
// wrong victim.
var samples = Math.Min(LruSampleSize, _keysList.Count);
for (var s = 0; s < samples; s++)
{
@ -459,11 +453,28 @@ public sealed class StepCache
private const int ChunkSize = 16;
/// <summary>
/// True if a multi (house / boat) covers (x, y) or any of its 8 neighbours. Multi-covered
/// cells — plus the 1-cell halo, because a cell's mask encodes the edges TO its neighbours, so
/// a neighbouring wall must block those edges — are served by the live movement path, not the
/// static chunk cache. Cheap: an interior cell checks only its own sector (chunk == sector);
/// only edge/corner cells additionally check the adjacent sector(s) the halo reaches.
/// All-zero mask carrying a Fallthrough_* kind. <see cref="StepMask.IsHit"/> is false for
/// these, so the caller ignores the payload and takes the slow path.
/// </summary>
private static StepMask Fallthrough(CacheHitKind kind) =>
new(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, kind);
/// <summary>Bumps the telemetry counter matching a served (non-fallthrough) hit kind.</summary>
private void RecordServed(CacheHitKind kind)
{
switch (kind)
{
case CacheHitKind.Miss_NotBuilt: { _missesNotBuilt++; break; }
case CacheHitKind.Miss_DirtyRebuild: { _missesDirtyRebuild++; break; }
case CacheHitKind.Hit: { _hits++; break; }
}
}
/// <summary>
/// True when a multi covers (x, y) or any of its 8 neighbours. The halo matters because a
/// cell's mask encodes the edges TO its neighbours, so a wall one cell over has to block those
/// edges. Since a chunk is a sector, an interior cell only inspects its own sector's HasMultis
/// flag; edge and corner cells additionally check whichever adjacent sectors the halo reaches.
/// </summary>
private static bool MultiInfluence(Map map, int x, int y)
{
@ -480,7 +491,7 @@ public sealed class StepCache
var south = (y & 15) == 15;
if (!(west || east || north || south))
{
return false; // interior cell — its whole halo is inside the (multi-free) own sector
return false; // interior cell: its whole halo lies in this sector, which has no multis
}
return west && map.GetRealSector(sx - 1, sy).HasMultis
@ -494,49 +505,25 @@ public sealed class StepCache
}
/// <summary>
/// Hot-path query. Returns the cached mask + 8 destination Z values + hit kind.
/// Inspect <see cref="StepMask.IsHit"/> to decide whether to use the result or fall
/// back to the slow path.
/// The hot-path query: one lookup yields the cell's 8-direction mask, its 8 destination Zs,
/// and the hit kind. Check <see cref="StepMask.IsHit"/> before trusting the payload — on any
/// fallthrough it is all zeroes and the caller must resolve the cell through MovementImpl.
/// </summary>
public StepMask TryGetMask(Map map, int x, int y, sbyte sourceZ)
{
if (map == null || map == Map.Internal || x < 0 || y < 0 || x >= map.Width || y >= map.Height)
{
_fallthroughOffMap++;
return new StepMask(
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
CacheHitKind.Fallthrough_OffMap
);
return Fallthrough(CacheHitKind.Fallthrough_OffMap);
}
// Multis (houses, boats) are not baked into the static chunk cache (they're dynamic
// content). If a multi covers this cell or its 1-cell halo, route to the live movement
// path, which is fully multi-aware. Gated on Sector.HasMultis, so the multi-free majority
// of the map pays a single (interior) sector lookup.
// Multis are dynamic, so they are never baked into a chunk. Cells they touch go to the
// multi-aware path instead. The check is gated on Sector.HasMultis, so the multi-free
// majority of the map pays one sector lookup for it.
if (MultiInfluence(map, x, y))
{
_fallthroughMulti++;
return new StepMask(
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
CacheHitKind.Fallthrough_Multi
);
return Fallthrough(CacheHitKind.Fallthrough_Multi);
}
var chunkX = x >> 4;
@ -546,8 +533,8 @@ public sealed class StepCache
var hitKindResult = CacheHitKind.Hit;
if (!_chunks.TryGetValue(key, out var chunk))
{
// Try lazy file first — file-loaded chunks bypass the miss tracker because
// the .swb represents an explicit prior decision to keep this chunk warm.
// The .swb is consulted before the promotion gate: a baked chunk is already an explicit
// decision to keep this area warm, and loading it is far cheaper than building it.
chunk = TryLoadFromLazyReader(map, chunkX, chunkY);
if (chunk != null)
{
@ -565,94 +552,44 @@ public sealed class StepCache
else
{
_fallthroughNotBuilt++;
return new StepMask(
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
CacheHitKind.Fallthrough_NotBuilt
);
return Fallthrough(CacheHitKind.Fallthrough_NotBuilt);
}
}
// A resident chunk is static-only — it never goes stale from multis (multi-covered cells
// fall through to the live path above).
// No staleness check: a resident chunk holds only static terrain, and every cell a multi
// could have changed already fell through above.
chunk.LastTouchedTicks = Core.TickCount;
var cellIndex = ((y - (chunkY << 4)) << 4) | (x - (chunkX << 4));
if (chunk.IsCellMultiZ(cellIndex))
{
// Tier 4: try the per-cell strata. Each stratum is keyed by its bake-time
// standing-Z; a query matches when |sourceZ - stratum.zCenter| <= StepHeight.
// Stacked surfaces: pick the stratum baked nearest the query Z. Multi-Z cells are
// served only from strata, never from the main mask.
if (TryStratumHit(chunk, cellIndex, sourceZ, hitKindResult, out var stratumResult))
{
switch (hitKindResult)
{
case CacheHitKind.Miss_NotBuilt: { _missesNotBuilt++; break; }
case CacheHitKind.Miss_DirtyRebuild: { _missesDirtyRebuild++; break; }
case CacheHitKind.Hit: { _hits++; break; }
}
RecordServed(hitKindResult);
return stratumResult;
}
_fallthroughMultiZ++;
return new StepMask(
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
CacheHitKind.Fallthrough_MultiZ
);
return Fallthrough(CacheHitKind.Fallthrough_MultiZ);
}
// Source-Z guard: the cache stores one answer per cell baked at SourceZ.
// StepHeight tolerance accepts incremental Z jitter; loosening it breaks parity
// because tile reachability shifts at step-height boundaries.
// Source-Z guard. A cell holds one answer, baked at one standing Z, so a query from too far
// above or below it would get an answer that doesn't apply. The StepHeight tolerance
// absorbs ordinary Z jitter and cannot be widened: reachability flips at exactly that
// boundary, so a looser guard would serve answers that disagree with MovementImpl.
if (Math.Abs(sourceZ - chunk.SourceZ[cellIndex]) > StepHeight)
{
// Swim-layer fallback for shore cells: if the chunk has the layer and this
// cell's water-surface Z is within StepHeight of the query, serve from the
// swim layer (computed at swim-perspective Z). Walker queries on shore cells
// fall through this branch via their Z mismatch with SwimSourceZ.
// Unless this is a shore cell and the query is coming from the water, in which case the
// swim layer holds the answer baked from the water surface.
if (chunk.HasSwimLayer)
{
var swimSrc = chunk.SwimSourceZ[cellIndex];
if (swimSrc != StepChunk.NoSwimLayerCell && Math.Abs(sourceZ - swimSrc) <= StepHeight)
{
switch (hitKindResult)
{
case CacheHitKind.Miss_NotBuilt: { _missesNotBuilt++; break; }
case CacheHitKind.Miss_DirtyRebuild: { _missesDirtyRebuild++; break; }
case CacheHitKind.Hit: { _hits++; break; }
}
RecordServed(hitKindResult);
return new StepMask(
0, chunk.SwimMask[cellIndex],
0, 0, 0, 0, 0, 0, 0, 0,
@ -670,35 +607,10 @@ public sealed class StepCache
}
_fallthroughSourceZMismatch++;
return new StepMask(
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
CacheHitKind.Fallthrough_SourceZMismatch
);
return Fallthrough(CacheHitKind.Fallthrough_SourceZMismatch);
}
switch (hitKindResult)
{
case CacheHitKind.Miss_NotBuilt: { _missesNotBuilt++; break; }
case CacheHitKind.Miss_DirtyRebuild: { _missesDirtyRebuild++; break; }
case CacheHitKind.Hit: { _hits++; break; }
}
RecordServed(hitKindResult);
return new StepMask(
chunk.WalkMask[cellIndex],
@ -724,41 +636,32 @@ public sealed class StepCache
}
/// <summary>
/// Returns a fresh StepChunk loaded from the lazy file reader, or null if there's no
/// open reader for the map / no record at (chunkX, chunkY) / the loaded snapshot is
/// stale relative to the live sector's <see cref="Map.Sector.MultisVersion"/>. A null
/// return means the caller should consult the miss tracker; a stale return means
/// "rebuild, the .swb is out of date and a future SaveToFile will overwrite it."
/// Loads a chunk from the map's .swb, or null if no reader is open or the file has no record at
/// (chunkX, chunkY). No staleness check is needed here — the fingerprint was validated when the
/// file was opened, and the chunks are static-only.
/// </summary>
private StepChunk TryLoadFromLazyReader(Map map, int chunkX, int chunkY)
{
if (!_lazyReaders.TryGetValue(map.MapID, out var reader))
{
return null;
}
// Static-only chunks are valid once the file fingerprint matched at open time; multi-covered
// cells fall through before reaching here. Returns null when the file lacks this chunk.
return reader.TryReadChunk(chunkX, chunkY);
}
private StepChunk TryLoadFromLazyReader(Map map, int chunkX, int chunkY) =>
_lazyReaders.TryGetValue(map.MapID, out var reader) ? reader.TryReadChunk(chunkX, chunkY) : null;
/// <summary>
/// Records a miss for <paramref name="chunkKey"/> and decides whether to build now
/// or defer to slow path. Counts distinct Find generations, not raw calls — multiple
/// TryGetMask calls within one Find (BeginFindGeneration scope) count as one touch.
/// Returns true when DISTINCT-FIND misses within the window cross
/// <see cref="MissPromotionThreshold"/>; caller should run BuildChunk and serve.
/// Returns false otherwise; caller should return Fallthrough_NotBuilt so the algorithm
/// uses the slow path. Generation 0 ("no Find active") treats every call as distinct,
/// preserving legacy semantics for callers that don't call BeginFindGeneration.
/// Records a miss and answers whether the chunk has now earned a build. True means build and
/// serve; false means return Fallthrough_NotBuilt and let the caller take the slow path.
///
/// A miss only counts once per Find (see <see cref="BeginFindGeneration"/>). With no Find open
/// — a direct caller, or a bake — every call counts separately.
/// </summary>
private bool ShouldPromoteAfterMiss(long chunkKey)
{
// Environment.TickCount, not Core.TickCount: tests/bench fixtures may not advance
// the game-loop tick. The promotion window is wall-clock anyway.
// Environment.TickCount rather than Core.TickCount: the window is wall-clock, and test and
// benchmark fixtures don't necessarily advance the game loop's tick.
var now = (uint)Environment.TickCount;
var gen = CurrentFindGeneration;
if (_chunkMissTracker.TryGetValue(chunkKey, out var state))
// One hash lookup for the whole update — the entry is mutated through the ref instead
// of being re-hashed and re-probed by an indexer assignment. Safe to hold across the
// Remove below only because nothing reads it afterwards.
ref var state = ref CollectionsMarshal.GetValueRefOrNullRef(_chunkMissTracker, chunkKey);
if (!Unsafe.IsNullRef(ref state))
{
// Same Find generation as the last touch — A* expansion is probing this chunk
// multiple times in one pathfind. Don't increment; the gate counts distinct
@ -772,12 +675,11 @@ public sealed class StepCache
var elapsed = now - state.LastMissTickStamp;
if (elapsed > MissPromotionWindowMs)
{
_chunkMissTracker[chunkKey] = new ChunkMissState
{
MissCount = 1,
LastMissTickStamp = now,
LastFindGeneration = gen
};
// Outside the window — restart the count. Never promotes on this call, even at
// threshold 1, matching the pre-existing gate semantics.
state.MissCount = 1;
state.LastMissTickStamp = now;
state.LastFindGeneration = gen;
return false;
}
@ -788,12 +690,9 @@ public sealed class StepCache
return true;
}
_chunkMissTracker[chunkKey] = new ChunkMissState
{
MissCount = newCount,
LastMissTickStamp = now,
LastFindGeneration = gen
};
state.MissCount = newCount;
state.LastMissTickStamp = now;
state.LastFindGeneration = gen;
return false;
}
@ -817,27 +716,30 @@ public sealed class StepCache
}
/// <summary>
/// Drop tracker entries older than the promotion window. Called when the tracker hits
/// its capacity ceiling. If the prune doesn't reclaim anything (every entry is in
/// window), the cap is enforced by clearing — the worst case is a few extra
/// Fallthrough_NotBuilt returns until traffic re-establishes hot chunks.
/// Drops tracker entries that have aged out of the promotion window, once the tracker hits its
/// capacity ceiling. When nothing has aged out, the whole tracker is cleared to enforce the cap
/// — that costs a few extra Fallthrough_NotBuilt returns while traffic re-establishes the hot
/// chunks, which is cheaper than letting the tracker grow without bound.
/// </summary>
private void PruneMissTracker(uint now)
{
var window = MissPromotionWindowMs;
var beforeCount = _chunkMissTracker.Count;
var toRemove = new List<long>();
using var toRemove = PooledRefQueue<long>.Create();
foreach (var kvp in _chunkMissTracker)
{
if (now - kvp.Value.LastMissTickStamp > window)
{
toRemove.Add(kvp.Key);
toRemove.Enqueue(kvp.Key);
}
}
foreach (var k in toRemove)
while (toRemove.Count > 0)
{
_chunkMissTracker.Remove(k);
_chunkMissTracker.Remove(toRemove.Dequeue());
}
if (_chunkMissTracker.Count == beforeCount)
{
_chunkMissTracker.Clear();
@ -851,14 +753,15 @@ public sealed class StepCache
var baseX = chunkX << 4;
var baseY = chunkY << 4;
// Tier 4 strata accumulator. Lazily allocated when the first multi-Z cell
// appears; otherwise the chunk has zero strata overhead.
// Strata accumulator, created on the first multi-Z cell so single-Z chunks pay nothing.
// strataData is rented scratch; the chunk receives an exact-size copy, so the pooled array
// never escapes this method.
ushort[] strataOffsetByCell = null;
List<byte> strataData = null;
byte[] strataData = null;
var strataLen = 0;
// Reused per cell: the standable surface Zs (walkway / bridge / floor levels). 16 is
// generous — clearance forces standable surfaces >= PersonHeight apart, so a 256-tall
// Z range admits at most ~16 anyway.
// Standable surface Zs for the current cell. 16 slots is generous: clearance forces
// surfaces at least PersonHeight apart, so an sbyte Z range can't hold more than ~16.
Span<sbyte> surfaceZs = stackalloc sbyte[16];
for (var dy = 0; dy < ChunkSize; dy++)
@ -871,16 +774,13 @@ public sealed class StepCache
map.GetAverageZ(x, y, out _, out var avgZ, out _);
// Anchor the cell at the surface a creature actually STANDS on, not the land
// average. For plain overworld that's the land; for static-over-land terrain
// (sewer/dungeon walkways, bridges, stair treads, raised foundations, upper
// building floors) it's the walkable static surface — which the old
// ComputeStandingZ(avgZ) anchor missed, producing source-Z fallthroughs (or,
// within the StepHeight tolerance band on stairs, a wrong vertical-neighbor
// answer baked at the adjacent tread). ComputeStandableSurfaceZs returns the
// standable surfaces ascending; the lowest is the primary anchor and A* tracks
// newZ to match it. Cells with no standable walk surface (deep water, solid
// rock) fall back to the land avg so the swim layer / wetMask still bake.
// Anchor the cell at the surface a creature stands on, not the land average. On
// open terrain those coincide, but on static-over-land geometry — walkways,
// bridges, stair treads, upper floors — the walkable surface is the static, and
// anchoring at the land below it would make every query fall through the source-Z
// guard. Surfaces come back ascending and the lowest is the anchor; A* tracks its
// per-cell Z to match. A cell with no standable surface at all (deep water, solid
// rock) falls back to the land average so its swim data still bakes.
var surfaceCount = StepProbe.ComputeStandableSurfaceZs(map, x, y, surfaceZs);
var standingZ = surfaceCount > 0 ? surfaceZs[0] : (sbyte)Math.Clamp(avgZ, sbyte.MinValue, sbyte.MaxValue);
@ -906,15 +806,12 @@ public sealed class StepCache
chunk.SwimZW[cell] = result.SwimZ_W;
chunk.SwimZNW[cell] = result.SwimZ_NW;
// Shore-cell handling: if the cell has BOTH a walk surface (standing Z)
// AND a water surface (Wet land tile or wet static) at a Z separated by
// > StepHeight, populate the swim layer at swim-perspective Z. Only when
// ComputeMaskAt produces a non-zero swim mask — bridges/docks/piers with
// insufficient vertical clearance for a swim creature's body envelope
// produce wetMask=0 (StaticsBlockAt rejects them), and we skip those cells
// rather than baking a stratum that always answers "no movement." The
// sentinel NoSwimLayerCell stays in SwimSourceZ for skipped cells; the
// chunk only sets HasSwimLayer when at least one cell got a usable entry.
// Shore cell: a walkable surface and a water surface more than StepHeight apart.
// The main mask is baked at the walk surface, so a swimmer querying from the water
// would fail the source-Z guard; bake it a second answer from the water surface.
// An empty swim mask means the water is unreachable anyway — a dock or pier with
// too little clearance for a swimmer's body — so leave those cells at the
// NoSwimLayerCell sentinel rather than store an answer that always says "blocked".
var swimZRaw = StepProbe.ComputeSwimStandingZ(map, x, y);
if (swimZRaw != int.MinValue && Math.Abs(swimZRaw - standingZ) > StepHeight)
{
@ -939,36 +836,33 @@ public sealed class StepCache
}
}
// Stacked walkable surfaces at one cell (ground + 1st + 2nd building floors,
// a bridge over a walkable path, etc.): bake a stratum per standable surface
// so a query at any floor's Z hits. The primary (lowest) surface is also in
// the main mask above, but multi-Z cells are served exclusively from strata,
// so every standable surface — including the primary — must appear here.
// Single-surface cells (the common case, incl. stair treads and sewer
// walkways) skip this entirely and stay on the fast single-mask path.
// Stacked walkable surfaces — a bridge over a path, the floors of a building —
// need one stratum each so a query at any of their Zs finds an answer. Every
// surface goes in, including the lowest, because a multi-Z cell is served only
// from its strata and never from the main mask baked above.
if (surfaceCount >= 2)
{
if (strataOffsetByCell == null)
{
strataOffsetByCell = new ushort[StepChunk.CellsPerChunk];
for (var i = 0; i < strataOffsetByCell.Length; i++)
{
strataOffsetByCell[i] = StepChunk.NoStrata;
}
strataData = new List<byte>(256);
strataOffsetByCell.AsSpan().Fill(StepChunk.NoStrata);
// NoStrata bounds the packed data to NoStrata bytes (see StepChunk), so
// renting that much up front leaves the record guard below as the only
// bound the writes need.
strataData = STArrayPool<byte>.Shared.Rent(StepChunk.NoStrata);
}
// Cap at 65,535 byte offsets — well above realistic per-chunk strata
// volume. If we ever blow past this we silently leave the cell single-Z
// (it keeps the land-anchored main mask and falls through off-surface).
if (strataData.Count <= ushort.MaxValue - StepChunk.StratumByteLength * 8)
// One count byte plus a record per surface. A cell whose record won't fit stays
// single-Z: it keeps the main mask and falls through off its anchor surface.
var recordLength = 1 + surfaceCount * StepChunk.StratumByteLength;
if (strataLen + recordLength <= StepChunk.NoStrata)
{
strataOffsetByCell[cell] = (ushort)strataData.Count;
strataData.Add((byte)surfaceCount);
strataOffsetByCell[cell] = (ushort)strataLen;
strataData[strataLen++] = (byte)surfaceCount;
for (var i = 0; i < surfaceCount; i++)
{
var sz = surfaceZs[i];
AppendStratumBytes(strataData, new StepProbe.ComputedStratum(sz, StepProbe.ComputeMaskAt(map, x, y, sz)));
WriteStratum(strataData, ref strataLen, sz, StepProbe.ComputeMaskAt(map, x, y, sz));
}
}
}
@ -977,7 +871,8 @@ public sealed class StepCache
if (strataOffsetByCell != null)
{
chunk.SetStrata(strataOffsetByCell, strataData.ToArray());
chunk.SetStrata(strataOffsetByCell, strataData.AsSpan(0, strataLen).ToArray());
STArrayPool<byte>.Shared.Return(strataData);
}
_buildsTotal++;
@ -985,10 +880,10 @@ public sealed class StepCache
}
/// <summary>
/// Tier 4 strata lookup. Walks the cell's stratum list, returns true with the first
/// stratum whose <c>zCenter</c> is within StepHeight of <paramref name="sourceZ"/>.
/// Layout matches <see cref="StepChunk.StrataData"/>: u8 count, then count × 19-byte
/// stratum (sbyte zCenter, byte walkMask, byte wetMask, 8 sbyte walkZ, 8 sbyte swimZ).
/// Finds the cell's stratum matching <paramref name="sourceZ"/> — the first whose zCenter is
/// within StepHeight — and builds its mask. False when the cell has no strata or none of them
/// sit near enough, in which case the caller falls through. Reads the layout
/// <see cref="WriteStratum"/> writes.
/// </summary>
private static bool TryStratumHit(
StepChunk chunk, int cellIndex, sbyte sourceZ, CacheHitKind hitKind, out StepMask result
@ -1050,29 +945,33 @@ public sealed class StepCache
return false;
}
private static void AppendStratumBytes(List<byte> dst, in StepProbe.ComputedStratum s)
/// <summary>
/// Packs one stratum into <paramref name="dst"/> at <paramref name="pos"/>, advancing it by
/// <see cref="StepChunk.StratumByteLength"/>. Layout must stay in lockstep with
/// <see cref="TryStratumHit"/> and <see cref="StepCacheFile"/>.
/// </summary>
private static void WriteStratum(Span<byte> dst, ref int pos, sbyte zCenter, in StepMask mask)
{
dst.Add((byte)s.ZCenter);
dst.Add(s.Mask.WalkMask);
dst.Add(s.Mask.WetMask);
dst.Add((byte)s.Mask.WalkZ_N);
dst.Add((byte)s.Mask.WalkZ_NE);
dst.Add((byte)s.Mask.WalkZ_E);
dst.Add((byte)s.Mask.WalkZ_SE);
dst.Add((byte)s.Mask.WalkZ_S);
dst.Add((byte)s.Mask.WalkZ_SW);
dst.Add((byte)s.Mask.WalkZ_W);
dst.Add((byte)s.Mask.WalkZ_NW);
dst.Add((byte)s.Mask.SwimZ_N);
dst.Add((byte)s.Mask.SwimZ_NE);
dst.Add((byte)s.Mask.SwimZ_E);
dst.Add((byte)s.Mask.SwimZ_SE);
dst.Add((byte)s.Mask.SwimZ_S);
dst.Add((byte)s.Mask.SwimZ_SW);
dst.Add((byte)s.Mask.SwimZ_W);
dst.Add((byte)s.Mask.SwimZ_NW);
dst[pos++] = (byte)zCenter;
dst[pos++] = mask.WalkMask;
dst[pos++] = mask.WetMask;
dst[pos++] = (byte)mask.WalkZ_N;
dst[pos++] = (byte)mask.WalkZ_NE;
dst[pos++] = (byte)mask.WalkZ_E;
dst[pos++] = (byte)mask.WalkZ_SE;
dst[pos++] = (byte)mask.WalkZ_S;
dst[pos++] = (byte)mask.WalkZ_SW;
dst[pos++] = (byte)mask.WalkZ_W;
dst[pos++] = (byte)mask.WalkZ_NW;
dst[pos++] = (byte)mask.SwimZ_N;
dst[pos++] = (byte)mask.SwimZ_NE;
dst[pos++] = (byte)mask.SwimZ_E;
dst[pos++] = (byte)mask.SwimZ_SE;
dst[pos++] = (byte)mask.SwimZ_S;
dst[pos++] = (byte)mask.SwimZ_SW;
dst[pos++] = (byte)mask.SwimZ_W;
dst[pos++] = (byte)mask.SwimZ_NW;
}
private const int PersonHeight = 16;
private const int StepHeight = 2;
}

297
Projects/UOContent/Engines/Pathing/Cache/StepCacheFile.cs
View file

@ -9,40 +9,36 @@ using Server.Compression;
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Binary serializer + lazy reader for the step cache. Persists chunk records to disk
/// so a server warm-starts without paying chunk-build cost on the first pathfind through
/// a region. Lazy: opening a file reads only the header + chunk-offset index (~few KB
/// for tens of thousands of chunks), then individual chunks are seeked + deserialized
/// only when the cache asks for them. RAM stays bounded by MaxResidentChunks regardless
/// of file size.
/// Binary serializer and reader for the step cache, so a shard can warm-start instead of building
/// chunks on the first pathfind through each region. Opening a file reads only the header and chunk
/// index; a chunk record is seeked and inflated when the cache actually asks for it, which keeps
/// resident memory bounded by MaxResidentChunks no matter how large the file is.
///
/// File layout v8 (little-endian, BufferWriter / BufferReader convention):
/// File layout (little-endian, BufferWriter / BufferReader convention):
///
/// Header (40 bytes):
/// u32 Magic = 0x42575300 ('SWB\0')
/// u32 Version = current FormatVersion (9)
/// u32 Version = FormatVersion
/// u32 MapId
/// u64 Fingerprint XxHash3 over (1) LandTable + ItemTable flags AND (2) the
/// on-disk bytes of mapX.mul / .uop, staidxX.mul, staticsX.mul.
/// Rejects a load when EITHER tile flags shifted (client patch)
/// OR the map data was rewritten (CentredSharp / UOFiddler edit).
/// The .mul format has no built-in CRC; this is the only way
/// to detect those mutations.
/// u64 BakeTimestamp DateTime.UtcNow.Ticks at write time (informational).
/// u64 Fingerprint XxHash3 over tiledata.mul and the map's own .mul / .uop files.
/// Detects both a client patch that shifts tile flags and a map edit
/// that rewrites the terrain; see ComputeFingerprint. The .mul format
/// carries no CRC of its own, so hashing is the only way to catch either.
/// u64 BakeTimestamp DateTime.UtcNow.Ticks at write time. Informational.
/// u32 ChunkCount
/// u64 IndexOffset File position where the chunk index begins.
/// u64 IndexOffset Where the index trailer begins.
///
/// Per chunk (ChunkCount times, variable size):
/// u32 UncompressedLen Size of the inflated record body below.
/// byte[] Payload The record body (the v6 layout that follows), libdeflate-
/// compressed. If the on-disk payload length (index recordLength −
/// 4) equals UncompressedLen, the body was stored raw because
/// compression did not shrink it (tiny Uniform records).
/// byte[] Payload The record body, libdeflate-compressed — or stored raw when
/// compression didn't shrink it, as happens with tiny Uniform
/// records. The reader tells the two apart by comparing the payload
/// length against UncompressedLen.
///
/// Record body (after inflate — the v6 layout):
/// Record body (after inflate):
/// u16 ChunkX
/// u16 ChunkY
/// u32 BuiltMultisVersion (reserved since v9 — always 0; chunks are static-only)
/// u32 BuiltMultisVersion Reserved, always 0 — chunks are static-only.
/// u8 Kind 0 = Full; 2 = Uniform
/// // Uniform (Kind == 2): ~28-byte record — all 256 cells share these single values:
/// byte walkMask, wetMask; sbyte sourceZ; sbyte walkZ_N..NW (8); sbyte swimZ_N..NW (8)
@ -78,34 +74,29 @@ namespace Server.Engines.Pathing.Cache;
/// The file offset is not stored — reconstructed as a cumulative sum of recordLength
/// starting at HeaderSize (the first record sits immediately after the header).
///
/// Per-chunk fixed portion (Kind + flags + ZArrayMask + WalkMask + WetMask + SourceZ):
/// ~783 bytes; each present base Z array adds 256 bytes (0..16 present, so up to ~4 KB).
/// A fully-flat Full chunk stores no residual blocks. Strata trailer: 516 + N × ~30 bytes
/// for a chunk with N multi-Z cells averaging ~2 strata each. LRU bookkeeping
/// (LastTouchedTicks) is intentionally not persisted.
/// A chunk's fixed portion runs ~783 bytes, and each directional-Z array that survives prediction
/// adds 256 more, so a Full record lands between ~783 bytes and ~4 KB. The strata trailer adds
/// 516 bytes plus roughly 30 per multi-Z cell. LastTouchedTicks is deliberately not persisted —
/// LRU state means nothing across a restart.
///
/// Files with version &lt; <see cref="MinSupportedVersion"/> are silently rejected
/// at open time (treated as missing) and overwritten on the next save.
/// Files below <see cref="MinSupportedVersion"/> are treated as missing and overwritten on the
/// next save. The cache regenerates from the map data, so a format bump only costs a one-time
/// re-bake.
/// </summary>
internal static class StepCacheFile
{
public const uint Magic = 0x42575300; // 'SWB\0'
// v9: chunks are STATIC-ONLY (land + statics.mul, no multis). v8 and earlier baked multis
// (houses/boats) into chunks, which is unsafe to persist — multis are dynamic, and the
// BuiltMultisVersion they were tagged with is a non-persisted session counter. Bumping the
// version rejects those old files so they re-bake static-only. The BuiltMultisVersion record
// field is retained as a reserved (always-0) u32 to avoid a layout change.
public const uint FormatVersion = 9;
/// <summary>
/// Lowest format version this binary can load. Files below it are treated as missing
/// (silently rejected) and overwritten by the next SaveToFile / BakeMap. The cache is
/// fully regenerable, so a format bump just forces a one-time re-bake of stale files.
/// Oldest format this binary will load. Anything older is treated as missing rather than
/// migrated: the cache is fully regenerable from the map data, so a re-bake is always
/// available and always correct.
/// </summary>
public const uint MinSupportedVersion = 9;
// Per-chunk record discriminator (first byte after BuiltMultisVersion). 1 is reserved.
// Record discriminator. 1 is reserved.
private const byte KindFull = 0;
private const byte KindUniform = 2;
@ -118,12 +109,12 @@ internal static class StepCacheFile
+ sizeof(uint) // ChunkCount
+ sizeof(ulong); // IndexOffset
// Index entry (v8 compact): u32 packedKey ((chunkX << 16) | chunkY) + u32 recordLength.
// The file offset is NOT stored — entries are in record write order, so the reader
// reconstructs each offset by cumulative sum of record lengths starting at HeaderSize.
// One index entry: u32 packedKey ((chunkX << 16) | chunkY) + u32 recordLength. The file offset
// isn't stored — entries sit in record write order, so the reader rebuilds each offset as a
// running sum of the lengths before it, starting at HeaderSize.
private const int IndexEntryBytes = sizeof(uint) + sizeof(uint);
/// <summary>Fixed-size portion of a chunk record (everything except the optional strata + swim trailers).</summary>
/// <summary>A chunk record minus its optional strata and swim trailers.</summary>
private const int BytesPerChunkBase =
sizeof(ushort) + sizeof(ushort) + sizeof(uint)
+ sizeof(byte) + sizeof(byte) + sizeof(byte) // Kind + HasStrata + HasSwimLayer
@ -135,17 +126,8 @@ internal static class StepCacheFile
+ 8 * StepChunk.CellsPerChunk; // SwimZ[8]
/// <summary>
/// Byte offset of the IndexOffset u64 within the header
/// (Magic+Version+MapId+Fingerprint+BakeTimestamp+ChunkCount = 32). Patched after chunks land.
/// </summary>
private const int IndexOffsetFieldPosition = 32;
public delegate bool ChunkEnumerator(out int chunkX, out int chunkY, out StepChunk chunk);
/// <summary>
/// Peek at a .swb file's Fingerprint field (header byte offset 12) without
/// reading any chunk data. Returns false on missing file, bad magic, or wrong
/// version. Cheap — reads 20 bytes total.
/// Reads just a .swb file's fingerprint — 20 bytes, no chunk data. False if the file is
/// missing, isn't a .swb, or is a version this binary can't load.
/// </summary>
public static bool TryReadFingerprint(string path, out ulong fingerprint)
{
@ -183,31 +165,28 @@ internal static class StepCacheFile
}
/// <summary>
/// Combined XxHash3 fingerprint over (1) the on-disk <c>tiledata.mul</c> file and (2) the
/// per-map .mul / .uop file contents (via <see cref="TileMatrix.MapFilesFingerprint"/>).
/// Bake files carry this hash so a load can refuse to populate the cache when EITHER the
/// tile data shifted (client patch) OR the map data was rewritten (CentredSharp / UOFiddler
/// edit). The .mul format has no built-in CRC; this is the only way to detect those mutations.
/// Hashes the inputs a bake depends on: <c>tiledata.mul</c> and the map's own .mul / .uop
/// files. A file carrying a stale hash is refused at open time, which is what catches a client
/// patch that shifts tile flags or a map editor that rewrites the terrain. Neither format has a
/// CRC of its own, so hashing is the only signal available.
///
/// IMPORTANT: hash the FILES, never the in-memory <see cref="TileData.LandTable"/> /
/// This must hash the FILES, never the in-memory <see cref="TileData.LandTable"/> /
/// <see cref="TileData.ItemTable"/>. The server patches those tables at runtime (ItemFixes,
/// LOSBlocker, PotionKeg, CTF, ...) at nondeterministic lifecycle points, so a fingerprint over
/// the live tables varies with WHEN it is taken; the file hash is the only lifecycle-stable
/// "did the client's tile data change?" signal. Server-side tile patches are applied identically
/// every boot and intentionally do NOT invalidate the cache — change one and you must
/// [PathCacheClear or bump the format.
/// LOSBlocker, PotionKeg, CTF), so a hash of the live tables changes depending on when it is
/// taken — useless as a fingerprint. Those server-side patches apply identically every boot and
/// deliberately do NOT invalidate the cache; if you change one, run [PathCacheClear or bump
/// <see cref="FormatVersion"/> yourself.
/// </summary>
public static ulong ComputeFingerprint(int mapId)
{
var hasher = HashUtility.CreateXxHash3();
// (1) tiledata.mul — hashed once, cached. The authoritative source for tile flags/heights.
Span<byte> tileDataBytes = stackalloc byte[sizeof(ulong)];
BinaryPrimitives.WriteUInt64LittleEndian(tileDataBytes, TileDataFileFingerprint());
hasher.Append(tileDataBytes);
// (2) Map files (mapX.mul / .uop, staidxX.mul, staticsX.mul). TileMatrix already
// streamed them through XxHash3 once at construction; mix the result in.
// TileMatrix already streamed the map files through XxHash3 when it was built; reuse that
// rather than re-reading them.
var map = Map.Maps[mapId];
if (map != null && map != Map.Internal && map.Tiles != null)
{
@ -223,10 +202,9 @@ internal static class StepCacheFile
private static bool _tileDataFileFingerprintComputed;
/// <summary>
/// XxHash3 over the raw <c>tiledata.mul</c> bytes, computed once and cached — the file never
/// changes during a run. Mirrors <see cref="TileMatrix.MapFilesFingerprint"/> for the map
/// files. Returns 0 if the file can't be found (the server can't run without it anyway, so
/// this only matters in stripped test hosts, where 0 is a fine deterministic constant).
/// XxHash3 of the raw <c>tiledata.mul</c> bytes, computed once — the file can't change while
/// the server runs. Returns 0 when the file is absent, which only happens in stripped test
/// hosts; a real server can't boot without it, and 0 is a fine deterministic stand-in.
/// </summary>
private static ulong TileDataFileFingerprint()
{
@ -249,84 +227,68 @@ internal static class StepCacheFile
}
/// <summary>
/// Writes the file: header (with placeholder IndexOffset) → chunks (offsets recorded)
/// → index trailer → patches the header IndexOffset. <paramref name="chunkCount"/> must
/// equal the actual number of chunks <paramref name="next"/> will yield.
/// Writes the map's chunks to <paramref name="path"/>: header, then one record per chunk, then
/// the index trailer. IndexOffset isn't known until the records are down, so it goes in as a
/// placeholder and gets patched by seeking back to it.
/// </summary>
public static void Write(string path, uint mapId, uint chunkCount, ChunkEnumerator next)
public static void Write(string path, uint mapId, ReadOnlySpan<(int chunkX, int chunkY, StepChunk chunk)> chunks)
{
Directory.CreateDirectory(Path.GetDirectoryName(path) ?? ".");
// Initial estimate: base record + a modest strata budget per chunk. Coastline
// chunks add another ~2.5 KB (swim layer) but they're a small fraction of any
// map; the writer grows on overflow so under-estimating just causes a few
// realloc/copy cycles during the bake — not a correctness issue.
var capacity = HeaderSize + (BytesPerChunkBase + 256) * (int)chunkCount + IndexEntryBytes * (int)chunkCount;
var buffer = new byte[capacity];
var w = new BufferWriter(buffer, prefixStr: false);
// A rough estimate: the base record plus a small strata budget per chunk. Coastline chunks
// run ~2.5 KB over it for their swim layer, but they're a small share of any map, and the
// writer grows on overflow — under-estimating costs a few reallocs during a bake, nothing more.
var capacity = HeaderSize + (BytesPerChunkBase + 256 + IndexEntryBytes) * chunks.Length;
var w = new BufferWriter(new byte[capacity], prefixStr: false);
w.Write(Magic);
w.Write(FormatVersion);
w.Write(mapId);
w.Write(ComputeFingerprint((int)mapId));
w.Write((ulong)DateTime.UtcNow.Ticks);
w.Write(chunkCount);
w.Write(0UL); // IndexOffset placeholder, patched after chunks
w.Write((uint)chunks.Length);
var indexOffsetPosition = w.Position;
w.Write(0UL); // patched below, once the records are written and the index position is known
// Each record is built uncompressed into recordScratch, then libdeflate-compressed into
// compScratch and framed as [u32 uncompressedLen][payload].
// Each record is built into recordScratch, compressed into compScratch, then framed as
// [u32 uncompressedLen][payload].
var packer = Deflate.Maximum;
var recordScratch = new byte[BytesPerChunkBase + 1024];
var compScratch = new byte[packer.MaxPackSize(recordScratch.Length)];
var indexEntries = new (ulong key, ulong offset, uint length)[chunkCount];
var written = 0u;
while (next(out var chunkX, out var chunkY, out var chunk))
// Record lengths only — the index stores no offsets, so the reader rebuilds them by
// summing these in order.
var lengths = new uint[chunks.Length];
for (var i = 0; i < chunks.Length; i++)
{
if (written >= chunkCount)
{
throw new InvalidOperationException(
$"StepCacheFile.Write: enumerator yielded more than the declared {chunkCount} chunks"
);
}
var chunkOffset = (ulong)w.Position;
var (chunkX, chunkY, chunk) = chunks[i];
var start = w.Position;
WriteChunk(w, chunkX, chunkY, chunk, packer, ref recordScratch, ref compScratch);
var chunkLength = (uint)((ulong)w.Position - chunkOffset);
indexEntries[written] = (PackChunkKey(chunkX, chunkY), chunkOffset, chunkLength);
written++;
}
if (written != chunkCount)
{
throw new InvalidOperationException(
$"StepCacheFile.Write: declared {chunkCount} chunks but enumerator yielded {written}"
);
lengths[i] = (uint)(w.Position - start);
}
var indexOffset = (ulong)w.Position;
for (var i = 0u; i < chunkCount; i++)
for (var i = 0; i < chunks.Length; i++)
{
// v8 compact entry: u32 packedKey ((chunkX << 16) | chunkY) + u32 recordLength.
// Offset is omitted; entries are in record write order so the reader derives it.
var key = indexEntries[i].key;
var packedKey = ((uint)(key >> 32) << 16) | (uint)(key & 0xFFFF);
w.Write(packedKey);
w.Write(indexEntries[i].length);
var (chunkX, chunkY, _) = chunks[i];
w.Write((uint)((chunkX & 0xFFFF) << 16 | chunkY & 0xFFFF));
w.Write(lengths[i]);
}
// Patch IndexOffset on the writer's current backing buffer (BufferWriter may
// have grown during chunk writes; the original `buffer` ref is stale after grow).
var liveBuffer = w.Buffer;
BinaryPrimitives.WriteUInt64LittleEndian(liveBuffer.AsSpan(IndexOffsetFieldPosition, 8), indexOffset);
var totalBytes = (int)w.Position;
File.WriteAllBytes(path, liveBuffer.AsSpan(0, totalBytes).ToArray());
w.Seek(indexOffsetPosition, SeekOrigin.Begin);
w.Write(indexOffset);
// w.Buffer, not the array handed to the constructor: BufferWriter reallocates on growth,
// which leaves that original reference pointing at a stale array.
File.WriteAllBytes(path, w.Buffer.AsSpan(0, totalBytes).ToArray());
}
/// <summary>
/// Opens a .swb file and reads only its header + chunk-offset index. Returns null on
/// missing file, magic / version mismatch, or Fingerprint mismatch (a stale bake
/// against a freshly patched client). Callers own disposal of the returned reader.
/// Opens a .swb file, reading only its header and chunk index. Null if the file is missing,
/// isn't a loadable .swb, or is a stale bake whose fingerprint no longer matches the live tile
/// and map data. The caller owns the returned reader.
/// </summary>
public static LazyReader OpenForLazy(string path)
{
@ -361,8 +323,6 @@ internal static class StepCacheFile
var version = BinaryPrimitives.ReadUInt32LittleEndian(headerBuf[4..]);
if (version < MinSupportedVersion || version > FormatVersion)
{
// Below the minimum supported version: treat as missing. Older files
// get silently overwritten on the next SaveToFile / BakeMap.
stream.Dispose();
return null;
}
@ -379,7 +339,7 @@ internal static class StepCacheFile
return null;
}
// Read the chunk-offset index in one shot.
// Pull the whole index in one read.
var indexBytes = (int)chunkCount * IndexEntryBytes;
var indexBuf = new byte[indexBytes];
stream.Position = (long)indexOffset;
@ -389,9 +349,8 @@ internal static class StepCacheFile
return null;
}
// v8 compact index: { u32 packedKey, u32 length } per chunk, in record write order.
// The file offset is not stored — reconstruct it by cumulative record length starting
// at the first record (immediately after the header).
// Entries are in record write order and carry no offset, so rebuild each one as a
// running sum of the record lengths, starting just past the header.
var offsets = new Dictionary<ulong, (ulong offset, uint length)>((int)chunkCount);
var runningOffset = (ulong)HeaderSize;
for (var i = 0; i < chunkCount; i++)
@ -416,27 +375,27 @@ internal static class StepCacheFile
private static ulong PackChunkKey(int chunkX, int chunkY) => ((ulong)(uint)chunkX << 32) | (uint)chunkY;
/// <summary>
/// Predicted directional-Z for one cell/direction: the cell's own SourceZ when the
/// direction is walkable/wet (mask bit set), else 0 — matching the baker, which leaves
/// non-walkable directional slots at their zero-initialized default
/// (StepProbe.ComputeMaskAt clears walkZs/swimZs and writes only on a successful step).
/// Guesses a cell's destination Z for one direction: on flat ground a step lands at the Z you
/// left from, so predict SourceZ where the direction is passable and 0 where it isn't. The
/// zero matches the baker, which only writes a slot on a successful step and leaves the rest
/// cleared. Most terrain is flat, so most predictions are exact and most residuals are 0 —
/// which is what makes the residual arrays compress away to nothing.
/// </summary>
internal static sbyte Predict(byte dirMaskByte, int bit, sbyte sourceZ) =>
(dirMaskByte >> bit & 1) != 0 ? sourceZ : (sbyte)0;
/// <summary>
/// Residual of an absolute directional-Z against its prediction. Unchecked two's-complement
/// so the transform is byte-exact for ALL sbyte inputs (no value-range constraint).
/// A destination Z's difference from its prediction. Wraps deliberately: two's-complement
/// round-trips exactly for every sbyte input, so no value range is off-limits.
/// </summary>
internal static sbyte EncodeResidual(sbyte z, sbyte predict) => unchecked((sbyte)(z - predict));
/// <summary>Inverse of <see cref="EncodeResidual"/>: absolute directional-Z = predict + residual.</summary>
/// <summary>Inverse of <see cref="EncodeResidual"/>.</summary>
internal static sbyte DecodeZ(sbyte predict, sbyte residual) => unchecked((sbyte)(predict + residual));
/// <summary>
/// The base directional-Z array for direction index d in canonical order: walk N..NW (0-7),
/// then swim N..NW (8-15). Index d uses WalkMask (d &lt; 8) or WetMask (d &gt;= 8) with
/// direction bit (d &amp; 7).
/// The destination-Z array for direction index d, in the canonical order the format stores them:
/// walk N..NW as 0-7, then swim N..NW as 8-15.
/// </summary>
private static sbyte[] GetBaseZArray(StepChunk c, int d) => d switch
{
@ -448,10 +407,9 @@ internal static class StepCacheFile
};
/// <summary>
/// Builds the uncompressed v6 record for one chunk into <paramref name="w"/>, libdeflate-
/// compresses it, and writes it framed as [u32 uncompressedLen][payload]. The payload is the
/// compressed bytes, or — when compression does not shrink the record (tiny Uniform records) —
/// the raw record itself; the reader distinguishes the two by payload length vs uncompressedLen.
/// Builds one chunk's record, compresses it, and frames it as [u32 uncompressedLen][payload].
/// When compression fails to shrink the record — as it does on the tiny Uniform ones — the raw
/// record is stored instead, and the reader tells the two apart by payload length.
/// </summary>
private static void WriteChunk(
BufferWriter w, int chunkX, int chunkY, StepChunk chunk,
@ -460,7 +418,7 @@ internal static class StepCacheFile
{
var rw = new BufferWriter(recordScratch, prefixStr: false);
BuildRecord(rw, chunkX, chunkY, chunk);
recordScratch = rw.Buffer; // may have grown; keep the larger buffer for reuse
recordScratch = rw.Buffer; // may have grown; hold onto the larger buffer for the next chunk
var recordLen = (int)rw.Position;
var bound = packer.MaxPackSize(recordLen);
@ -478,8 +436,8 @@ internal static class StepCacheFile
}
else
{
// Incompressible (or expanded): store the record raw. The reader detects this when
// the on-disk payload length equals the uncompressed length.
// Compression didn't help, so store the record raw. Payload length == uncompressedLen
// is how the reader recognizes that.
w.Write(recordScratch.AsSpan(0, recordLen));
}
}
@ -490,8 +448,8 @@ internal static class StepCacheFile
w.Write((ushort)chunkY);
w.Write((uint)chunk.BuiltMultisVersion);
// Kind: 0 = Full, 2 = Uniform. A uniform chunk (no strata, no swim layer, all 19 base
// arrays constant) stores one cell's worth of data (~28-byte record total).
// A uniform chunk — every cell identical — collapses to one cell's worth of data, ~28 bytes.
// Open water and solid rock make up a lot of a map, so this is worth the branch.
if (chunk.IsUniform())
{
w.Write(KindUniform);
@ -517,7 +475,7 @@ internal static class StepCacheFile
return;
}
w.Write(KindFull); // Full
w.Write(KindFull);
var strataOffsetByCell = chunk.GetStrataOffsetByCellForSerialization();
var strataData = chunk.GetStrataDataForSerialization();
@ -526,9 +484,9 @@ internal static class StepCacheFile
w.Write((byte)(hasStrata ? 1 : 0));
w.Write((byte)(hasSwimLayer ? 1 : 0));
// Predictive-Z: each base directional Z array is stored as a masked residual against
// SourceZ. Bit d of ZArrayMask is set only when array d differs from its prediction
// somewhere; cleared arrays are omitted and rebuilt from mask+SourceZ at read.
// Each destination-Z array is stored as residuals against its prediction (see Predict). An
// array that matches its prediction everywhere — the common case on flat terrain — is
// omitted entirely, and its ZArrayMask bit stays clear so the reader synthesizes it.
ushort zArrayMask = 0;
for (var d = 0; d < 16; d++)
{
@ -583,7 +541,6 @@ internal static class StepCacheFile
if (hasStrata)
{
// 256 × u16 offsets, then u32 length-prefixed strata byte array.
for (var i = 0; i < StepChunk.CellsPerChunk; i++)
{
w.Write(strataOffsetByCell[i]);
@ -600,7 +557,7 @@ internal static class StepCacheFile
private static StepChunk ReadChunk(byte[] buffer)
{
var r = new BufferReader(buffer);
// Skip ChunkX + ChunkY (already known via the index lookup).
// ChunkX + ChunkY — already known from the index lookup that got us here.
r.ReadUShort();
r.ReadUShort();
var multisVersion = (int)r.ReadUInt();
@ -608,7 +565,7 @@ internal static class StepCacheFile
var chunk = new StepChunk { BuiltMultisVersion = multisVersion };
if (kind == KindUniform) // Uniform — one cell's worth of the 19 base arrays, fill all 256 cells.
if (kind == KindUniform) // one cell's values, broadcast to all 256
{
Array.Fill(chunk.WalkMask, r.ReadByte());
Array.Fill(chunk.WetMask, r.ReadByte());
@ -640,8 +597,8 @@ internal static class StepCacheFile
r.Read(chunk.WetMask);
ReadSBytes(r, chunk.SourceZ);
// Predictive-Z reconstruction: present arrays carry residuals (z = predict + residual);
// absent arrays are synthesized from mask+SourceZ (z = predict, residual implicitly 0).
// Inverse of the write path: a stored array carries residuals to add back to the
// prediction, an omitted one IS the prediction.
Span<sbyte> residual = stackalloc sbyte[StepChunk.CellsPerChunk];
for (var d = 0; d < 16; d++)
{
@ -706,16 +663,15 @@ internal static class StepCacheFile
r.Read(MemoryMarshal.Cast<sbyte, byte>(arr.AsSpan()));
/// <summary>
/// Open handle on a .swb file. Holds the FileStream + chunk-offset index. Chunks are
/// fetched on demand via <see cref="TryReadChunk"/>; only the records actually queried
/// are ever materialized. Dispose releases the underlying stream.
/// An open .swb file: the stream plus the chunk index. Only the records actually asked for are
/// ever read or inflated. Dispose releases the stream.
/// </summary>
internal sealed class LazyReader : IDisposable
{
private FileStream _stream;
private readonly Dictionary<ulong, (ulong offset, uint length)> _offsets;
private byte[] _buffer; // raw on-disk record: [u32 uncompressedLen][payload]
private byte[] _bodyBuffer; // decompressed v6 record, parsed by ReadChunk
private byte[] _buffer; // the raw framed record as it sits on disk
private byte[] _bodyBuffer; // that record, inflated, ready for ReadChunk
public uint MapId { get; }
public ulong Fingerprint { get; }
@ -725,11 +681,7 @@ internal static class StepCacheFile
public bool Has(int chunkX, int chunkY) => _offsets.ContainsKey(PackChunkKey(chunkX, chunkY));
/// <summary>
/// Enumerates every (chunkX, chunkY) coordinate the file holds. Used by
/// <see cref="StepCache"/> when preload is enabled to materialize all chunks
/// upfront instead of on first query.
/// </summary>
/// <summary>Every (chunkX, chunkY) the file holds. Used to preload the whole file.</summary>
public IEnumerable<(int chunkX, int chunkY)> EnumerateChunkCoords()
{
foreach (var key in _offsets.Keys)
@ -754,9 +706,8 @@ internal static class StepCacheFile
}
/// <summary>
/// Returns the chunk record at (<paramref name="chunkX"/>, <paramref name="chunkY"/>)
/// from the file, or null if the file doesn't contain it. Single seek + bulk read,
/// sized exactly to the chunk's recorded length (which varies with strata size).
/// Reads one chunk from the file, or null if the file has no record for it. One seek and
/// one read, sized to the record's indexed length.
/// </summary>
public StepChunk TryReadChunk(int chunkX, int chunkY)
{
@ -771,7 +722,6 @@ internal static class StepCacheFile
return null;
}
// Grow the on-disk scratch buffer if this chunk's record is larger than what we have.
if (entry.length > _buffer.Length)
{
_buffer = new byte[entry.length];
@ -784,8 +734,8 @@ internal static class StepCacheFile
return null;
}
// Frame: [u32 uncompressedLen][payload]. payload is libdeflate-compressed, unless its
// length equals uncompressedLen, in which case it was stored raw (incompressible).
// [u32 uncompressedLen][payload], where the payload is compressed unless its length
// already equals uncompressedLen — then it was stored raw.
var uncompressedLen = (int)BinaryPrimitives.ReadUInt32LittleEndian(_buffer);
var payloadLen = (int)entry.length - sizeof(uint);
if (_bodyBuffer.Length < uncompressedLen)
@ -799,7 +749,8 @@ internal static class StepCacheFile
}
else
{
// Decompression is level-independent, so reuse the shared per-thread binding.
// Deflate.Standard, not .Maximum: the level only affects packing, and inflate has
// to accept whatever the writer produced regardless.
var result = Deflate.Standard.Unpack(
_bodyBuffer.AsSpan(0, uncompressedLen),
_buffer.AsSpan(sizeof(uint), payloadLen),

74
Projects/UOContent/Engines/Pathing/Cache/StepChunk.cs
View file

@ -3,20 +3,19 @@ using System;
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Per-chunk storage backing StepCache. Holds raw walk + swim masks and destination Z
/// values for each of 256 cells in a 16x16 chunk, plus build-time metadata (multis version, multi-Z strata)
/// and LRU bookkeeping.
/// Per-chunk storage backing <see cref="StepCache"/>: walk + swim masks and destination Zs for
/// each of the 256 cells in a 16x16 chunk, plus the optional multi-Z strata and swim layers and
/// the LRU timestamp.
/// </summary>
internal sealed class StepChunk
{
public const int CellsPerChunk = 256; // 16 x 16
/// <summary>Bit i of WalkMask[c] = "default walker can step from cell c to neighbor (Direction)i".
/// Raw — no diagonal corner-cut applied here.</summary>
/// <summary>Bit i of WalkMask[c]: a default walker can step from cell c to neighbour (Direction)i.
/// Raw — no diagonal corner-cut applied, so callers must AND the partner bits themselves.</summary>
public readonly byte[] WalkMask = new byte[CellsPerChunk];
/// <summary>Bit i of WetMask[c] = "swim-only mob can step from cell c to neighbor (Direction)i".
/// Layered with WalkMask via canSwim/cantWalk capability flags.</summary>
/// <summary>Bit i of WetMask[c]: a swim-only mob can step from cell c to neighbour (Direction)i.</summary>
public readonly byte[] WetMask = new byte[CellsPerChunk];
public readonly sbyte[] SourceZ = new sbyte[CellsPerChunk];
@ -40,14 +39,13 @@ internal sealed class StepChunk
public readonly sbyte[] SwimZNW = new sbyte[CellsPerChunk];
/// <summary>
/// Swim layer — populated only for chunks containing at least one shore cell (a cell
/// with both a walkable land surface and a water surface separated by > StepHeight).
/// On shore cells, queries from the swim source Z miss the primary source-Z guard;
/// the swim layer carries the correct wetMask + per-direction destination Zs computed
/// from the water surface's perspective. For non-shore cells in a chunk that has the
/// layer, <see cref="SwimSourceZ"/>[cell] = <see cref="NoSwimLayerCell"/> sentinel.
/// All swim-layer arrays are null on chunks with no shore cells (~90% of map chunks
/// on Trammel) — zero memory cost on the common case.
/// Marks a cell with no swim-layer entry, in a chunk that has the layer.
///
/// The swim layer exists only on chunks holding at least one shore cell — a cell with both a
/// walkable surface and a water surface more than StepHeight apart. A swim query there sits
/// too far from the primary SourceZ to pass the source-Z guard, so the layer carries a second
/// mask and destination-Z set computed from the water surface instead. Chunks with no shore
/// cells leave every swim-layer array null.
/// </summary>
public const sbyte NoSwimLayerCell = sbyte.MinValue;
@ -79,9 +77,8 @@ internal sealed class StepChunk
public sbyte[] SwimZNW_Layer => _swimZNW_extra;
/// <summary>
/// Lazily allocates the swim-layer arrays and seeds <see cref="SwimSourceZ"/> with
/// the <see cref="NoSwimLayerCell"/> sentinel. Called at bake time the first time a
/// shore cell is detected in this chunk.
/// Allocates the swim-layer arrays and seeds <see cref="SwimSourceZ"/> with
/// <see cref="NoSwimLayerCell"/>. Called on the first shore cell found in this chunk.
/// </summary>
internal void AllocateSwimLayer()
{
@ -105,28 +102,30 @@ internal sealed class StepChunk
}
}
/// <summary>Sentinel: cell has no strata — single-Z, use the main Walk/Wet arrays.</summary>
/// <summary>
/// Marks a single-Z cell: no strata, read the main Walk/Wet arrays instead. Because this
/// takes ushort.MaxValue, a real strata offset is at most NoStrata - 1, which bounds
/// <see cref="StrataData"/> to NoStrata bytes.
/// </summary>
public const ushort NoStrata = ushort.MaxValue;
/// <summary>
/// Length-256 offset table: <c>StrataOffsetByCell[cell] = byte offset</c> into
/// <see cref="StrataData"/> where this cell's strata begin, or <see cref="NoStrata"/>
/// for cells without multi-Z. Null when the chunk has zero multi-Z cells.
/// Length-256 table mapping a cell to the byte offset in <see cref="StrataData"/> where its
/// strata begin, or <see cref="NoStrata"/>. Null when no cell in the chunk is multi-Z.
/// </summary>
private ushort[] _strataOffsetByCell;
/// <summary>
/// Packed per-cell strata. For each cell with strata:
/// u8 stratumCount, then stratumCount × Stratum (19 bytes each):
/// sbyte zCenter, byte walkMask, byte wetMask,
/// sbyte walkZ_N..NW (8), sbyte swimZ_N..NW (8)
/// Packed strata for the multi-Z cells. Per cell: u8 stratumCount, then stratumCount records
/// of <see cref="StratumByteLength"/> bytes — sbyte zCenter, byte walkMask, byte wetMask,
/// sbyte walkZ_N..NW (8), sbyte swimZ_N..NW (8).
/// </summary>
private byte[] _strataData;
/// <summary>Snapshot of Sector.MultisVersion at the time this chunk was built.</summary>
/// <summary>Reserved. Chunks are static-only, so this is always 0.</summary>
public int BuiltMultisVersion;
/// <summary>Updated on every cache hit/miss. Used by LRU fallback eviction.</summary>
/// <summary>Refreshed on every query that reaches this chunk. Drives LRU eviction.</summary>
public long LastTouchedTicks;
/// <summary>Size in bytes of one Stratum record in StrataData.</summary>
@ -140,10 +139,8 @@ internal sealed class StepChunk
_strataData == null ? ReadOnlySpan<byte>.Empty : _strataData.AsSpan();
/// <summary>
/// Single-shot setter for the chunk's strata. Pass null/null to clear (chunk becomes
/// "no multi-Z"). Otherwise <paramref name="offsetByCell"/> must be length 256 with
/// <see cref="NoStrata"/> for cells without strata, and <paramref name="data"/> the
/// packed strata records.
/// Sets the chunk's strata in one shot. <paramref name="offsetByCell"/> must be length 256,
/// carrying <see cref="NoStrata"/> for single-Z cells. Pass null/null to clear.
/// </summary>
internal void SetStrata(ushort[] offsetByCell, byte[] data)
{
@ -151,18 +148,17 @@ internal sealed class StepChunk
_strataData = data;
}
/// <summary>Serialization hook: returns the raw offset array (or null if no strata).</summary>
/// <summary>Serialization hook: the raw offset array, or null if the chunk has no strata.</summary>
internal ushort[] GetStrataOffsetByCellForSerialization() => _strataOffsetByCell;
/// <summary>Serialization hook: returns the raw data array (or null if no strata).</summary>
/// <summary>Serialization hook: the raw data array, or null if the chunk has no strata.</summary>
internal byte[] GetStrataDataForSerialization() => _strataData;
/// <summary>
/// True when every cell shares one value across WalkMask, WetMask, SourceZ, and all 16
/// directional-Z arrays, and the chunk has neither multi-Z strata nor a swim layer. Such a
/// chunk serializes to a ~28-byte uniform record (StepCacheFile v5) instead of the full
/// record. Chunks with a swim layer (shore cells) are never uniform — their per-cell swim
/// data must be preserved via the Full record.
/// True when all 256 cells share one value across WalkMask, WetMask, SourceZ and every
/// directional-Z array, with no strata and no swim layer — open water or solid rock, mostly.
/// <see cref="StepCacheFile"/> collapses such a chunk to a ~28-byte record. A swim layer
/// disqualifies a chunk outright: its per-cell shore data would not survive the collapse.
/// </summary>
internal bool IsUniform() => _strataOffsetByCell == null
&& !HasSwimLayer

12
Projects/UOContent/Engines/Pathing/Cache/StepMask.cs
View file

@ -1,10 +1,10 @@
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Per-cell, per-direction static walkability data baked by <see cref="StepProbe"/>
/// and stored by <see cref="StepCache"/>. WalkMask + WalkZ_* applies under default-walker
/// rules (cantWalk=false, canSwim=false). WetMask + SwimZ_* applies under swim-only rules
/// (cantWalk=true, canSwim=true). Algorithms layer the right rules per mobile.
/// Per-cell, per-direction walkability baked by <see cref="StepProbe"/> and stored by
/// <see cref="StepCache"/>. Two rule sets travel together: WalkMask + WalkZ_* for a default
/// walker (cantWalk=false, canSwim=false), WetMask + SwimZ_* for a swim-only mob
/// (cantWalk=true, canSwim=true). Callers overlay whichever applies to the mobile.
/// </summary>
public readonly struct StepMask(
byte walkMask,
@ -49,8 +49,8 @@ public readonly struct StepMask(
public readonly CacheHitKind HitKind = hitKind;
/// <summary>
/// True when the cache produced a usable answer (Hit / Miss_NotBuilt / Miss_DirtyRebuild).
/// False on Fallthrough_*, in which case the caller must use the slow path for this cell.
/// True when the cache produced a usable answer. False on any Fallthrough_*, where the
/// payload is all zeroes and the caller must resolve this cell via the slow path.
/// </summary>
public bool IsHit => HitKind <= CacheHitKind.Miss_DirtyRebuild;

158
Projects/UOContent/Engines/Pathing/Cache/StepProbe.cs
View file

@ -4,123 +4,37 @@ using CalcMoves = Server.Movement.Movement;
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Computes static-only walkability for a single cell — the per-cell, per-direction
/// "can step" mask and destination Z, based purely on land + statics.mul tiles (NOT
/// multis). Mirrors <see cref="MovementImpl"/>.Check minus the item and mobile collision
/// phases. Multis (houses, boats) are intentionally excluded: they're dynamic content, so
/// cells they cover route to the live movement path via <see cref="StepCache"/>'s
/// multi-halo fallthrough rather than being baked into the static chunk cache.
/// Computes the 8-direction "can step" mask and destination Zs for a single cell from land and
/// statics alone. Mirrors <see cref="MovementImpl"/>.Check minus the item and mobile collision
/// phases, which belong to the caller's dynamic-obstacle pass.
///
/// Multis (houses, boats) are excluded from the static bake because they are dynamic content;
/// cells they cover route to the live movement path via <see cref="StepCache"/>'s multi halo.
/// <see cref="ComputeMultiMaskAt"/> is the opt-in exception for those cells.
///
/// Each call bakes both rule sets: walker (canSwim=false, cantWalk=false) and swim-only
/// (canSwim=true, cantWalk=true). Diagonal corner-cut is not applied — callers hold the partner
/// bits in the same mask byte and combine them at query time.
/// </summary>
/// <remarks>
/// Bakes two rule sets per cell: walker (canSwim=false, cantWalk=false) and swim-only
/// (canSwim=true, cantWalk=true). Item / mobile collision phases are omitted (they're
/// the dynamic-obstacle pass's job). Diagonal corner-cut is NOT applied here; callers
/// must AND the partner-cell results at query time.
/// </remarks>
public static class StepProbe
{
private const int PersonHeight = 16;
private const int StepHeight = 2;
public readonly struct ComputedStratum(sbyte zCenter, StepMask mask)
{
public readonly sbyte ZCenter = zCenter;
public readonly StepMask Mask = mask;
}
/// <summary>
/// Tier 4 strata builder: enumerates the distinct walkable standing-Zs at (x, y)
/// — one per land surface plus one per walkable static — and runs
/// <see cref="ComputeMaskAt"/> at each, producing a per-stratum walkability snapshot.
/// Returns null when the cell has 0 or 1 strata (single-Z; the caller should use
/// the chunk's main mask).
/// </summary>
public static ComputedStratum[] ComputeStrataAt(Map map, int x, int y)
{
if (map == null || map == Map.Internal)
{
return null;
}
if (x < 0 || y < 0 || x >= map.Width || y >= map.Height)
{
return null;
}
// Collect candidate Zs. 16 slots is generous — multi-Z cells in practice rarely
// exceed 3-4 surfaces (bridge over land, paver-over-ground, multi-floor stairs).
Span<int> zs = stackalloc int[16];
var count = 0;
var landTile = map.Tiles.GetLandTile(x, y);
var landFlags = TileData.LandTable[landTile.ID & TileData.MaxLandValue].Flags;
if (!landTile.Ignored && (landFlags & TileFlag.Impassable) == 0)
{
map.GetAverageZ(x, y, out _, out var landCenter, out _);
zs[count++] = landCenter;
}
foreach (var tile in map.Tiles.GetStaticTiles(x, y))
{
if (count >= zs.Length)
{
break;
}
var data = TileData.ItemTable[tile.ID & TileData.MaxItemValue];
if (!data.Surface || data.Impassable)
{
continue;
}
zs[count++] = tile.Z + data.CalcHeight;
}
if (count <= 1)
{
return null;
}
// Sort and merge near-equal Zs. Two Zs separated by less than 2*StepHeight collapse
// into a single stratum — the slow path's tolerance treats them as the same surface.
zs[..count].Sort();
Span<int> distinct = stackalloc int[16];
var distinctCount = 0;
for (var i = 0; i < count; i++)
{
if (distinctCount == 0 || zs[i] - distinct[distinctCount - 1] > 2 * StepHeight)
{
distinct[distinctCount++] = zs[i];
}
}
if (distinctCount <= 1)
{
return null;
}
var strata = new ComputedStratum[distinctCount];
for (var i = 0; i < distinctCount; i++)
{
var z = (sbyte)Math.Clamp(distinct[i], sbyte.MinValue, sbyte.MaxValue);
strata[i] = new ComputedStratum(z, ComputeMaskAt(map, x, y, z));
}
return strata;
}
/// <summary>
/// Writes the distinct surface Zs at (x, y) that a default walker (PersonHeight envelope)
/// can actually STAND on — each candidate surface (walkable land center + every walkable
/// static top) that has PersonHeight of vertical clearance free of impassable statics —
/// into <paramref name="zs"/>, ascending, and returns the count.
/// Writes the surface Zs at (x, y) a default walker can actually stand on into
/// <paramref name="zs"/>, ascending, and returns the count. A candidate surface — the
/// walkable land centre, or any walkable static's top — qualifies only if a PersonHeight
/// envelope above it is clear of impassable statics.
///
/// This is the clearance-aware counterpart to <see cref="ComputeStrataAt"/>'s candidate
/// gather: it drops surfaces a creature cannot occupy (land under a sewer walkway, ground
/// under a low bridge), so the result is exactly the set of standing Zs the slow path can
/// resolve to. Two standable surfaces are inherently &gt;= PersonHeight apart (an upper
/// surface within PersonHeight of a lower one removes the lower one's clearance), so a
/// single ascending pass with an exact-duplicate skip is sufficient.
/// The clearance test is what makes this the exact set of standing Zs the slow path can
/// resolve to: it drops surfaces a creature cannot occupy, like the land beneath a sewer
/// walkway or a low bridge. That in turn means two surviving surfaces are always at least
/// PersonHeight apart (an upper surface any closer would have taken the lower one's
/// clearance away), so one ascending pass with a duplicate skip suffices.
///
/// Used by the baker to capture walkable static-over-land surfaces (sewer/dungeon
/// walkways, bridges, raised foundations, upper building floors) that the land-anchored
/// main mask would otherwise miss.
/// The baker anchors each cell here so static-over-land geometry — walkways, bridges, raised
/// foundations, upper floors — bakes at the Z a creature stands on rather than the land average.
/// </summary>
public static int ComputeStandableSurfaceZs(Map map, int x, int y, Span<sbyte> zs)
{
@ -190,21 +104,17 @@ public static class StepProbe
ComputeMaskCore(map, x, y, sourceZ, includeMultis: false);
/// <summary>
/// Multi-aware counterpart to <see cref="ComputeMaskAt"/>: synthesizes the full 8-direction
/// walkability mask for a cell covered by (or adjacent to) a multi, folding house/boat component
/// tiles into the surface/step logic via GetStaticAndMultiTiles. Replaces the slow path's 8x
/// per-cell CheckMovement for Fallthrough_Multi cells. Item/mobile collision is still handled by
/// the caller's dynamic-obstacle pass.
/// Multi-aware counterpart to <see cref="ComputeMaskAt"/>, for cells a multi covers or
/// neighbours: folds house/boat component tiles into the same surface/step logic. Builds the
/// whole 8-direction mask in one pass, where the slow path would run CheckMovement eight times.
/// </summary>
public static StepMask ComputeMultiMaskAt(Map map, int x, int y, sbyte sourceZ) =>
ComputeMaskCore(map, x, y, sourceZ, includeMultis: true);
/// <summary>
/// Shared per-cell 8-direction mask builder. With includeMultis=false this reproduces the
/// static-only bake (land + statics.mul). With includeMultis=true it also folds in multi
/// (house/boat) component tiles via GetStaticAndMultiTiles — the multi-aware synthesizer used
/// for Fallthrough_Multi cells. Item/mobile collision phases are still omitted (the dynamic pass
/// owns them).
/// Shared 8-direction mask builder behind <see cref="ComputeMaskAt"/> and
/// <see cref="ComputeMultiMaskAt"/>. <paramref name="includeMultis"/> is the only difference:
/// it swaps the tile source to GetStaticAndMultiTiles so house and boat components participate.
/// </summary>
private static StepMask ComputeMaskCore(Map map, int x, int y, sbyte sourceZ, bool includeMultis)
{
@ -224,8 +134,8 @@ public static class StepProbe
byte wetMask = 0;
Span<sbyte> walkZs = stackalloc sbyte[8];
Span<sbyte> swimZs = stackalloc sbyte[8];
// stackalloc is NOT zero-initialized — unwritten slots hold whatever was on the
// stack. Clear before use; the loop only writes slots where the step succeeds.
// stackalloc is not zero-initialized, and the loop below writes a slot only where the
// step succeeds, so blocked directions would otherwise carry stack garbage.
walkZs.Clear();
swimZs.Clear();
@ -262,10 +172,10 @@ public static class StepProbe
}
/// <summary>
/// Returns the slow path's standing-Z for a default walker at (x, y). Mirrors
/// MovementImpl.Check's surface-selection — paver Z+1 for paver-over-ground,
/// landCenter for bare land. Used by <see cref="StepCache"/> to bake SourceZ so
/// A*'s tracked-per-cell Z matches the cache's bake-time assumption.
/// The standing-Z a default walker at (x, y) resolves to under the slow path's
/// surface-selection rules: paver Z+1 over paver-on-ground, land centre on bare land.
/// The baker anchors cells with <see cref="ComputeStandableSurfaceZs"/> instead, which is
/// clearance-aware; this remains the direct MovementImpl equivalent for parity checks.
/// </summary>
public static int ComputeStandingZ(Map map, int x, int y, int locZ)
{

78
Projects/UOContent/Engines/Pathing/PathCacheCommands.cs
View file

@ -8,23 +8,23 @@ namespace Server.Engines.Pathing;
/// <summary>
/// Admin commands for inspecting and operating the pathfinding step cache.
/// [PathCacheStats — current resident-chunk count + hit/miss/eviction telemetry.
/// [PathCacheClear — drop all cached chunks, close lazy readers, zero counters.
/// [PathBake — walk a whole map building the full static cache, then save it.
/// [PathCacheSave — persist resident chunks per map to Data/Pathfinding/&lt;mapId&gt;.swb.
/// [PathCacheLoad — open those files as lazy backing stores. Also runs at startup.
/// [PathRecord — toggle JSONL telemetry capture for replay / benchmark corpora.
/// [PathCacheStats — resident-chunk count and hit/miss/eviction telemetry.
/// [PathCacheClear — drop all cached chunks, close the .swb readers, zero the counters.
/// [PathBake — build a map's full static cache and save it.
/// [PathCacheSave — persist the resident chunks to Data/Pathfinding/&lt;mapId&gt;.swb.
/// [PathCacheLoad — open those files as backing stores. Also runs at startup.
/// [PathRecord — toggle capture of pathfind telemetry.
///
/// The step cache works WITHOUT any .swb file — chunks build on demand as creatures path.
/// A baked .swb is an optional optimization that removes first-pathfind-after-boot latency
/// for shard owners who want it; <see cref="OnPathBake"/> is how you produce one.
/// None of this is required: the cache builds chunks on demand as creatures path, with or without
/// a .swb on disk. Baking one is purely an optimization that trades disk and a few minutes of bake
/// time for the removal of first-pathfind-after-boot latency.
/// </summary>
public static class PathCacheCommands
{
private static readonly ILogger logger = LogFactory.GetLogger(typeof(PathCacheCommands));
// modernuo.json flag: when true, Initialize() bakes any missing/stale .swb at startup.
// The first-boot ConfigurePrompts() prompt writes it.
// When set, Initialize() bakes any missing or stale .swb at startup. ConfigurePrompts() asks
// for it on first boot.
private const string PrebakeSetting = "pathfinding.prebakeMaps";
private static string PathFor(int mapId) =>
@ -32,9 +32,8 @@ public static class PathCacheCommands
public static void Configure()
{
// Resident-chunk cap is shard-tunable. Default 8192 ≈ 40 MB; small shards may
// want lower, large shards (or full-map bakes) may want higher. Setting is
// written back to server.cfg on first boot for discoverability.
// Resident-chunk cap, shard-tunable — the default works out to roughly 40 MB. Written back
// to server.cfg on first boot so it's discoverable.
StepCache.Instance.MaxResidentChunks = ServerConfiguration.GetOrUpdateSetting(
"pathfinding.maxResidentChunks",
8192
@ -52,13 +51,13 @@ public static class PathCacheCommands
}
/// <summary>
/// First-boot prompt, auto-invoked by <c>AssemblyHandler.Invoke("ConfigurePrompts")</c> in
/// the startup sequence — after assemblies load (so content can prompt) but before Serilog
/// starts, so the console prompt isn't interleaved with async log output. Offers to pre-bake
/// the pathfinding <c>.swb</c> cache for the selected maps; the answer persists in
/// modernuo.json (<see cref="PrebakeSetting"/>), so it's asked exactly once. Skipped when the
/// setting already exists or when input is redirected (headless/CI) — operators can set the
/// flag directly. The bake itself happens later in <see cref="Initialize"/>.
/// Asks, once, whether to pre-bake the .swb cache; <see cref="Initialize"/> does the work later.
/// The answer persists, so the question is never repeated, and it's skipped entirely when input
/// is redirected — a headless or CI boot sets <see cref="PrebakeSetting"/> directly instead.
///
/// Runs in the ConfigurePrompts phase because that's the one window where content can prompt:
/// assemblies are loaded, but Serilog hasn't started, so console output won't interleave with
/// async log writes.
/// </summary>
public static void ConfigurePrompts()
{
@ -81,16 +80,15 @@ public static class PathCacheCommands
}
/// <summary>
/// Auto-invoked by <c>AssemblyHandler.Invoke("Initialize")</c> after the tile matrix and
/// world are loaded. When <see cref="PrebakeSetting"/> is set, bakes any map whose
/// <c>.swb</c> is missing or stale, so the first pathfind on each region is already warm. A
/// fresh cache makes this a no-op, so only first boot — or a client/map update that changes
/// the fingerprint — pays the cost.
/// Bakes any map whose <c>.swb</c> is missing or stale, when <see cref="PrebakeSetting"/> is
/// set. Runs in the Initialize phase, once the tile matrix and world are loaded. An up-to-date
/// cache makes it a no-op, so the cost lands only on a first boot or after a client or map
/// update moves the fingerprint.
///
/// Validity is decided by <see cref="StepCache.HasLazyReader"/>: <see cref="Configure"/> runs
/// <see cref="AutoLoadAtStartup"/> in the earlier Configure phase, opening (and fingerprint-
/// validating) a reader for every up-to-date <c>.swb</c>. So a map with an open reader is
/// already good and we skip it — no need to recompute the fingerprint a second time here.
/// A map is judged up-to-date by whether it has an open reader. <see cref="AutoLoadAtStartup"/>
/// already ran in the earlier Configure phase and only opens a reader for a .swb whose
/// fingerprint validates, so an open reader is proof of a good bake — no need to fingerprint
/// the map a second time here.
/// </summary>
public static void Initialize()
{
@ -110,7 +108,7 @@ public static class PathCacheCommands
if (StepCache.Instance.HasLazyReader(map.MapID))
{
continue; // AutoLoadAtStartup already opened a fingerprint-valid .swb for this map
continue; // already has a fingerprint-valid .swb open
}
var path = PathFor(map.MapID);
@ -127,15 +125,14 @@ public static class PathCacheCommands
if (baked > 0)
{
logger.Information("PathBake: pre-bake complete ({Count} map(s) written).", baked);
AutoLoadAtStartup(); // (re)open the freshly written files as lazy backing stores
AutoLoadAtStartup(); // reopen what we just wrote
}
}
/// <summary>
/// Open Data/Pathfinding/&lt;mapId&gt;.swb as a lazy backing store for every map.
/// Reads only the header + chunk-offset index up front (~16 bytes per chunk);
/// individual chunk records are fetched on demand when the cache asks for them.
/// RAM stays bounded by MaxResidentChunks regardless of file size.
/// Opens Data/Pathfinding/&lt;mapId&gt;.swb as a backing store for every map. Only the header and
/// index are read up front; chunk records are fetched as the cache asks for them, so resident
/// memory stays bounded by the LRU cap however large the files are.
/// </summary>
private static void AutoLoadAtStartup()
{
@ -196,9 +193,9 @@ public static class PathCacheCommands
continue;
}
// BakeMap walks the whole map (building every chunk) and writes the .swb. The
// chunks are left resident afterward; drop them so peak memory is bounded to one
// map at a time and the post-command footprint returns to the LRU cap.
// BakeMap leaves every chunk it built resident. Drop them between maps so peak memory
// is one map's worth rather than all of them, and the footprint afterwards is back
// under the LRU cap.
var written = StepCache.Instance.BakeMap(map.MapID, PathFor(map.MapID));
StepCache.Instance.ClearResidentChunks();
@ -218,8 +215,7 @@ public static class PathCacheCommands
return;
}
// Reopen the freshly written files as lazy backing stores so they're usable now
// without a restart (resident memory stays bounded by the LRU cap).
// Reopen what we just wrote, so the bake is usable immediately without a restart.
AutoLoadAtStartup();
from.SendMessage($"PathBake: {totalChunks} chunks across {totalMaps} map(s) in {sw.Elapsed.TotalSeconds:F1}s; lazy readers reopened.");
}

44
Projects/UOContent/Engines/Pathing/PathDiag.cs
View file

@ -8,26 +8,19 @@ using Server.Targeting;
namespace Server.Engines.Pathing;
/// <summary>
/// Developer diagnostic for the bitmap A* step cache. Stand where a creature would start,
/// run <c>[PathDiag</c>, and target the goal. The detailed report is appended to
/// <c>Logs/pathdiag.log</c>; a short summary is sent to the invoking client. For the route
/// it records:
/// 1. the raw tile makeup of the start and goal cells (land + statics) and the
/// clearance-aware standable surfaces the baker anchors to — the ground truth for
/// "why does the cache (not) serve this cell";
/// 2. one warm <see cref="StepCache.TryGetMask"/>-served Find with the per-pathfind cache
/// hit/fallthrough breakdown and fallthrough fraction — a high fallthrough fraction
/// means the cache isn't helping the route (it pays the lookup then uses the slow path);
/// 3. warm timing over many iterations.
/// Diagnoses why the step cache does or doesn't serve a given route. Stand where the creature
/// would start, run <c>[PathDiag</c>, target the goal; the full report lands in
/// <c>Logs/pathdiag.log</c> and a summary goes to the client. It reports the tile makeup of the
/// start and goal cells alongside the standable surfaces the baker anchors to, the cache
/// hit/fallthrough breakdown for one warm Find, and warm timings.
///
/// Primarily useful when bringing up custom maps / facets: it shows whether static-over-land
/// geometry (dungeon walkways, bridges, stairs, raised foundations, stacked floors) is being
/// baked at the right Z.
/// The fallthrough fraction is the number to read: a high one means the cache is paying for a
/// lookup on every cell and then taking the slow path anyway. That usually points at
/// static-over-land geometry — dungeon walkways, bridges, stairs, stacked floors — baking at the
/// wrong Z, which is why this is most useful when bringing up a custom map or facet.
///
/// Output goes to a log file rather than the console because the live server uses Serilog and
/// raw Console writes interleave badly with it. The promotion gate is forced to eager
/// (threshold 1) for the duration so the cache builds on first touch and the numbers reflect
/// its best case; the previous threshold is restored afterward.
/// The promotion gate is forced eager for the duration, so the numbers reflect the cache's best
/// case rather than an artifact of chunks not having been built yet.
/// </summary>
public static class PathDiag
{
@ -67,7 +60,7 @@ public static class PathDiag
var cache = StepCache.Instance;
var previousThreshold = cache.MissPromotionThreshold;
cache.MissPromotionThreshold = 1; // eager build — measure the cache's best case
cache.MissPromotionThreshold = 1; // build eagerly, so we measure the cache's best case
StreamWriter log = null;
try
@ -99,9 +92,8 @@ public static class PathDiag
}
/// <summary>
/// Writes the raw tile makeup of one cell plus the surfaces the baker anchors to. A large
/// gap between the query Z and the standable surfaces is the signature of a route the
/// cache can't serve (the creature stands on a static surface far from the land average).
/// Dumps one cell's tiles and the surfaces the baker anchors to. A wide gap between the query Z
/// and every standable surface is the signature of a cell the cache can't serve.
/// </summary>
private static void DumpCell(TextWriter log, Map map, int x, int y, int queryZ, string label)
{
@ -133,9 +125,8 @@ public static class PathDiag
}
/// <summary>
/// Runs one warm Find and records the StepCache counter delta for it — the per-pathfind
/// cache hit/fallthrough mix and the fallthrough fraction. Returns a summary for the
/// caller to relay to the player.
/// Runs one Find against a warm cache and reports the counter delta it produced — the
/// hit/fallthrough mix for that single pathfind.
/// </summary>
private static (string result, double fallthroughPct, long total) RunInstrumentedFind(
TextWriter log, Mobile from, Map map, Point3D start, Point3D goal
@ -143,7 +134,8 @@ public static class PathDiag
{
var cache = StepCache.Instance;
// Warm every chunk the route touches before measuring.
// Build every chunk the route touches first, so the measured Find below reports steady-state
// behaviour rather than first-touch misses.
for (var i = 0; i < 3; i++)
{
BitmapAStarAlgorithm.Instance.Find(from, map, start, goal);

45
Projects/UOContent/Engines/Pathing/PathfindRecorder.cs
View file

@ -7,25 +7,14 @@ using Server.Text;
namespace Server.Engines.Pathing;
/// <summary>
/// Admin-toggled telemetry: appends a JSONL line per pathfind request to a file.
/// One record per BitmapAStarAlgorithm.Find call, capturing the inputs (start, goal,
/// map, capability flags) needed to replay the scenario in benchmarks. Output format
/// matches the corpus the BDN harness consumes.
/// Appends one JSONL record per pathfind, capturing the inputs — start, goal, map, capability
/// flags — needed to replay it later in a benchmark. Toggled at runtime with [PathRecord;
/// <see cref="Configure"/> only seeds the initial state from server.cfg.
///
/// Hot-toggleable at runtime via the [PathRecord admin command — no restart needed.
/// <see cref="Configure"/> only seeds the initial state from server.cfg
/// (pathfinding.recorder.enable, default false).
///
/// Holds a single StreamWriter open while recording; its internal buffer absorbs
/// per-record writes without per-call File.Open / File.Append. Each record is built
/// in a stack-allocated ValueStringBuilder (zero per-int allocation for the field
/// formatting), then handed to the writer as a ReadOnlySpan&lt;char&gt;.
///
/// <b>Workload note:</b> intended for short bursts of capture (turn on, walk a region
/// or trigger a scenario, turn off). On a busy server with hundreds of pathfinds
/// per second, sustained recording can saturate the StreamWriter's 4 KB buffer and
/// block the game thread on disk writes. A backpressure-aware async sink is a
/// future enhancement if 24/7 capture becomes a use case.
/// Meant for short bursts: turn it on, walk the region or trigger the scenario, turn it off. The
/// writes go through a StreamWriter's buffer on the game thread, so a busy shard doing hundreds of
/// pathfinds a second can saturate that buffer and stall the loop on disk I/O. Sustained capture
/// would need an async sink with backpressure.
/// </summary>
public static class PathfindRecorder
{
@ -52,9 +41,8 @@ public static class PathfindRecorder
}
/// <summary>
/// Toggle recording. When enabling, opens an append-mode StreamWriter; when
/// disabling, flushes + disposes it. Idempotent — calling twice with the same
/// state is a no-op.
/// Toggles recording, opening the file on enable and flushing and closing it on disable.
/// Idempotent.
/// </summary>
public static void SetEnabled(bool enabled)
{
@ -98,9 +86,8 @@ public static class PathfindRecorder
}
/// <summary>
/// Force a flush of the writer's internal buffer to disk. Safe to call when
/// disabled (no-op). Useful after a burst of recording when an admin wants to
/// inspect the file without waiting for buffer fill or disable.
/// Pushes the writer's buffer to disk, so a capture can be inspected without disabling first.
/// No-op when disabled.
/// </summary>
public static void Flush()
{
@ -115,9 +102,8 @@ public static class PathfindRecorder
}
/// <summary>
/// Capture one Find call. Hot path: cheap when disabled (single bool check).
/// When enabled, formats one JSONL line and writes it through the StreamWriter's
/// internal buffer — flush is amortized across many calls.
/// Records one Find. Sits on the pathfinding hot path, so it costs a single bool check when
/// disabled.
/// </summary>
public static void RecordIfEnabled(Mobile m, Map map, Point3D start, Point3D goal)
{
@ -140,9 +126,8 @@ public static class PathfindRecorder
try
{
// One interpolation handles every numeric field with no per-int ToString
// allocation; bool fields use explicit literal spans because JSON wants
// lowercase "true"/"false" and bool.ToString() yields "True"/"False".
// One interpolation covers every numeric field without a per-field ToString. The bools
// are appended as literals because JSON wants lowercase and bool.ToString() capitalizes.
using var vsb = ValueStringBuilder.Create(192);
vsb.Append(
$"{{\"Name\":\"recorded\",\"MapId\":{map.MapID},\"StartX\":{start.X},\"StartY\":{start.Y},\"StartZ\":{start.Z},\"GoalX\":{goal.X},\"GoalY\":{goal.Y},\"GoalZ\":{goal.Z},\"CanSwim\":"