warrentyexpired/ModernUO
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(pathing): rewrite the comments across the pathing engine for publication

Browse source 
The pathing and step-cache comments had accumulated as development notes rather
than documentation: internal phase jargon ("Tier 4", "the Phase-2 synthesizer"),
change narration aimed at a reviewer ("which the old ComputeStandingZ anchor
missed", "legacy behavior"), benchmark anecdotes ("benchmarked as near-optimal",
"a ~20 ns lookup"), and multi-paragraph blocks restating what the code says.

Rewritten to keep the rationale a reader cannot derive from the code - why the
source-Z guard cannot be loosened, why multis fall through, why the promotion gate
counts Finds instead of calls, why the fingerprint hashes files rather than the
live tile tables - and to drop the history that got us there.

Also corrects comments that had gone stale:

- CacheEvictionTimer and CacheStats documented a class named
  StaticWalkabilityCache, which no longer exists; it is StepCache.
- StepCacheFile's header said "File layout v8" while FormatVersion is 9, and the
  body called the current record layout "the v6 layout" throughout. The layout
  descriptions are now unversioned, since they describe whatever FormatVersion
  currently is.
- StepProbe.ComputeStandingZ claimed StepCache uses it to bake SourceZ. It hasn't
  since the baker moved to the clearance-aware ComputeStandableSurfaceZs; only a
  parity test calls it now.

Two small code changes came along with the comment work, both behavior-preserving:
_lazyReaders now uses a collection expression like its neighbours, and
TryLoadFromLazyReader collapses to an expression body once its inline comment moved
to the doc comment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kamron Batman 2026-07-12 19:33:58 -07:00
parent 6149b4ef21
commit 4bbc4589e7
No known key found for this signature in database
GPG key ID: 7D81DF26D9A5D94A
13 changed files with 494 additions and 637 deletions
Download patch file Download diff file Expand all files Collapse all files

140
Projects/UOContent/Engines/Pathing/BitmapAStarAlgorithm.cs
View file

@ -26,12 +26,11 @@ using MoveImpl = Server.Movement.MovementImpl;
namespace Server.PathAlgorithms;
/// <summary>
/// A* pathfinder with a single bitmap-cache lookup per cell expansion. Default walkers
/// take one <see cref="StepCache.TryGetMask"/> call returning the 8-direction
/// mask + per-direction Z. Non-default walkers (non-GM players, creatures with swim/fly/
/// door/clip capabilities) and per-cell cache fallthroughs route through
/// <see cref="GetSuccessorsSlowPath"/>, which runs the per-direction
/// <see cref="CalcMoves.CheckMovement"/> loop for that one cell.
/// A* pathfinder that expands a cell with a single <see cref="StepCache.TryGetMask"/> lookup,
/// which returns all 8 directions' walkability and destination Zs at once. Where the cache can't
/// answer — a fallthrough on that cell, or a flying creature the static cache can't model —
/// <see cref="GetSuccessorsSlowPath"/> runs the per-direction <see cref="CalcMoves.CheckMovement"/>
/// loop for that one cell instead, so a partial cache miss costs only the cells it affects.
/// </summary>
public class BitmapAStarAlgorithm : PathAlgorithm
{
@ -50,61 +49,50 @@ public class BitmapAStarAlgorithm : PathAlgorithm
private const int PlaneOffset = 128;
private const int PlaneCount = 13;
private const int PlaneHeight = 20;
// Default shared singleton (MaxSearchNodes = 1000, set from config in Configure). Typed
// as the concrete class so Configure can set its instance config; assignable anywhere a
// PathAlgorithm is expected. Specialized variants are just additional instances.
// The shared default. A differently-configured variant is just another instance.
public static readonly BitmapAStarAlgorithm Instance = new();
// Scratch buffers — reused across every Find on THIS instance. Per-instance (not static)
// so independently-configured algorithms don't share state. ~320 KB per instance; create
// specialized instances once (static readonly), never per-call. Safe to reuse per Find
// because the game loop is single-threaded and Find is never re-entered.
// Scratch reused across every Find on this instance — roughly 320 KB of it, so create
// instances once and hold them, never per call. Per-instance rather than static so two
// differently-configured algorithms don't share state. Reuse is safe because the game loop is
// single-threaded and Find never re-enters.
private readonly Direction[] _path = new Direction[AreaSize * AreaSize];
private readonly PathNode[] _nodes = new PathNode[NodeCount];
private readonly byte[] _nodeStates = new byte[NodeCount];
private readonly int[] _successors = new int[8];
private readonly PriorityQueue<int, int> _openQueue = new();
// A* node-expansion budget: the search bails (returning null) after this many node
// expansions. Benchmarked as near-optimal: above the ~500 needed to solve walled-off
// indoor routes, below the ~1500 window-exhaustion cost ceiling where a failed
// (unreachable) search's worst-case cost spikes for no solving benefit. Successful
// searches terminate on goal-found, so this never touches the common open-terrain case.
// Per-instance so specialized algorithms (e.g. a wider-budget variant for special NPCs)
// can coexist; the shared default lives on Instance and is set from config in Configure.
// Expansion budget: the search gives up and returns null past this many nodes. It bounds the
// cost of an unreachable goal, which would otherwise exhaust the whole search window. A
// successful search stops when it finds the goal, so the budget only binds on hard or hopeless
// routes — it needs to stay high enough to solve walled-off indoor ones.
public int MaxSearchNodes { get; set; } = 1000;
private int _xOffset;
private int _yOffset;
// When set, GetSuccessors delegates to the per-cell slow path on every expansion
// (creature has CanFly — Z-jumping is beyond the cache's static-only scope).
// Every expansion goes to the slow path: the creature can fly, and arbitrary Z-jumping is
// outside what a static cache can model.
private bool _currentMobileNeedsSlowPath;
// When set, diagonal corner-cut uses the strict AND-rule (BOTH cardinal partners
// must be walkable) instead of the lenient creature OR-rule. Cache still applies —
// partner bits live in the same source-cell mask byte. Non-GM players only.
// Diagonal corner-cut uses the strict rule — both cardinal partners walkable, not just one.
// Non-GM players only. The cache still applies; the partner bits are in the same mask byte.
private bool _currentMobilePlayerStrict;
// Capability overlay applied to cache results. Layered each cell:
// Capability overlay on the cache's two rule sets, applied per cell as
// effective = (walkMask & !cantWalk) | (wetMask & canSwim)
// Reset at end of Find.
private bool _currentMobileCanSwim;
private bool _currentMobileCantWalk;
// Dynamic-obstacle pass capability flags (per-mobile, captured in Find).
// Mirrors MovementImpl.Check's per-mobile derivations so per-cell items/mobiles
// checks can be evaluated without re-deriving.
// Per-mobile flags for the dynamic-obstacle pass, derived once in Find rather than per cell.
private bool _currentMobileIgnoreDoors;
private bool _currentMobileIgnoreSpellFields;
private bool _currentMobileIgnoreMovableImpassables;
public static void Configure()
{
// A* node-expansion budget. Default 1000 is benchmarked near-optimal (see
// MaxSearchNodes). Applied to the shared singleton; specialized instances pass their
// own value. Written back to server.cfg on first boot. Auto-invoked at startup via
// AssemblyHandler.Invoke("Configure").
// Shard-tunable expansion budget for the shared instance; see MaxSearchNodes. Written back
// to server.cfg on first boot so it's discoverable.
Instance.MaxSearchNodes = ServerConfiguration.GetOrUpdateSetting(
"pathfinding.maxSearchNodes",
1000
@ -136,10 +124,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm
return null;
}
// Mark a new Find generation so the StepCache promotion gate counts THIS pathfind
// as one touch per chunk regardless of how many times the expansion frontier
// probes a given chunk. Without this, A* hits each visited chunk dozens of times
// and trips the threshold immediately.
// The frontier probes a given chunk dozens of times over one search; opening a generation
// is what makes the cache's promotion gate count all of that as a single touch.
StepCache.Instance.BeginFindGeneration();
PathfindRecorder.RecordIfEnabled(m, map, start, goal);
@ -161,7 +147,7 @@ public class BitmapAStarAlgorithm : PathAlgorithm
_currentMobileIgnoreMovableImpassables = false;
}
// Mirrors MovementImpl: dead/spectral mobiles also ignore doors.
// Dead and spectral mobiles pass through doors too. Mirrors MovementImpl.
_currentMobileIgnoreDoors |= !m.Alive || m.Body.BodyID == 0x3DB || m.IsDeadBondedPet;
_currentMobileIgnoreSpellFields = m is PlayerMobile && map != Map.Felucca;
@ -209,7 +195,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm
_nodeStates[bestNode] = 2;
// Set MovementImpl globals so per-cell slow-path fallthroughs see the right state.
// MovementImpl reads these statics, so a slow-path fallthrough on any cell below needs
// them set for this mobile.
if (bc != null)
{
MoveImpl.AlwaysIgnoreDoors = bc.CanOpenDoors;
@ -326,11 +313,10 @@ public class BitmapAStarAlgorithm : PathAlgorithm
}
/// <summary>
/// One <see cref="StepCache.TryGetMask"/> call returns the 8-direction
/// walkable mask + destination Zs. Diagonal corner-cut applies the lenient creature
/// OR-rule using partner bits in the same mask byte — no neighbor-chunk lookup needed.
/// On cache fallthrough or for non-default walkers, defers to
/// <see cref="GetSuccessorsSlowPath"/> for THIS cell only.
/// Expands one cell into its walkable neighbours. A single cache lookup covers all 8
/// directions, including the partner bits the diagonal corner-cut needs, so no neighbouring
/// cell has to be consulted. Falls back to <see cref="GetSuccessorsSlowPath"/> for this cell
/// alone when the cache can't answer.
/// </summary>
private int GetSuccessors(int p, Mobile m, Map map)
{
@ -353,16 +339,12 @@ public class BitmapAStarAlgorithm : PathAlgorithm
if (!lookup.IsHit)
{
// Multi-covered cells: synthesize a multi-aware mask in ONE pass (over land + statics +
// house/boat component tiles) instead of the slow path's 8x per-cell CheckMovement.
// Fliers and cache-off already returned at the top of GetSuccessors, so this only runs
// for cacheable walkers/swimmers. The synthesized mask flows through the SAME
// capability-overlay + diagonal corner-cut + dynamic-obstacle loop below as a static hit.
// A multi-covered cell still gets a whole-cell mask, synthesized over the house or boat
// components rather than looked up. That keeps it out of the slow path's 8 separate
// CheckMovement calls, and the result flows through the same overlay, corner-cut and
// dynamic-obstacle logic below as a cache hit would.
if (lookup.HitKind == CacheHitKind.Fallthrough_Multi)
{
// Multi-covered cell: the per-multiID interior cache serves a ~20 ns lookup for
// interior cells (and records the right counter); it falls back internally to the
// Phase-2 live synthesizer for perimeter / terrain-dirty / foundation cells.
lookup = MultiMaskCache.Instance.GetMask(map, p3D.X, p3D.Y, (sbyte)p3D.Z);
}
else
@ -371,8 +353,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm
}
}
// Capability overlay: walking allowed unless cantWalk; swimming allowed if canSwim.
// Partner bits used for diagonal corner-cut also use the effective mask.
// Overlay the mobile's capabilities onto the cache's two rule sets. The corner-cut below
// reads its partner bits from this effective mask, not the raw one.
var walkBits = _currentMobileCantWalk ? (byte)0 : lookup.WalkMask;
var swimBits = _currentMobileCanSwim ? lookup.WetMask : (byte)0;
var mask = (byte)(walkBits | swimBits);
@ -393,9 +375,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm
continue;
}
// Diagonal corner-cut. Creatures (default): OR-rule — at least one cardinal
// partner walkable. Non-GM players: AND-rule — BOTH partners must be walkable.
// Partner bits live in the same source-cell mask byte either way.
// Diagonal corner-cut: a creature needs at least one of the two flanking cardinals to
// be walkable, a non-GM player needs both.
if ((i & 1) == 1)
{
var leftBit = 1 << ((i - 1) & 0x7);
@ -408,9 +389,9 @@ public class BitmapAStarAlgorithm : PathAlgorithm
}
}
// Walking takes precedence over swimming when both apply (matches MovementImpl's
// surface-selection: closest-to-startZ wins, and walk surface is always closer
// when the creature is currently standing on land).
// Walking wins over swimming where both are possible. MovementImpl picks the surface
// closest to the start Z, and for a creature standing on land that is always the walk
// surface.
var useWalkZ = (walkBits & (1 << i)) != 0;
var z = useWalkZ
? i switch
@ -441,8 +422,8 @@ public class BitmapAStarAlgorithm : PathAlgorithm
var absX = x + _xOffset;
var absY = y + _yOffset;
// Dynamic-obstacle pass: items + mobiles at the target cell. Cache only
// covers static walkability; dynamic state has to be checked at query time.
// The cache only knows static terrain, so items and mobiles at the target cell have to
// be checked live.
if (IsBlockedByDynamic(m, map, absX, absY, z))
{
continue;
@ -464,11 +445,9 @@ public class BitmapAStarAlgorithm : PathAlgorithm
private const int MobileHeight = 15;
/// <summary>
/// Mirrors MovementImpl's dynamic-item / mobile collision phase for a target cell.
/// Items: ImpassableSurface that overlap (z, z+PersonHeight), respecting capability
/// overrides (CanOpenDoors → ignore door items; CanMoveOverObstacles → ignore movables;
/// non-Felucca players → ignore spell fields). Mobiles: any other mobile whose Z range
/// overlaps and which we can't move over.
/// MovementImpl's item and mobile collision phase for one target cell: impassable items
/// overlapping the mobile's vertical envelope block it, subject to the capability overrides,
/// as does any other mobile it can't move over.
/// </summary>
private bool IsBlockedByDynamic(Mobile m, Map map, int x, int y, int z)
{
@ -509,9 +488,9 @@ public class BitmapAStarAlgorithm : PathAlgorithm
}
}
// A* must be able to plan a path to the goal cell even when the target mobile is
// standing on it (the follower stops within range short of it). Skip the mob-block
// check at the goal cell ONLY; everywhere else dynamic mobiles still block.
// The goal cell is usually occupied by whatever the mobile is chasing, so blocking on it
// would fail every pursuit. The follower stops short of the goal anyway. Every other cell
// still blocks on mobiles.
var skipMobCheck = x == MoveImpl.Goal.X && y == MoveImpl.Goal.Y;
if (!skipMobCheck)
@ -534,18 +513,16 @@ public class BitmapAStarAlgorithm : PathAlgorithm
}
/// <summary>
/// Mirrors MovementImpl.CanMoveOver — true when m can step onto t's cell (dead bodies,
/// hidden staff, etc.).
/// True when m can step onto t's cell — a corpse, hidden staff, and so on. Mirrors
/// MovementImpl.CanMoveOver.
/// </summary>
private static bool CanMoveOver(Mobile m, Mobile t) =>
!t.Alive || !m.Alive || t.IsDeadBondedPet || m.IsDeadBondedPet
|| t.Hidden && t.AccessLevel > AccessLevel.Player;
/// <summary>
/// Per-direction <see cref="CalcMoves.CheckMovement"/> loop for a single source cell.
/// Runs on cache fallthrough or when <see cref="_currentMobileNeedsSlowPath"/> is set.
/// CheckMovement validates land/statics/items via MovementImpl; dynamic mobile blocking
/// is layered on top because MovementImpl doesn't iterate same-cell mobiles.
/// Expands one cell the long way, with a CheckMovement call per direction. The same-cell
/// mobile check is layered on top because MovementImpl doesn't iterate those.
/// </summary>
private int GetSuccessorsSlowPath(Mobile m, Map map, int px, int py, Point3D p3D, int[] vals)
{
@ -586,11 +563,10 @@ public class BitmapAStarAlgorithm : PathAlgorithm
}
/// <summary>
/// True for creatures whose movement rules the static cache can't model. Currently
/// only CanFly — flying creatures Z-jump arbitrarily and the cache's source-Z guard
/// would over-fire. CanSwim / CantWalk are handled via the capability overlay (walkMask
/// + wetMask). CanOpenDoors / CanMoveOverObstacles only affect dynamic items and don't
/// disqualify the cache.
/// True for creatures the static cache can't model at all. Only flying ones qualify: they
/// Z-jump freely, so the cache's source-Z guard would reject nearly every cell anyway. Swim
/// and cant-walk are handled by the capability overlay, and the door / obstacle capabilities
/// only affect dynamic items, so none of those disqualify the cache.
/// </summary>
private static bool RequiresSlowPath(Mobile m) => m is BaseCreature bc && bc.CanFly;
}

5
Projects/UOContent/Engines/Pathing/Cache/CacheEvictionTimer.cs
View file

@ -3,9 +3,8 @@ using System;
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Periodic backstop that enforces the StaticWalkabilityCache resident-chunk cap.
/// Steady-state cost is a single early-return; only fires real work when the cache
/// has overflowed MaxResidentChunks. Runs on the game thread; no locking required.
/// Periodic backstop that enforces <see cref="StepCache"/>'s resident-chunk cap. Costs a
/// single early-return unless the cache has overflowed MaxResidentChunks.
/// </summary>
public class CacheEvictionTimer : Timer
{

18
Projects/UOContent/Engines/Pathing/Cache/CacheHitKind.cs
View file

@ -1,18 +1,18 @@
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Outcome categories for StepCache.TryGetMask. Used for telemetry and to drive
/// the slow-path fallthrough decision in callers. Ordering is load-bearing:
/// values 0-2 are hits, values 3+ are fallthroughs (see StepMask.IsHit).
/// Outcome of <see cref="StepCache.TryGetMask"/>. Drives both telemetry and the caller's
/// decision to fall back to the slow path. Ordering is load-bearing: 0-2 are usable answers,
/// 3+ are fallthroughs, and <see cref="StepMask.IsHit"/> tests that boundary.
/// </summary>
public enum CacheHitKind : byte
{
Hit = 0, // clean default-walker answer from the resident chunk
Hit = 0, // served from the resident chunk
Miss_NotBuilt = 1, // chunk wasn't resident; built and returned
Miss_DirtyRebuild = 2, // version mismatch; rebuilt and returned
Fallthrough_MultiZ = 3, // cell has multiple walkable surfaces; caller must use slow path
Miss_DirtyRebuild = 2, // chunk was stale; rebuilt and returned
Fallthrough_MultiZ = 3, // stacked walkable surfaces, none matching the query Z
Fallthrough_OffMap = 4, // out of bounds
Fallthrough_SourceZMismatch = 5, // |loc.Z - BakedSourceZ| > StepHeight; cache answer would diverge
Fallthrough_NotBuilt = 6, // first-touch miss without lazy file hit; build deferred until second touch
Fallthrough_Multi = 7, // a multi (house/boat) covers this cell or its halo; use the live path
Fallthrough_SourceZMismatch = 5, // |query Z - baked SourceZ| > StepHeight; a cached answer would diverge
Fallthrough_NotBuilt = 6, // first touch of an unbuilt chunk; the promotion gate defers the build
Fallthrough_Multi = 7, // a multi (house/boat) covers this cell or its halo
}

5
Projects/UOContent/Engines/Pathing/Cache/CacheStats.cs
View file

@ -1,8 +1,9 @@
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Snapshot of StaticWalkabilityCache counters. Returned by GetStats() and consumed
/// by the [PathCacheStats admin command. All counters are monotonic except ResidentChunks.
/// Snapshot of <see cref="StepCache"/>'s counters, as returned by GetStats() and reported by
/// the [PathCacheStats command. Every counter is monotonic except ResidentChunks, and all of
/// them reset on Clear() — they count since the last clear, not since startup.
/// </summary>
public readonly struct CacheStats(
int residentChunks,

84
Projects/UOContent/Engines/Pathing/Cache/MultiMaskCache.cs
View file

@ -6,13 +6,19 @@ using Server.Multis;
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Warm, in-memory cache of per-multiID local-frame walkability masks for INTERIOR multi cells
/// (cell + all 8 neighbours covered by the multi → terrain-neighbour-free → position-invariant).
/// Wraps the Phase-2 synthesizer (StepProbe.ComputeMultiMaskAt). Cleanliness is decided ONCE per
/// instance (BaseMulti.PathInteriorCacheState, via ComputeFootprintClean): a clean instance — whole
/// footprint terrain below the floor — serves interior cells from the shared per-multiID cache;
/// dirty instances, boats (movers), and HouseFoundation (runtime-mutable) fall back to live-synth.
/// Keyed by multiID &amp; 0x3FFF.
/// Caches walkability masks for the interior cells of a multi, shared across every instance of the
/// same multiID.
///
/// An interior cell — one whose 8 neighbours are all covered by the multi — has no terrain
/// neighbour, so its mask depends only on the multi's own component tiles and is identical at every
/// position the design is placed. That makes it cacheable in the multi's local frame and reusable
/// across instances; perimeter cells are not, and fall back to <see cref="StepProbe.ComputeMultiMaskAt"/>.
///
/// The catch is terrain intruding into the multi's floor envelope, which would make a cell's mask
/// position-dependent after all. <see cref="ComputeFootprintClean"/> rules that out per instance,
/// once: if the whole footprint's terrain sits below the lowest floor, no interior cell can see it.
/// A dirty instance, or a <see cref="HouseFoundation"/> (whose design mutates at runtime), always
/// synthesizes live rather than risk serving a wrong mask.
/// </summary>
public sealed class MultiMaskCache
{
@ -25,26 +31,20 @@ public sealed class MultiMaskCache
public void Clear() => _byMultiId.Clear();
/// <summary>
/// Returns the multi-aware StepMask for a covered cell (x,y,sourceZ). Serves a cached interior
/// mask when available and the guards pass (counted as a MultiMaskCacheHit); otherwise falls
/// back to the Phase-2 live synthesizer ComputeMultiMaskAt (counted as a MultiLocalHit), caching
/// the result if the cell is interior and clean. Always returns a usable mask (HitKind == Hit).
/// The multi-aware mask for a covered cell, always usable (HitKind is always Hit). Served from
/// the shared cache when the cell is a clean interior one, synthesized live otherwise.
/// </summary>
public StepMask GetMask(Map map, int x, int y, sbyte sourceZ)
{
if (!TryResolveCoveringMulti(map, x, y, out var multi, out var lx, out var ly)
|| multi is HouseFoundation) // runtime-mutable per-instance DesignState MCL
|| multi is HouseFoundation) // its DesignState MCL changes at runtime
{
return LiveSynth(map, x, y, sourceZ);
}
// Boats are cached too: their per-multiID deck masks are movement-invariant (built once per
// heading), and the per-instance clean gate below + the ItemID/location/map resets keep a
// moving/turning boat correct. Narrow boats have little interior; wide galleons gain a lot.
// Boats are cached despite moving: a deck mask is built in the local frame and is invariant
// under translation, and the clean gate plus the ItemID/location/map resets cover turning.
// Per-instance footprint cleanliness (computed once, stored on the multi; reset on move).
// Clean ⇒ no terrain intrusion anywhere in the footprint ⇒ interior cells are exact from the
// shared per-multiID cache. Dirty ⇒ degrade to the live synthesizer (never serve a wrong mask).
if (multi.PathInteriorCacheState == MultiInteriorCacheState.Unknown)
{
multi.PathInteriorCacheState =
@ -62,7 +62,7 @@ public sealed class MultiMaskCache
if (state == MultiLocalMask.CellState.Cached)
{
// Footprint is clean, so only the source-Z match matters (terrain can't intrude).
// A clean footprint rules terrain out, so the source-Z match is the only guard left.
var worldFloorZ = local.FloorZAt(lx, ly) + multi.Z;
if (Math.Abs(sourceZ - worldFloorZ) <= StepHeight)
{
@ -78,8 +78,7 @@ public sealed class MultiMaskCache
return LiveSynth(map, x, y, sourceZ);
}
// Unknown → classify + (if interior) build & cache. No per-cell terrain guard needed: the
// instance is clean, so every interior cell's 3x3 terrain is below the floor.
// First touch of this cell: synthesize, then classify and cache if it's interior.
var mask = LiveSynth(map, x, y, sourceZ);
if (IsInteriorLocalCell(mcl, lx, ly)
&& TryToLocalZ(mask, multi.Z, out var localMask)
@ -113,8 +112,7 @@ public sealed class MultiMaskCache
}
/// <summary>
/// Finds the multi covering (x,y) and the local cell indices into its MCL. Mirrors
/// Map.StaticTileEnumerator / BaseMulti.Contains. Returns false if no multi covers the cell.
/// Finds the multi covering (x,y) and the cell's indices into its MCL, or false if none does.
/// </summary>
public static bool TryResolveCoveringMulti(Map map, int x, int y, out BaseMulti multi, out int lx, out int ly)
{
@ -138,9 +136,9 @@ public sealed class MultiMaskCache
}
/// <summary>
/// True iff local cell (lx,ly) and all 8 neighbours are covered by the multi (have MCL tiles).
/// Such a cell's 8-direction transition is fully determined by the multi (no terrain neighbour),
/// so its mask is position-invariant. A pure function of the MCL.
/// True when (lx,ly) and all 8 of its neighbours carry MCL tiles. Such a cell has no terrain
/// neighbour, so the multi alone determines its transitions and its mask is position-invariant.
/// A pure function of the MCL.
/// </summary>
public static bool IsInteriorLocalCell(MultiComponentList mcl, int lx, int ly)
{
@ -161,9 +159,9 @@ public sealed class MultiMaskCache
}
/// <summary>
/// Converts a world-frame mask's per-direction Zs to local Z (subtract multiZ). Returns false
/// if any local Z doesn't fit sbyte (caller must then NOT cache the cell — rare; only when
/// |multiZ| is large enough to push a world Z out of range). Mask (walk/wet) bits are copied.
/// Rebases a world-frame mask's per-direction Zs into the multi's local frame. Returns false
/// when a local Z overflows sbyte — only reachable at extreme |multiZ| — and the caller must
/// then leave the cell uncached.
/// </summary>
public static bool TryToLocalZ(StepMask world, int multiZ, out StepMask local)
{
@ -191,9 +189,8 @@ public sealed class MultiMaskCache
}
/// <summary>
/// True iff all terrain (land + statics) at (x,y) sits strictly below <paramref name="floorZ"/>,
/// so a creature standing on the multi floor never sees terrain in its envelope and the cached
/// (terrain-free) mask is exact. Cheap: one land-top read + the cell's static-tile array scan.
/// True when all terrain (land + statics) at (x,y) sits strictly below <paramref name="floorZ"/>,
/// so a creature standing on the multi's floor never sees terrain in its envelope.
/// </summary>
public static bool TerrainTopBelow(Map map, int x, int y, sbyte floorZ)
{
@ -216,7 +213,7 @@ public sealed class MultiMaskCache
return true;
}
/// <summary>Highest terrain (land + statics) top at (x,y). Building block for the cleanliness check.</summary>
/// <summary>Highest terrain (land + statics) top at (x,y).</summary>
public static int TerrainTop(Map map, int x, int y)
{
map.GetAverageZ(x, y, out _, out _, out var top);
@ -234,10 +231,10 @@ public sealed class MultiMaskCache
}
/// <summary>
/// True iff the multi's WHOLE footprint terrain sits below its lowest standable floor — i.e.
/// maxTerrain &lt; minFloor over all covered cells. When true, no covered cell's terrain (nor any
/// neighbour's) can intrude into a creature's floor envelope, so interior cells of this design are
/// safe to serve from the shared per-multiID cache for THIS instance. One-time per instance.
/// True when the multi's entire footprint terrain sits below its lowest standable floor. No
/// covered cell's terrain — nor any neighbour's — can then intrude into a creature's floor
/// envelope, which is what makes this instance's interior cells safe to serve from the shared
/// per-multiID cache. Evaluated once per instance and cached on the multi.
/// </summary>
public static bool ComputeFootprintClean(Map map, BaseMulti multi)
{
@ -252,7 +249,7 @@ public sealed class MultiMaskCache
var col = mcl.Tiles[lx][ly];
if (col.Length == 0)
{
continue; // uncovered local cell
continue;
}
foreach (var tile in col)
@ -278,7 +275,7 @@ public sealed class MultiMaskCache
if (minFloorLocal == int.MaxValue)
{
return false; // no standable floor anywhere → don't cache (defensive)
return false; // no standable floor anywhere; refuse to cache rather than guess
}
return maxTerrain < minFloorLocal + multi.Z;
@ -304,8 +301,9 @@ public sealed class MultiMaskCache
}
/// <summary>
/// Per-multiID lazily-filled grid of interior-cell masks. Cell state: Unknown (not yet classified),
/// Cached (interior + clean → mask valid), NonInterior (perimeter/edge/terrain-dirty → live-synth).
/// One multiID's grid of interior-cell masks, filled in as cells are first touched. A cell is
/// Unknown until classified, then either Cached (interior — the mask is valid) or NonInterior
/// (perimeter — synthesize live).
/// </summary>
internal sealed class MultiLocalMask
{
@ -314,8 +312,8 @@ internal sealed class MultiLocalMask
private readonly int _width;
private readonly int _height;
private readonly CellState[] _state;
private readonly StepMask[] _mask; // local-Z mask, valid when state == Cached
private readonly sbyte[] _floorZ; // local floor Z, valid when state == Cached
private readonly StepMask[] _mask; // local-frame mask, valid only when state == Cached
private readonly sbyte[] _floorZ; // local floor Z, valid only when state == Cached
public MultiLocalMask(int width, int height)
{

322
Projects/UOContent/Engines/Pathing/Cache/StepCache.cs
View file

@ -10,13 +10,15 @@ using Server.Logging;
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Singleton store of per-chunk static walkability data. Chunks correspond to
/// Map.SectorSize = 16; key encoding packs (mapId, chunkX, chunkY) into a long.
/// Lazily built on first query; invalidated by version-check vs Sector.MultisVersion;
/// memory bounded by MaxResidentChunks via probabilistic LRU eviction.
/// Singleton store of static walkability, keyed by 16x16 chunk (one per map sector). Chunks build
/// on demand and memory stays bounded by MaxResidentChunks through probabilistic LRU eviction, so
/// the cache is usable with no on-disk bake at all; a baked .swb file only removes the first-touch
/// build cost.
///
/// Default-walker scope only. Cells with multi-Z surfaces and queries for non-default
/// walkers route to the MovementImpl slow path via the Fallthrough_* hit kinds.
/// The cache answers for a default walker on static terrain. Anything outside that — a multi
/// covering the cell, a query Z that doesn't match what the cell was baked at, stacked surfaces
/// with no matching stratum — returns a Fallthrough_* kind, and the caller resolves that cell
/// through MovementImpl instead. Callers must check <see cref="StepMask.IsHit"/>.
/// </summary>
public sealed class StepCache
{
@ -25,25 +27,22 @@ public sealed class StepCache
public static StepCache Instance { get; } = new();
private readonly Dictionary<long, StepChunk> _chunks = [];
// Parallel list of keys for O(1) random sampling during eviction. Kept in lockstep
// with _chunks: append on Miss_NotBuilt, swap-and-pop on eviction.
// Keys of _chunks, kept in lockstep with it, so eviction can sample a random resident chunk
// in O(1). Appended on insert, swap-and-popped on eviction.
private readonly List<long> _keysList = [];
// Second-touch promotion tracker. A chunk's first miss within the window returns
// Fallthrough_NotBuilt; the caller takes the slow path. The Nth DISTINCT-FIND miss
// within the same window (where N = MissPromotionThreshold) promotes to BuildChunk +
// serve. We count distinct Find generations, not raw TryGetMask calls — A* expansion
// hits each visited chunk many times in one Find, so per-call counting hits threshold
// immediately and defeats the gate. Per-Find counting filters single-Find pass-throughs
// (pet following a moving player) while still promoting chunks revisited by multiple
// Finds (NPC patrolling fixed territory).
// Promotion gate. A chunk's first miss returns Fallthrough_NotBuilt and the caller takes the
// slow path; only once misses reach MissPromotionThreshold within MissPromotionWindowMs does
// the chunk get built and served. This keeps one-off traffic — a pet trailing a player across
// the map — from building chunks nothing will query again, while a creature working a fixed
// territory still warms the chunks it revisits.
//
// The gate counts distinct Finds, not TryGetMask calls: A* probes each chunk it visits dozens
// of times within a single pathfind, so per-call counting would cross any threshold instantly
// and gate nothing.
private readonly Dictionary<long, ChunkMissState> _chunkMissTracker = [];
private const int MaxMissTrackerEntries = 4096;
// Generation counter incremented by BeginFindGeneration(). Sentinel 0 = "no Find started
// yet"; treated as a distinct generation per call so callers that bypass BeginFindGeneration
// (single-call tests, BakeMap with threshold=1) get sensible behavior.
private struct ChunkMissState
{
public byte MissCount;
@ -83,24 +82,21 @@ public sealed class StepCache
public bool PreloadOnLazyOpen { get; set; }
/// <summary>
/// Number of misses on the same chunk within <see cref="MissPromotionWindowMs"/>
/// required to trigger a build. 1 = eager (legacy behavior). 2 = second-touch (default,
/// filters single-touch pass-throughs).
/// Misses on the same chunk, within <see cref="MissPromotionWindowMs"/>, needed to build it.
/// 1 builds eagerly on first touch; the default 2 waits for a second Find to show interest.
/// </summary>
public int MissPromotionThreshold { get; set; } = 2;
/// <summary>
/// Window over which misses against the same chunk accumulate toward promotion.
/// Misses spaced wider than this restart the count. Default 30s.
/// How long misses on a chunk accumulate toward promotion. A gap wider than this restarts
/// the count.
/// </summary>
public uint MissPromotionWindowMs { get; set; } = 30_000;
/// <summary>
/// Marks the start of a new pathfind. The promotion gate counts distinct Find
/// generations per chunk, not raw TryGetMask calls — call this once at the top of
/// each pathfind invocation so multiple cell expansions within one Find don't trip
/// the threshold. Wraps at uint.MaxValue back to 1 (0 is reserved as the
/// "no Find started yet" sentinel).
/// Opens a new pathfind for the promotion gate. Call once per pathfind: the gate counts
/// distinct Finds, so without this every cell expansion would count separately and the
/// threshold would be met immediately. Wraps back to 1, since 0 means "no Find open".
/// </summary>
public void BeginFindGeneration()
{
@ -112,12 +108,11 @@ public sealed class StepCache
}
}
/// <summary>Test-only: read the current Find generation.</summary>
/// <summary>The open pathfind's generation, or 0 if none. See <see cref="BeginFindGeneration"/>.</summary>
internal uint CurrentFindGeneration { get; private set; }
/// <summary>
/// Pack (mapId, chunkX, chunkY) into a single long key.
/// Layout: [reserved 16][mapId 16][chunkX 16][chunkY 16].
/// Packs (mapId, chunkX, chunkY) into one key: [reserved 16][mapId 16][chunkX 16][chunkY 16].
/// </summary>
internal static long EncodeKey(int mapId, int chunkX, int chunkY) =>
((long)(mapId & 0xFFFF) << 32) | ((long)(chunkX & 0xFFFF) << 16) | (long)(chunkY & 0xFFFF);
@ -139,9 +134,8 @@ public sealed class StepCache
);
/// <summary>
/// Drop all cached chunks AND zero every telemetry counter. Used by tests and
/// benchmarks that need a known cold-start state. Counter reset is intentional —
/// counters are since-last-clear, not since-startup.
/// Returns the cache to a cold-start state: drops every chunk, closes the .swb readers, and
/// zeroes the counters.
/// </summary>
public void Clear()
{
@ -151,10 +145,8 @@ public sealed class StepCache
}
/// <summary>
/// Drop all resident chunks AND zero counters, but keep lazy readers open.
/// Useful in benchmark loops that want to measure "first query after boot" cost
/// without paying the lazy-reader reopen overhead each iteration. Same intent as
/// <see cref="Clear"/> minus the file-handle teardown.
/// <see cref="Clear"/> without the file-handle teardown: drops the resident chunks and zeroes
/// the counters, but leaves the .swb readers open so the next query can refill from them.
/// </summary>
public void ClearResidentChunks()
{
@ -176,16 +168,14 @@ public sealed class StepCache
_buildsTotal = 0;
}
// Per-map open .swb readers, populated by TryOpenLazyReader at startup. Chunks are
// fetched on demand from the file when ResolveMissingChunk fires; resident memory
// stays bounded by MaxResidentChunks regardless of file size.
private readonly Dictionary<int, StepCacheFile.LazyReader> _lazyReaders = new();
// Open .swb readers, one per map. Chunks are pulled from them on demand, so resident memory
// stays bounded by MaxResidentChunks no matter how large the file is.
private readonly Dictionary<int, StepCacheFile.LazyReader> _lazyReaders = [];
/// <summary>
/// Walk every chunk in <paramref name="mapId"/>, populate the resident set, then
/// save to <paramref name="path"/>. Returns the number of chunks written.
/// Designed for offline / fixture use; blocks the calling thread for many seconds
/// on a full Trammel walk.
/// Builds every chunk in the map and writes them to <paramref name="path"/>, returning the
/// number written. Blocks the caller for many seconds on a full-size map — run it offline or
/// during maintenance, not on a live shard at peak.
/// </summary>
public int BakeMap(int mapId, string path)
{
@ -195,9 +185,9 @@ public sealed class StepCache
return 0;
}
// BakeMap is an explicit decision to populate every chunk; the promotion gate
// would otherwise return Fallthrough_NotBuilt for every chunk (each touched once)
// and the bake would write an empty file. Force eager build for the duration.
// A bake touches each chunk exactly once, so the promotion gate would defer every one of
// them and write an empty file. Baking is an explicit decision to populate everything, so
// build eagerly for the duration.
var prevThreshold = MissPromotionThreshold;
MissPromotionThreshold = 1;
try
@ -216,8 +206,8 @@ public sealed class StepCache
{
for (var cx = 0; cx < chunkCols; cx++)
{
// Any sourceZ works — the chunk is built on first access regardless of
// whether the query returns Hit or Fallthrough_SourceZMismatch.
// The sourceZ is irrelevant here: the chunk gets built on first access whether
// the query ends up a Hit or a Fallthrough_SourceZMismatch.
TryGetMask(map, cx * ChunkSize, cy * ChunkSize, sourceZ: 0);
}
@ -245,9 +235,9 @@ public sealed class StepCache
}
/// <summary>
/// Persist all resident chunks for <paramref name="mapId"/> to a .swb file. Returns
/// the number of chunks written. The file embeds a TileData fingerprint so a stale
/// file (built before a client patch) can be detected and rejected at open time.
/// Writes the map's resident chunks to a .swb file and returns the count. The file carries a
/// fingerprint of the tile and map data, so a bake made before a client patch is detected and
/// rejected when it is next opened.
/// </summary>
public int SaveToFile(string path, int mapId)
{
@ -285,10 +275,9 @@ public sealed class StepCache
}
/// <summary>
/// Open a .swb file as a lazy backing store for <paramref name="mapId"/>. Reads only
/// header + chunk-offset index (~16 bytes per chunk); individual records are fetched
/// on demand by <see cref="ResolveMissingChunk"/>. Returns false on missing file,
/// magic / version mismatch, or TileData hash mismatch (stale bake).
/// Opens a .swb file as a backing store for the map, reading only the header and chunk index
/// up front; records are pulled as queries ask for them. Returns false if the file is missing,
/// unreadable, or a stale bake whose fingerprint no longer matches the live tile data.
/// </summary>
public bool TryOpenLazyReader(string path, int mapId)
{
@ -328,10 +317,7 @@ public sealed class StepCache
}
/// <summary>
/// Materializes every chunk in <paramref name="reader"/> into the resident set.
/// Called from <see cref="TryOpenLazyReader"/> when <see cref="PreloadOnLazyOpen"/>
/// is set. Skips chunks whose live <see cref="Map.Sector.MultisVersion"/> doesn't
/// match the file's snapshot — those will rebake on first query.
/// Loads every chunk in the file into the resident set, for <see cref="PreloadOnLazyOpen"/>.
/// </summary>
private void PreloadFromLazyReader(int mapId, StepCacheFile.LazyReader reader)
{
@ -366,29 +352,22 @@ public sealed class StepCache
);
}
/// <summary>
/// Number of .swb readers currently open. Mostly for tests / telemetry.
/// </summary>
/// <summary>Number of .swb readers currently open.</summary>
public int OpenLazyReaderCount => _lazyReaders.Count;
/// <summary>
/// True if a valid .swb reader is open for <paramref name="mapId"/>. A reader only opens via
/// <see cref="TryOpenLazyReader"/> after <see cref="StepCacheFile.OpenForLazy"/> validates the
/// file's fingerprint against the live tile data, so "has reader" already means "present and
/// up-to-date" — the boot prebake uses this to skip baking maps that don't need it, instead of
/// recomputing the fingerprint a second time.
/// True when a .swb reader is open for the map. A reader only opens after its fingerprint
/// validates against the live tile data, so this already answers "is there an up-to-date bake
/// for this map?" — the boot prebake leans on that to skip maps rather than fingerprint them
/// a second time.
/// </summary>
public bool HasLazyReader(int mapId) => _lazyReaders.ContainsKey(mapId);
/// <summary>Test-only diagnostic: does the lazy reader for <paramref name="mapId"/> hold an offset for (chunkX, chunkY)?</summary>
/// <summary>Diagnostic: does the map's .swb hold a record for (chunkX, chunkY)?</summary>
internal bool LazyReaderHasChunk(int mapId, int chunkX, int chunkY) =>
_lazyReaders.TryGetValue(mapId, out var r) && r.Has(chunkX, chunkY);
/// <summary>
/// Closes all open lazy readers, releasing their underlying file streams. Called from
/// <see cref="Clear"/> so test cleanup can delete .swb files (they're held with
/// FileShare.Read | FileShare.Delete, so this is mostly belt-and-suspenders).
/// </summary>
/// <summary>Closes every open .swb reader, releasing the underlying file streams.</summary>
public void CloseLazyReaders()
{
foreach (var reader in _lazyReaders.Values)
@ -399,18 +378,16 @@ public sealed class StepCache
}
/// <summary>
/// Probabilistic LRU sample size — picks SampleSize random resident chunks per
/// eviction and evicts the oldest of that sample. Approximates true LRU at a tiny
/// fraction of the cost (no full sort). Redis uses the same approach (`maxmemory-samples`).
/// 5 yields ~quality-of-true-LRU for cache eviction; higher values trade speed for accuracy.
/// How many random resident chunks each eviction samples before dropping the oldest of them.
/// Sampling approximates true LRU closely enough at a fraction of the cost, since it needs no
/// sort and no access-ordered structure. Raising it trades speed for accuracy.
/// </summary>
private const int LruSampleSize = 5;
/// <summary>
/// If resident chunk count exceeds MaxResidentChunks, evict via probabilistic LRU
/// until the count is at or below the cap. Per-eviction cost is O(LruSampleSize),
/// independent of resident count — sustained cap pressure has no perpetual perf hit.
/// Called from CacheEvictionTimer; also callable directly from tests.
/// Evicts chunks until the resident count is back within MaxResidentChunks. Each eviction costs
/// O(<see cref="LruSampleSize"/>) regardless of how many chunks are resident, so sustained cap
/// pressure doesn't degrade. Driven by <see cref="CacheEvictionTimer"/>.
/// </summary>
public void EnforceLruCap()
{
@ -426,8 +403,8 @@ public sealed class StepCache
long oldestTouched = long.MaxValue;
long oldestKey = 0;
// Sample LruSampleSize random keys; track the oldest by LastTouchedTicks.
// With replacement is fine — collisions are rare and don't break correctness.
// Sampling with replacement: a repeated key just wastes one sample, it can't pick a
// wrong victim.
var samples = Math.Min(LruSampleSize, _keysList.Count);
for (var s = 0; s < samples; s++)
{
@ -482,11 +459,10 @@ public sealed class StepCache
}
/// <summary>
/// True if a multi (house / boat) covers (x, y) or any of its 8 neighbours. Multi-covered
/// cells — plus the 1-cell halo, because a cell's mask encodes the edges TO its neighbours, so
/// a neighbouring wall must block those edges — are served by the live movement path, not the
/// static chunk cache. Cheap: an interior cell checks only its own sector (chunk == sector);
/// only edge/corner cells additionally check the adjacent sector(s) the halo reaches.
/// True when a multi covers (x, y) or any of its 8 neighbours. The halo matters because a
/// cell's mask encodes the edges TO its neighbours, so a wall one cell over has to block those
/// edges. Since a chunk is a sector, an interior cell only inspects its own sector's HasMultis
/// flag; edge and corner cells additionally check whichever adjacent sectors the halo reaches.
/// </summary>
private static bool MultiInfluence(Map map, int x, int y)
{
@ -503,7 +479,7 @@ public sealed class StepCache
var south = (y & 15) == 15;
if (!(west || east || north || south))
{
return false; // interior cell — its whole halo is inside the (multi-free) own sector
return false; // interior cell: its whole halo lies in this sector, which has no multis
}
return west && map.GetRealSector(sx - 1, sy).HasMultis
@ -517,9 +493,9 @@ public sealed class StepCache
}
/// <summary>
/// Hot-path query. Returns the cached mask + 8 destination Z values + hit kind.
/// Inspect <see cref="StepMask.IsHit"/> to decide whether to use the result or fall
/// back to the slow path.
/// The hot-path query: one lookup yields the cell's 8-direction mask, its 8 destination Zs,
/// and the hit kind. Check <see cref="StepMask.IsHit"/> before trusting the payload — on any
/// fallthrough it is all zeroes and the caller must resolve the cell through MovementImpl.
/// </summary>
public StepMask TryGetMask(Map map, int x, int y, sbyte sourceZ)
{
@ -529,10 +505,9 @@ public sealed class StepCache
return Fallthrough(CacheHitKind.Fallthrough_OffMap);
}
// Multis (houses, boats) are not baked into the static chunk cache (they're dynamic
// content). If a multi covers this cell or its 1-cell halo, route to the live movement
// path, which is fully multi-aware. Gated on Sector.HasMultis, so the multi-free majority
// of the map pays a single (interior) sector lookup.
// Multis are dynamic, so they are never baked into a chunk. Cells they touch go to the
// multi-aware path instead. The check is gated on Sector.HasMultis, so the multi-free
// majority of the map pays one sector lookup for it.
if (MultiInfluence(map, x, y))
{
_fallthroughMulti++;
@ -546,8 +521,8 @@ public sealed class StepCache
var hitKindResult = CacheHitKind.Hit;
if (!_chunks.TryGetValue(key, out var chunk))
{
// Try lazy file first — file-loaded chunks bypass the miss tracker because
// the .swb represents an explicit prior decision to keep this chunk warm.
// The .swb is consulted before the promotion gate: a baked chunk is already an explicit
// decision to keep this area warm, and loading it is far cheaper than building it.
chunk = TryLoadFromLazyReader(map, chunkX, chunkY);
if (chunk != null)
{
@ -569,16 +544,16 @@ public sealed class StepCache
}
}
// A resident chunk is static-only — it never goes stale from multis (multi-covered cells
// fall through to the live path above).
// No staleness check: a resident chunk holds only static terrain, and every cell a multi
// could have changed already fell through above.
chunk.LastTouchedTicks = Core.TickCount;
var cellIndex = ((y - (chunkY << 4)) << 4) | (x - (chunkX << 4));
if (chunk.IsCellMultiZ(cellIndex))
{
// Tier 4: try the per-cell strata. Each stratum is keyed by its bake-time
// standing-Z; a query matches when |sourceZ - stratum.zCenter| <= StepHeight.
// Stacked surfaces: pick the stratum baked nearest the query Z. Multi-Z cells are
// served only from strata, never from the main mask.
if (TryStratumHit(chunk, cellIndex, sourceZ, hitKindResult, out var stratumResult))
{
RecordServed(hitKindResult);
@ -589,15 +564,14 @@ public sealed class StepCache
return Fallthrough(CacheHitKind.Fallthrough_MultiZ);
}
// Source-Z guard: the cache stores one answer per cell baked at SourceZ.
// StepHeight tolerance accepts incremental Z jitter; loosening it breaks parity
// because tile reachability shifts at step-height boundaries.
// Source-Z guard. A cell holds one answer, baked at one standing Z, so a query from too far
// above or below it would get an answer that doesn't apply. The StepHeight tolerance
// absorbs ordinary Z jitter and cannot be widened: reachability flips at exactly that
// boundary, so a looser guard would serve answers that disagree with MovementImpl.
if (Math.Abs(sourceZ - chunk.SourceZ[cellIndex]) > StepHeight)
{
// Swim-layer fallback for shore cells: if the chunk has the layer and this
// cell's water-surface Z is within StepHeight of the query, serve from the
// swim layer (computed at swim-perspective Z). Walker queries on shore cells
// fall through this branch via their Z mismatch with SwimSourceZ.
// Unless this is a shore cell and the query is coming from the water, in which case the
// swim layer holds the answer baked from the water surface.
if (chunk.HasSwimLayer)
{
var swimSrc = chunk.SwimSourceZ[cellIndex];
@ -650,37 +624,24 @@ public sealed class StepCache
}
/// <summary>
/// Returns a fresh StepChunk loaded from the lazy file reader, or null if there's no
/// open reader for the map / no record at (chunkX, chunkY) / the loaded snapshot is
/// stale relative to the live sector's <see cref="Map.Sector.MultisVersion"/>. A null
/// return means the caller should consult the miss tracker; a stale return means
/// "rebuild, the .swb is out of date and a future SaveToFile will overwrite it."
/// Loads a chunk from the map's .swb, or null if no reader is open or the file has no record at
/// (chunkX, chunkY). No staleness check is needed here — the fingerprint was validated when the
/// file was opened, and the chunks are static-only.
/// </summary>
private StepChunk TryLoadFromLazyReader(Map map, int chunkX, int chunkY)
{
if (!_lazyReaders.TryGetValue(map.MapID, out var reader))
{
return null;
}
// Static-only chunks are valid once the file fingerprint matched at open time; multi-covered
// cells fall through before reaching here. Returns null when the file lacks this chunk.
return reader.TryReadChunk(chunkX, chunkY);
}
private StepChunk TryLoadFromLazyReader(Map map, int chunkX, int chunkY) =>
_lazyReaders.TryGetValue(map.MapID, out var reader) ? reader.TryReadChunk(chunkX, chunkY) : null;
/// <summary>
/// Records a miss for <paramref name="chunkKey"/> and decides whether to build now
/// or defer to slow path. Counts distinct Find generations, not raw calls — multiple
/// TryGetMask calls within one Find (BeginFindGeneration scope) count as one touch.
/// Returns true when DISTINCT-FIND misses within the window cross
/// <see cref="MissPromotionThreshold"/>; caller should run BuildChunk and serve.
/// Returns false otherwise; caller should return Fallthrough_NotBuilt so the algorithm
/// uses the slow path. Generation 0 ("no Find active") treats every call as distinct,
/// preserving legacy semantics for callers that don't call BeginFindGeneration.
/// Records a miss and answers whether the chunk has now earned a build. True means build and
/// serve; false means return Fallthrough_NotBuilt and let the caller take the slow path.
///
/// A miss only counts once per Find (see <see cref="BeginFindGeneration"/>). With no Find open
/// — a direct caller, or a bake — every call counts separately.
/// </summary>
private bool ShouldPromoteAfterMiss(long chunkKey)
{
// Environment.TickCount, not Core.TickCount: tests/bench fixtures may not advance
// the game-loop tick. The promotion window is wall-clock anyway.
// Environment.TickCount rather than Core.TickCount: the window is wall-clock, and test and
// benchmark fixtures don't necessarily advance the game loop's tick.
var now = (uint)Environment.TickCount;
var gen = CurrentFindGeneration;
@ -743,10 +704,10 @@ public sealed class StepCache
}
/// <summary>
/// Drop tracker entries older than the promotion window. Called when the tracker hits
/// its capacity ceiling. If the prune doesn't reclaim anything (every entry is in
/// window), the cap is enforced by clearing — the worst case is a few extra
/// Fallthrough_NotBuilt returns until traffic re-establishes hot chunks.
/// Drops tracker entries that have aged out of the promotion window, once the tracker hits its
/// capacity ceiling. When nothing has aged out, the whole tracker is cleared to enforce the cap
/// — that costs a few extra Fallthrough_NotBuilt returns while traffic re-establishes the hot
/// chunks, which is cheaper than letting the tracker grow without bound.
/// </summary>
private void PruneMissTracker(uint now)
{
@ -780,17 +741,15 @@ public sealed class StepCache
var baseX = chunkX << 4;
var baseY = chunkY << 4;
// Tier 4 strata accumulator. Both the offset table and the packed data buffer are
// created lazily on the first multi-Z cell; single-Z chunks (the vast majority) never
// touch the pool. strataData is a rented scratch buffer — the chunk gets an exact-size
// copy at the end, so the pooled array never escapes this method.
// Strata accumulator, created on the first multi-Z cell so single-Z chunks pay nothing.
// strataData is rented scratch; the chunk receives an exact-size copy, so the pooled array
// never escapes this method.
ushort[] strataOffsetByCell = null;
byte[] strataData = null;
var strataLen = 0;
// Reused per cell: the standable surface Zs (walkway / bridge / floor levels). 16 is
// generous — clearance forces standable surfaces >= PersonHeight apart, so a 256-tall
// Z range admits at most ~16 anyway.
// Standable surface Zs for the current cell. 16 slots is generous: clearance forces
// surfaces at least PersonHeight apart, so an sbyte Z range can't hold more than ~16.
Span<sbyte> surfaceZs = stackalloc sbyte[16];
for (var dy = 0; dy < ChunkSize; dy++)
@ -803,16 +762,13 @@ public sealed class StepCache
map.GetAverageZ(x, y, out _, out var avgZ, out _);
// Anchor the cell at the surface a creature actually STANDS on, not the land
// average. For plain overworld that's the land; for static-over-land terrain
// (sewer/dungeon walkways, bridges, stair treads, raised foundations, upper
// building floors) it's the walkable static surface — which the old
// ComputeStandingZ(avgZ) anchor missed, producing source-Z fallthroughs (or,
// within the StepHeight tolerance band on stairs, a wrong vertical-neighbor
// answer baked at the adjacent tread). ComputeStandableSurfaceZs returns the
// standable surfaces ascending; the lowest is the primary anchor and A* tracks
// newZ to match it. Cells with no standable walk surface (deep water, solid
// rock) fall back to the land avg so the swim layer / wetMask still bake.
// Anchor the cell at the surface a creature stands on, not the land average. On
// open terrain those coincide, but on static-over-land geometry — walkways,
// bridges, stair treads, upper floors — the walkable surface is the static, and
// anchoring at the land below it would make every query fall through the source-Z
// guard. Surfaces come back ascending and the lowest is the anchor; A* tracks its
// per-cell Z to match. A cell with no standable surface at all (deep water, solid
// rock) falls back to the land average so its swim data still bakes.
var surfaceCount = StepProbe.ComputeStandableSurfaceZs(map, x, y, surfaceZs);
var standingZ = surfaceCount > 0 ? surfaceZs[0] : (sbyte)Math.Clamp(avgZ, sbyte.MinValue, sbyte.MaxValue);
@ -838,15 +794,12 @@ public sealed class StepCache
chunk.SwimZW[cell] = result.SwimZ_W;
chunk.SwimZNW[cell] = result.SwimZ_NW;
// Shore-cell handling: if the cell has BOTH a walk surface (standing Z)
// AND a water surface (Wet land tile or wet static) at a Z separated by
// > StepHeight, populate the swim layer at swim-perspective Z. Only when
// ComputeMaskAt produces a non-zero swim mask — bridges/docks/piers with
// insufficient vertical clearance for a swim creature's body envelope
// produce wetMask=0 (StaticsBlockAt rejects them), and we skip those cells
// rather than baking a stratum that always answers "no movement." The
// sentinel NoSwimLayerCell stays in SwimSourceZ for skipped cells; the
// chunk only sets HasSwimLayer when at least one cell got a usable entry.
// Shore cell: a walkable surface and a water surface more than StepHeight apart.
// The main mask is baked at the walk surface, so a swimmer querying from the water
// would fail the source-Z guard; bake it a second answer from the water surface.
// An empty swim mask means the water is unreachable anyway — a dock or pier with
// too little clearance for a swimmer's body — so leave those cells at the
// NoSwimLayerCell sentinel rather than store an answer that always says "blocked".
var swimZRaw = StepProbe.ComputeSwimStandingZ(map, x, y);
if (swimZRaw != int.MinValue && Math.Abs(swimZRaw - standingZ) > StepHeight)
{
@ -871,29 +824,24 @@ public sealed class StepCache
}
}
// Stacked walkable surfaces at one cell (ground + 1st + 2nd building floors,
// a bridge over a walkable path, etc.): bake a stratum per standable surface
// so a query at any floor's Z hits. The primary (lowest) surface is also in
// the main mask above, but multi-Z cells are served exclusively from strata,
// so every standable surface — including the primary — must appear here.
// Single-surface cells (the common case, incl. stair treads and sewer
// walkways) skip this entirely and stay on the fast single-mask path.
// Stacked walkable surfaces — a bridge over a path, the floors of a building —
// need one stratum each so a query at any of their Zs finds an answer. Every
// surface goes in, including the lowest, because a multi-Z cell is served only
// from its strata and never from the main mask baked above.
if (surfaceCount >= 2)
{
if (strataOffsetByCell == null)
{
strataOffsetByCell = new ushort[StepChunk.CellsPerChunk];
strataOffsetByCell.AsSpan().Fill(StepChunk.NoStrata);
// Offsets are ushort and NoStrata takes ushort.MaxValue, so a reachable
// offset is at most NoStrata - 1 and the packed data can never exceed
// NoStrata bytes. Renting that much up front means the guard below is
// the only bound the writes need.
// NoStrata bounds the packed data to NoStrata bytes (see StepChunk), so
// renting that much up front leaves the record guard below as the only
// bound the writes need.
strataData = STArrayPool<byte>.Shared.Rent(StepChunk.NoStrata);
}
// A record is one count byte plus surfaceCount strata. Skip the cell if it
// doesn't fit — it keeps the land-anchored main mask and falls through
// off-surface. Well above realistic per-chunk strata volume either way.
// One count byte plus a record per surface. A cell whose record won't fit stays
// single-Z: it keeps the main mask and falls through off its anchor surface.
var recordLength = 1 + surfaceCount * StepChunk.StratumByteLength;
if (strataLen + recordLength <= StepChunk.NoStrata)
{
@ -920,10 +868,10 @@ public sealed class StepCache
}
/// <summary>
/// Tier 4 strata lookup. Walks the cell's stratum list, returns true with the first
/// stratum whose <c>zCenter</c> is within StepHeight of <paramref name="sourceZ"/>.
/// Layout matches <see cref="StepChunk.StrataData"/>: u8 count, then count × 19-byte
/// stratum (sbyte zCenter, byte walkMask, byte wetMask, 8 sbyte walkZ, 8 sbyte swimZ).
/// Finds the cell's stratum matching <paramref name="sourceZ"/> — the first whose zCenter is
/// within StepHeight — and builds its mask. False when the cell has no strata or none of them
/// sit near enough, in which case the caller falls through. Reads the layout
/// <see cref="WriteStratum"/> writes.
/// </summary>
private static bool TryStratumHit(
StepChunk chunk, int cellIndex, sbyte sourceZ, CacheHitKind hitKind, out StepMask result

230
Projects/UOContent/Engines/Pathing/Cache/StepCacheFile.cs
View file

@ -9,40 +9,36 @@ using Server.Compression;
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Binary serializer + lazy reader for the step cache. Persists chunk records to disk
/// so a server warm-starts without paying chunk-build cost on the first pathfind through
/// a region. Lazy: opening a file reads only the header + chunk-offset index (~few KB
/// for tens of thousands of chunks), then individual chunks are seeked + deserialized
/// only when the cache asks for them. RAM stays bounded by MaxResidentChunks regardless
/// of file size.
/// Binary serializer and reader for the step cache, so a shard can warm-start instead of building
/// chunks on the first pathfind through each region. Opening a file reads only the header and chunk
/// index; a chunk record is seeked and inflated when the cache actually asks for it, which keeps
/// resident memory bounded by MaxResidentChunks no matter how large the file is.
///
/// File layout v8 (little-endian, BufferWriter / BufferReader convention):
/// File layout (little-endian, BufferWriter / BufferReader convention):
///
/// Header (40 bytes):
/// u32 Magic = 0x42575300 ('SWB\0')
/// u32 Version = current FormatVersion (9)
/// u32 Version = FormatVersion
/// u32 MapId
/// u64 Fingerprint XxHash3 over (1) LandTable + ItemTable flags AND (2) the
/// on-disk bytes of mapX.mul / .uop, staidxX.mul, staticsX.mul.
/// Rejects a load when EITHER tile flags shifted (client patch)
/// OR the map data was rewritten (CentredSharp / UOFiddler edit).
/// The .mul format has no built-in CRC; this is the only way
/// to detect those mutations.
/// u64 BakeTimestamp DateTime.UtcNow.Ticks at write time (informational).
/// u64 Fingerprint XxHash3 over tiledata.mul and the map's own .mul / .uop files.
/// Detects both a client patch that shifts tile flags and a map edit
/// that rewrites the terrain; see ComputeFingerprint. The .mul format
/// carries no CRC of its own, so hashing is the only way to catch either.
/// u64 BakeTimestamp DateTime.UtcNow.Ticks at write time. Informational.
/// u32 ChunkCount
/// u64 IndexOffset File position where the chunk index begins.
/// u64 IndexOffset Where the index trailer begins.
///
/// Per chunk (ChunkCount times, variable size):
/// u32 UncompressedLen Size of the inflated record body below.
/// byte[] Payload The record body (the v6 layout that follows), libdeflate-
/// compressed. If the on-disk payload length (index recordLength −
/// 4) equals UncompressedLen, the body was stored raw because
/// compression did not shrink it (tiny Uniform records).
/// byte[] Payload The record body, libdeflate-compressed — or stored raw when
/// compression didn't shrink it, as happens with tiny Uniform
/// records. The reader tells the two apart by comparing the payload
/// length against UncompressedLen.
///
/// Record body (after inflate — the v6 layout):
/// Record body (after inflate):
/// u16 ChunkX
/// u16 ChunkY
/// u32 BuiltMultisVersion (reserved since v9 — always 0; chunks are static-only)
/// u32 BuiltMultisVersion Reserved, always 0 — chunks are static-only.
/// u8 Kind 0 = Full; 2 = Uniform
/// // Uniform (Kind == 2): ~28-byte record — all 256 cells share these single values:
/// byte walkMask, wetMask; sbyte sourceZ; sbyte walkZ_N..NW (8); sbyte swimZ_N..NW (8)
@ -78,34 +74,29 @@ namespace Server.Engines.Pathing.Cache;
/// The file offset is not stored — reconstructed as a cumulative sum of recordLength
/// starting at HeaderSize (the first record sits immediately after the header).
///
/// Per-chunk fixed portion (Kind + flags + ZArrayMask + WalkMask + WetMask + SourceZ):
/// ~783 bytes; each present base Z array adds 256 bytes (0..16 present, so up to ~4 KB).
/// A fully-flat Full chunk stores no residual blocks. Strata trailer: 516 + N × ~30 bytes
/// for a chunk with N multi-Z cells averaging ~2 strata each. LRU bookkeeping
/// (LastTouchedTicks) is intentionally not persisted.
/// A chunk's fixed portion runs ~783 bytes, and each directional-Z array that survives prediction
/// adds 256 more, so a Full record lands between ~783 bytes and ~4 KB. The strata trailer adds
/// 516 bytes plus roughly 30 per multi-Z cell. LastTouchedTicks is deliberately not persisted —
/// LRU state means nothing across a restart.
///
/// Files with version &lt; <see cref="MinSupportedVersion"/> are silently rejected
/// at open time (treated as missing) and overwritten on the next save.
/// Files below <see cref="MinSupportedVersion"/> are treated as missing and overwritten on the
/// next save. The cache regenerates from the map data, so a format bump only costs a one-time
/// re-bake.
/// </summary>
internal static class StepCacheFile
{
public const uint Magic = 0x42575300; // 'SWB\0'
// v9: chunks are STATIC-ONLY (land + statics.mul, no multis). v8 and earlier baked multis
// (houses/boats) into chunks, which is unsafe to persist — multis are dynamic, and the
// BuiltMultisVersion they were tagged with is a non-persisted session counter. Bumping the
// version rejects those old files so they re-bake static-only. The BuiltMultisVersion record
// field is retained as a reserved (always-0) u32 to avoid a layout change.
public const uint FormatVersion = 9;
/// <summary>
/// Lowest format version this binary can load. Files below it are treated as missing
/// (silently rejected) and overwritten by the next SaveToFile / BakeMap. The cache is
/// fully regenerable, so a format bump just forces a one-time re-bake of stale files.
/// Oldest format this binary will load. Anything older is treated as missing rather than
/// migrated: the cache is fully regenerable from the map data, so a re-bake is always
/// available and always correct.
/// </summary>
public const uint MinSupportedVersion = 9;
// Per-chunk record discriminator (first byte after BuiltMultisVersion). 1 is reserved.
// Record discriminator. 1 is reserved.
private const byte KindFull = 0;
private const byte KindUniform = 2;
@ -118,12 +109,12 @@ internal static class StepCacheFile
+ sizeof(uint) // ChunkCount
+ sizeof(ulong); // IndexOffset
// Index entry (v8 compact): u32 packedKey ((chunkX << 16) | chunkY) + u32 recordLength.
// The file offset is NOT stored — entries are in record write order, so the reader
// reconstructs each offset by cumulative sum of record lengths starting at HeaderSize.
// One index entry: u32 packedKey ((chunkX << 16) | chunkY) + u32 recordLength. The file offset
// isn't stored — entries sit in record write order, so the reader rebuilds each offset as a
// running sum of the lengths before it, starting at HeaderSize.
private const int IndexEntryBytes = sizeof(uint) + sizeof(uint);
/// <summary>Fixed-size portion of a chunk record (everything except the optional strata + swim trailers).</summary>
/// <summary>A chunk record minus its optional strata and swim trailers.</summary>
private const int BytesPerChunkBase =
sizeof(ushort) + sizeof(ushort) + sizeof(uint)
+ sizeof(byte) + sizeof(byte) + sizeof(byte) // Kind + HasStrata + HasSwimLayer
@ -135,17 +126,16 @@ internal static class StepCacheFile
+ 8 * StepChunk.CellsPerChunk; // SwimZ[8]
/// <summary>
/// Byte offset of the IndexOffset u64 within the header
/// (Magic+Version+MapId+Fingerprint+BakeTimestamp+ChunkCount = 32). Patched after chunks land.
/// Where the header's IndexOffset u64 sits. It's written as a placeholder and patched once the
/// chunks are down and the index position is known.
/// </summary>
private const int IndexOffsetFieldPosition = 32;
public delegate bool ChunkEnumerator(out int chunkX, out int chunkY, out StepChunk chunk);
/// <summary>
/// Peek at a .swb file's Fingerprint field (header byte offset 12) without
/// reading any chunk data. Returns false on missing file, bad magic, or wrong
/// version. Cheap — reads 20 bytes total.
/// Reads just a .swb file's fingerprint — 20 bytes, no chunk data. False if the file is
/// missing, isn't a .swb, or is a version this binary can't load.
/// </summary>
public static bool TryReadFingerprint(string path, out ulong fingerprint)
{
@ -183,31 +173,28 @@ internal static class StepCacheFile
}
/// <summary>
/// Combined XxHash3 fingerprint over (1) the on-disk <c>tiledata.mul</c> file and (2) the
/// per-map .mul / .uop file contents (via <see cref="TileMatrix.MapFilesFingerprint"/>).
/// Bake files carry this hash so a load can refuse to populate the cache when EITHER the
/// tile data shifted (client patch) OR the map data was rewritten (CentredSharp / UOFiddler
/// edit). The .mul format has no built-in CRC; this is the only way to detect those mutations.
/// Hashes the inputs a bake depends on: <c>tiledata.mul</c> and the map's own .mul / .uop
/// files. A file carrying a stale hash is refused at open time, which is what catches a client
/// patch that shifts tile flags or a map editor that rewrites the terrain. Neither format has a
/// CRC of its own, so hashing is the only signal available.
///
/// IMPORTANT: hash the FILES, never the in-memory <see cref="TileData.LandTable"/> /
/// This must hash the FILES, never the in-memory <see cref="TileData.LandTable"/> /
/// <see cref="TileData.ItemTable"/>. The server patches those tables at runtime (ItemFixes,
/// LOSBlocker, PotionKeg, CTF, ...) at nondeterministic lifecycle points, so a fingerprint over
/// the live tables varies with WHEN it is taken; the file hash is the only lifecycle-stable
/// "did the client's tile data change?" signal. Server-side tile patches are applied identically
/// every boot and intentionally do NOT invalidate the cache — change one and you must
/// [PathCacheClear or bump the format.
/// LOSBlocker, PotionKeg, CTF), so a hash of the live tables changes depending on when it is
/// taken — useless as a fingerprint. Those server-side patches apply identically every boot and
/// deliberately do NOT invalidate the cache; if you change one, run [PathCacheClear or bump
/// <see cref="FormatVersion"/> yourself.
/// </summary>
public static ulong ComputeFingerprint(int mapId)
{
var hasher = HashUtility.CreateXxHash3();
// (1) tiledata.mul — hashed once, cached. The authoritative source for tile flags/heights.
Span<byte> tileDataBytes = stackalloc byte[sizeof(ulong)];
BinaryPrimitives.WriteUInt64LittleEndian(tileDataBytes, TileDataFileFingerprint());
hasher.Append(tileDataBytes);
// (2) Map files (mapX.mul / .uop, staidxX.mul, staticsX.mul). TileMatrix already
// streamed them through XxHash3 once at construction; mix the result in.
// TileMatrix already streamed the map files through XxHash3 when it was built; reuse that
// rather than re-reading them.
var map = Map.Maps[mapId];
if (map != null && map != Map.Internal && map.Tiles != null)
{
@ -223,10 +210,9 @@ internal static class StepCacheFile
private static bool _tileDataFileFingerprintComputed;
/// <summary>
/// XxHash3 over the raw <c>tiledata.mul</c> bytes, computed once and cached — the file never
/// changes during a run. Mirrors <see cref="TileMatrix.MapFilesFingerprint"/> for the map
/// files. Returns 0 if the file can't be found (the server can't run without it anyway, so
/// this only matters in stripped test hosts, where 0 is a fine deterministic constant).
/// XxHash3 of the raw <c>tiledata.mul</c> bytes, computed once — the file can't change while
/// the server runs. Returns 0 when the file is absent, which only happens in stripped test
/// hosts; a real server can't boot without it, and 0 is a fine deterministic stand-in.
/// </summary>
private static ulong TileDataFileFingerprint()
{
@ -257,10 +243,9 @@ internal static class StepCacheFile
{
Directory.CreateDirectory(Path.GetDirectoryName(path) ?? ".");
// Initial estimate: base record + a modest strata budget per chunk. Coastline
// chunks add another ~2.5 KB (swim layer) but they're a small fraction of any
// map; the writer grows on overflow so under-estimating just causes a few
// realloc/copy cycles during the bake — not a correctness issue.
// A rough estimate: the base record plus a small strata budget per chunk. Coastline chunks
// run ~2.5 KB over it for their swim layer, but they're a small share of any map, and the
// writer grows on overflow — under-estimating costs a few reallocs during a bake, nothing more.
var capacity = HeaderSize + (BytesPerChunkBase + 256) * (int)chunkCount + IndexEntryBytes * (int)chunkCount;
var buffer = new byte[capacity];
var w = new BufferWriter(buffer, prefixStr: false);
@ -273,8 +258,8 @@ internal static class StepCacheFile
w.Write(chunkCount);
w.Write(0UL); // IndexOffset placeholder, patched after chunks
// Each record is built uncompressed into recordScratch, then libdeflate-compressed into
// compScratch and framed as [u32 uncompressedLen][payload].
// Each record is built into recordScratch, compressed into compScratch, then framed as
// [u32 uncompressedLen][payload].
var packer = Deflate.Maximum;
var recordScratch = new byte[BytesPerChunkBase + 1024];
var compScratch = new byte[packer.MaxPackSize(recordScratch.Length)];
@ -306,16 +291,14 @@ internal static class StepCacheFile
var indexOffset = (ulong)w.Position;
for (var i = 0u; i < chunkCount; i++)
{
// v8 compact entry: u32 packedKey ((chunkX << 16) | chunkY) + u32 recordLength.
// Offset is omitted; entries are in record write order so the reader derives it.
var key = indexEntries[i].key;
var packedKey = ((uint)(key >> 32) << 16) | (uint)(key & 0xFFFF);
w.Write(packedKey);
w.Write(indexEntries[i].length);
}
// Patch IndexOffset on the writer's current backing buffer (BufferWriter may
// have grown during chunk writes; the original `buffer` ref is stale after grow).
// Patch IndexOffset on the writer's CURRENT buffer: BufferWriter may have grown during the
// chunk writes, which leaves the original `buffer` reference pointing at a stale array.
var liveBuffer = w.Buffer;
BinaryPrimitives.WriteUInt64LittleEndian(liveBuffer.AsSpan(IndexOffsetFieldPosition, 8), indexOffset);
@ -324,9 +307,9 @@ internal static class StepCacheFile
}
/// <summary>
/// Opens a .swb file and reads only its header + chunk-offset index. Returns null on
/// missing file, magic / version mismatch, or Fingerprint mismatch (a stale bake
/// against a freshly patched client). Callers own disposal of the returned reader.
/// Opens a .swb file, reading only its header and chunk index. Null if the file is missing,
/// isn't a loadable .swb, or is a stale bake whose fingerprint no longer matches the live tile
/// and map data. The caller owns the returned reader.
/// </summary>
public static LazyReader OpenForLazy(string path)
{
@ -361,8 +344,6 @@ internal static class StepCacheFile
var version = BinaryPrimitives.ReadUInt32LittleEndian(headerBuf[4..]);
if (version < MinSupportedVersion || version > FormatVersion)
{
// Below the minimum supported version: treat as missing. Older files
// get silently overwritten on the next SaveToFile / BakeMap.
stream.Dispose();
return null;
}
@ -379,7 +360,7 @@ internal static class StepCacheFile
return null;
}
// Read the chunk-offset index in one shot.
// Pull the whole index in one read.
var indexBytes = (int)chunkCount * IndexEntryBytes;
var indexBuf = new byte[indexBytes];
stream.Position = (long)indexOffset;
@ -389,9 +370,8 @@ internal static class StepCacheFile
return null;
}
// v8 compact index: { u32 packedKey, u32 length } per chunk, in record write order.
// The file offset is not stored — reconstruct it by cumulative record length starting
// at the first record (immediately after the header).
// Entries are in record write order and carry no offset, so rebuild each one as a
// running sum of the record lengths, starting just past the header.
var offsets = new Dictionary<ulong, (ulong offset, uint length)>((int)chunkCount);
var runningOffset = (ulong)HeaderSize;
for (var i = 0; i < chunkCount; i++)
@ -416,27 +396,27 @@ internal static class StepCacheFile
private static ulong PackChunkKey(int chunkX, int chunkY) => ((ulong)(uint)chunkX << 32) | (uint)chunkY;
/// <summary>
/// Predicted directional-Z for one cell/direction: the cell's own SourceZ when the
/// direction is walkable/wet (mask bit set), else 0 — matching the baker, which leaves
/// non-walkable directional slots at their zero-initialized default
/// (StepProbe.ComputeMaskAt clears walkZs/swimZs and writes only on a successful step).
/// Guesses a cell's destination Z for one direction: on flat ground a step lands at the Z you
/// left from, so predict SourceZ where the direction is passable and 0 where it isn't. The
/// zero matches the baker, which only writes a slot on a successful step and leaves the rest
/// cleared. Most terrain is flat, so most predictions are exact and most residuals are 0 —
/// which is what makes the residual arrays compress away to nothing.
/// </summary>
internal static sbyte Predict(byte dirMaskByte, int bit, sbyte sourceZ) =>
(dirMaskByte >> bit & 1) != 0 ? sourceZ : (sbyte)0;
/// <summary>
/// Residual of an absolute directional-Z against its prediction. Unchecked two's-complement
/// so the transform is byte-exact for ALL sbyte inputs (no value-range constraint).
/// A destination Z's difference from its prediction. Wraps deliberately: two's-complement
/// round-trips exactly for every sbyte input, so no value range is off-limits.
/// </summary>
internal static sbyte EncodeResidual(sbyte z, sbyte predict) => unchecked((sbyte)(z - predict));
/// <summary>Inverse of <see cref="EncodeResidual"/>: absolute directional-Z = predict + residual.</summary>
/// <summary>Inverse of <see cref="EncodeResidual"/>.</summary>
internal static sbyte DecodeZ(sbyte predict, sbyte residual) => unchecked((sbyte)(predict + residual));
/// <summary>
/// The base directional-Z array for direction index d in canonical order: walk N..NW (0-7),
/// then swim N..NW (8-15). Index d uses WalkMask (d &lt; 8) or WetMask (d &gt;= 8) with
/// direction bit (d &amp; 7).
/// The destination-Z array for direction index d, in the canonical order the format stores them:
/// walk N..NW as 0-7, then swim N..NW as 8-15.
/// </summary>
private static sbyte[] GetBaseZArray(StepChunk c, int d) => d switch
{
@ -448,10 +428,9 @@ internal static class StepCacheFile
};
/// <summary>
/// Builds the uncompressed v6 record for one chunk into <paramref name="w"/>, libdeflate-
/// compresses it, and writes it framed as [u32 uncompressedLen][payload]. The payload is the
/// compressed bytes, or — when compression does not shrink the record (tiny Uniform records) —
/// the raw record itself; the reader distinguishes the two by payload length vs uncompressedLen.
/// Builds one chunk's record, compresses it, and frames it as [u32 uncompressedLen][payload].
/// When compression fails to shrink the record — as it does on the tiny Uniform ones — the raw
/// record is stored instead, and the reader tells the two apart by payload length.
/// </summary>
private static void WriteChunk(
BufferWriter w, int chunkX, int chunkY, StepChunk chunk,
@ -460,7 +439,7 @@ internal static class StepCacheFile
{
var rw = new BufferWriter(recordScratch, prefixStr: false);
BuildRecord(rw, chunkX, chunkY, chunk);
recordScratch = rw.Buffer; // may have grown; keep the larger buffer for reuse
recordScratch = rw.Buffer; // may have grown; hold onto the larger buffer for the next chunk
var recordLen = (int)rw.Position;
var bound = packer.MaxPackSize(recordLen);
@ -478,8 +457,8 @@ internal static class StepCacheFile
}
else
{
// Incompressible (or expanded): store the record raw. The reader detects this when
// the on-disk payload length equals the uncompressed length.
// Compression didn't help, so store the record raw. Payload length == uncompressedLen
// is how the reader recognizes that.
w.Write(recordScratch.AsSpan(0, recordLen));
}
}
@ -490,8 +469,8 @@ internal static class StepCacheFile
w.Write((ushort)chunkY);
w.Write((uint)chunk.BuiltMultisVersion);
// Kind: 0 = Full, 2 = Uniform. A uniform chunk (no strata, no swim layer, all 19 base
// arrays constant) stores one cell's worth of data (~28-byte record total).
// A uniform chunk — every cell identical — collapses to one cell's worth of data, ~28 bytes.
// Open water and solid rock make up a lot of a map, so this is worth the branch.
if (chunk.IsUniform())
{
w.Write(KindUniform);
@ -517,7 +496,7 @@ internal static class StepCacheFile
return;
}
w.Write(KindFull); // Full
w.Write(KindFull);
var strataOffsetByCell = chunk.GetStrataOffsetByCellForSerialization();
var strataData = chunk.GetStrataDataForSerialization();
@ -526,9 +505,9 @@ internal static class StepCacheFile
w.Write((byte)(hasStrata ? 1 : 0));
w.Write((byte)(hasSwimLayer ? 1 : 0));
// Predictive-Z: each base directional Z array is stored as a masked residual against
// SourceZ. Bit d of ZArrayMask is set only when array d differs from its prediction
// somewhere; cleared arrays are omitted and rebuilt from mask+SourceZ at read.
// Each destination-Z array is stored as residuals against its prediction (see Predict). An
// array that matches its prediction everywhere — the common case on flat terrain — is
// omitted entirely, and its ZArrayMask bit stays clear so the reader synthesizes it.
ushort zArrayMask = 0;
for (var d = 0; d < 16; d++)
{
@ -583,7 +562,6 @@ internal static class StepCacheFile
if (hasStrata)
{
// 256 × u16 offsets, then u32 length-prefixed strata byte array.
for (var i = 0; i < StepChunk.CellsPerChunk; i++)
{
w.Write(strataOffsetByCell[i]);
@ -600,7 +578,7 @@ internal static class StepCacheFile
private static StepChunk ReadChunk(byte[] buffer)
{
var r = new BufferReader(buffer);
// Skip ChunkX + ChunkY (already known via the index lookup).
// ChunkX + ChunkY — already known from the index lookup that got us here.
r.ReadUShort();
r.ReadUShort();
var multisVersion = (int)r.ReadUInt();
@ -608,7 +586,7 @@ internal static class StepCacheFile
var chunk = new StepChunk { BuiltMultisVersion = multisVersion };
if (kind == KindUniform) // Uniform — one cell's worth of the 19 base arrays, fill all 256 cells.
if (kind == KindUniform) // one cell's values, broadcast to all 256
{
Array.Fill(chunk.WalkMask, r.ReadByte());
Array.Fill(chunk.WetMask, r.ReadByte());
@ -640,8 +618,8 @@ internal static class StepCacheFile
r.Read(chunk.WetMask);
ReadSBytes(r, chunk.SourceZ);
// Predictive-Z reconstruction: present arrays carry residuals (z = predict + residual);
// absent arrays are synthesized from mask+SourceZ (z = predict, residual implicitly 0).
// Inverse of the write path: a stored array carries residuals to add back to the
// prediction, an omitted one IS the prediction.
Span<sbyte> residual = stackalloc sbyte[StepChunk.CellsPerChunk];
for (var d = 0; d < 16; d++)
{
@ -706,16 +684,15 @@ internal static class StepCacheFile
r.Read(MemoryMarshal.Cast<sbyte, byte>(arr.AsSpan()));
/// <summary>
/// Open handle on a .swb file. Holds the FileStream + chunk-offset index. Chunks are
/// fetched on demand via <see cref="TryReadChunk"/>; only the records actually queried
/// are ever materialized. Dispose releases the underlying stream.
/// An open .swb file: the stream plus the chunk index. Only the records actually asked for are
/// ever read or inflated. Dispose releases the stream.
/// </summary>
internal sealed class LazyReader : IDisposable
{
private FileStream _stream;
private readonly Dictionary<ulong, (ulong offset, uint length)> _offsets;
private byte[] _buffer; // raw on-disk record: [u32 uncompressedLen][payload]
private byte[] _bodyBuffer; // decompressed v6 record, parsed by ReadChunk
private byte[] _buffer; // the raw framed record as it sits on disk
private byte[] _bodyBuffer; // that record, inflated, ready for ReadChunk
public uint MapId { get; }
public ulong Fingerprint { get; }
@ -725,11 +702,7 @@ internal static class StepCacheFile
public bool Has(int chunkX, int chunkY) => _offsets.ContainsKey(PackChunkKey(chunkX, chunkY));
/// <summary>
/// Enumerates every (chunkX, chunkY) coordinate the file holds. Used by
/// <see cref="StepCache"/> when preload is enabled to materialize all chunks
/// upfront instead of on first query.
/// </summary>
/// <summary>Every (chunkX, chunkY) the file holds. Used to preload the whole file.</summary>
public IEnumerable<(int chunkX, int chunkY)> EnumerateChunkCoords()
{
foreach (var key in _offsets.Keys)
@ -754,9 +727,8 @@ internal static class StepCacheFile
}
/// <summary>
/// Returns the chunk record at (<paramref name="chunkX"/>, <paramref name="chunkY"/>)
/// from the file, or null if the file doesn't contain it. Single seek + bulk read,
/// sized exactly to the chunk's recorded length (which varies with strata size).
/// Reads one chunk from the file, or null if the file has no record for it. One seek and
/// one read, sized to the record's indexed length.
/// </summary>
public StepChunk TryReadChunk(int chunkX, int chunkY)
{
@ -771,7 +743,6 @@ internal static class StepCacheFile
return null;
}
// Grow the on-disk scratch buffer if this chunk's record is larger than what we have.
if (entry.length > _buffer.Length)
{
_buffer = new byte[entry.length];
@ -784,8 +755,8 @@ internal static class StepCacheFile
return null;
}
// Frame: [u32 uncompressedLen][payload]. payload is libdeflate-compressed, unless its
// length equals uncompressedLen, in which case it was stored raw (incompressible).
// [u32 uncompressedLen][payload], where the payload is compressed unless its length
// already equals uncompressedLen — then it was stored raw.
var uncompressedLen = (int)BinaryPrimitives.ReadUInt32LittleEndian(_buffer);
var payloadLen = (int)entry.length - sizeof(uint);
if (_bodyBuffer.Length < uncompressedLen)
@ -799,7 +770,8 @@ internal static class StepCacheFile
}
else
{
// Decompression is level-independent, so reuse the shared per-thread binding.
// Deflate.Standard, not .Maximum: the level only affects packing, and inflate has
// to accept whatever the writer produced regardless.
var result = Deflate.Standard.Unpack(
_bodyBuffer.AsSpan(0, uncompressedLen),
_buffer.AsSpan(sizeof(uint), payloadLen),

74
Projects/UOContent/Engines/Pathing/Cache/StepChunk.cs
View file

@ -3,20 +3,19 @@ using System;
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Per-chunk storage backing StepCache. Holds raw walk + swim masks and destination Z
/// values for each of 256 cells in a 16x16 chunk, plus build-time metadata (multis version, multi-Z strata)
/// and LRU bookkeeping.
/// Per-chunk storage backing <see cref="StepCache"/>: walk + swim masks and destination Zs for
/// each of the 256 cells in a 16x16 chunk, plus the optional multi-Z strata and swim layers and
/// the LRU timestamp.
/// </summary>
internal sealed class StepChunk
{
public const int CellsPerChunk = 256; // 16 x 16
/// <summary>Bit i of WalkMask[c] = "default walker can step from cell c to neighbor (Direction)i".
/// Raw — no diagonal corner-cut applied here.</summary>
/// <summary>Bit i of WalkMask[c]: a default walker can step from cell c to neighbour (Direction)i.
/// Raw — no diagonal corner-cut applied, so callers must AND the partner bits themselves.</summary>
public readonly byte[] WalkMask = new byte[CellsPerChunk];
/// <summary>Bit i of WetMask[c] = "swim-only mob can step from cell c to neighbor (Direction)i".
/// Layered with WalkMask via canSwim/cantWalk capability flags.</summary>
/// <summary>Bit i of WetMask[c]: a swim-only mob can step from cell c to neighbour (Direction)i.</summary>
public readonly byte[] WetMask = new byte[CellsPerChunk];
public readonly sbyte[] SourceZ = new sbyte[CellsPerChunk];
@ -40,14 +39,13 @@ internal sealed class StepChunk
public readonly sbyte[] SwimZNW = new sbyte[CellsPerChunk];
/// <summary>
/// Swim layer — populated only for chunks containing at least one shore cell (a cell
/// with both a walkable land surface and a water surface separated by > StepHeight).
/// On shore cells, queries from the swim source Z miss the primary source-Z guard;
/// the swim layer carries the correct wetMask + per-direction destination Zs computed
/// from the water surface's perspective. For non-shore cells in a chunk that has the
/// layer, <see cref="SwimSourceZ"/>[cell] = <see cref="NoSwimLayerCell"/> sentinel.
/// All swim-layer arrays are null on chunks with no shore cells (~90% of map chunks
/// on Trammel) — zero memory cost on the common case.
/// Marks a cell with no swim-layer entry, in a chunk that has the layer.
///
/// The swim layer exists only on chunks holding at least one shore cell — a cell with both a
/// walkable surface and a water surface more than StepHeight apart. A swim query there sits
/// too far from the primary SourceZ to pass the source-Z guard, so the layer carries a second
/// mask and destination-Z set computed from the water surface instead. Chunks with no shore
/// cells leave every swim-layer array null.
/// </summary>
public const sbyte NoSwimLayerCell = sbyte.MinValue;
@ -79,9 +77,8 @@ internal sealed class StepChunk
public sbyte[] SwimZNW_Layer => _swimZNW_extra;
/// <summary>
/// Lazily allocates the swim-layer arrays and seeds <see cref="SwimSourceZ"/> with
/// the <see cref="NoSwimLayerCell"/> sentinel. Called at bake time the first time a
/// shore cell is detected in this chunk.
/// Allocates the swim-layer arrays and seeds <see cref="SwimSourceZ"/> with
/// <see cref="NoSwimLayerCell"/>. Called on the first shore cell found in this chunk.
/// </summary>
internal void AllocateSwimLayer()
{
@ -105,28 +102,30 @@ internal sealed class StepChunk
}
}
/// <summary>Sentinel: cell has no strata — single-Z, use the main Walk/Wet arrays.</summary>
/// <summary>
/// Marks a single-Z cell: no strata, read the main Walk/Wet arrays instead. Because this
/// takes ushort.MaxValue, a real strata offset is at most NoStrata - 1, which bounds
/// <see cref="StrataData"/> to NoStrata bytes.
/// </summary>
public const ushort NoStrata = ushort.MaxValue;
/// <summary>
/// Length-256 offset table: <c>StrataOffsetByCell[cell] = byte offset</c> into
/// <see cref="StrataData"/> where this cell's strata begin, or <see cref="NoStrata"/>
/// for cells without multi-Z. Null when the chunk has zero multi-Z cells.
/// Length-256 table mapping a cell to the byte offset in <see cref="StrataData"/> where its
/// strata begin, or <see cref="NoStrata"/>. Null when no cell in the chunk is multi-Z.
/// </summary>
private ushort[] _strataOffsetByCell;
/// <summary>
/// Packed per-cell strata. For each cell with strata:
/// u8 stratumCount, then stratumCount × Stratum (19 bytes each):
/// sbyte zCenter, byte walkMask, byte wetMask,
/// sbyte walkZ_N..NW (8), sbyte swimZ_N..NW (8)
/// Packed strata for the multi-Z cells. Per cell: u8 stratumCount, then stratumCount records
/// of <see cref="StratumByteLength"/> bytes — sbyte zCenter, byte walkMask, byte wetMask,
/// sbyte walkZ_N..NW (8), sbyte swimZ_N..NW (8).
/// </summary>
private byte[] _strataData;
/// <summary>Snapshot of Sector.MultisVersion at the time this chunk was built.</summary>
/// <summary>Reserved. Chunks are static-only, so this is always 0.</summary>
public int BuiltMultisVersion;
/// <summary>Updated on every cache hit/miss. Used by LRU fallback eviction.</summary>
/// <summary>Refreshed on every query that reaches this chunk. Drives LRU eviction.</summary>
public long LastTouchedTicks;
/// <summary>Size in bytes of one Stratum record in StrataData.</summary>
@ -140,10 +139,8 @@ internal sealed class StepChunk
_strataData == null ? ReadOnlySpan<byte>.Empty : _strataData.AsSpan();
/// <summary>
/// Single-shot setter for the chunk's strata. Pass null/null to clear (chunk becomes
/// "no multi-Z"). Otherwise <paramref name="offsetByCell"/> must be length 256 with
/// <see cref="NoStrata"/> for cells without strata, and <paramref name="data"/> the
/// packed strata records.
/// Sets the chunk's strata in one shot. <paramref name="offsetByCell"/> must be length 256,
/// carrying <see cref="NoStrata"/> for single-Z cells. Pass null/null to clear.
/// </summary>
internal void SetStrata(ushort[] offsetByCell, byte[] data)
{
@ -151,18 +148,17 @@ internal sealed class StepChunk
_strataData = data;
}
/// <summary>Serialization hook: returns the raw offset array (or null if no strata).</summary>
/// <summary>Serialization hook: the raw offset array, or null if the chunk has no strata.</summary>
internal ushort[] GetStrataOffsetByCellForSerialization() => _strataOffsetByCell;
/// <summary>Serialization hook: returns the raw data array (or null if no strata).</summary>
/// <summary>Serialization hook: the raw data array, or null if the chunk has no strata.</summary>
internal byte[] GetStrataDataForSerialization() => _strataData;
/// <summary>
/// True when every cell shares one value across WalkMask, WetMask, SourceZ, and all 16
/// directional-Z arrays, and the chunk has neither multi-Z strata nor a swim layer. Such a
/// chunk serializes to a ~28-byte uniform record (StepCacheFile v5) instead of the full
/// record. Chunks with a swim layer (shore cells) are never uniform — their per-cell swim
/// data must be preserved via the Full record.
/// True when all 256 cells share one value across WalkMask, WetMask, SourceZ and every
/// directional-Z array, with no strata and no swim layer — open water or solid rock, mostly.
/// <see cref="StepCacheFile"/> collapses such a chunk to a ~28-byte record. A swim layer
/// disqualifies a chunk outright: its per-cell shore data would not survive the collapse.
/// </summary>
internal bool IsUniform() => _strataOffsetByCell == null
&& !HasSwimLayer

12
Projects/UOContent/Engines/Pathing/Cache/StepMask.cs
View file

@ -1,10 +1,10 @@
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Per-cell, per-direction static walkability data baked by <see cref="StepProbe"/>
/// and stored by <see cref="StepCache"/>. WalkMask + WalkZ_* applies under default-walker
/// rules (cantWalk=false, canSwim=false). WetMask + SwimZ_* applies under swim-only rules
/// (cantWalk=true, canSwim=true). Algorithms layer the right rules per mobile.
/// Per-cell, per-direction walkability baked by <see cref="StepProbe"/> and stored by
/// <see cref="StepCache"/>. Two rule sets travel together: WalkMask + WalkZ_* for a default
/// walker (cantWalk=false, canSwim=false), WetMask + SwimZ_* for a swim-only mob
/// (cantWalk=true, canSwim=true). Callers overlay whichever applies to the mobile.
/// </summary>
public readonly struct StepMask(
byte walkMask,
@ -49,8 +49,8 @@ public readonly struct StepMask(
public readonly CacheHitKind HitKind = hitKind;
/// <summary>
/// True when the cache produced a usable answer (Hit / Miss_NotBuilt / Miss_DirtyRebuild).
/// False on Fallthrough_*, in which case the caller must use the slow path for this cell.
/// True when the cache produced a usable answer. False on any Fallthrough_*, where the
/// payload is all zeroes and the caller must resolve this cell via the slow path.
/// </summary>
public bool IsHit => HitKind <= CacheHitKind.Miss_DirtyRebuild;

74
Projects/UOContent/Engines/Pathing/Cache/StepProbe.cs
View file

@ -4,39 +4,37 @@ using CalcMoves = Server.Movement.Movement;
namespace Server.Engines.Pathing.Cache;
/// <summary>
/// Computes static-only walkability for a single cell — the per-cell, per-direction
/// "can step" mask and destination Z, based purely on land + statics.mul tiles (NOT
/// multis). Mirrors <see cref="MovementImpl"/>.Check minus the item and mobile collision
/// phases. Multis (houses, boats) are intentionally excluded: they're dynamic content, so
/// cells they cover route to the live movement path via <see cref="StepCache"/>'s
/// multi-halo fallthrough rather than being baked into the static chunk cache.
/// Computes the 8-direction "can step" mask and destination Zs for a single cell from land and
/// statics alone. Mirrors <see cref="MovementImpl"/>.Check minus the item and mobile collision
/// phases, which belong to the caller's dynamic-obstacle pass.
///
/// Multis (houses, boats) are excluded from the static bake because they are dynamic content;
/// cells they cover route to the live movement path via <see cref="StepCache"/>'s multi halo.
/// <see cref="ComputeMultiMaskAt"/> is the opt-in exception for those cells.
///
/// Each call bakes both rule sets: walker (canSwim=false, cantWalk=false) and swim-only
/// (canSwim=true, cantWalk=true). Diagonal corner-cut is not applied — callers hold the partner
/// bits in the same mask byte and combine them at query time.
/// </summary>
/// <remarks>
/// Bakes two rule sets per cell: walker (canSwim=false, cantWalk=false) and swim-only
/// (canSwim=true, cantWalk=true). Item / mobile collision phases are omitted (they're
/// the dynamic-obstacle pass's job). Diagonal corner-cut is NOT applied here; callers
/// must AND the partner-cell results at query time.
/// </remarks>
public static class StepProbe
{
private const int PersonHeight = 16;
private const int StepHeight = 2;
/// <summary>
/// Writes the distinct surface Zs at (x, y) that a default walker (PersonHeight envelope)
/// can actually STAND on — each candidate surface (walkable land center + every walkable
/// static top) that has PersonHeight of vertical clearance free of impassable statics —
/// into <paramref name="zs"/>, ascending, and returns the count.
/// Writes the surface Zs at (x, y) a default walker can actually stand on into
/// <paramref name="zs"/>, ascending, and returns the count. A candidate surface — the
/// walkable land centre, or any walkable static's top — qualifies only if a PersonHeight
/// envelope above it is clear of impassable statics.
///
/// Clearance-aware: it drops surfaces a creature cannot occupy (land under a sewer walkway,
/// ground under a low bridge), so the result is exactly the set of standing Zs the slow path
/// can resolve to. Two standable surfaces are inherently &gt;= PersonHeight apart (an upper
/// surface within PersonHeight of a lower one removes the lower one's clearance), so a
/// single ascending pass with an exact-duplicate skip is sufficient.
/// The clearance test is what makes this the exact set of standing Zs the slow path can
/// resolve to: it drops surfaces a creature cannot occupy, like the land beneath a sewer
/// walkway or a low bridge. That in turn means two surviving surfaces are always at least
/// PersonHeight apart (an upper surface any closer would have taken the lower one's
/// clearance away), so one ascending pass with a duplicate skip suffices.
///
/// Used by the baker to capture walkable static-over-land surfaces (sewer/dungeon
/// walkways, bridges, raised foundations, upper building floors) that the land-anchored
/// main mask would otherwise miss.
/// The baker anchors each cell here so static-over-land geometry — walkways, bridges, raised
/// foundations, upper floors — bakes at the Z a creature stands on rather than the land average.
/// </summary>
public static int ComputeStandableSurfaceZs(Map map, int x, int y, Span<sbyte> zs)
{
@ -106,21 +104,17 @@ public static class StepProbe
ComputeMaskCore(map, x, y, sourceZ, includeMultis: false);
/// <summary>
/// Multi-aware counterpart to <see cref="ComputeMaskAt"/>: synthesizes the full 8-direction
/// walkability mask for a cell covered by (or adjacent to) a multi, folding house/boat component
/// tiles into the surface/step logic via GetStaticAndMultiTiles. Replaces the slow path's 8x
/// per-cell CheckMovement for Fallthrough_Multi cells. Item/mobile collision is still handled by
/// the caller's dynamic-obstacle pass.
/// Multi-aware counterpart to <see cref="ComputeMaskAt"/>, for cells a multi covers or
/// neighbours: folds house/boat component tiles into the same surface/step logic. Builds the
/// whole 8-direction mask in one pass, where the slow path would run CheckMovement eight times.
/// </summary>
public static StepMask ComputeMultiMaskAt(Map map, int x, int y, sbyte sourceZ) =>
ComputeMaskCore(map, x, y, sourceZ, includeMultis: true);
/// <summary>
/// Shared per-cell 8-direction mask builder. With includeMultis=false this reproduces the
/// static-only bake (land + statics.mul). With includeMultis=true it also folds in multi
/// (house/boat) component tiles via GetStaticAndMultiTiles — the multi-aware synthesizer used
/// for Fallthrough_Multi cells. Item/mobile collision phases are still omitted (the dynamic pass
/// owns them).
/// Shared 8-direction mask builder behind <see cref="ComputeMaskAt"/> and
/// <see cref="ComputeMultiMaskAt"/>. <paramref name="includeMultis"/> is the only difference:
/// it swaps the tile source to GetStaticAndMultiTiles so house and boat components participate.
/// </summary>
private static StepMask ComputeMaskCore(Map map, int x, int y, sbyte sourceZ, bool includeMultis)
{
@ -140,8 +134,8 @@ public static class StepProbe
byte wetMask = 0;
Span<sbyte> walkZs = stackalloc sbyte[8];
Span<sbyte> swimZs = stackalloc sbyte[8];
// stackalloc is NOT zero-initialized — unwritten slots hold whatever was on the
// stack. Clear before use; the loop only writes slots where the step succeeds.
// stackalloc is not zero-initialized, and the loop below writes a slot only where the
// step succeeds, so blocked directions would otherwise carry stack garbage.
walkZs.Clear();
swimZs.Clear();
@ -178,10 +172,10 @@ public static class StepProbe
}
/// <summary>
/// Returns the slow path's standing-Z for a default walker at (x, y). Mirrors
/// MovementImpl.Check's surface-selection — paver Z+1 for paver-over-ground,
/// landCenter for bare land. Used by <see cref="StepCache"/> to bake SourceZ so
/// A*'s tracked-per-cell Z matches the cache's bake-time assumption.
/// The standing-Z a default walker at (x, y) resolves to under the slow path's
/// surface-selection rules: paver Z+1 over paver-on-ground, land centre on bare land.
/// The baker anchors cells with <see cref="ComputeStandableSurfaceZs"/> instead, which is
/// clearance-aware; this remains the direct MovementImpl equivalent for parity checks.
/// </summary>
public static int ComputeStandingZ(Map map, int x, int y, int locZ)
{

78
Projects/UOContent/Engines/Pathing/PathCacheCommands.cs
View file

@ -8,23 +8,23 @@ namespace Server.Engines.Pathing;
/// <summary>
/// Admin commands for inspecting and operating the pathfinding step cache.
/// [PathCacheStats — current resident-chunk count + hit/miss/eviction telemetry.
/// [PathCacheClear — drop all cached chunks, close lazy readers, zero counters.
/// [PathBake — walk a whole map building the full static cache, then save it.
/// [PathCacheSave — persist resident chunks per map to Data/Pathfinding/&lt;mapId&gt;.swb.
/// [PathCacheLoad — open those files as lazy backing stores. Also runs at startup.
/// [PathRecord — toggle JSONL telemetry capture for replay / benchmark corpora.
/// [PathCacheStats — resident-chunk count and hit/miss/eviction telemetry.
/// [PathCacheClear — drop all cached chunks, close the .swb readers, zero the counters.
/// [PathBake — build a map's full static cache and save it.
/// [PathCacheSave — persist the resident chunks to Data/Pathfinding/&lt;mapId&gt;.swb.
/// [PathCacheLoad — open those files as backing stores. Also runs at startup.
/// [PathRecord — toggle capture of pathfind telemetry.
///
/// The step cache works WITHOUT any .swb file — chunks build on demand as creatures path.
/// A baked .swb is an optional optimization that removes first-pathfind-after-boot latency
/// for shard owners who want it; <see cref="OnPathBake"/> is how you produce one.
/// None of this is required: the cache builds chunks on demand as creatures path, with or without
/// a .swb on disk. Baking one is purely an optimization that trades disk and a few minutes of bake
/// time for the removal of first-pathfind-after-boot latency.
/// </summary>
public static class PathCacheCommands
{
private static readonly ILogger logger = LogFactory.GetLogger(typeof(PathCacheCommands));
// modernuo.json flag: when true, Initialize() bakes any missing/stale .swb at startup.
// The first-boot ConfigurePrompts() prompt writes it.
// When set, Initialize() bakes any missing or stale .swb at startup. ConfigurePrompts() asks
// for it on first boot.
private const string PrebakeSetting = "pathfinding.prebakeMaps";
private static string PathFor(int mapId) =>
@ -32,9 +32,8 @@ public static class PathCacheCommands
public static void Configure()
{
// Resident-chunk cap is shard-tunable. Default 8192 ≈ 40 MB; small shards may
// want lower, large shards (or full-map bakes) may want higher. Setting is
// written back to server.cfg on first boot for discoverability.
// Resident-chunk cap, shard-tunable — the default works out to roughly 40 MB. Written back
// to server.cfg on first boot so it's discoverable.
StepCache.Instance.MaxResidentChunks = ServerConfiguration.GetOrUpdateSetting(
"pathfinding.maxResidentChunks",
8192
@ -52,13 +51,13 @@ public static class PathCacheCommands
}
/// <summary>
/// First-boot prompt, auto-invoked by <c>AssemblyHandler.Invoke("ConfigurePrompts")</c> in
/// the startup sequence — after assemblies load (so content can prompt) but before Serilog
/// starts, so the console prompt isn't interleaved with async log output. Offers to pre-bake
/// the pathfinding <c>.swb</c> cache for the selected maps; the answer persists in
/// modernuo.json (<see cref="PrebakeSetting"/>), so it's asked exactly once. Skipped when the
/// setting already exists or when input is redirected (headless/CI) — operators can set the
/// flag directly. The bake itself happens later in <see cref="Initialize"/>.
/// Asks, once, whether to pre-bake the .swb cache; <see cref="Initialize"/> does the work later.
/// The answer persists, so the question is never repeated, and it's skipped entirely when input
/// is redirected — a headless or CI boot sets <see cref="PrebakeSetting"/> directly instead.
///
/// Runs in the ConfigurePrompts phase because that's the one window where content can prompt:
/// assemblies are loaded, but Serilog hasn't started, so console output won't interleave with
/// async log writes.
/// </summary>
public static void ConfigurePrompts()
{
@ -81,16 +80,15 @@ public static class PathCacheCommands
}
/// <summary>
/// Auto-invoked by <c>AssemblyHandler.Invoke("Initialize")</c> after the tile matrix and
/// world are loaded. When <see cref="PrebakeSetting"/> is set, bakes any map whose
/// <c>.swb</c> is missing or stale, so the first pathfind on each region is already warm. A
/// fresh cache makes this a no-op, so only first boot — or a client/map update that changes
/// the fingerprint — pays the cost.
/// Bakes any map whose <c>.swb</c> is missing or stale, when <see cref="PrebakeSetting"/> is
/// set. Runs in the Initialize phase, once the tile matrix and world are loaded. An up-to-date
/// cache makes it a no-op, so the cost lands only on a first boot or after a client or map
/// update moves the fingerprint.
///
/// Validity is decided by <see cref="StepCache.HasLazyReader"/>: <see cref="Configure"/> runs
/// <see cref="AutoLoadAtStartup"/> in the earlier Configure phase, opening (and fingerprint-
/// validating) a reader for every up-to-date <c>.swb</c>. So a map with an open reader is
/// already good and we skip it — no need to recompute the fingerprint a second time here.
/// A map is judged up-to-date by whether it has an open reader. <see cref="AutoLoadAtStartup"/>
/// already ran in the earlier Configure phase and only opens a reader for a .swb whose
/// fingerprint validates, so an open reader is proof of a good bake — no need to fingerprint
/// the map a second time here.
/// </summary>
public static void Initialize()
{
@ -110,7 +108,7 @@ public static class PathCacheCommands
if (StepCache.Instance.HasLazyReader(map.MapID))
{
continue; // AutoLoadAtStartup already opened a fingerprint-valid .swb for this map
continue; // already has a fingerprint-valid .swb open
}
var path = PathFor(map.MapID);
@ -127,15 +125,14 @@ public static class PathCacheCommands
if (baked > 0)
{
logger.Information("PathBake: pre-bake complete ({Count} map(s) written).", baked);
AutoLoadAtStartup(); // (re)open the freshly written files as lazy backing stores
AutoLoadAtStartup(); // reopen what we just wrote
}
}
/// <summary>
/// Open Data/Pathfinding/&lt;mapId&gt;.swb as a lazy backing store for every map.
/// Reads only the header + chunk-offset index up front (~16 bytes per chunk);
/// individual chunk records are fetched on demand when the cache asks for them.
/// RAM stays bounded by MaxResidentChunks regardless of file size.
/// Opens Data/Pathfinding/&lt;mapId&gt;.swb as a backing store for every map. Only the header and
/// index are read up front; chunk records are fetched as the cache asks for them, so resident
/// memory stays bounded by the LRU cap however large the files are.
/// </summary>
private static void AutoLoadAtStartup()
{
@ -196,9 +193,9 @@ public static class PathCacheCommands
continue;
}
// BakeMap walks the whole map (building every chunk) and writes the .swb. The
// chunks are left resident afterward; drop them so peak memory is bounded to one
// map at a time and the post-command footprint returns to the LRU cap.
// BakeMap leaves every chunk it built resident. Drop them between maps so peak memory
// is one map's worth rather than all of them, and the footprint afterwards is back
// under the LRU cap.
var written = StepCache.Instance.BakeMap(map.MapID, PathFor(map.MapID));
StepCache.Instance.ClearResidentChunks();
@ -218,8 +215,7 @@ public static class PathCacheCommands
return;
}
// Reopen the freshly written files as lazy backing stores so they're usable now
// without a restart (resident memory stays bounded by the LRU cap).
// Reopen what we just wrote, so the bake is usable immediately without a restart.
AutoLoadAtStartup();
from.SendMessage($"PathBake: {totalChunks} chunks across {totalMaps} map(s) in {sw.Elapsed.TotalSeconds:F1}s; lazy readers reopened.");
}

44
Projects/UOContent/Engines/Pathing/PathDiag.cs
View file

@ -8,26 +8,19 @@ using Server.Targeting;
namespace Server.Engines.Pathing;
/// <summary>
/// Developer diagnostic for the bitmap A* step cache. Stand where a creature would start,
/// run <c>[PathDiag</c>, and target the goal. The detailed report is appended to
/// <c>Logs/pathdiag.log</c>; a short summary is sent to the invoking client. For the route
/// it records:
/// 1. the raw tile makeup of the start and goal cells (land + statics) and the
/// clearance-aware standable surfaces the baker anchors to — the ground truth for
/// "why does the cache (not) serve this cell";
/// 2. one warm <see cref="StepCache.TryGetMask"/>-served Find with the per-pathfind cache
/// hit/fallthrough breakdown and fallthrough fraction — a high fallthrough fraction
/// means the cache isn't helping the route (it pays the lookup then uses the slow path);
/// 3. warm timing over many iterations.
/// Diagnoses why the step cache does or doesn't serve a given route. Stand where the creature
/// would start, run <c>[PathDiag</c>, target the goal; the full report lands in
/// <c>Logs/pathdiag.log</c> and a summary goes to the client. It reports the tile makeup of the
/// start and goal cells alongside the standable surfaces the baker anchors to, the cache
/// hit/fallthrough breakdown for one warm Find, and warm timings.
///
/// Primarily useful when bringing up custom maps / facets: it shows whether static-over-land
/// geometry (dungeon walkways, bridges, stairs, raised foundations, stacked floors) is being
/// baked at the right Z.
/// The fallthrough fraction is the number to read: a high one means the cache is paying for a
/// lookup on every cell and then taking the slow path anyway. That usually points at
/// static-over-land geometry — dungeon walkways, bridges, stairs, stacked floors — baking at the
/// wrong Z, which is why this is most useful when bringing up a custom map or facet.
///
/// Output goes to a log file rather than the console because the live server uses Serilog and
/// raw Console writes interleave badly with it. The promotion gate is forced to eager
/// (threshold 1) for the duration so the cache builds on first touch and the numbers reflect
/// its best case; the previous threshold is restored afterward.
/// The promotion gate is forced eager for the duration, so the numbers reflect the cache's best
/// case rather than an artifact of chunks not having been built yet.
/// </summary>
public static class PathDiag
{
@ -67,7 +60,7 @@ public static class PathDiag
var cache = StepCache.Instance;
var previousThreshold = cache.MissPromotionThreshold;
cache.MissPromotionThreshold = 1; // eager build — measure the cache's best case
cache.MissPromotionThreshold = 1; // build eagerly, so we measure the cache's best case
StreamWriter log = null;
try
@ -99,9 +92,8 @@ public static class PathDiag
}
/// <summary>
/// Writes the raw tile makeup of one cell plus the surfaces the baker anchors to. A large
/// gap between the query Z and the standable surfaces is the signature of a route the
/// cache can't serve (the creature stands on a static surface far from the land average).
/// Dumps one cell's tiles and the surfaces the baker anchors to. A wide gap between the query Z
/// and every standable surface is the signature of a cell the cache can't serve.
/// </summary>
private static void DumpCell(TextWriter log, Map map, int x, int y, int queryZ, string label)
{
@ -133,9 +125,8 @@ public static class PathDiag
}
/// <summary>
/// Runs one warm Find and records the StepCache counter delta for it — the per-pathfind
/// cache hit/fallthrough mix and the fallthrough fraction. Returns a summary for the
/// caller to relay to the player.
/// Runs one Find against a warm cache and reports the counter delta it produced — the
/// hit/fallthrough mix for that single pathfind.
/// </summary>
private static (string result, double fallthroughPct, long total) RunInstrumentedFind(
TextWriter log, Mobile from, Map map, Point3D start, Point3D goal
@ -143,7 +134,8 @@ public static class PathDiag
{
var cache = StepCache.Instance;
// Warm every chunk the route touches before measuring.
// Build every chunk the route touches first, so the measured Find below reports steady-state
// behaviour rather than first-touch misses.
for (var i = 0; i < 3; i++)
{
BitmapAStarAlgorithm.Instance.Find(from, map, start, goal);

45
Projects/UOContent/Engines/Pathing/PathfindRecorder.cs
View file

@ -7,25 +7,14 @@ using Server.Text;
namespace Server.Engines.Pathing;
/// <summary>
/// Admin-toggled telemetry: appends a JSONL line per pathfind request to a file.
/// One record per BitmapAStarAlgorithm.Find call, capturing the inputs (start, goal,
/// map, capability flags) needed to replay the scenario in benchmarks. Output format
/// matches the corpus the BDN harness consumes.
/// Appends one JSONL record per pathfind, capturing the inputs — start, goal, map, capability
/// flags — needed to replay it later in a benchmark. Toggled at runtime with [PathRecord;
/// <see cref="Configure"/> only seeds the initial state from server.cfg.
///
/// Hot-toggleable at runtime via the [PathRecord admin command — no restart needed.
/// <see cref="Configure"/> only seeds the initial state from server.cfg
/// (pathfinding.recorder.enable, default false).
///
/// Holds a single StreamWriter open while recording; its internal buffer absorbs
/// per-record writes without per-call File.Open / File.Append. Each record is built
/// in a stack-allocated ValueStringBuilder (zero per-int allocation for the field
/// formatting), then handed to the writer as a ReadOnlySpan&lt;char&gt;.
///
/// <b>Workload note:</b> intended for short bursts of capture (turn on, walk a region
/// or trigger a scenario, turn off). On a busy server with hundreds of pathfinds
/// per second, sustained recording can saturate the StreamWriter's 4 KB buffer and
/// block the game thread on disk writes. A backpressure-aware async sink is a
/// future enhancement if 24/7 capture becomes a use case.
/// Meant for short bursts: turn it on, walk the region or trigger the scenario, turn it off. The
/// writes go through a StreamWriter's buffer on the game thread, so a busy shard doing hundreds of
/// pathfinds a second can saturate that buffer and stall the loop on disk I/O. Sustained capture
/// would need an async sink with backpressure.
/// </summary>
public static class PathfindRecorder
{
@ -52,9 +41,8 @@ public static class PathfindRecorder
}
/// <summary>
/// Toggle recording. When enabling, opens an append-mode StreamWriter; when
/// disabling, flushes + disposes it. Idempotent — calling twice with the same
/// state is a no-op.
/// Toggles recording, opening the file on enable and flushing and closing it on disable.
/// Idempotent.
/// </summary>
public static void SetEnabled(bool enabled)
{
@ -98,9 +86,8 @@ public static class PathfindRecorder
}
/// <summary>
/// Force a flush of the writer's internal buffer to disk. Safe to call when
/// disabled (no-op). Useful after a burst of recording when an admin wants to
/// inspect the file without waiting for buffer fill or disable.
/// Pushes the writer's buffer to disk, so a capture can be inspected without disabling first.
/// No-op when disabled.
/// </summary>
public static void Flush()
{
@ -115,9 +102,8 @@ public static class PathfindRecorder
}
/// <summary>
/// Capture one Find call. Hot path: cheap when disabled (single bool check).
/// When enabled, formats one JSONL line and writes it through the StreamWriter's
/// internal buffer — flush is amortized across many calls.
/// Records one Find. Sits on the pathfinding hot path, so it costs a single bool check when
/// disabled.
/// </summary>
public static void RecordIfEnabled(Mobile m, Map map, Point3D start, Point3D goal)
{
@ -140,9 +126,8 @@ public static class PathfindRecorder
try
{
// One interpolation handles every numeric field with no per-int ToString
// allocation; bool fields use explicit literal spans because JSON wants
// lowercase "true"/"false" and bool.ToString() yields "True"/"False".
// One interpolation covers every numeric field without a per-field ToString. The bools
// are appended as literals because JSON wants lowercase and bool.ToString() capitalizes.
using var vsb = ValueStringBuilder.Create(192);
vsb.Append(
$"{{\"Name\":\"recorded\",\"MapId\":{map.MapID},\"StartX\":{start.X},\"StartY\":{start.Y},\"StartZ\":{start.Z},\"GoalX\":{goal.X},\"GoalY\":{goal.Y},\"GoalZ\":{goal.Z},\"CanSwim\":"