Regression test that deserializes every spawned JSON file under
Data/Spawns/ via SpawnerDto, calls ToSpawner() to build the live
spawner, and deletes it to prevent world leaks. Catches both JSON
deserialization failures and exceptions during spawner construction,
collecting failures per file. Any failure names the offending file
and is a real migration defect, not a test weakness.
All 484 tests pass including the new AllSpawnFilesLoadTests.
- Rename "type" -> "$type" (first key, required for STJ polymorphism) in
all 109 spawn JSON files (5,612 records).
- Convert "homeRange" + "location" -> "spawnBounds" object, dropping homeRange.
Runtime keeps reading homeRange so un-migrated/external files still load.
- Add BoundsEquivalenceTests (hr=0/1/3/7) validating that the script's toBounds
formula produces the same Rectangle3D as the runtime homeRange path.
- Add MigratedDataLoadTests loading a real migrated file as List<SpawnerDto>
and round-tripping each DTO through ToSpawner().
- Add tools/spawner-json-migrate/migrate.mjs (offline Node script).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add ImportLocation/ImportMap read-only accessors to BaseSpawner.Json.cs
exposing _jsonLocation/_jsonMap for the importer post-STJ-deserialization.
- Rewrite ImportJsonSpawners to deserialize List<BaseSpawner> directly via
SpawnerJsonSerializer.Options (polymorphic on $type), eliminating
DynamicJson + AssemblyHandler.FindTypeByName + CreateInstance<ISpawner>.
- Enforce cleanup contract: every deserialized spawner (registered in World
at Map.Internal by STJ) is either MoveToWorld'd or Delete()'d — map==null/
Internal guard Deletes, inner catch Deletes on placement failure.
- Add internal ImportFile seam for testability (from=null safe; only used in
JsonException path which valid files never hit).
- Add ImportCleanupTests: valid placement, no-map orphan rejected, duplicate
location replaced — all with rigorous finally cleanup.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rewire ExportSpawnersCommand.ExecuteList to build List<BaseSpawner>
and serialize via SpawnerJsonSerializer.Options instead of the
DynamicJson.Create + spawner.ToJson loop. Add file serialize→deserialize
round-trip test with exception-safe cleanup.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add regression tests for homeRange property conversion on deserialize.
Tests verify that legacy JSON with homeRange field (no spawnBounds)
deserializes to centered Rectangle3D bounds as implemented in
BaseSpawner.OnAfterJsonDeserialize.
- homeRange: 3 -> Rectangle3D(97, 197, -128, 7, 7, 256)
- homeRange: 0 -> Rectangle3D(100, 200, 5, 1, 1, 0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add ProximitySpawner.Json.cs with shadow properties for triggerRange,
spawnMessage, and instant; OnAfterJsonDeserialize calls base then applies them
- Mark ProximitySpawner [JsonDiscoverableType] and its parameterless ctor
[JsonConstructor] for STJ polymorphism discovery
- Add ProximitySpawnerRoundTripTests with test-hygiene finally cleanup
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move spawner?.Delete() into the finally block so a placed (MoveToWorld'd)
spawner cannot leak into shared "Sequential UOContent Tests" map state if
an assertion throws. Declare spawner/s before the try and null-init them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add RegionSpawner.Json.cs with a `region` shadow property (serializes
SpawnRegion.Name) and OnAfterJsonDeserialize override that resolves the
region by name using _jsonMap from BaseSpawner. Mark RegionSpawner with
[JsonDiscoverableType] and [JsonConstructor] on its parameterless ctor
so SpawnerJsonSerializer discovers it for STJ polymorphism.
Add RegionSpawnerRoundTripTests: create+register a test BaseRegion on
Felucca (game regions are not loaded in the test environment), serialize
a RegionSpawner, assert $type and region name in JSON, deserialize, and
assert SpawnRegion.Name round-trips correctly. RED → GREEN confirmed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements Task 2 of the spawner STJ migration:
- SpawnerJsonSerializer: assembly discovery of [JsonDiscoverableType]-marked
BaseSpawner subclasses, STJ polymorphism on $type, property pruning to
[JsonPropertyName]-only, and OnDeserialized→OnAfterJsonDeserialize() hook.
- BaseSpawner.Json.cs: shadow JSON properties for all BaseSpawner fields,
OnAfterJsonDeserialize() virtual method that applies deserialized state.
- Spawner.Json.cs: JsonSpawnBounds shadow property, override of
OnAfterJsonDeserialize() to apply spawnBounds after base.
- Spawner.cs: [JsonDiscoverableType] + [JsonConstructor] on parameterless ctor.
- TestServerInitializer: SpawnerJsonSerializer.Configure() wired in.
- SpawnerRoundTripTests: 2 tests (round-trip core fields, omits domain defaults).
Old DynamicJson ctors and ToJson() left in place; removal is Task 9.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
## Problem
Houses and boats (multis) were pathed correctly only by **delegation to the slow path**: `StepCache.TryGetMask` returns `Fallthrough_Multi` for any multi-covered cell, and `GetSuccessors` ran `CheckMovement` **8× per cell** (each re-resolving the tile stack via `GetStaticAndMultiTiles`) — a sustained per-step cost near every house/boat. There was also no automated test pinning multi pathfinding.
This branch is the full multi-pathfinding effort in phases on one branch.
## Phase 1 — characterization tests (the oracle)
Implementation-agnostic invariants: a cache-on≡cache-off whole-path invariant, a per-cell sweep vs `CheckMovement` over footprint+halo (incl. destination Z), hand-verified routing (around walls, demolish-reopens, foundation-redesign-honored), classic-house / foundation / boat fixtures, non-vacuity guards. These gate every later phase byte-for-byte.
## Phase 2 — live single-pass synthesizer
`StepProbe.ComputeMultiMaskAt` synthesizes a covered cell's full 8-direction `StepMask` in one pass (the existing surface/step logic over `GetStaticAndMultiTiles` instead of 8× `CheckMovement`). `GetSuccessors` routes `Fallthrough_Multi` cells through it. No new cache, no `.swb` change. **~1.5×**, zero added allocations.
## Phase 3 / 3.1 — warm per-`multiID` interior cache (airtight)
`MultiMaskCache` caches each fixed multi's local-frame `StepMask` for **interior** cells (cell + all 8 neighbours covered → terrain-neighbour-free → position-invariant), keyed by `multiID & 0x3FFF`, built lazily from the MCL. Interior cells become ~20 ns lookups.
The cache is gated on a **per-instance footprint-clean flag** (`BaseMulti.PathInteriorCacheState`): an instance whose whole footprint terrain is below its floor (`maxTerrain < minFloor`) serves from the cache; a **dirty** instance (terrain intrudes — a contrived/GM placement) **degrades to live-synth, never a wrong mask**. This closes a cross-instance soundness gap (the cached mask depends on neighbour terrain too) found in a holistic review. The gate resets whenever the footprint's world-terrain relationship can change — **location, map, or ItemID** (a boat's heading swaps the MCL).
**Boats are cached too.** Their per-`multiID` deck masks are movement-invariant (built once per heading), so a sailing boat never rebuilds them; only the cheap clean-flag rescan repeats per move (and only when pathed near). Narrow existing boats have little interior; wide galleons (`multi.mul`) would gain Castle-class. `HouseFoundation` (per-instance runtime `DesignState`) is the one type that stays on the live path.
## Verification
- `UOContent.Tests` **454/454**, `Server.Tests` **708/708**, 0 failures.
- The Phase-1 oracle (`MultiPathInvariantTests`, cache-on ≡ cache-off) stays **byte-identical** with the synthesizer + interior cache active.
- Tests pin: footprint-cleanliness (clean vs sunk), dirty/cluttered placement degrades to live-synth while still pathing, clean placement serves, and the gate resets on move/ItemID change.
## Performance (modernuo/ModernUO-Benchmarks#8, full-fixture)
Houses at **Green Acres** (flat staff region → clean footprints, the legit-placement case):
| Route | Slow path | Phase 3.1 (interior cache) | Speedup |
|-------|----------:|---------------------------:|--------:|
| `around_a` (29 steps) | 238.3 µs | **49.1 µs** | **4.85×** |
| `around_b` (29 steps) | 224.3 µs | **49.5 µs** | **4.53×** |
~130 of ~167 multi cells/route serve from the cache (~20 ns) vs 37 live-synth. Per-cell, the slow path's 8× `CheckMovement` grows with multi complexity (GuildHouse ~857 ns → Castle ~1,194 ns), the synthesizer is a flat ~780 ns, and the cache serve is ~20 ns — so big/tall multis (and wide galleons) gain most. Identical allocations throughout.
Fixes#1690. Addresses Blood Oath holistically — three bugs found while researching the spell against RunUO, ServUO, the UODemise/uo.com guides, and the archived UOGuide page.
## Bugs fixed
### 1. Expiry timing (the filed issue)
The `ExpireTimer` polled every 1s, so expiry and death/delete cleanup lagged up to ~1s. Replaced with a **single-shot** timer plus centralized `[OnEvent]` handlers on `PlayerDeathEvent`/`PlayerDeletedEvent`/`CreatureDeathEvent`/`CreatureDeletedEvent` — the oath now breaks immediately on death/delete of either party.
### 2. Duration formula
Used `/80` (the bugged in-game tooltip value) instead of the real OSI formula `((SpiritSpeak - Resist) / 8) + 8`. Confirmed by RunUO, ServUO, the emulator guides, and the code's own fixed-point comment. At GM Spirit Speak this changes duration from ~9.5s to 23s and makes Spirit Speak actually affect duration.
### 3. Damage reflection (`BaseCreature.Damage` vs `PlayerMobile.Damage`)
`BaseCreature.Damage` diverged: it attributed the reflected hit to the attacker itself (`from.Damage(amount, from)`) instead of the caster, reflected the bonused (not original) amount, used `×1.1` vs `×1.2`, lacked the caster-survival guard, and had no Publish 48 resist mitigation.
Unified both paths: reflect the **original** damage attributed to the **caster** at `×1.2`. Publish 48 resist mitigation now applies only to creature casters and is gated behind `Core.SA`.
## Internals
- Collapsed the parallel `_oathTable` into a single `_table` keyed by both participants → shared timer, so `RemoveCurse` resolves from either side (required by the event handlers).
- Extracted `GetDurationSeconds` and `ComputeReflectedDamage` as testable statics.
## Tests
13 new tests (duration formula, reflection mitigation, oath lifecycle, end-to-end event-driven removal). Full suite: **436/436 pass**.
## Summary
Fixes the pathfinding step-cache (`.swb`) prebake so it bakes **once** and skips when a valid cache already exists, instead of re-baking on every boot. The root cause was the staleness fingerprint hashing mutable in-memory tile data rather than the on-disk files. This PR makes the fingerprint a pure function of the client data files and separates dynamic multis (houses/boats) from the static cache.
> This branch builds on the `ConfigurePrompts` first-boot-prompt unification (commit `5df8d0bd`, also included here) — that commit accounts for the `ServerConfiguration.cs` and `dev-docs/server-lifecycle.md` changes in the diff.
## The bug
With `pathfinding.prebakeMaps` set, the cache re-baked on **every** boot. The `.swb` staleness fingerprint hashed the live `TileData.LandTable`/`ItemTable` flags, which the server patches at runtime (`ItemFixes`, `LOSBlocker`, `PotionKeg`, `CTF`) at nondeterministic lifecycle points (Initialize-phase methods share a priority; static ctors fire lazily). So a fingerprint stamped at runtime (`[PathBake`) never matched the one recomputed during startup `Initialize()`, and the cache rebaked every time.
## Changes
**1. Fingerprint the files, not the in-memory tables** (`fix`)
Hash `tiledata.mul` (cached, computed once) plus the per-map `.mul`/`.uop` files — never the runtime-mutated `TileData` tables. The fingerprint is now lifecycle-stable. Existing `.swb` files rebake once after deploy, then stay stable.
**2. Compute the fingerprint once per boot** (`refactor`)
`Configure()`'s `AutoLoadAtStartup()` already opens and fingerprint-validates a reader for every up-to-date `.swb`. `Initialize()` now skips baking any map that already has an open reader (`StepCache.HasLazyReader`) instead of recomputing the fingerprint a second time.
**3. Bake static-only; route multis to the live path** (`refactor`)
Multis (houses/boats) are dynamic, so they're no longer baked into the static chunk cache — they were tagged with `BuiltMultisVersion`, a non-persisted session counter, which made persisting them unsafe (false matches / wasted re-bakes).
- Chunks bake land + `statics.mul` only.
- At query time, any cell whose sector (or its 1-cell halo) contains a multi routes to `Fallthrough_Multi` → the existing live, multi-aware `CheckMovement` path. The halo prevents a cell proposing a walkable edge into a neighbouring wall; interior (multi-free) cells pay one sector lookup.
- Adds `Sector.HasMultis` (one engine accessor); `.swb` format → v9 (rejects old multi-baked files); new `Fallthrough_Multi` telemetry.
- Behaviour-preserving: multis use the same live path the engine used before the cache existed.
**4. Comment polish** (`style`) — no behaviour change.
## Testing
All green: **92** pathfinding (incl. a new fingerprint-stability test and a multi-halo fallthrough test), **423** UOContent, **708** Server.
## Follow-ups (not in this PR)
- **Background bake worker** — make `[PathBake` and the boot prebake non-blocking (game thread serves tile reads to an off-thread worker).
- **Per-multi MCL cache** — cache walkability in each multi's own frame (keyed by multiID, movement-invariant) so houses/boats get a fast path instead of the live fallback.
- **House-pathfinding equivalence tests** — the one area not yet covered by a dedicated automated test; multi pathing is currently correct by delegation to the live path.
## Summary
Adds authentic **T2A-era (pre-UO:Third-Dawn) packet-based crafting menus**, enabled via the **`t2aCraftMenus` server setting** (read once at startup; default **`!Core.UOTD`**, so a pre-UO:TD shard gets them automatically). When enabled, double-clicking a crafting tool opens the classic `0x7C`/`0x7D` item-list menu — skill- and material-filtered — instead of the modern gump, covering all 8 tool/skill crafts (blacksmithy, tailoring, tinkering, carpentry, alchemy, bowcraft/fletching, inscription, cartography). It is **not** a runtime/admin-flippable feature flag.
This is the **definitive, reconciled** branch and **supersedes**:
- **#2181** (Delphi — `T2A_CraftingMenus`): the original effort.
- **#2381** (Jack/UOLL — `t2a_crafting_menus`): the research-grounded superset (Delphi's base + 12 corrections), rebased onto current `main`.
Original authorship is preserved across the cherry-picked history: foundation commit **@Delphi79**, mechanic fixes **@jackuoll (Jack Ward)**, reconciliation/fixes/docs mine.
## How it was built
1. Cherry-picked Jack's 13 commits onto current `main` (superset of Delphi's; only 2 trivial FeatureFlags conflicts).
2. Applied targeted fixes (below) with tests.
3. Full convention audit, build, and test pass.
Grounded in independent historical research plus Jack's deep dive. Maintainer reference: `dev-docs/t2a-crafting.md`.
## Mechanics (highlights)
- Double-click tool → target resource → skill/material-filtered menu → craft. Resource pre-selection per skill; make-last by targeting the tool.
- **Stacked-gem jewelry:** target a gem stack → the **full stack** is consumed and the piece is named by count ("a 1000 diamond ring"); count persists (`BaseJewel` serialization **v4 → v5**, new `_gemCount`).
- **Tool-less inscription & cartography** (skill-list invoked; no pen/sextant); inscription consumes reagents+scroll on success and failure, mana only on success.
- **Tailoring matching-hue consumption:** targeting hued cloth/leather consumes only that hue. Crafted items take color from their **`CraftResource`** (not the dyed hue), so dyed leather/cloth don't tint the product; in T2A only colored ingots/ore color items (metal armor/shields).
- **Half-resources on failed non-scroll crafts** (pre-UO:TD).
- **Maker's mark** always prompted for exceptional items, via the shared `QueryMakersMarkGump`.
- Server-side menu infra changes are additive (`ItemListEntry.CraftIndex`, `Entries` setter, `HasSent`).
## Notable changes on top of the cherry-pick
- **Toggle is a startup server setting, not a feature flag.** Removed `ContentFeatureFlags.T2ACraftMenus` (and its admin-flippable plumbing); the value is read once via `ServerConfiguration.GetSetting("t2aCraftMenus", !Core.UOTD)` into `T2ACraftSystem.Enabled`. Since the default tracks the era and it can't be flipped at runtime, there's no incoherent "menus-on / UO:TD-era" state.
- **Stacked-gem consumption (B3a/B3):** consume the full `PendingGemCount` (was deliberately consuming 1 while naming by the stack), null-safe gem type, plain-piece fallback + message. New `T2AJewelGemCraftTests`.
- **Convention audit:** `new List<Item>()` → `PooledRefList<Item>` on the hue-aware consume path; removed dead code.
## Decisions & deviations
- `make-last` kept as **QoL** (post-T2A gump-era feature).
- `half-on-failure` (non-scroll) kept as a **reconstruction** (not OSI-confirmed).
- **Stacked-gem** behavior set per shard authority (overrides the "single gem" reconstruction).
- **Cooking** out of scope (no T2A crafting menu existed for it).
- **No colored items from dyed materials:** crafted color comes from the `CraftResource` type. Pre-AOS leather has no colored variant, so leather is always uncolored; weapons retain resource color only in AOS+ (unchanged, intended).
## Test plan
- Automated: `dotnet build ModernUO.slnx -c Debug` clean; `dotnet test Projects/UOContent.Tests` → **421 passed** (incl. 3 new jewelry tests).
- Manual (needs a running T2A shard + client):
- [ ] Each of the 8 skills opens the correct menu; empty-menu guard fires.
- [ ] Make-last repeats the last craft (jewelry re-prompts gem).
- [ ] Jewelry consumes the full targeted gem stack and names by count.
- [ ] Cartography consumes blank maps only with T2A enabled / maps+scrolls when disabled.
- [ ] Tailoring consumes only the targeted-hue material; crafted items are not tinted by dyed cloth/leather.
- [ ] Maker's-mark prompt on exceptional.
- [ ] Failed non-scroll craft consumes half resources.
- [ ] Inscription: reagents+scroll on success/failure, mana only on success.
- [ ] T2A disabled: gump crafting unchanged.
## Credits
Co-authored-by: @Delphi79
Co-authored-by: @jackuoll
## Summary
Fixes two related pet-behavior bugs and the underlying design flaw behind both:
1. **Post-combat erratic** — after a pet killed its `all kill` target it milled around erratically at the kill site (or failed to return to the master) until the player issued `all follow`/`all stop`.
2. **`all stop` returns home** — a pet with a non-zero `Home` walked back toward that location on `all stop` (on ML; on non-ML the old `DoOrderStop` was a no-op, so the sighting there came from a residual `Stay`).
### Root cause
`ControlOrder` and the wild-creature `Home` field were overloaded to express several distinct ideas, mutated/read inconsistently across order transitions:
- `Home` doubled as the controlled-pet "stay anchor" (`HandleStayOrder` set `Home = Location`) but nothing cleared it when the pet left the staying state; `HandleStopOrder` was the only handler that never touched it.
- `DoOrderStop` had dropped RunUO's `Home = Location` re-anchor, so on ML it walked to a stale anchor.
- The post-combat fallback was a fragile `_lastPetOrder` hack in `DoOrderNone` that re-anchored a resumed `Stay` at the corpse.
- Controlled idle-wander bypassed the `CheckIdle()` rest gate that every non-controlled creature uses, so idling pets jittered every AI tick.
## Approach
Separate three concepts that were tangled together:
- **`ControlOrder`** — the active order (may be transient: Come/Attack/Drop).
- **Persistent command** (`PersistentOrder` ∈ `{None, Stay, Follow, Guard}`) — the standing directive a pet falls back to when a transient order completes. Runtime-only (not serialized; reset to `None` on load) and **derived from master proximity on login** (near → Follow, far → Stay).
- **Anchor** (`Home`) — a pure function of the persistent command, set only when that command changes (never on transient transitions or fallback-resume), so it can't go stale.
### Behavior
- **Stop** is resolved immediately from what the pet was doing: Attack/Come → resume the persistent command; Follow/Guard → cancel to idle where it stands; Stay → stay put.
- **Stay** holds its post (returns only if displaced, e.g. after a fight) — no shuffle.
- **Idle** (`None`) is a gentle wander routed through `CheckMove/CanMoveNow/CheckIdle`, so idling pets take the same 15–25s rest periods as other creatures, on both ML and non-ML.
- **Post-combat** the pet resumes its persistent command (a staying pet returns to its original post, not the corpse).
- **Release** without a spawner anchors where the pet stands instead of pathing to a stale anchor.
This restores the RunUO-intended behavior (verified against the RunUO reference) while fixing the ModernUO regressions.
## Tests
New `PetOrderTests` (13 deterministic xUnit tests) cover: anchor lifecycle, the full Stop truth table, report 1 (post-combat return to post), report 2 (no stale-anchor walk-home), frozen-Stay/gated-idle wiring, release fix, derive-on-login, and a non-ML spot-check. The subjective wander *feel* is covered by a manual-QA checklist in the implementation plan.
## Notes
- Engine project (`Projects/Server`) untouched; the one `BaseCreature.cs` change is the `ControlOrder` setter passing the previous order to `OnCurrentOrderChanged`.
- `DoOrderCome` keeps auto-converting to `Stay` on arrival, which under the new model cleanly means "come and hold near me."
- Commits in this PR are temporarily **unsigned** (the signing agent's passphrase cache expired mid-session); happy to re-sign / amend on request.
## Summary
Phase #3b (final roadmap item), stacked on #2470. Compacts the index trailer from 20 to 8 bytes/chunk.
Trammel: 19.2 MB → 17.9 MB. Roadmap total: 565 MB → 17.9 MB (−96.8%).
## Details
- Trailer stores `{ u32 packedKey = (ChunkX << 16) | ChunkY, u32 recordLength }` per chunk, in record write order; the file offset is dropped and reconstructed by cumulative recordLength from HeaderSize.
- No record reordering, no varint; fixed-stride, TryReadChunk unchanged.
- Also simplifies the accumulated `.swb` code comments across the stack.
- Format v8; v7 files rejected and re-baked once.
## Tests
v8 multi-chunk round-trip (cumulative offset reconstruction) + the v6/v7 suite; full pathfinding suite green; Release build clean.
## Summary
Phase #3a, stacked on #2469. Compresses each chunk record independently with libdeflate (random access preserved).
Trammel: 124.7 MB → 19.2 MB (−85%).
## Details
- Whole-record framing `[u32 UncompressedLen][payload]`; records that don't shrink (tiny Uniform) are stored raw, detected as payload length == UncompressedLen.
- Codec chosen by full-Trammel spike: libdeflate VeryHigh (16.5 MB, 1.83 µs/chunk decompress) over zstd L19/22 (17.4 MB) and managed Brotli q11 (16.4 MB) — best native ratio, fastest decompress, already the repo's packet codec (no new dependency).
- Reuses cached thread-static bindings: `Deflate.Maximum` for bake, `Deflate.Standard` for reads.
- Compression is bake-time only; decompression is one-time per chunk (LRU-cached).
- Format v7; v6 files rejected and re-baked once.
## Tests
v7 unit tests + the v6 suite run through the compression path; full pathfinding suite green; Release build clean.
## Summary
Phase #2 of the `.swb` step-cache size-reduction roadmap (after #2465, v5 uniform elision). Stores the 16 base directional Z arrays as masked residuals against each cell's own SourceZ and omits any array that matches its prediction. Lossless, byte-identical reconstruction.
Trammel: 231.9 MB → 124.7 MB (−46%).
## Details
- Predictor: `predict = mask bit ? SourceZ : 0` (matches the baker's 0 on unwalkable directions); residual `Z − predict` via unchecked two's-complement (byte-exact for all inputs); reconstruct `Z = predict + residual`.
- A `u16 ZArrayMask` flags which of the 16 base arrays differ from prediction; matching arrays are omitted and synthesized from mask + SourceZ at read.
- Serializer-layer only: StepChunk, the cache, the algorithm, and the baker are unchanged.
- Format v6; v5 files rejected and re-baked once.
## Tests
21 v6 unit tests; full pathfinding suite green; Release build clean.
## Issue
Fixes#2452. A player with 30 Ninjitsu reported that the Animal Form menu showed **every** form; selecting one above their skill (e.g. Dog, req 40) **consumed mana** and returned "you need at least 40 skill", and afterwards the **gump never reopened** — every recast silently re-attempted the unusable form and drained more mana.
## Root cause
Three linked bugs, all reproduced from the code:
1. **Gump not gated by skill.** `AnimalFormGump.BuildLayout` compared `Skill.Fixed` (which is `Value * 10`, so 30 skill → `300`) against the raw 0–100 `ReqSkill` (Dog = `40`). `300 >= 40` is always true, so all forms were shown. `Morph` itself correctly uses `.Value`.
2. **Mana charged on a no-skill cast.** `Morph` returns `MorphResult.NoSkill` for an under-skilled form, but both call sites (`OnCast`, `OnResponse`) only special-cased `MorphResult.Fail`; `NoSkill` fell through to the branch that deducts mana.
3. **Menu never reopened.** Per OSI ([uo.com](https://uo.com/wiki/ultima-online-wiki/skills/ninjitsu/), [uoguide](https://www.uoguide.com/Animal_Form)), casting while **standing still always opens the selection menu**, and casting while **moving** quick-transforms into the last selected form. ModernUO only opened the menu when `lastAnimalForm == -1`, so once any form was selected a stationary recast skipped the menu.
## Fix
- Add `AnimalForm.CanSelectEntry` (compares `Skill.Value` to `ReqSkill`, plus the talisman check) and use it for the gump's per-entry enable check.
- `OnCast`: standing still always opens the menu; moving quick-transforms into the last form. `NoSkill` no longer costs mana.
- `OnResponse`: handle `Success` / `Fail` / `NoSkill` explicitly so `NoSkill` costs no mana.
## Tests
Adds `AnimalFormTests`:
- `CanSelectEntry` rejects forms above skill, accepts forms at/below skill, and requires a talisman for talisman-gated forms.
- `Morph` returns `NoSkill` (without transforming) when under-skilled, and `Success` when sufficiently skilled.
Verified the gating test catches the regression (reintroducing `.Fixed` fails it). Full solution build is clean; the 5 new tests plus 284 other UOContent tests pass (the pathfinding/AI sequential tests were excluded only because they deadlock under concurrent local runs — they are unrelated to this change).
## Summary
Sub-project #1 of the `.swb` step-cache size-reduction roadmap (`dev-docs/pathfinding.md` § Future work). Adds **uniform-chunk elision** to the `StepCacheFile` format, bumping it **v4 → v5**.
A fully-uniform 16×16 chunk — no strata, **no swim layer**, all 19 base arrays constant (open ocean, Green Acres, void) — serializes to a **~28-byte record** (`KindUniform`) instead of ~5,393, and reconstructs **byte-identically** via `Array.Fill`. Non-uniform chunks use the existing v4 body (`KindFull`) with the swim-layer and strata trailers **fully preserved** — the Kind byte is just prepended.
## Calibrated result (measured, not projected)
Baked Trammel via `SaveToFile`:
| | |
|---|---:|
| Chunks | 114,688 |
| Uniform (swim-aware) → elided | 62.7% |
| Swim-layer chunks (stay Full) | 8.9% |
| Strata chunks (stay Full) | 1.8% |
| Baseline (full records) | 592.2 MB |
| **Actual v5 `.swb`** | **231.9 MB (−61%)** |
The residual is ~150 MB of non-uniform land Z-blocks (targeted by #2 predictive-Z) + ~81 MB of swim-layer trailers (#2/#3). #2 and #3 are separate follow-up PRs.
## Implementation
- `StepChunk.IsUniform()` — false if it has strata **or a swim layer**, else true only when all 19 base arrays are constant (the "all-same" check uses the SIMD-accelerated `ContainsAnyExcept`).
- `StepCacheFile` v5 — `Kind` byte (`KindFull=0`/`KindUniform=2`, 1 reserved); uniform write/read; `FormatVersion`/`MinSupportedVersion` → 5 (v4 files rejected on open and re-baked). No `StepCache`/algorithm/index changes; fingerprint logic untouched.
## Tests
7 `StepCacheFileV5Tests` (uniform round-trip + `<200 B` compactness, varied-full, swim-layer-full, strata-full, swim+strata combined, v4 version-gate rejection) + the existing StepCache/pathfinding suite — **70 pass**, including the prior `SwimLayer_RoundTrips`. An independent review verified write/read symmetry, cast round-tripping, swim/strata preservation, and the version gate (READY TO MERGE).
Fixes#2462
## Summary
Removes the per-object `VirtualHairInfo` heap wrapper for mobile/corpse hair. Hair is now stored **inline** on `Mobile` and `Corpse` as `int _hairItemId` / `int _hairHue` plus a lazily-allocated, **non-serialized** ephemeral `Serial _hairSerial` (in the high virtual-serial range) — and likewise for facial hair. The `VirtualHairInfo` class is deleted, with a **lossless** save migration.
This delivers three things:
1. **Fixes a hair-removal bug.** `Delta(MobileDelta.Hair)` is deferred (it enqueues; `ProcessDeltaQueue` runs later in the tick). The old `HairItemID = 0` setter nulled `_hair` *immediately*, so by the time `ProcessDelta` built the remove packet the equipped virtual serial was already gone — the old `??=` code then re-materialized a **fresh** serial (≠ the equipped one), so clients never removed the right entity, and it left a phantom ItemId-0 object behind. The serial now lives on the entity and **persists across removal**, so remove packets carry the correct serial.
2. **Lightens the entity.** No heap hair object; bald mobiles allocate nothing (the serial is minted lazily only when hair is present). This was the original reason `HairItemID`/`HairHue` exist.
3. **Removes `VirtualHairInfo` entirely**, keeping the high-range virtual serial behavior.
## How
- **Mobile** (manual serialization): inline `_hairItemId/_hairHue/_hairSerial` (+facial); lazy `HairSerial`/`FacialHairSerial`; `ProcessDelta` reads those. Serialization **v36 → v37** — the v30-v37 deserialize is unified, reading the legacy per-hair `VirtualHairInfo` version int only when `version < 37`. Setting item id to 0 clears the hue (matching the old object-nulling) while retaining the serial.
- **Corpse** (codegen serialization): decomposed to `[SerializableField] int _hairItemId/_hairHue` (+facial) + ephemeral serial; **v16 → v17** with `MigrateFrom(V16Content)`.
- **Lossless migration:** the loader validates exact byte length, and the old corpse hair is a presence-bool-gated block, so a tiny **migration-only** `LegacyHairInfo` reader (no runtime role) consumes the legacy `[bool][int ver][int itemId][int hue]` bytes. Frozen `Corpse.v14/v15/v16.json` are retyped to it; `v17.json` describes the new int fields.
- All consumers updated to discrete accessors: `OutgoingMobilePackets`, `CorpsePackets`, corpse subclasses (`MilitiaFighterCorpse`, `SchmendrickApprenticeCorpse`), and the packet test mirrors.
- `VirtualHair.cs` renamed to `OutgoingVirtualHairPackets.cs` (the only type left in it after `VirtualHairInfo` was removed).
## Test Plan
- [x] Full solution build: **0 warnings, 0 errors** (`TreatWarningsAsErrors`).
- [x] `Server.Tests`: **708 passed** (incl. new `RemoveHairUsesEquippedSerial` / `RemoveFacialHairUsesEquippedSerial` proving the serial survives removal + hue clears).
- [x] `UOContent.Tests` corpse/hair: **6 passed** (incl. `CorpseHairMigrationTests` asserting the legacy hair bytes are consumed exactly — the loader's length invariant).
- [x] Generated migration code inspected: V14/V15/V16 readers consume the legacy block byte-for-byte; serial never written to disk.
## Upgrade notes
- Old Mobile (v30–v36) and Corpse (v13–v16) saves load losslessly.
- Minor cosmetic-only change: `SchmendrickApprenticeCorpse` hair/facial-hair RNG draws shift order within each pair (same draw count); irrelevant for a quest NPC corpse.
## Summary
Closes the Cold-cache regression flagged in PR #2450. `StepCache.TryGetMask` no longer eagerly runs `BuildChunk` on the first miss for a chunk that isn't in a `.swb` lazy reader. Instead it returns `Fallthrough_NotBuilt` and the caller (`BitmapAStarAlgorithm`) takes the per-cell slow path. The chunk is only promoted to the bitmap fast path after the **second** miss within a 30-second window, filtering single-touch pass-throughs.
This makes BitmapAStar's worst-case (cold cache + short hops) collapse from **12–47× slower** than FastAStar to **roughly the same**, which is the floor the slow path can deliver. Steady-state warm performance (the actual deliverable) is unchanged from PR-5 — it was always the cache fast path.
## The pet-follow scenario this fixes
A mounted player at ~4 tiles/sec with a pet/hireable following will trigger an NPC pathfind every 100–300 ms. Each pathfind is 1–6 tiles. As the player crosses chunk boundaries (~4 sec/chunk), the pet's first pathfind in the new chunk under the previous behavior triggered a full ~700 µs `BuildChunk` for a chunk the player would leave shortly after. At 50–100 mobiles per shard, this exceeded the 8 ms tick budget. PR-5 BDN data showed scenarios 6–9 (2–8 tile NPC perception) at 2,300–3,700 µs Cold vs FastAStar's 80–200 µs.
Under the new gate:
- First miss → `Fallthrough_NotBuilt` → caller uses slow path (~30–50 µs short path). No `BuildChunk`. No allocation.
- Player keeps moving → chunk never gets a second touch within window → never promoted, no rot.
- NPC patrolling a fixed territory → repeatedly hits the same chunks → second touch within window → promote → cache fast path on subsequent calls.
## What changed
- **`CacheHitKind.Fallthrough_NotBuilt = 6`** + **`CacheStats.FallthroughNotBuilt`** counter. `IsHit=false`, so the caller routes to slow path.
- **`StepCache._chunkMissTracker`** — `Dictionary<long, ChunkMissState>` capped at 4096 entries. State is `(byte missCount, uint lastMissTickStamp)` keyed by chunk key. Window-expired entries reset count to 1; capacity overflow prunes window-old entries first.
- **`StepCache.MissPromotionThreshold`** (default `2`) and **`StepCache.MissPromotionWindowMs`** (default `30_000`) — tunable, can be wired through `ServerConfiguration` if shards want different policy. Setting threshold to `1` restores legacy eager-build behavior (used by tests that prime chunks via single `TryGetMask` call).
- **`StepCache.TryGetMask` miss branch** — try lazy reader first (file-loaded chunks bypass the tracker entirely; an `.swb` represents an explicit prior decision to keep the chunk warm). Otherwise consult the tracker.
- **`BitmapAStarAlgorithm.GetSuccessorsSlowPath`** now layers `IsBlockedByDynamic` on top of `CalcMoves.CheckMovement`. Previously the slow path only ran for `CanFly` creatures and rare cache fallthroughs — `CheckMovement` doesn't iterate same-cell mobiles, so the bitmap fast path's `IsBlockedByDynamic` was the only mobile-blocking check. Now first-touch pathfinds run through the slow path, so the gap had to close.
## Tests
50 pathfinding tests pass (was 47). New / updated:
- **`TryGetMask_FirstTouchOnUnbuiltChunk_DefersBuildAndReturnsFallthrough`** — single TryGetMask call returns `Fallthrough_NotBuilt`, no chunk built, no allocation.
- **`TryGetMask_SecondTouchWithinWindow_PromotesAndBuilds`** — second call inside the 30s window builds + serves.
- **`TryGetMask_SecondTouchAfterWindow_RestartsCounterAndDefers`** — second call outside the window restarts the count, returns Fallthrough again.
- **`TryGetMask_DistinctChunks_TrackedIndependently`** — counters are per-chunk; one touch on each of two adjacent chunks both stay in fallthrough.
- **`LazyReaderHit_BypassesMissTrackerOnFirstTouch`** — open `.swb` + first touch hits without consulting the tracker. Production with `.swb` loaded skips the gate entirely.
- **`MultisVersion_Bump_TriggersDirtyRebuild`** — updated to reflect the new 3-step flow (Fallthrough → Miss_NotBuilt → Miss_DirtyRebuild).
- Tests that prime chunks via a single `TryGetMask` call (multi-Z, Tier4, lifecycle, parity, BitmapAStar uses-cache) set `MissPromotionThreshold = 1` to opt into eager behavior.
## Expected BDN impact
The Cold column from PR-5's BDN should change as follows once the bench's submodule pointer is updated to this branch:
| # | Scenario | Cold (PR-5) | Cold (PR-6 expected) | FastAStar Cold |
|--:|-----------------|-------------:|---------------------:|---------------:|
| 2 | sewer corridor | 1,627 µs | ~36 µs | 36 µs |
| 4 | causeway | 1,533 µs | ~39 µs | 39 µs |
| 6 | pet 2-tile | 2,364 µs | ~80 µs | 81 µs |
| 8 | npc 5-tile | 3,708 µs | ~140 µs | 141 µs |
| 9 | npc 8-tile | 2,386 µs | ~200 µs | 197 µs |
WarmNoFile and LazyWarm rows should be unchanged — they were always cache-warm. The miss tracker only fires when neither resident chunks nor the lazy reader can satisfy the request.
## Future work (not in this PR)
- **Background-thread bake**: builds outside the game thread so even promoted chunks don't pay the 700 µs build cost on the main thread. Rule 10 (no Task.Run) applies, so this needs careful design — the bake is a pure data transform but main-thread synchronization on chunk-state transitions has to be threaded through. Defer to a follow-up.
- **Long-traverse BDN scenario**: a multi-Find benchmark simulating 50 pet repaths across chunk transitions. Requires restructuring the bench harness; the existing 10-scenario corpus + Cold provider already exercises the gate.
- **Swim sourceZ bake**: scenario 5 (sea serpent) shows 56 B alloc on warm paths because the cache's SourceZ is computed under default-walker rules. Swim creatures fall through to slow path. Independent of this PR.
## Summary
Creatures (pets following, monsters chasing, NPCs approaching) would **oscillate — "pace back and forth really fast"** at concave obstacles (reported at the Britain Inn L-desk: a pet at `(1493,1614,20)` never reaching its master at `(1494,1605,21)`) instead of routing around them.
**Root cause** (the A* pathfinder itself was correct): all goal-seeking funnels through `MoveTo` and `WalkMobileRange` → `MoveTowardsOrAwayFrom`, which step greedily via `DoMove(dir, badStateOk:true)`. `DoMove` returns `true` even when the direct step was blocked and the creature merely **auto-turned and sidestepped** (`MoveResult.SuccessAutoTurn`), and the caller then set `Path = null`, discarding the `PathFollower`. So at a concave obstacle a non-progressing sidestep was mistaken for progress and the creature never committed to a route. (AOS pet-follow runs at `CurrentSpeed = 0.1`, hence the "really fast" shuffle.)
## What changed
- **New centralized `BaseAI.ApproachTarget(target, run, range)` primitive.** A greedy step is committed only when it **fully succeeds (`MoveResult.Success`) and actually gets closer**; otherwise the creature commits to a **persistent `PathFollower`** that routes around the obstacle and is never discarded by a greedy step. The open-terrain fast path (one greedy step, no pathfinding) is preserved. `MoveTo`, `MoveTowardsOrAwayFrom`, and `MoveToWithCollisionAvoidance` all delegate to it — public signatures unchanged, so no AI-class call site changes.
- **Best-distance give-up + idle.** A creature that cannot reach a **stationary** in-range goal stops shuffling and idles after `ApproachGiveUpTicks` (40) ticks without lowering its closest-ever distance; a **moving** goal (active chase) never gives up. It resumes the moment the goal moves.
- **Pathfinder fix (required):** `BitmapAStarAlgorithm.IsBlockedByDynamic` now skips the dynamic mobile-block check **at the goal cell only** (`MoveImpl.Goal`). Previously A* returned `null` whenever the target mobile stood on the goal cell, so creatures could never pathfind *toward* another mobile — only toward empty ground. The follower stops within `range` short of it. Static/item blocking and all non-goal mobile blocking are unchanged.
## Tests
New AI-loop integration tests in `ApproachTargetTests.cs` drive the real `BaseAI` primitives against live Britain Inn map statics: exact-repro pet follow, open-terrain (asserts zero pathfinding), `MoveTo` chase (static + walking-away target), route-around-a-dynamic-wall, and walled-off give-up-and-idle.
- Pathfinding + AI subset: **52/52** pass.
- Full `UOContent.Tests`: **301/301** pass. (Note: the test host lingers on shutdown — a pre-existing infra quirk unrelated to this change; all tests complete and pass.)
- Full solution build: clean (0 warnings / 0 errors).
## Notes
- Branched off `main`; independent of the in-flight step-cache work.
- Out of scope (future work): proactive "SmartAI" look-ahead pathfinding so clever creatures plan a route before walking into the obstacle, rather than reacting after they hit it.
## Test Plan
- [X] In-game: order a pet to `follow`/`come` across the Britain Inn L-desk; confirm it routes around and reaches you instead of pacing.
- [X] Aggro a monster and kite it around a building/treeline; confirm it chases around obstacles.
- [X] Confirm open-terrain following/chasing feels unchanged (no extra latency).
- [X] Confirm a creature with a genuinely unreachable target idles rather than shuffling forever.
## Summary
Multi-Z cells (bridges, stairs, paver-over-ground, multi-floor structures) now carry **per-stratum walkability data** in the cache instead of falling through to the slow path. The data is computed at chunk-build time, persisted in the `.swb` file, and selected at query time by matching the request's `sourceZ` against each stratum's `zCenter` (within `StepHeight` tolerance).
This is the Tier 4 strata feature, deferred from PR #2447 / PR #2448 / PR #2449. Builds on PR #2449's lazy backing store and public bake helpers.
## Wire format change (v1 → v2)
`StepCacheFile.FormatVersion = 2`. `MinSupportedVersion = 2`. v1 `.swb` files are silently rejected at open time (treated as missing) and overwritten on the next `SaveToFile` / `BakeMap`. **No migration** — older files just get re-baked.
The `MinSupportedVersion` sentinel is the model going forward: bump the constant when an incompatible change lands; admins re-bake on the next deploy. No matrix of v1↔v2↔v3 migration logic to maintain.
## What changed
- **`StepProbe.ComputeStrataAt(map, x, y)`** — enumerates walkable standing-Zs at the cell (one per land surface plus one per walkable static), collapses Zs within `2*StepHeight`, runs `ComputeMaskAt` at each surviving Z. Returns `null` for single-Z cells (caller uses the chunk's main mask).
- **`StepChunk`** — replaces the old `MultiZCells` bitmap with a **strata storage pair**:
- `ushort[256] StrataOffsetByCell` (sentinel `NoStrata = 0xFFFF` = "no strata for that cell")
- `byte[] StrataData` packed: `u8 stratumCount`, then `count × 19-byte stratum`
- `sbyte zCenter, byte walkMask, byte wetMask, sbyte walkZ_N..NW (8), sbyte swimZ_N..NW (8)`
- `IsCellMultiZ` derives from `StrataOffsetByCell[cell] != NoStrata` — same semantics, single source of truth.
- **`StepCache.BuildChunk`** — populates strata for cells flagged multi-Z via `SetStrata`. Chunks with zero multi-Z cells pay zero strata overhead (offset array + data array stay null).
- **`StepCache.TryGetMask`** — for multi-Z cells, scans strata with `TryStratumHit`; returns the matching one with `HitKind=Hit`. Falls through to slow path only when no stratum matches the query `sourceZ`.
- **`StepCacheFile`** — v2 serialization with strata trailer per chunk + `recordLength` in index entry. Lazy reader sizes scratch per-chunk-record using the recorded length, growing on demand for multi-Z-heavy chunks. Patches `IndexOffset` on `w.Buffer` (BufferWriter's current backing array) since it grows during variable-size chunk writes.
## File layout v2
```
Header (48 bytes):
u32 Magic = 0x42575300 ('SWB\0')
u32 Version = 2
u32 MapId
u64 Fingerprint XxHash3 over LandTable + ItemTable flags + map files (mapX.mul/.uop, staidxX.mul, staticsX.mul)
u64 BakeTimestamp informational
u32 ChunkCount
u64 IndexOffset position where chunk index begins
Per chunk (variable size):
u16 ChunkX, ChunkY
u32 BuiltMultisVersion
u8 HasStrata 0 = no strata trailer; 1 = strata trailer follows
byte WalkMask[256], WetMask[256]
sbyte SourceZ[256], WalkZN..WalkZNW[256], SwimZN..SwimZNW[256]
// Strata trailer (only when HasStrata == 1):
u16 StrataOffsetByCell[256] // NoStrata sentinel = 0xFFFF
u32 StrataDataLength
byte StrataData[StrataDataLength]
Per multi-Z cell: u8 count, then count × Stratum (19 bytes)
Index trailer (20 × ChunkCount bytes):
per chunk: { u64 chunkKey, u64 fileOffset, u32 recordLength }
```
## Summary
Adds two pieces of pathfinding tooling on top of PR #2448's lazy `.swb` infrastructure:
- **`PathfindRecorder`** — admin-toggled JSONL telemetry capture; one record per `BitmapAStarAlgorithm.Find` call. Output format matches the BDN harness corpus, so production traffic can be captured and replayed in benchmarks without an adapter.
- **Public bake helpers on `StepCache`** — `ComputeLiveTileDataHash`, `TryReadTileDataHashFromFile`, `BakeMap`, `ClearResidentChunks`. Lets the benchmark project (and any future bake utility) drive cache fill + persist without exposing internal types.
The companion BDN harness update lives in [ModernUO-Benchmarks#kb/pathfinding-pr4-bench](https://github.com/modernuo/ModernUO-Benchmarks/tree/kb/pathfinding-pr4-bench): porting `Benchmarks/PathfindInGame/` from the `kb/ai_pathfinding` branch to the API shipped in #2446–#2448.
## What's in this PR
### `PathfindRecorder` (`PathfindRecorder.cs`)
- Holds a single `StreamWriter` open while recording; its internal buffer absorbs per-record writes without per-call `File.AppendAllText`.
- Single `bool` check on the hot path; cheap when disabled.
- Disabling flushes + disposes; an IO failure during write also disables the recorder.
- Server config:
- `pathfinding.recorder.enable` — bool, default `false`. Read on boot via `GetOrUpdateSetting`.
- `pathfinding.recorder.path` — default `<basedir>/Data/Pathfinding/recordings/pathfinds.jsonl`.
- Hooked into `BitmapAStarAlgorithm.Find` — runs once per call, does nothing when disabled.
- Admin command: `[PathRecord [on|off|flush|status]` (default `status`).
### Public cache helpers
- `static ulong StepCache.ComputeLiveTileDataHash()` — wraps the file module's hash function for staleness checks.
- `static bool StepCache.TryReadTileDataHashFromFile(string, out ulong)` — peeks at a `.swb` file's hash field (20 bytes).
- `int StepCache.BakeMap(int, string)` — walks every chunk in the map, populates resident set, saves. Offline / fixture use; blocks for many seconds on a full-map walk.
- `void StepCache.ClearResidentChunks()` — drops chunks + zeros counters but keeps lazy readers open. Lets benchmark loops measure "first query after boot" cost across iterations without the lazy-reader reopen overhead.
## Summary
Adds a binary disk format + lazy reader so the step cache can warm-start from a precomputed file without paying chunk-build cost on the first pathfind through a region. **Resident memory stays bounded by `MaxResidentChunks` regardless of file size** — opening a `.swb` reads only the header + chunk-offset index (~16 bytes per indexed chunk), and individual chunks are seeked + deserialized only when `ResolveMissingChunk` asks for them.
The lazy design (vs. an eager bulk load): a 250 MB bake on a RAM-constrained shard never materializes more than the LRU cap (~40 MB at the default 8192-chunk cap), and unwanted regions never enter memory at all.
Builds on PR #2447.
## What changed
- **`StepCacheFile`** — binary reader/writer module. Writer emits header → chunks (offsets recorded) → index trailer, then patches the header's `IndexOffset` field. Reader is `OpenForLazy(path)` returning a `LazyReader` that holds an open `FileStream` + offset dictionary.
- **`StepCacheFile.LazyReader`** — `TryReadChunk(chunkX, chunkY)` does a single seek + bulk read for one record. `Dispose` releases the underlying stream. Files are opened with `FileShare.Read | FileShare.Delete` so admin tooling can replace them.
- **TileData fingerprint via XxHash3.** The `.swb` header carries a hash of `LandTable + ItemTable` flags. Load rejects any file whose hash doesn't match the running server. Computed via `HashUtility.ComputeHash64` (engine-blessed hasher) — adds a `ReadOnlySpan<byte>` overload alongside the existing `ReadOnlySpan<char>` one for parity.
- **`StepCache.SaveToFile(path, mapId)`** — writes resident chunks for the given map.
- **`StepCache.TryOpenLazyReader(path, mapId)`** — opens the file, validates header, holds the reader for the map's lifetime.
- **`StepCache.ResolveMissingChunk`** — now consults the lazy reader before invoking the runtime baker. A loaded chunk whose `BuiltMultisVersion` doesn't match the live sector falls through to the baker (snapshot was made before a multi was added/removed in that sector).
- **`StepCache.Clear` closes lazy readers.** Test cleanup can delete `.swb` files cleanly.
- **Auto-load at startup.** `PathCacheCommands.Configure()` opens `Data/Pathfinding/<mapId>.swb` as a lazy reader for every map.
- **`[PathCacheSave`** / **`[PathCacheLoad`** — admin commands for the same workflow.
- **`pathfinding.maxResidentChunks` shard-tunable.** Read from `server.cfg` at boot via `ServerConfiguration.GetOrUpdateSetting` (default 8192 ≈ 40 MB). Small shards can tune down; large shards with substantial bakes can tune up to reduce eviction churn. Default is written back to `server.cfg` on first boot, matching the engine pattern used by other settings.
## File layout (v1)
```
Header (48 bytes):
u32 Magic = 0x42575300 ('SWB\0')
u32 Version = 1
u32 MapId
u64 TileDataHash XxHash3 over LandTable + ItemTable flags (HashUtility)
u64 BakeTimestamp informational
u32 ChunkCount
u64 IndexOffset file position where the chunk index begins
Chunk records (fixed size, ~5,393 bytes each, +32 if multi-Z):
u16 ChunkX
u16 ChunkY
u32 BuiltMultisVersion
u8 HasMultiZ
byte WalkMask[256], WetMask[256]
sbyte SourceZ[256], WalkZN..WalkZNW[256], SwimZN..SwimZNW[256]
[byte MultiZCells[32] when HasMultiZ == 1]
Index trailer (16 × ChunkCount bytes):
(u64 chunkKey, u64 fileOffset)
```
## Memory math
| Scenario | Disk file | RAM at boot | Notes |
|---|---|---|---|
| Empty / no `.swb` files | — | 0 | Silent; cache builds on demand. |
| Admin-curated towns (5K chunks) | 25 MB | 0 + per-query | Index ≈ 80 KB. Resident grows to the configured cap under steady-state queries. |
| Full-map bake (50K chunks) | 250 MB | 0 + per-query | Index ≈ 800 KB. Same configured cap. Cold areas never load. |
| All 5 maps fully baked | 1.25 GB | 0 + per-query | Index ≈ 4 MB total. Same configured cap. |
## Hash choice (FNV-1a → XxHash3)
The original draft used inlined FNV-1a-64. Switched to XxHash3 via `HashUtility`:
- ~30× faster on this workload (~30 GB/s SIMD vs ~2 GB/s byte-by-byte). Boot-time only, so absolute saving is microseconds — the real wins are elsewhere.
- Stronger collision resistance and distribution.
- Drops ~25 lines of inlined hash code; matches the rest of the codebase's hashing pattern.
- Hash is stable as long as `HashUtility`'s `xxHash3Seed` constant doesn't change (already marked `// DO NOT CHANGE THIS NUMBER`).
## Summary
Builds on PR #2446's cache-direct A*. The previous PR conservatively routed players + creatures with capability flags entirely through the slow path. This PR pushes that line: most mobile classes now use the cache, with the right rule set layered on top per-mobile, and the cache fast-path now does the dynamic items / mobiles check that PR #2446 had silently skipped.
## What changed
- **Non-GM players** now use the cache. Diagonal corner-cut applies the strict AND-rule (BOTH cardinal partners walkable) by reading the same source-cell mask byte the creature OR-rule reads — both rules are evaluable from one byte.
- **Creatures with `CanOpenDoors` / `CanMoveOverObstacles`** now use the cache. Reading `MovementImpl` confirmed those flags only affect dynamic items, never static tiles, so they were over-conservatively excluded before.
- **Swim creatures** now use the cache via a capability overlay. `StepProbe` bakes a second rule set (`canSwim=true, cantWalk=true`) producing `WetMask` + `SwimZ_*`. The algorithm composes `effectiveMask = (walkMask & !cantWalk) | (wetMask & canSwim)` per direction; walk Z preferred when both apply.
- **Dynamic-obstacle pass.** Cache fast-path now mirrors `MovementImpl`'s per-cell items + mobiles collision check (`GetItemsAt` / `GetMobilesAt` at the target cell, with `CanOpenDoors` / `CanMoveOverObstacles` / spell-field overrides). This closes a correctness gap from PR #2446 — the cache fast-path was silently skipping dynamic obstacles entirely.
- **`StepCache.TryGetMask` returns `StepMask` struct** instead of 11 out parameters. `HitKind` rolls into the struct with an `IsHit` accessor. Sets up wet/swim without ballooning the call site.
- **`StepChunk.MultiZCells` is lazy-init.** Most chunks are entirely single-Z; allocating the 32-byte bitmap up-front wasted ~256KB at full cap.
- **Admin commands.** `[PathCacheStats` (resident chunks + hit/miss/eviction counters) and `[PathCacheClear` (drop everything, zero counters).
- **Feature flag.** `bitmap_pathfinding_cache` (default true) gates the cache fast-path. Flipped off, every cell expansion routes to `MovementImpl` — equivalent to PR #2446's slow-path-only behavior. Safety net for shipping the new behavior.
`RequiresSlowPath` shrinks to just `CanFly` — flying creatures Z-jump arbitrarily, which the cache's static-Z model can't accommodate.
## Summary
Replaces `FastAStarAlgorithm` with `BitmapAStarAlgorithm`: one cache lookup per cell expansion (8-direction mask + per-direction destination Z) instead of 8 separate `MovementImpl.CheckMovement` calls. Adds the supporting cache infrastructure to back it.
Public API unchanged — `MovementPath` / `Mobile.Move` / `CalcMoves.Find` return the same shapes; the algorithm swap is internal.
## What's in this PR
- **`BitmapAStarAlgorithm`** — A* that issues one `StepCache.TryGetMask` call per cell expansion. Inline fallthrough to the per-cell slow path for multi-Z, off-map, source-Z mismatch, and non-default walkers.
- **`StepCache`** — singleton chunk store keyed by `(mapId, chunkX, chunkY)`. Lazily built on first query, invalidated by `Sector.MultisVersion` mismatch, memory-bounded by sampled probabilistic LRU.
- **`StepProbe`** — computes static-only walkability for a single cell, mirroring `MovementImpl.Check` minus the item / mobile collision phases.
- **`StepMask` / `StepChunk`** — value / storage types for the per-cell results.
- **`CacheEvictionTimer`** — periodic cap backstop (60s interval; early-returns when not over cap).
- **`Map.Sector.MultisVersion`** promoted to `public` so the cache can detect dynamic-static invalidations cheaply.
## Eviction strategy
Sampled probabilistic LRU (Redis-style). Per eviction, sample 5 random keys from a parallel `List<long>` kept in lockstep with the chunk dictionary; evict the oldest of the sample via swap-and-pop. O(1) per eviction regardless of resident count, so sustained cap pressure has no perpetual perf hit.
## Capability handling (interim)
Non-default walkers (non-GM players, creatures with `CanSwim` / `CanFly` / `CanOpenDoors` / `CanMoveOverObstacles`) route entirely through the per-cell slow path via `BitmapAStarAlgorithm.GetSuccessorsSlowPath`. The 2-pass design (cache + capability overlay + dynamic-obstacle pass) lands in the follow-up PR.
> [!IMPORTANT]
> **Breaking Changes**
> - DecodePacket and EncodePacket delegates replaced with IClientEncryption interface
> - NetState.Connection (Socket) replaced with internal RingSocket management
> - NetState.RecvPipe and NetState.SendPipe removed (buffers managed internally)
## Summary
Upgrades the networking stack from PollGroup-based I/O to io_uring, significantly improving I/O performance on Linux.
This also adds native client encryption support for encrypted UO clients.
## Major Changes
io_uring Networking Architecture
- Replaced PollGroup with IORingGroup for async socket I/O operations
- Removed Pipe.cs (mirrored ring buffer) and TcpServer.cs in favor of RingSocketManager
- Added NetState.Network.cs - centralized network infrastructure handling accept, recv, send, and disconnect
completions
- Added SocketHelper.cs - platform-specific socket utilities for raw socket handle operations (getpeername,
getsockname)
- Buffer management now handled by RingSocketManager with configurable slab allocation
### Client Encryption Support
- Added full encryption stack in Network/Encryption/:
- EncryptionConfig.cs - configurable encryption modes (None, Unencrypted, Encrypted, Both)
- EncryptionManager.cs - encryption detection and initialization for login/game packets
- LoginEncryption.cs - handles login packet encryption with version-derived keys
- GameEncryption.cs - handles game server encryption using Twofish
- TwofishEngine.cs - optimized Twofish block cipher implementation
- LoginKeys.cs - encryption key table for client versions
- IClientEncryption.cs - interface for client encryption implementations
### NetState Improvements
- Replaced Socket Connection with RingSocket _socket for managed socket lifecycle
- Changed from GCHandle polling to event-based completion processing
- Disconnect handling now properly waits for pending sends to flush
- Simplified connecting socket management using lazy queue removal
### Configuration
- New settings: network.encryptionMode and network.encryptionDebug
- Encryption mode flags: Unencrypted, Encrypted, or Both
### Dependencies
- Replaced PollGroup NuGet package with IORingGroup
- Linux requires liburing-dev / liburing-devel package
### Test plan
- Verify server starts and accepts connections on Linux with io_uring
- Verify server starts and accepts connections on Windows (fallback to IOCP)
- Test unencrypted client connections (ClassicUO with encryption disabled)
- Test encrypted client connections if available
- Verify graceful disconnect flushes pending data
- Confirm CI builds pass on all target platforms
## Summary
- Adds proper Latin1 encoding support, replacing CP1252 usage throughout the codebase
- Adds specialized, optimized string decoding methods with safe string filtering for each encoding type
- Filters invalid Unicode characters (C0/C1 control codes, non-characters) by removal rather than replacement since
the UO client renders nothing for these characters
- Fixes UTF-16 null terminator position handling to correctly advance by 2 bytes
## Changes
TextEncoding.cs
- Added SearchValues-based invalid byte/char detection for efficient filtering
- Added encoding-specific GetString methods: GetStringAscii, GetStringLatin1, GetStringUtf8, GetStringBigUni,
GetStringLittleUni
- Each method supports a safeString parameter for filtering invalid characters
- Little-endian UTF-16 uses direct memory cast for zero-copy decoding on LE systems
- Invalid characters are removed (not replaced with U+FFFD) since the client renders nothing for them
SpanReader.cs
- Added ReadLatin1() and ReadLatin1Safe() methods
- Rewrote encoding-specific read methods to use optimized TextEncoding.GetString* methods
- Fixed UTF-16 null terminator handling: position now correctly advances by byteLength (2) instead of 1
SpanWriter.cs
- Added WriteLatin1 and WriteLatin1Null methods
## Packet Updates
- Updated all packet code to use Latin1 encoding instead of CP1252
- Affected: account packets, equipment packets, menu packets, message packets, mobile packets, player packets, secure
trade packets, vendor packets, gump packets, book packets, mahjong packets
## Filtering Behavior
Invalid characters filtered in safe mode:
```
┌───────────────┬────────────────────────┐
│ Range │ Description │
├───────────────┼────────────────────────┤
│ 0x00-0x1F │ C0 control codes │
├───────────────┼────────────────────────┤
│ 0x7F │ DEL │
├───────────────┼────────────────────────┤
│ 0x80-0x9F │ C1 control codes │
├───────────────┼────────────────────────┤
│ 0xFFFE-0xFFFF │ Unicode non-characters │
└───────────────┴────────────────────────┘
```
Note: Surrogate pairs (0xD800-0xDFFF) are not filtered because proper validation requires context checking for paired
vs unpaired surrogates. The UO client renders nothing for these anyway.
## Test Plan
- All 631 Server.Tests pass
- Verified client rendering behavior using TestUnicodeGump command (pages 1-5)
- Confirmed U+FFFD, unpaired surrogates, and non-characters all render as blank in client
- Verified Latin1 characters (0xA0-0xFF) display correctly
- Verified C1 control codes (0x80-0x9F) are filtered and don't display
### Summary
- Create BitMask256 struct with scalar operations (benchmarks showed AVX2 vectorization provides no benefit for this size)
- Refactor Map.cs to use BitMask256 for full Z range support (-128 to 127)
- Remove SectorSpawnCache struct, use BitMask256 directly in manager
- Update tests to use BitMask256 directly
- Remove unused test assertions for old 64-bit behavior
> [!IMPORTANT]
> **Developer Note**
> THIS IS A BREAKING CHANGE TO THE NEW API GUMP.
> Please give us feedback in [discord ](https://muo.gg/discord) if you have issues, need help, or have ideas for a better API change!
### Summary
* Adds support for size/style to dynamic/static builder.
* Drastically simplifies the dynamic/static builder api for AddHtml.
* Cleans up some legacy gump files.
### Summary
Refactors ScheduledEvent and EventScheduler API to use TimeOnly so recurrence offset is explicit.
API:
```cs
public ScheduledEvent(
DateTime startAfter,
DateTime endOn,
TimeOnly time,
IRecurrencePattern recurrence,
TimeZoneInfo timeZone = null
)
```
Example:
```cs
// Schedule a daily event at 8:00 AM UTC
EventScheduler.DailyAt(
new DateTime(2024, 6, 1, 8, 0, 0, DateTimeKind.Utc),
() => Console.WriteLine("Daily event triggered!")
);
// Schedule a custom recurring event at 3:30 PM UTC every Monday
var recurrence = new WeeklyRecurrencePattern(1, DaysOfWeek.Monday);
EventScheduler.Shared.ScheduleEvent(
DateTime.UtcNow,
new TimeOnly(15, 30),
() => Console.WriteLine("Weekly Monday event!"),
recurrence
);
```
### Summary
* Adds an event scheduler.
* Adds conveniences for hourly, daily, weekly, biweekly, monthly, ordinal monthly, and yearly recurrences
Example:
```cs
var tz = TimeZoneInfo.FindSystemTimeZoneById("America/New_York");
// Specify the time of the day, and the day of the week you want it to occur. Make sure it is translated into Utc.
// The next occurrence will be _after_ the specified date/time.
var scheduledEvent = EventScheduler.WeeklyAt(new DateTime(2025, 04, 26, 17, 00, 00), StartEvent, tz);
void StartEvent()
{
World.Broadcast(0x30, false, "The event has started!");
}
Console.WriteLine("Event starts on {0}", scheduledEvent.NextOccurrence);
```
In this example, on _Saturday, May 3rd, 2025 @ 5pm ET_, the message "The event has started!" will be broadcasted.
