Follow-up to #2729.
## Problem
At load, `ReadAnchoredTime` shifted **every** anchored value forward by the downtime, including values that were already in the past when the save started. That caused two problems.
**An expired deadline came back.** A deadline that passed 5 minutes before a 30-minute outage loaded 25 minutes in the future. For example:
- `Sheep.NextWoolTime`: a sheep that was ready to shear stopped being ready.
- `PlayerVendor.NextPayTime`: an overdue payment was pushed back.
- `BaseCreature.SummonEnd`, the spell fields' `End`, `TransientItem.Expiration`, and the ChampionSpawn times could move the same way.
**Records were rewritten for nothing.** The first save after every restart rewrote:
- stamps of things that had already happened: VirtueContext's `LastHonorUse`, `LastSacrificeGain`, the four `Last*Loss` stamps, and `Corpse.TimeOfDeath`
- every deadline that had expired long ago.
## Change
- **`IGenericReader.AnchoredTimeOrigin`** is the loaded save's start time.
- `GenericEntityPersistence` and `GenericPersistence` set it from `World.LoadedSaveStartTime`.
- `BufferReader`, `UnmanagedDataReader` and `BinaryFileReader` expose it.
- **`ReadAnchoredTime` returns any value at or before the origin exactly as saved.**
- A deadline still running at save start is shifted as before and keeps its remaining time.
- Readers without an origin keep the old behaviour.
- **This covers every anchored field, including generated ones.** Generated `[AnchoredDateTime]` fields call `reader.ReadAnchoredTime()`, so the one change covers all 26 generated fields as well as the manual reads. `[DeserializeTimer]` only restores timers that were running at save time, so it is unaffected.
- **The `[AnchoredDateTime]` section of `dev-docs/serialization.md` and the `WriteAnchoredTime` doc comment describe the new rule.**
## Behaviour change
Downtime now counts toward windows measured from a past stamp:
| Window | Length |
|---|---|
| Honor use | 5 minutes |
| Sacrifice gain | 1 day |
| Virtue atrophy | 7 days |
| Instanced corpse looting | 3 minutes |
A 30-minute outage makes a 7-day atrophy fire about 0.3% sooner. No format change.
## Tests
New tests in `AnchoredTimeTests`:
- A deadline running at save start keeps its remaining time.
- A deadline that passed before save start stays passed.
- A past stamp is not shifted.
- A value exactly at save start is not shifted.
`Server.Tests`: 845 passed. `UOContent.Tests`: 1140 passed.
ReadAnchoredTime shifted Item.LastMoved, Item.DecayResetTime and
Mobile.LastStr/Int/DexGain forward by the downtime at load. The bytes
were stable from save to save, but the first save after every restart
rewrote one timestamp in nearly every item record and most mobile
records, which defeats any per-record backup or delta save.
- LastMoved and DecayResetTime are read as saved. The downtime credit
is applied after load by Item.CreditDowntimeToDecay, which moves
DecayResetTime only for items whose decay countdown was still running
when the save started. Every other item keeps its bytes, and decay
timing is unchanged.
- World.LoadedSaveStartTime exposes the loaded save's anchor so the
credit can tell which countdowns were running.
- Stat-gain stamps are read raw. The cooldown is minutes long, so
downtime counting toward it costs nothing.
No format change: these fields were already absolute ticks.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Follow-up to #2721. It fixes how the remaining tests behave and changes no production code.
## CI
- Both `dotnet test` steps pass `--filter "Category!=Manual"`. Before this, Manual-tagged tests ran on CI anyway.
- The one surviving movement-throttle latency test is the only coverage of burst forgiveness over a full, wrapped history. It loses its Manual tag, so it runs on CI.
## Tests that passed on CI without testing anything
- **Corpse and throwing tests.** They relied on tiledata for the weapon `Layer`, so on CI the equip silently failed and they asserted against empty lists. They now set `Layer` explicitly and assert the equip took effect. `TestThrown` also pins its own strength and range values, so a shard retuning SoulGlaive no longer breaks them.
- **Healthbar poison rows.** Server.Tests registers no poisons, so every row tested `null`. They now use real poison levels.
- **Account packet tests.** The mock accounts reported `Length` 0, so the character-list tests only ever encoded empty slots. They now encode the real characters.
- **Pathing and AI tests that only mean something on real Trammel/Felucca terrain** are now guarded with `TileDataRequirement.SkipIfMissing()`: Britain desk and clutter scenes, house footprints (no MultiData on CI) and fingerprints. Tests that are valid on flat blank terrain stay unguarded. (A synthetic tiledata fixture to run these on CI is a planned follow-up.)
- **Assertions that couldn't fail now check their claim.** Affected tests include the `DefaultWalker` cache use, cache on/off parity requiring a path, the recorder capability flags, the idle-pet gate, the monthly DST recurrence, the AdvancedSearch `Record.Exception` wrappers, `CanFitItem` with no branch-guarded assert, the Firewall callback, and `Import_NoMap`. The Geometry Parse asserts also now use the same input as their TryParse.
## Shared state
Every test that changes process-wide state now restores it, even when an assert fails:
- **The clock.** A new `TimerWheelScope` saves and restores `Core._tickCount`, `Core._now` and the timer wheel. `SimulatedClockScope` covers decay tests.
- **StepCache settings and caches**, via `StepCacheStateScope`.
- **Network state.** The ManualAllowlist, LoginAllowlist, firewall and BanChannel reporters are restored. Global packet handlers are now registered per test and removed afterwards, so they no longer stay registered for later tests.
- **Data and config tables.** `ArchiveJournal` statics, `TileData.ItemTable`, `ObjectPropertyList.Enabled`, `Core.Expansion` and the localization table are restored. One `MobileStatusVersion` restore was resetting a different expansion than the one the test changed.
- **Entities.** Tests with fixed serials now delete what they create. Fixed mobile serials moved to `World.NewMobile`, because deleting a hard-coded serial could unregister a live entity.
- **Collection race.** `SpiralScanTests` no longer clears the spawn cache from a separate collection that could run in parallel with the cache-manager tests.
- **Shared statics.** Classes reading shared static allowlists joined the sequential collection.
## Duplication
New shared helpers replace copy-pasted harness code:
- **Server.Tests:** `PacketAssert.Sends`, used by about 130 packet tests; `TestClientHarness`, `PersistencePipelineScope`, `NetTestHelpers`, `EntityCleanup` and one `TestMulti`.
- **UOContent.Tests:** `TimerWheelScope`, which also holds one wraparound-safe `RunFor`/`RunUntil`; `AiTestStubs`, which merges six walker copies and five identical creature stubs; `MobileTestHarness`, `CommandTestHarness` and `TempDirectory`.
Dead oracle packet classes are removed.
## Cost
| Test | Before | After |
|---|---|---|
| `VirtualSerialTests` | 1.57s | 2ms (starts near the wrap via reflection instead of drawing ~17.9M serials) |
| Event scheduler recurrence loop | 473ms | 1ms |
| CrowdSec drain | 517ms | 16ms (event-driven instead of a fixed 500ms wait) |
| AuthId tests | 248ms | 99ms (SHA2 except the two GameLogin tests) |
| Container cap tests | 5,000 items | 3,449 / 3,277 (one past each protocol cap) |
| Bulk removal | 100k children | 20k children |
| OPL hash distribution | 60k builds | 10k builds |
## Shard-tuning decoupling
- AI assertions read `ReacquireDelay` and `FollowMoveSpeed` instead of hard-coding production tuning.
- The guard friendly-fire test uses an `AI_Mage` stub instead of WhiteWyrm.
- Argon2 assertions pin the `argon2id` variant and `v=19` but read the cost parameters from the configured hasher.
## Tests
| Run | Server.Tests | UOContent.Tests |
|---|---|---|
| With client files | 812 pass | 1,123 pass |
| Without client files (CI simulation) | 800 pass, 12 skip | 1,021 pass, 102 skip |
- The UOContent fixture reads `MODERNUO_TEST_DATA_DIR`; Server.Tests reads `MODERNUO_CLIENT_PATH`.
- The 15 new UOContent skips are the map-dependent guards above.
- Total run time with client files is about 5s, down from about 8s.
## Summary
Many tests proved a specific change when it landed but don't need to run on every CI build forever. This removes **202 test methods and 9 files: 313 test cases, 2,247 → 1,934**. UOContent.Tests drops from about 14s to about 5s locally.
Removed:
- **Tautologies.** Asserting a constant, a constructor readback or a trivial getter, or testing BCL behavior.
- **Exact duplicates** of a surviving test on the same code path.
- **Obsolete regressions** whose failure can no longer be expressed.
- **Game-content values** shard owners customize (era drop rates, loot and skill tables, single-creature tuning). Each customization would otherwise force a test fix.
## How it was decided
1. Every test was classified, with a reason, by a per-shard sweep over all 287 test files.
2. Each proposed deletion was then checked by a second, adversarial reviewer against the surviving tests and the production code.
3. Anything that was the only coverage of a separate code path was **kept**, even where it looked like a duplicate. Examples: the unsigned `SpanReader`/`SpanWriter` methods, the at-location versus bounds enumerators, null-map enumerables, the security filters, archive-journal state transitions. About 130 proposed deletions were rejected this way.
Whole files removed:
- `FishingTests`, apart from 3 tests: the map-edge scan, serpent cleanup, and SOS chest consumption. The SOS test now stops at the first chest instead of measuring a catch rate. The multi-million-cast rate simulations are gone.
- `LordOaksTests` and `MeerEnragedTests` (single-creature tests).
- `AutoArchiveHelperTests`, `JsonDiscoverableTypeAttributeTests`, `DecimalSerializationTests` and `EnumConversionTests` (tautologies, or tests of their own local helpers).
- `NetworkStatsTests`.
- `MigratedDataLoadTests` (a subset of `AllSpawnFilesLoadTests`).
- `SpawnerFixtureCapture`: a capture tool, not a test, and its `version == 12` assert can no longer pass. The `.bin` fixtures it produced stay.
## Tests
- Server.Tests: 811 pass.
- UOContent.Tests: 1,123 pass.
A follow-up PR, stacked on this one, fixes how the remaining tests behave: shared state, CI-vacuous tests, harness duplication.
## Summary
- `AnchoredItemSerializationTests.LastMovedAndDecayReset_RoundTripExactly` never deleted its deserialized copy. `Item.Deserialize` enters the item into its map's sectors, so the copy stayed live on Felucca at (121,100). `ItemEnumerator_FiltersByBoundsAndOrder`, `ItemByDistanceEnumerator_Bounds_FindsItemsInBounds` and `ItemByDistanceEnumerator_Bounds_MakeBoundsInclusive` scan that area and found an extra item whenever the serialization test ran before them, so they failed depending on test order. The copy is now deleted in a `finally`.
- Bump MailKit 4.17.0 → 4.18.0 and Microsoft.NET.Test.Sdk 18.10.0 → 18.10.1.
## Test plan
- [x] Probe test: a deserialized copy shows up in `GetItemsInBounds` (reproduced), and after the fix the area is empty
- [x] Server.Tests: 907/907 pass
- [x] UOContent.Tests: 1126 pass, 2 skipped
- [x] Solution builds with 0 errors
Three pre-existing world-save failure-handling bugs, found while reviewing the delta-saves engine (#2610), split out so they land and get adopted on their own before the larger change. #2610 will be rebased on top once this merges. Tracked in project Delta Saves (#7).
## 1. A snapshot missing a segment was published
`GenericEntityPersistence.WriteSnapshot` logged a segment (or self-payload) write error and carried on. The save was published without those records, and every entity in the failed segment was deleted at the next load. The error now propagates: `WriteFiles` never moves a partial snapshot over `Saves/`, the previous save stays authoritative, and staff are told.
## 2. A serializer exception killed the process, or published a partial freeze
An entity whose `Serialize` threw on a worker thread was an unhandled exception on that thread, which terminates the process. On the inline (main-thread) drain it was caught, but the snapshot was still queued for writing with whatever the workers had produced. Workers now record the first exception and keep draining so the queue empties and the wake/pause handshake completes; after every worker paused, `Snapshot` treats a recorded error (or a failure of the drain itself) as a failed save: nothing is written, the previous save stays, staff are told, and the world resumes. A `WorldSave` handler throwing is logged but does not invalidate the serialized snapshot.
## 3. Publication was not transactional
Publishing moved the previous `Saves/` away (`AutoArchive` in `WorldSavePostSnapshot`) and only then moved the new files in. A subscriber throwing after the archive, or the final move failing, left nothing at `Saves/` until the next save; a crash in that window lost the newest save at the next boot.
The snapshot now moves to `Saves.next` as soon as its files are complete; only then do subscribers archive the previous save (same event, same `OldSavePath`, so `AutoArchive` and shard subscribers are unchanged), and the staged directory is renamed into place, which is atomic on one volume (file-by-file move across volumes). A staged directory is by construction a complete save newer than `Saves/`, so an interrupted publish is finished at the next boot (before load) or before the next save: whatever sits at `Saves/` is set aside as `Saves.previous-<timestamp>` (never deleted) and the staged save is published, with a warning naming the directory to archive or delete by hand.
## Tests
Server.Tests 860 / UOContent.Tests green. New: a throwing serializer is recorded on the worker and the next drain starts clean; a segment that cannot be indexed fails `WriteSnapshot` instead of dropping records; staged-save recovery with and without an existing `Saves/`, and as a no-op.
## Summary
Phase 3 of the anchored-time work: **every actively-written delta-time value in the engine now stores an anchored timestamp** — absolute on the wire, shifted forward by the downtime at load. Remaining time survives restarts (as delta did), and unlike delta, the bytes do not change on every save, so an idle world serializes identically save after save.
The answer to "is it possible everywhere": **yes** — including the one case that looked impossible.
## The GenericPersistence problem, solved
`GenericPersistence` bins (`Virtues.bin`, `StealableArtifacts.bin`, …) are raw payloads with no idx header, so they have no anchor of their own — anchored reads there would silently apply zero shift. But the anchor is a property of the **save**, not the file: every file in one save shares one `World.SaveStartTime`, and `Persistence.Load` reads **all** entity indexes (phase 1) before **any** persistence payload (phase 2). So the idx v5 header stamps a save-wide `World.LoadTimeShift`, and generic persistence readers inherit it. No file-format change, no per-bin header, old bins unaffected.
## Converted
- **Item v10 → v11**: `LastMoved` — previously whole-minute delta, rewritten every save for every item, the single largest source of idle-save churn — and `DecayResetTime` (retiring the TODO from #2583). **Mobile v37 → v38**: the three stat-gain stamps. **BaseCreature v20 → v21**: `SummonEnd`.
- **17 code-generated classes** (`[DeltaDateTime]` → `[AnchoredDateTime]`, version bump + `MigrateFrom` each): the five field spells, TransientItem, VirtueContext (×7 fields), PuzzleChestSolutionAndTime, BaseCamp, BaseBoat, RentedVendor, PlayerVendor, Ethics Player, Sheep, StarRoomGate, ChampionSpawn (×3), Corpse (`TimeOfDeath`, v19). The `MigrateFrom` bodies were generated from each class's current migration schema and are compiler-verified; VirtueContext's save-flagged nullables fall back to the same defaults the old deserialize left in place. Corpse's six migrations moved to a new `Corpse.Migrations.cs`.
- **Hand-written sites**: StealableArtifacts (v2), VendorInventory (v1), ML quest objectives (persistence v3) — each gated on its own version.
**Not converted, deliberately**: the ~25 read-only `ReadDeltaTime` sites in legacy version fallbacks and migration replays — they decode existing old bytes and must never change. `[DeltaDateTime]`/`WriteDeltaTime` remain available for them.
## Verification
- Build 0 errors / 0 warnings; **837 + 708 tests green**.
- Schema regeneration produced exactly the 17 expected new `vN.json` files (all `AnchoredTime` rule args), nothing else touched.
- **New acceptance tests** pin the point of the whole effort: serializing the same item at two save times **5 hours apart produces byte-identical output**, and `LastMoved`/`DecayResetTime` round-trip **exactly** at sub-minute precision (the old minutes encoding destroyed both properties).
## Notes for review
- `LastMoved` grows from a 1–3 byte encoded minutes value to 8-byte ticks per item — the price of byte-stability; it repays itself in incremental-save behavior since unchanged items now produce unchanged bytes.
- BaseEscortable-style semantics are unchanged: anchored shift preserves *remaining* time exactly, the same contract delta provided, so no gameplay-visible behavior changes — deadlines simply stop being consumed by downtime that delta already protected against, now with stable bytes.
## Enforcement
`WriteDeltaTime` is now `[Obsolete]` (interface + implementation). With the repo's warnings-as-errors, any new delta-time write — hand-written or emitted by a still-unconverted `[DeltaDateTime]` field — fails the build, with the migration instructions in the message. That the full solution still builds with **zero warnings** is itself the proof no active delta writer survived the conversion. `ReadDeltaTime` deliberately stays un-attributed: its remaining callers decode existing old bytes and are correct forever; its XML docs now state the legacy-decode-only contract.
## Summary
The save-stability infrastructure consumed by generator v3's `[AnchoredDateTime]`: anchored timestamps are written as **absolute values** and re-based once at load by the elapsed time since the save started — so downtime doesn't age them, and an unchanged entity serializes to identical bytes (the prerequisite for replacing delta-time encodings, which rewrite every entity on every save).
## Design
- **`WriteAnchoredTime` / `ReadAnchoredTime`** on `IGenericWriter`/`IGenericReader`. The read side applies the reader's `AnchoredTimeShift`; `Min/MaxValue` sentinels pass through unshifted, and shifts saturate instead of overflowing.
- **`World.SaveStartTime`** is stamped the moment the world freezes for a snapshot — one anchor for the entire save, no per-persistence skew.
- **idx v5**: the anchor ticks sit in the header right after the version. The anchor travels with the file it re-anchors, so a single idx+bin pair restored from a backup is self-describing, and anchor presence is guaranteed by the same version gate as the record format — there is no separate anchor file to lose.
- **The shift rides the reader instance** (`BufferReader`, `UnmanagedDataReader`, `BinaryFileReader` delegating), not a static — parallel per-persistence loads and ad-hoc restores each see their own file's anchor. idx v4 and older read with a zero shift.
## Scope
Behavior-neutral: nothing serializes anchored values yet (`Item.DecayResetTime` and the `[DeltaDateTime]` field migrations come separately, with their own version bumps). Saves written from this branch are idx v5; loading v4/v3 saves is unchanged and remains pinned by the existing hand-written-header tests.
## Testing
- Unit round-trips: exact with zero shift, shifted read, sentinel passthrough, saturation, Local→UTC normalization.
- End-to-end through the real worker/segment-log pipeline: an anchored timestamp re-bases across a simulated two-hour downtime via the idx v5 header.
- Full suites green: Server.Tests 835/835, UOContent.Tests 708/708 (including the existing v4/v3 idx loading tests).
Reduces the world-save freeze window from ~740ms to ~78ms (measured on a synthetic 10M-entity / 1.7GB world, 24 cores, through the real pipeline classes) by removing the per-entity handoff between the game loop and the serialization workers, fixing how large indivisible payloads are scheduled, rewriting the BufferWriter hot path, removing per-entity placement state entirely, and finally replacing the global serialized-types tracking with a per-file type table (idx v4) that also shrinks idx files by ~21% and speeds the background write phase. The pipeline has also been validated end-to-end on live-copy worlds in the multi-million-entity range, where the freeze is drain-bound (real `Serialize()` costs far more CPU per byte than synthetic writes) — the same structural wins hold, and entity/file round-trips are byte-clean across both load paths.
## The problem
The freeze window is `max(main-thread handoff, slowest worker drain)`:
1. **The producer was the bottleneck.** The main thread round-robined every entity through per-worker `ConcurrentQueue`s — two interlocked ops per entity, ~740ms of freeze floor at 10M entities before any serialization happened.
2. **Round-robin distributes count, not cost.** "Deep" systems (50MB generic persistence blobs) and "thick" entities (100K-item storage keys) landed on arbitrary workers, producing lopsided drain times on large worlds.
3. **Worst-case scheduling.** `GenericEntityPersistence.Serialize` pushed its self-payload *after* all entities, and generic persistences sort last in the registry — so the biggest indivisible blobs started serializing at the very end, extending the freeze by their entire duration.
## The fix
**Commit 1 — chunked handoff + LPT scheduling + heap pre-sizing:**
- Pooled 4096-entity chunks published to one shared queue; workers pull chunks and load-balance dynamically (a worker busy with a thick entity simply takes fewer chunks).
- `Persistence.SerializeAll` pushes systems largest-first (LPT) using the previous save's payload size (or loaded file size on first boot); self-payloads get dedicated single-entity chunks so they overlap the entity stream instead of ending it.
- Worker heaps pre-size from the loaded save's `.bin` totals, eliminating copy-on-grow inside the first save's freeze.
- `SpinWait` backoff in the drain loop (never `Sleep(1)`), per-worker balance stats logged in debug builds (the call site is compiled out of Release), 1MB snapshot write buffer.
**Commit 2 — workers iterate the dictionaries directly + main thread joins the drain:**
- `GenericEntityPersistence` publishes 4096-slot ranges over its dictionary's backing entries array; workers serialize occupied slots (`value != null`) directly through a `ShadowEntry<TValue>` struct mirroring the runtime's private `Entry` layout. Safe because the dictionary is frozen during `Saving` (mutations divert to the pending safety queues).
- The layout is **proven at startup before any code reads through it**: validation measures the true `Entry` stride via precise allocation accounting (guaranteeing all shadow reads are in-bounds), then verifies every key/value of a churned, resized, freelist-exercised dictionary — reading value slots as raw pointer bits only, never materializing a managed reference until the layout is proven. If a future runtime changes `Dictionary` internals, validation fails with a logged warning and saves fall back to the (fully maintained) enumerate-and-push path.
- The main thread joins the drain via an inline worker after publishing, instead of idling — worth a full worker share, proportionally more on low-core hosts.
**Commit 3 — 2.2x faster BufferWriter write path, single-pass short strings:**
- PGO already devirtualizes and inlines every `IGenericWriter.Write` callsite (interface vs concrete measured identical) — the real per-write cost was the non-inlinable `Index` setter (range-check throw path + per-write high-water tracking) plus span bounds checks. Writes now reserve capacity once, then do an unaligned store through a ref with a raw index increment; the high-water mark folds at Seek/Resize instead of per write.
- Class-level implementations of the hottest default interface methods keep nested writes inlined (a DIM re-dispatches on `this` even at a devirtualized callsite).
- Strings of 85 chars or fewer encode once into a stack scratch instead of walking the string twice (`GetByteCount` + `GetBytes`). Byte output is identical.
- Measured: 34.4 → 15.7 ns/entity on a generated-style write mix; end-to-end freeze ~99ms → ~74-82ms.
**Commit 4 — branch-free fallback push loop:**
- A bare `foreach { PushToCache(entity); }` runs at 2.3ns/entity; the same loop carrying a per-entity heavy-check runs 2.3x slower — the cost is the fatter loop body defeating tight-loop codegen. Entity-level >1MB payloads are rare enough to ride in shared chunks; system self-payloads (the large ones) are still explicitly scheduled largest-first.
**Commit 5 — drop the 9-byte per-entity placement state; snapshots write from worker segment logs:**
- Every `ISerializable` carried `SerializedThread/SerializedPosition/SerializedLength` so `WriteSnapshot` could gather each entity's bytes from the worker heaps in dictionary order. But the idx records absolute positions — bin order is free — so the snapshot is now written in worker-heap order and the join inverts: workers log segments (owner, slot range, heap start) plus one length per record as they serialize; positions are implicit because a worker's writes are contiguous, and identity comes from re-walking the same snapshot slots in the same order (stable until `PostWorldSave`).
- Chunks are persistence-homogeneous (the partial chunk publishes at each `SerializeAll` boundary) so segments route to files by owner with zero per-entity state. Self-payloads keep placement as three private fields on the handful of persistence instances.
- Net: 9 bytes (plus padding) of resident state removed from every item, mobile, guild, and account on every shard; three interface-property stores per entity leave the drain hot path (stamping dirtied one cache line per entity mid-freeze — the lengths log is one sequential stream); each segment's bytes hit the bin as a single span write instead of one copy per entity, speeding the background write phase; and `IGenericSerializable` shrinks to just `Serialize(IGenericWriter)`. Transient cost: ~4 bytes per entity in pooled per-worker logs, released after each write. The save format was unchanged at this point (idx v3, same loader); the v4 bump comes later in the branch.
**Commit 6 — staged file writes replace memory-mapped snapshot writing:**
- `MemoryMapFileWriter` is removed. `FileBufferWriter` composes through the full `BufferWriter` raw write path into a pooled staging block that drains to the file as large sequential positional writes (`RandomAccess.Write`); seeks flush the block and move the file offset, so backwards patches (the idx entity count) become small positional writes.
- Memory-mapped composition paid a soft page fault on every composed page plus unpredictable dirty-section teardown stalls at dispose — measured ~4x slower end-to-end than staged writes at snapshot sizes.
**Commits 7–11 — idx v4: per-file type table replaces SerializedTypes.db and all runtime type tracking:**
- Previously every `Write(Type)` from every worker enqueued into a shared `ConcurrentQueue<Type>` during the freeze (interlocked writes on a shared cache line, millions of mostly-duplicate entries), the background phase drained and deduped it all into a `HashSet`, and the snapshot recomputed `xxHash64(Type.FullName)` once per entity record (~5.4M redundant hashes per save on a large world) to write 9-byte tag+hash idx records plus a global `SerializedTypes.db`.
- The db's only real job was diagnostics: the string name behind "Type `<X>` was not found. Delete all of those types?" during idx loading. That map now lives in the idx itself: each `GenericEntityPersistence<T>` keeps an insertion-ordered `Type -> ushort` table, hydrated at `AddEntity` and on every deserialize path — one dictionary `TryAdd` per entity add on the game thread, amortized across gameplay, and provably immutable while the background writer reads it (adds divert to the pending queues during saves).
- idx v4 layout: the table (names only) is written before the records; records reference it by 2-byte index, shrinking from 33 to 26 bytes (−21%). The loader resolves each table name **once** (`FindTypeByHash(ComputeHash64(name))` — semantically identical to v3 resolution, `TypeAlias` included) into a constructor array, and each record becomes an array index instead of an 8-byte hash read plus dictionary probe. The unresolved-type prompt now surfaces once per type, with the name.
- Deleted outright: `World.SerializedTypes`, the drain/dedupe pass in `WriteFiles`, `BufferWriter`'s type tracking (its `Write(Type)` is now pure — payload format unchanged: tag byte + xxHash64), `FileBufferWriter`'s typeSet parameter, `Persistence.WriteSerializedTypesSnapshot`, and the adhoc db write. SerializedTypes.db is no longer produced.
- **Backward compatibility:** v0–v3 saves (including their SerializedTypes.db and legacy tdb files) load exactly as before, and every legacy load path hydrates the new table so the first v4 save after an upgrade is complete. Stale db files in existing save folders are simply ignored. Verified live: a v3 save boots, saves as v4 (Items.idx −20.2% on a dev world), and reloads with identical entity counts.
## Measured (synthetic 10M entities / 1.7GB, 64+64+32MB system blobs, 24x 2MB thick entities, dense write profile, 24 cores)
| Metric | Before | After |
|---|---|---|
| Steady-state freeze | ~740 ms | **~78 ms** |
| Main-thread publish cost | ~740 ms | **~0.1 ms** |
| Steady-state allocations | 0 | 0 (by iter 2) |
| Worker byte-load spread | 2x | ~1.15x |
The freeze is now bound by pure serialize throughput (payload / cores).
## Tests
- 779 Server.Tests + 501 UOContent.Tests pass.
- New across the branch: chunk fill/flush/owner-boundary tests, pool reuse/clear tests, an end-to-end multi-worker drain through the real wake/push/flush/pause protocol, a 50K-entry churn equivalence test for the shadow iteration re-walk (the exact pairing the snapshot writer relies on), byte-level BufferWriter output/position pins, `RuntimeLayoutIsSupported` so a silent fallback on a future runtime upgrade fails loudly in CI, and a full snapshot **round-trip test** that serializes 25K entities plus a self-payload through real workers, writes the idx/bin from the segment logs, and reloads them through the standard loader (now in v4 format).
- For idx v4 specifically: `FileBufferWriter` staging/drain/seek-patch and oversized-item tests, type-table registration tests, a hand-written v4 fixture proving an unresolvable type name skips only its own records through the console confirmation flow, and a hand-written **legacy v3 fixture** proving old saves still load and hydrate the type table for their next save.
## Trade-offs
- Chunk scheduling is nondeterministic, so worker heaps ratchet to each worker's max-ever draw rather than a fixed share. With slot ranges the balance is tight (~1.15x), so the effect is small; a shared slab pool remains an option if production shows retention creep.
- Entity-level heavy items inside slot ranges are serialized wherever they're encountered (no LPT for them); worst-case tail is one thick entity's serialize time (~ms). System self-payloads — the large ones — are still explicitly scheduled largest-first.
- Snapshot-write error granularity is per segment rather than per entity (heap-bounds bugs were the only thing the per-entity catch ever caught; idx metadata reads keep per-record granularity).
- idx v4 is a save-format version bump: old saves load unchanged through the preserved legacy paths, but saves written by this branch require this loader. Per-persistence type tables cap at 65,535 distinct entity types per boot (hard throw, orders of magnitude of headroom), and a type's table slot persists until restart even if its last entity is deleted — a few stale name entries per file, by design.
### Summary
* Fixes spawner timer deserialization
* Adds a check for a null timer and allows the timer to get recreated
* Adds PotionKeg reverse lookup
* Heavily optimizes decimal serialize/deserialize
### Features
* Fully abstracts serialization by using compile-time attributes.
* Supports serializing the following:
- Primitives (integers, strings, etc)
- IP Addresses
- BigDecimal
- DateTime, Delta DateTimes
- TimeSpan
- Server.Race
- Server.Map
- Point2D, Point3D, Rect2D, Rect3D
- Existing/New `ISerializable` references
- Lists/Sets of serializable types
- Type with a `Serialize` method and constructor that takes an `IGenericReader`
* Supports forward-only migration
* Supports existing RunUO deserialization for older versions by changing to the following signature:
- `public void OldDeserialize(IGenericReader reader, int version)`
- Must remove deserializing the version since this is already done
* Supports serializing from private fields or custom made properties.
* Types do not require inheriting Item/Mobile. Code gen will fully create `ISerializable` information.
- This is not recommended yet, since it requires wiring to `Persistence` which will cause lots of unresolved symbol errors until code gen is built.
### Example
```cs
using System.Collections.Generic;
namespace Server.Items
{
[Serializable(1)]
public partial class TestItem1 : Item
{
[SerializableField(1)]
[SerializableFieldAttr("[CommandProperty(AccessLevel.Administrator)]")]
private List<Item> _someProperty;
private void Deserialize(IGenericReader reader, int version)
{
}
}
}
```
Generates this:
```cs
namespace Server.Items
{
public partial class TestItem1
{
#pragma warning disable 0414
private const int _version = 1;
#pragma warning restore 0414
[CommandProperty(AccessLevel.Administrator)]
public System.Collections.Generic.List<Server.Item> SomeProperty
{
get => _someProperty;
set
{
if (value != _someProperty)
{
((ISerializable)this).MarkDirty();
_someProperty = value;
}
}
}
public TestItem1(Serial serial) : base(serial)
{
}
public override void Serialize(IGenericWriter writer)
{
var savePosition = ((Server.ISerializable)this).SavePosition;
if (savePosition > -1)
{
writer.Seek(savePosition, System.IO.SeekOrigin.Begin);
return;
}
writer.WriteEncodedInt(_version);
writer.Write(_someProperty);
}
public override void Deserialize(IGenericReader reader)
{
var version = reader.ReadEncodedInt();
if (version < 1)
{
OldDeserialize(reader, version);
((Server.ISerializable)this).MarkDirty();
return;
}
SomeProperty = reader.ReadEntityList<Server.Item>();
}
}
}
```
And this:
```json
{
"version": 1,
"type": "TestItem1",
"properties": [
{
"name": "SomeProperty",
"type": "System.Collections.Generic.List\u003CServer.Item\u003E",
"rule": "ListMigrationRule",
"ruleArguments": [
"Server.Item",
"SerializableInterfaceMigrationRule"
]
}
]
}
```
Kamron Batman