Commit graph

98 commits

Author SHA1 Message Date
Kamron Batman
2935eafe24
feat: convert all delta-time serialization to anchored time (#2589)
## Summary

Phase 3 of the anchored-time work: **every actively-written delta-time value in the engine now stores an anchored timestamp** — absolute on the wire, shifted forward by the downtime at load. Remaining time survives restarts (as delta did), and unlike delta, the bytes do not change on every save, so an idle world serializes identically save after save.

The answer to "is it possible everywhere": **yes** — including the one case that looked impossible.

## The GenericPersistence problem, solved

`GenericPersistence` bins (`Virtues.bin`, `StealableArtifacts.bin`, …) are raw payloads with no idx header, so they have no anchor of their own — anchored reads there would silently apply zero shift. But the anchor is a property of the **save**, not the file: every file in one save shares one `World.SaveStartTime`, and `Persistence.Load` reads **all** entity indexes (phase 1) before **any** persistence payload (phase 2). So the idx v5 header stamps a save-wide `World.LoadTimeShift`, and generic persistence readers inherit it. No file-format change, no per-bin header, old bins unaffected.

## Converted

- **Item v10 → v11**: `LastMoved` — previously whole-minute delta, rewritten every save for every item, the single largest source of idle-save churn — and `DecayResetTime` (retiring the TODO from #2583). **Mobile v37 → v38**: the three stat-gain stamps. **BaseCreature v20 → v21**: `SummonEnd`.
- **17 code-generated classes** (`[DeltaDateTime]` → `[AnchoredDateTime]`, version bump + `MigrateFrom` each): the five field spells, TransientItem, VirtueContext (×7 fields), PuzzleChestSolutionAndTime, BaseCamp, BaseBoat, RentedVendor, PlayerVendor, Ethics Player, Sheep, StarRoomGate, ChampionSpawn (×3), Corpse (`TimeOfDeath`, v19). The `MigrateFrom` bodies were generated from each class's current migration schema and are compiler-verified; VirtueContext's save-flagged nullables fall back to the same defaults the old deserialize left in place. Corpse's six migrations moved to a new `Corpse.Migrations.cs`.
- **Hand-written sites**: StealableArtifacts (v2), VendorInventory (v1), ML quest objectives (persistence v3) — each gated on its own version.

**Not converted, deliberately**: the ~25 read-only `ReadDeltaTime` sites in legacy version fallbacks and migration replays — they decode existing old bytes and must never change. `[DeltaDateTime]`/`WriteDeltaTime` remain available for them.

## Verification

- Build 0 errors / 0 warnings; **837 + 708 tests green**.
- Schema regeneration produced exactly the 17 expected new `vN.json` files (all `AnchoredTime` rule args), nothing else touched.
- **New acceptance tests** pin the point of the whole effort: serializing the same item at two save times **5 hours apart produces byte-identical output**, and `LastMoved`/`DecayResetTime` round-trip **exactly** at sub-minute precision (the old minutes encoding destroyed both properties).

## Notes for review

- `LastMoved` grows from a 1–3 byte encoded minutes value to 8-byte ticks per item — the price of byte-stability; it repays itself in incremental-save behavior since unchanged items now produce unchanged bytes.
- BaseEscortable-style semantics are unchanged: anchored shift preserves *remaining* time exactly, the same contract delta provided, so no gameplay-visible behavior changes — deadlines simply stop being consumed by downtime that delta already protected against, now with stable bytes.

## Enforcement

`WriteDeltaTime` is now `[Obsolete]` (interface + implementation). With the repo's warnings-as-errors, any new delta-time write — hand-written or emitted by a still-unconverted `[DeltaDateTime]` field — fails the build, with the migration instructions in the message. That the full solution still builds with **zero warnings** is itself the proof no active delta writer survived the conversion. `ReadDeltaTime` deliberately stays un-attributed: its remaining callers decode existing old bytes and are correct forever; its XML docs now state the legacy-decode-only contract.
2026-08-22 19:34:43 -07:00
Kamron Batman
126a10ce53
feat: anchored-time infrastructure with a save-start anchor in idx v5 (#2585)
## Summary

The save-stability infrastructure consumed by generator v3's `[AnchoredDateTime]`: anchored timestamps are written as **absolute values** and re-based once at load by the elapsed time since the save started — so downtime doesn't age them, and an unchanged entity serializes to identical bytes (the prerequisite for replacing delta-time encodings, which rewrite every entity on every save).

## Design

- **`WriteAnchoredTime` / `ReadAnchoredTime`** on `IGenericWriter`/`IGenericReader`. The read side applies the reader's `AnchoredTimeShift`; `Min/MaxValue` sentinels pass through unshifted, and shifts saturate instead of overflowing.
- **`World.SaveStartTime`** is stamped the moment the world freezes for a snapshot — one anchor for the entire save, no per-persistence skew.
- **idx v5**: the anchor ticks sit in the header right after the version. The anchor travels with the file it re-anchors, so a single idx+bin pair restored from a backup is self-describing, and anchor presence is guaranteed by the same version gate as the record format — there is no separate anchor file to lose.
- **The shift rides the reader instance** (`BufferReader`, `UnmanagedDataReader`, `BinaryFileReader` delegating), not a static — parallel per-persistence loads and ad-hoc restores each see their own file's anchor. idx v4 and older read with a zero shift.

## Scope

Behavior-neutral: nothing serializes anchored values yet (`Item.DecayResetTime` and the `[DeltaDateTime]` field migrations come separately, with their own version bumps). Saves written from this branch are idx v5; loading v4/v3 saves is unchanged and remains pinned by the existing hand-written-header tests.

## Testing

- Unit round-trips: exact with zero shift, shifted read, sentinel passthrough, saturation, Local→UTC normalization.
- End-to-end through the real worker/segment-log pipeline: an anchored timestamp re-bases across a simulated two-hour downtime via the idx v5 header.
- Full suites green: Server.Tests 835/835, UOContent.Tests 708/708 (including the existing v4/v3 idx loading tests).
2026-08-22 15:59:39 -07:00
Kamron Batman
e12cc5dd83
perf(saves): eliminate world-save freeze bottlenecks (~9.5x faster freeze) (#2525)
Reduces the world-save freeze window from ~740ms to ~78ms (measured on a synthetic 10M-entity / 1.7GB world, 24 cores, through the real pipeline classes) by removing the per-entity handoff between the game loop and the serialization workers, fixing how large indivisible payloads are scheduled, rewriting the BufferWriter hot path, removing per-entity placement state entirely, and finally replacing the global serialized-types tracking with a per-file type table (idx v4) that also shrinks idx files by ~21% and speeds the background write phase. The pipeline has also been validated end-to-end on live-copy worlds in the multi-million-entity range, where the freeze is drain-bound (real `Serialize()` costs far more CPU per byte than synthetic writes) — the same structural wins hold, and entity/file round-trips are byte-clean across both load paths.

## The problem

The freeze window is `max(main-thread handoff, slowest worker drain)`:

1. **The producer was the bottleneck.** The main thread round-robined every entity through per-worker `ConcurrentQueue`s — two interlocked ops per entity, ~740ms of freeze floor at 10M entities before any serialization happened.
2. **Round-robin distributes count, not cost.** "Deep" systems (50MB generic persistence blobs) and "thick" entities (100K-item storage keys) landed on arbitrary workers, producing lopsided drain times on large worlds.
3. **Worst-case scheduling.** `GenericEntityPersistence.Serialize` pushed its self-payload *after* all entities, and generic persistences sort last in the registry — so the biggest indivisible blobs started serializing at the very end, extending the freeze by their entire duration.

## The fix

**Commit 1 — chunked handoff + LPT scheduling + heap pre-sizing:**
- Pooled 4096-entity chunks published to one shared queue; workers pull chunks and load-balance dynamically (a worker busy with a thick entity simply takes fewer chunks).
- `Persistence.SerializeAll` pushes systems largest-first (LPT) using the previous save's payload size (or loaded file size on first boot); self-payloads get dedicated single-entity chunks so they overlap the entity stream instead of ending it.
- Worker heaps pre-size from the loaded save's `.bin` totals, eliminating copy-on-grow inside the first save's freeze.
- `SpinWait` backoff in the drain loop (never `Sleep(1)`), per-worker balance stats logged in debug builds (the call site is compiled out of Release), 1MB snapshot write buffer.

**Commit 2 — workers iterate the dictionaries directly + main thread joins the drain:**
- `GenericEntityPersistence` publishes 4096-slot ranges over its dictionary's backing entries array; workers serialize occupied slots (`value != null`) directly through a `ShadowEntry<TValue>` struct mirroring the runtime's private `Entry` layout. Safe because the dictionary is frozen during `Saving` (mutations divert to the pending safety queues).
- The layout is **proven at startup before any code reads through it**: validation measures the true `Entry` stride via precise allocation accounting (guaranteeing all shadow reads are in-bounds), then verifies every key/value of a churned, resized, freelist-exercised dictionary — reading value slots as raw pointer bits only, never materializing a managed reference until the layout is proven. If a future runtime changes `Dictionary` internals, validation fails with a logged warning and saves fall back to the (fully maintained) enumerate-and-push path.
- The main thread joins the drain via an inline worker after publishing, instead of idling — worth a full worker share, proportionally more on low-core hosts.

**Commit 3 — 2.2x faster BufferWriter write path, single-pass short strings:**
- PGO already devirtualizes and inlines every `IGenericWriter.Write` callsite (interface vs concrete measured identical) — the real per-write cost was the non-inlinable `Index` setter (range-check throw path + per-write high-water tracking) plus span bounds checks. Writes now reserve capacity once, then do an unaligned store through a ref with a raw index increment; the high-water mark folds at Seek/Resize instead of per write.
- Class-level implementations of the hottest default interface methods keep nested writes inlined (a DIM re-dispatches on `this` even at a devirtualized callsite).
- Strings of 85 chars or fewer encode once into a stack scratch instead of walking the string twice (`GetByteCount` + `GetBytes`). Byte output is identical.
- Measured: 34.4 → 15.7 ns/entity on a generated-style write mix; end-to-end freeze ~99ms → ~74-82ms.

**Commit 4 — branch-free fallback push loop:**
- A bare `foreach { PushToCache(entity); }` runs at 2.3ns/entity; the same loop carrying a per-entity heavy-check runs 2.3x slower — the cost is the fatter loop body defeating tight-loop codegen. Entity-level >1MB payloads are rare enough to ride in shared chunks; system self-payloads (the large ones) are still explicitly scheduled largest-first.

**Commit 5 — drop the 9-byte per-entity placement state; snapshots write from worker segment logs:**
- Every `ISerializable` carried `SerializedThread/SerializedPosition/SerializedLength` so `WriteSnapshot` could gather each entity's bytes from the worker heaps in dictionary order. But the idx records absolute positions — bin order is free — so the snapshot is now written in worker-heap order and the join inverts: workers log segments (owner, slot range, heap start) plus one length per record as they serialize; positions are implicit because a worker's writes are contiguous, and identity comes from re-walking the same snapshot slots in the same order (stable until `PostWorldSave`).
- Chunks are persistence-homogeneous (the partial chunk publishes at each `SerializeAll` boundary) so segments route to files by owner with zero per-entity state. Self-payloads keep placement as three private fields on the handful of persistence instances.
- Net: 9 bytes (plus padding) of resident state removed from every item, mobile, guild, and account on every shard; three interface-property stores per entity leave the drain hot path (stamping dirtied one cache line per entity mid-freeze — the lengths log is one sequential stream); each segment's bytes hit the bin as a single span write instead of one copy per entity, speeding the background write phase; and `IGenericSerializable` shrinks to just `Serialize(IGenericWriter)`. Transient cost: ~4 bytes per entity in pooled per-worker logs, released after each write. The save format was unchanged at this point (idx v3, same loader); the v4 bump comes later in the branch.

**Commit 6 — staged file writes replace memory-mapped snapshot writing:**
- `MemoryMapFileWriter` is removed. `FileBufferWriter` composes through the full `BufferWriter` raw write path into a pooled staging block that drains to the file as large sequential positional writes (`RandomAccess.Write`); seeks flush the block and move the file offset, so backwards patches (the idx entity count) become small positional writes.
- Memory-mapped composition paid a soft page fault on every composed page plus unpredictable dirty-section teardown stalls at dispose — measured ~4x slower end-to-end than staged writes at snapshot sizes.

**Commits 7–11 — idx v4: per-file type table replaces SerializedTypes.db and all runtime type tracking:**
- Previously every `Write(Type)` from every worker enqueued into a shared `ConcurrentQueue<Type>` during the freeze (interlocked writes on a shared cache line, millions of mostly-duplicate entries), the background phase drained and deduped it all into a `HashSet`, and the snapshot recomputed `xxHash64(Type.FullName)` once per entity record (~5.4M redundant hashes per save on a large world) to write 9-byte tag+hash idx records plus a global `SerializedTypes.db`.
- The db's only real job was diagnostics: the string name behind "Type `<X>` was not found. Delete all of those types?" during idx loading. That map now lives in the idx itself: each `GenericEntityPersistence<T>` keeps an insertion-ordered `Type -> ushort` table, hydrated at `AddEntity` and on every deserialize path — one dictionary `TryAdd` per entity add on the game thread, amortized across gameplay, and provably immutable while the background writer reads it (adds divert to the pending queues during saves).
- idx v4 layout: the table (names only) is written before the records; records reference it by 2-byte index, shrinking from 33 to 26 bytes (−21%). The loader resolves each table name **once** (`FindTypeByHash(ComputeHash64(name))` — semantically identical to v3 resolution, `TypeAlias` included) into a constructor array, and each record becomes an array index instead of an 8-byte hash read plus dictionary probe. The unresolved-type prompt now surfaces once per type, with the name.
- Deleted outright: `World.SerializedTypes`, the drain/dedupe pass in `WriteFiles`, `BufferWriter`'s type tracking (its `Write(Type)` is now pure — payload format unchanged: tag byte + xxHash64), `FileBufferWriter`'s typeSet parameter, `Persistence.WriteSerializedTypesSnapshot`, and the adhoc db write. SerializedTypes.db is no longer produced.
- **Backward compatibility:** v0–v3 saves (including their SerializedTypes.db and legacy tdb files) load exactly as before, and every legacy load path hydrates the new table so the first v4 save after an upgrade is complete. Stale db files in existing save folders are simply ignored. Verified live: a v3 save boots, saves as v4 (Items.idx −20.2% on a dev world), and reloads with identical entity counts.

## Measured (synthetic 10M entities / 1.7GB, 64+64+32MB system blobs, 24x 2MB thick entities, dense write profile, 24 cores)

| Metric | Before | After |
|---|---|---|
| Steady-state freeze | ~740 ms | **~78 ms** |
| Main-thread publish cost | ~740 ms | **~0.1 ms** |
| Steady-state allocations | 0 | 0 (by iter 2) |
| Worker byte-load spread | 2x | ~1.15x |

The freeze is now bound by pure serialize throughput (payload / cores).

## Tests

- 779 Server.Tests + 501 UOContent.Tests pass.
- New across the branch: chunk fill/flush/owner-boundary tests, pool reuse/clear tests, an end-to-end multi-worker drain through the real wake/push/flush/pause protocol, a 50K-entry churn equivalence test for the shadow iteration re-walk (the exact pairing the snapshot writer relies on), byte-level BufferWriter output/position pins, `RuntimeLayoutIsSupported` so a silent fallback on a future runtime upgrade fails loudly in CI, and a full snapshot **round-trip test** that serializes 25K entities plus a self-payload through real workers, writes the idx/bin from the segment logs, and reloads them through the standard loader (now in v4 format).
- For idx v4 specifically: `FileBufferWriter` staging/drain/seek-patch and oversized-item tests, type-table registration tests, a hand-written v4 fixture proving an unresolvable type name skips only its own records through the console confirmation flow, and a hand-written **legacy v3 fixture** proving old saves still load and hydrate the type table for their next save.

## Trade-offs

- Chunk scheduling is nondeterministic, so worker heaps ratchet to each worker's max-ever draw rather than a fixed share. With slot ranges the balance is tight (~1.15x), so the effect is small; a shared slab pool remains an option if production shows retention creep.
- Entity-level heavy items inside slot ranges are serialized wherever they're encountered (no LPT for them); worst-case tail is one thick entity's serialize time (~ms). System self-payloads — the large ones — are still explicitly scheduled largest-first.
- Snapshot-write error granularity is per segment rather than per entity (heap-bounds bugs were the only thing the per-entity catch ever caught; idx metadata reads keep per-record granularity).
- idx v4 is a save-format version bump: old saves load unchanged through the preserved legacy paths, but saves written by this branch require this loader. Per-persistence type tables cap at 65,535 distinct entity types per boot (hard throw, orders of magnitude of headroom), and a type's table slot persists until restart even if its last entity is deleted — a few stale name entries per file, by design.
2026-07-16 22:53:28 -07:00
Kamron Batman
5f9fa88220
perf: Zero-alloc interpolation for SendMessage/Overhead APIs (#2434)
## Summary

Phase 1 of a multi-phase optimization to eliminate intermediate string allocations between `$"..."` interpolation and the packet text region for ModernUO's player-facing message APIs.

- Adds `[InterpolatedStringHandler]` overloads to every `Send*`/`Public/Local/Private/NonlocalOverheadMessage`/`Say`/`Emote`/`Whisper`/`Yell`/`SendLocalizedMessageTo` API in `OutgoingMessagePackets`, `Mobile`, and `Item`. Each overload is a 3-line shim that forwards `handler.Text` to the existing span-based path then calls `handler.Clear()` to return the rented `STArrayPool<char>` buffer (matches the established `SpanWriter.WriteAscii(ref RawInterpolatedStringHandler)` precedent).
- Converts `string text/args/affix/name` parameters to `ReadOnlySpan<char>` for consistency with the handler path. `lang` intentionally stays `string` (it's never interpolated and the `??= "ENU"` fallback stays cleaner).
- Adds `int charCount` overloads of the three `GetMaxMessage*Length` helpers so stackalloc sizing can avoid the redundant `ROS<char>` round-trip.
- Moves `Mobile` (17 methods) and `Item` (4 methods) message methods into new partial-class files (`Mobile.Messages.cs`, `Item.Messages.cs`) for organization.

No UOContent call sites change in this PR — existing `string`/`ROS<char>` calls compile unchanged via implicit conversion. Phase 2 (intermediate-string audit) and Phase 3 (cleanup PRs) follow.

## Files

- `Projects/Server/Network/Packets/OutgoingMessagePackets.cs` — `string` → `ROS<char>` for text params, `int charCount` length helpers added, class made `partial`
- `Projects/Server/Network/Packets/OutgoingMessagePackets.Interpolated.cs` (new) — 3 `ref RawInterpolatedStringHandler` extension overloads
- `Projects/Server/Mobiles/Mobile.cs` — message methods extracted (-262 lines)
- `Projects/Server/Mobiles/Mobile.Messages.cs` (new, 463 lines) — moved + ROS-converted methods + 25 handler overloads
- `Projects/Server/Items/Item.cs` — message methods extracted (-93 lines)
- `Projects/Server/Items/Item.Messages.cs` (new, 142 lines) — moved + ROS-converted methods + 4 handler overloads
- `Projects/Server.Tests/Tests/Network/Packets/Outgoing/MessagePacketTests.cs` — 3 new regression tests verifying byte-equivalence for the handler overloads
2026-05-03 18:04:02 -07:00
Kamron Batman
04d438239d
feat: Adds robust speed hack detection and movement throttling (#2266)
## Summary

   Server-side movement throttle that prevents speed hacking while accurately identifying cheaters with detection of lagging connections.

   **Key features:**
   - Credit buffer (200ms) absorbs timing jitter from legitimate players
   - Movement queue handles larger bursts, draining at proper game-tick intervals
   - RTT measurement distinguishes network lag from speed hacks
   - Queue depth detection catches ACK-throttled speed hacks (going straight)

   ## How It Works

   **Throttle** (prevention): Movements arriving too early either consume credit or get queued. The queue drains at
   correct intervals, so speed hackers can't move faster regardless of what they send.

   **Detection** (identification): Combines multiple signals to identify cheaters:
   | Signal | What it catches |
   |--------|-----------------|
   | Queue depth ≥4 sustained | ACK-throttled speed hacks (client limits unacked moves to 5) |
   | Movement rate >1.05x | Direction-change speed hacks where timing is visible |
   | Stable RTT + high queue | Eliminates false positives from laggy players |

   **RTT-Aware Logic:**
   - Probes only sent to players actively moving (event-driven, not global loop)
   - Stable low-latency + problems = suspicious
   - Unstable/high-latency + problems = probably just lag, throttle handles it

   ## Configuration

   ```json
   {
     "movementThrottle.maxCredit": 200,
     "movementThrottle.softQueueLimit": 6,
     "movementThrottle.hardQueueLimit": 10,
     "movementThrottle.debugLogging": false
   }
   ```
2026-03-07 11:44:37 -08:00
Kamron Batman
e1e1a7c640
fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Kamron Batman
b191498569
feat: Adds Feature Flag System (#2328)
## Feature Flag System

A runtime feature flag system for dynamically enabling/disabling game systems and blocking specific game elements
without server restarts.

### Overview

Two types of controls:

1. Feature Flags - Named boolean flags that gate entire systems (trading, PvP, vendors, housing, etc.)
2. Dynamic Blocks - Block specific gumps, items, skills, or spells by type

All changes are persisted to JSON, broadcast to online staff, and bypass-able by administrators.

### Predefined Flags
```
┌─────────────────┬──────────┬─────────────────────────────────────┐
│      Flag       │ Category │             Description             │
├─────────────────┼──────────┼─────────────────────────────────────┤
│ player_trading  │ Economy  │ Allow secure trades between players │
├─────────────────┼──────────┼─────────────────────────────────────┤
│ vendor_purchase │ Economy  │ Allow purchasing from NPC vendors   │
├─────────────────┼──────────┼─────────────────────────────────────┤
│ vendor_sell     │ Economy  │ Allow selling to NPC vendors        │
├─────────────────┼──────────┼─────────────────────────────────────┤
│ player_vendors  │ Economy  │ Allow player vendor interactions    │
├─────────────────┼──────────┼─────────────────────────────────────┤
│ bank_access     │ Economy  │ Allow bank box access               │
├─────────────────┼──────────┼─────────────────────────────────────┤
│ house_placement │ Housing  │ Allow new house placements          │
├─────────────────┼──────────┼─────────────────────────────────────┤
│ boat_placement  │ Housing  │ Allow new boat placements           │
├─────────────────┼──────────┼─────────────────────────────────────┤
│ bulk_orders     │ Crafting │ Allow bulk order deeds              │
├─────────────────┼──────────┼─────────────────────────────────────┤
│ pvp_combat      │ Combat   │ Allow player vs player combat       │
└─────────────────┴──────────┴─────────────────────────────────────┘
```

### Commands

#### Feature Flags
```
┌────────────────────────────────────────────────────┬───────────────────────┐
│                      Command                       │      Description      │
├────────────────────────────────────────────────────┼───────────────────────┤
│ [FeatureFlag <key> on|off|toggle|info              │ Manage a feature flag │
├────────────────────────────────────────────────────┼───────────────────────┤
│ [FF <key> on|off                                   │ Shorthand alias       │
├────────────────────────────────────────────────────┼───────────────────────┤
│ [FeatureFlag <key> create <category> <desc>        │ Create a custom flag  │
├────────────────────────────────────────────────────┼───────────────────────┤
│ [FeatureFlag <key> delete                          │ Delete a custom flag  │
├────────────────────────────────────────────────────┼───────────────────────┤
│ [FeatureList [flags|gumps|items|skills|spells|all] │ List all flags/blocks │
├────────────────────────────────────────────────────┼───────────────────────┤
│ [FeatureAdmin                                      │ Open the admin gump   │
└────────────────────────────────────────────────────┴───────────────────────┘
```

#### Gump Blocks
```
┌────────────────────────────────┬──────────────────────────────────────────────────────┐
│            Command             │                     Description                      │
├────────────────────────────────┼──────────────────────────────────────────────────────┤
│ [BlockGump <typeName> [reason] │ Block a gump type from displaying                    │
├────────────────────────────────┼──────────────────────────────────────────────────────┤
│ [UnblockGump <typeName>        │ Remove a gump block                                  │
├────────────────────────────────┼──────────────────────────────────────────────────────┤
│ [ListGumps [self]              │ List open gumps on a player (for finding type names) │
└────────────────────────────────┴──────────────────────────────────────────────────────┘
```

#### Item Blocks
```
┌────────────────────────────────────────────────┬───────────────────────────────┐
│                    Command                     │          Description          │
├────────────────────────────────────────────────┼───────────────────────────────┤
│ [BlockItemUse <typeName|target> [reason]       │ Block item use                │
├────────────────────────────────────────────────┼───────────────────────────────┤
│ [BlockItemEquip <typeName|target> [reason]     │ Block item equipping          │
├────────────────────────────────────────────────┼───────────────────────────────┤
│ [BlockItemContainer <typeName|target> [reason] │ Block container access        │
├────────────────────────────────────────────────┼───────────────────────────────┤
│ [UnblockItemUse <typeName>                     │ Remove use block              │
├────────────────────────────────────────────────┼───────────────────────────────┤
│ [UnblockItemEquip <typeName>                   │ Remove equip block            │
├────────────────────────────────────────────────┼───────────────────────────────┤
│ [UnblockItemContainer <typeName>               │ Remove container access block │
└────────────────────────────────────────────────┴───────────────────────────────┘
```
Item block commands create an entry if one doesn't exist, or flip the relevant flag on an existing entry. Unblock
commands only clear their specific flag -- the entry is removed when all flags (use/equip/container) are off.

#### Skill & Spell Blocks
```
┌──────────────────────────────────────┬──────────────────────┐
│               Command                │     Description      │
├──────────────────────────────────────┼──────────────────────┤
│ [BlockSkill <SkillName> [reason]     │ Block a skill        │
├──────────────────────────────────────┼──────────────────────┤
│ [UnblockSkill <SkillName>            │ Remove a skill block │
├──────────────────────────────────────┼──────────────────────┤
│ [BlockSpell <SpellTypeName> [reason] │ Block a spell        │
├──────────────────────────────────────┼──────────────────────┤
│ [UnblockSpell <SpellTypeName>        │ Remove a spell block │
└──────────────────────────────────────┴──────────────────────┘
```

### Architecture

- Static bool classes for hot-path checks (no dictionary lookups): ServerFeatureFlags (Server project) and
ContentFeatureFlags (UOContent)
- Synced automatically by FeatureFlagManager.SyncStaticFlag() when flags change
- Predefined flags loaded from Distribution/Configuration/FeatureFlags/default-flags.json
- Runtime state persisted to Configuration/FeatureFlags/*.json (flags, gump-blocks, item-blocks, skill-blocks,
spell-blocks)
- Admin gump with tabbed UI for managing all block types, per-entry PropertiesGump editing, and pagination
- All staff at AccessLevel.GameMaster+ bypass dynamic blocks; AccessLevel.Administrator+ bypass feature flags

### Hook Points

Hook: Player trading
Location: Mobile.OpenTrade
Mechanism: ServerFeatureFlags.PlayerTrading
────────────────────────────────────────
Hook: PvP combat
Location: Mobile.CanBeHarmful
Mechanism: ServerFeatureFlags.PvPCombat
────────────────────────────────────────
Hook: Bank access
Location: BankBox.Open()
Mechanism: ServerFeatureFlags.BankAccess
────────────────────────────────────────
Hook: Vendor buy/sell
Location: BaseVendor
Mechanism: ContentFeatureFlags.VendorPurchase/Sell
────────────────────────────────────────
Hook: Player vendors
Location: PlayerVendor
Mechanism: ContentFeatureFlags.PlayerVendors
────────────────────────────────────────
Hook: House placement
Location: HousePlacement.Check()
Mechanism: ContentFeatureFlags.HousePlacement (returns BadRegionTemp)
────────────────────────────────────────
Hook: Boat placement
Location: BaseBoatDeed.OnDoubleClick/OnPlacement
Mechanism: ContentFeatureFlags.BoatPlacement
────────────────────────────────────────
Hook: Bulk orders
Location: SmallBOD/LargeBOD
Mechanism: ContentFeatureFlags.BulkOrders
────────────────────────────────────────
Hook: Gump display
Location: GumpSystem.SendGump
Mechanism: FeatureFlagManager.IsGumpBlocked
────────────────────────────────────────
Hook: Item use
Location: PlayerMobile.AllowItemUse
Mechanism: FeatureFlagManager.IsItemUseBlocked
────────────────────────────────────────
Hook: Item equip
Location: PlayerMobile.CheckEquip
Mechanism: FeatureFlagManager.IsItemEquipBlocked
────────────────────────────────────────
Hook: Container access
Location: BaseContainer.DisplayTo
Mechanism: FeatureFlagManager.IsContainerAccessBlocked
────────────────────────────────────────
Hook: Skill use
Location: PlayerMobile.AllowSkillUse
Mechanism: FeatureFlagManager.IsSkillBlocked
────────────────────────────────────────
Hook: Spell casting
Location: Spell.Cast / Spellbook.CastSpellRequest
Mechanism: FeatureFlagManager.IsSpellBlocked
2026-02-07 12:02:57 -08:00
Kamron Batman
ee2cd1d18d
feat: Adds new decay system and SkipSerialization (#2311)
## Summary

- Replaces scan-based decay checking during world saves with an event-driven timer wheel scheduler
- Removes virtual property checks from serialization hot path, achieving 6-8x faster world saves

## Changes

### DecayScheduler (new):
- Timer wheel with 12 HashSet buckets (5-min intervals) + PriorityQueue for active processing
- O(1) register/unregister vs O(n) scan of all items
- Auto start/stop when items exist/empty
- Configurable tick interval with jitter to prevent system synchronization

### Item.cs:
- Added ScheduledDecayTime computed property
- Added UpdateDecayRegistration() called from SetLastMoved(), property setters, AddItem()/RemoveItem()
- Hooks in Visible, Movable, Spawner setters and Delete()

### World.cs:
- Removed _decayQueue, EnqueueForDecay(), ProcessDecay()
- ItemPersistence.Serialize() now tight loop without virtual calls

## Performance

| Metric        | Before     | After     | Improvement |
|---------------|------------|-----------|-------------|
| Items (230K)  | 245K ticks | 34K ticks | 8x faster   |
| Mobiles (43K) | 56K ticks  | 6K ticks  | 9x faster   |
| Total         | 302K ticks | 40K ticks | 7.5x faster |

## Configuration

decay.maxItemsPerTick = 250       # Items processed per tick
decay.tickInterval = 256ms        # Base processing interval
decay.bucketInterval = 5min       # Timer wheel bucket size
decay.jitterMaxMs = 25            # ±25ms tick jitter
2026-01-08 11:43:51 -08:00
Kamron Batman
ebaf104935
chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
Kamron Batman
4aa272d429
feat: Adds an Item/Mobile memory leak detector. Fixes minor leak in doors. (#2159)
### Summary

Adds the command [TrackLeaks to enable tracking item/mobiles that have been deleted but still have dangling references. Requires adding the _TRACK_LEAKS_ define constant during build.
2025-04-15 19:14:45 -07:00
Kamron Batman
e64a632998
fix: Moves snapshot request synchronously (#2105) 2025-02-02 13:07:28 -08:00
Kamron Batman
717a1a062e
fix: Fixes race condition with world save snapshot request (#2104)
### Summary

* Fixes a race condition where the snapshot path isn't between the request snapshot being set on a background thread, and the main loop consuming that flag.


Closes #2102
2025-02-02 12:05:40 -08:00
Kamron Batman
465d3c8187
feat: Upgrades serialization v4 (Threaded Heap Serialization) (#1947)
### Summary

- `GenericEntityPersistence` is now a type of `GenericPersistence`. This allows developers to serialize both entities and non-entities in the same system. 🎉
- Each `SerializationThreadWorker` now allocates 1MB of heap for serialization _permanently_. If more memory is needed, that thread will double it's memory, not to exceed increments of 64MB.
- Several bugs with serialization introduced with the pure MMF implementation have been fixed.
- `BinaryFileReader` has been added back. 🎉
- Adds `world.useMultithreadedSaves` to allow disabling threaded saves.

> [!IMPORTANT]
> **Developer Note**
> The split file serialization has been deprecated and is no longer used. We have effectively gone back to the same file writing we had before the pure MMF implementation.
2024-09-14 09:57:43 -07:00
mdodkins
91e37fb8d4
fix: Hair and facial hair "teleporting" when mobile dies several times (#1901)
### Summary

* Added World.NewVirtual for creating virtual serial numbers
* Reserved range 0x7EEEEEEE to 0x7FFFFFFF for virtual serials
* Hair and Facial hair (for mobiles) now use virtual serials instead of FakeSerial() functions
* Consolidated virtual hair to a single `VirtualHairInfo` class.

Corpse hair and facial hair now persists across save/load and hair and facial hair no longer teleport to newest corpse.
2024-08-07 20:10:23 -07:00
Kamron Batman
fd067febd7
fix: Fixes exit threads preventing reboots (#1886) 2024-07-23 20:57:13 -07:00
Kamron Batman
5d18a194b8
fix: Deletes unused serialization files (#1848) 2024-06-26 07:57:00 -07:00
Kamron Batman
8ec203f387
feat: Updates serialization to use MMF (considerable memory savings) (#1841)
### Summary
Updates the serialization strategy to use `MemoryMappedFile` instead of thick buffers. This has the benefit of being on-par with the current implementation (based on hardware/OS), however won't incur the double-memory issue.

> [!Important]
> **Developer Note**
> The `BinaryFileWriter` and `BinaryFileReader` has been removed in favor of `MemoryMapFileWriter` and `UnmanagedDataReader`
2024-06-23 10:51:20 -07:00
Kamron Batman
9f1f314077
fix: Fixes infinite loop in BinaryFileWriter (#1792)
### Summary
- Fixes infinite loop with binary file writer
- Removes extra buffer copying with binary file writer
- Removes storing type counts during world save file writing
- Fixes display cache self-deletion warning during world load
2024-05-24 14:36:43 -07:00
Kamron Batman
05535ec9d1
fix: Fixes continuing after deserialization error (#1761) 2024-05-07 16:56:32 -07:00
Kamron Batman
e638fb7893
feat: Adds console commands (#1714)
### Summary
* Adds console command support.
* Adds `save`, `restart`, and `shutdown` commands.
2024-03-30 09:40:43 -07:00
Kamron Batman
3244704ea2
fix: Fixes directory copying (#1668) 2024-02-09 23:25:20 -08:00
Kamron Batman
3812a783ca
fix: Fixes issue with duping items and serialization, and sector lists. (#1662) 2024-01-23 23:08:52 -08:00
Kamron Batman
4cd668ef61
feat: Moves TcpServer to another thread. Rewrites Firewall (#1660)
## Breaking Changes
* The Firewall and IP Limiter have been rewritten. Please read the notes carefully!
* `TcpServer.Instances` moved back to `NetState.Instances` - sorry - it was stupid to move it to begin with.

> [!Note]
> Sockets that fail the IP Limiter or Firewall will be immediately and forcibly disconnected.
> This means they will be stuck at "Verifying account..." if it was a real client.

### Summary
- Removes firewall wildcard support.
- Removes `AccessRestrictions`.
- Moves Firewall/IPLimiter to the core.
- Moves `TcpServer` to its own thread.
- Removes the `SocketConnect` and `SocketDisconnect` event sinks.
- Moves `Instances` back to `NetState.Instances`.
- Fixes a long standing bug with bad handling of duplicate listener addresses.

#### Firewall
The firewall has been completely rewritten. There is now an "Admin Firewall" which saves to the config file. Secondarily, there is an internal firewall used exclusively by the TcpServer while processing sockets. The Admin firewall mirrors it's additions/deletions to the internal firewall by adding requests to a queue.

> [!IMPORTANT]  
> **Wildcard firewall entries, such as `X`, `*`, `?` are not allowed.**
> **Ranges in between IP classes or sextets are not allowed.**
> **Please make sure to use one of the following:**
> * IP Address - `192.168.1.1`
> * CIDR - `192.168.1.0/24`
> * Range - `192.168.1.1-192.168.1.100`

#### IP Limiter
The IP Limiter has been completely rewritten. The available configurations are:
```json
"ipLimiter.enable": "True",
"ipLimiter.maxConnectionsPerIP": 10,
"ipLimiter.clearConnectionAttemptsDuration": "00:00:00:10",
"ipLimiter.clearThrottledDuration": "00:00:02:00",
```

The IP Limiter is set up to prevent spamming connections from the same IP. Every time an IP connects, it is added to a connection list. After 10 attempts, the IP is added to the throttle list. To keep the system fast, the connection list is entirely wiped every 10 seconds, and the throttle list is entirely wiped every 2 minutes.
2024-01-20 14:25:12 -08:00
Kamron Batman
dba3ec2db5
fix: Use built-in RNG (#1599)
### Summary

.NET 8 supports Xoroshiro 256** off the shelf and added Shuffle. Switching to that implementation.

### Developer Notes

* Removed many convenience methods that weren't used.
2023-11-17 17:45:18 -08:00
Quick
c32aa39931
fix: Fixes crash on single-core machines (#1572) 2023-10-30 14:44:16 -07:00
Kamron Batman
28068c8241
fix: Fixes world save decay (#1540) 2023-10-11 14:04:13 -07:00
Kamron Batman
e1f22d9694
fix: Adds back missing world state (#1536) 2023-10-09 10:02:39 -07:00
Kamron Batman
629a5008c3
fix: Fixes hanging tests (#1535) 2023-10-09 00:22:59 -07:00
Kamron Batman
1f0acddc46
fix: Fixes serialization threading, moves world save to end of loop, and eliminates Parallel.ForEach. (#1530) 2023-10-03 00:42:49 -07:00
Kamron Batman
f2de2fbb77
fix: Cleans up entity persistence. Generalizes Mobiles/Items/Guilds (#1528)
### Summary
- [X] Fixed a bug where entity persistence was serialized out of order, causing world corruption.
- [X] Fixed LastSerialized not being utilized properly and dangling references still becoming an issue.
- [X] Added a new `GenericEntityPersistence<T>` type to encapsulate `ISerializable` serialization.
- [X] Removing the custom logic and moved Items, Mobiles, Guilds, and Accounts  to GenericEntityPersistence.
- [X] Changed serialization to use the singleton pattern to reduce calling methods from stored variables.
2023-10-01 22:10:47 -07:00
Kamron Batman
e0225c6e59
fix: Adds generic entity persistence support and BOBEntry as entities (#1527)
### Summary
Adds a generic entity persistence. This can be used to create new entity types that have a `Serial`.

Here is an example:

```cs
public class BOBEntries : GenericEntitySerialization<IBOBEntry>
{
    public static void Configure()
    {
        Configure("BOBEntries");
    }
}
```

The annotation tells the system what folder to serialize the entries to. The class/interface (`IBOBEntry`) is the root type that implements `ISerializable`.
2023-10-01 17:38:52 -07:00
Kamron Batman
8389bfacfe
chore: Updates copyright (#1448) 2023-08-09 09:09:26 -07:00
Kamron Batman
51ecee6caa
fix: Overhauls champion titles & codegen champion system (#1430)
## MAJOR CHANGE

Added a champion title system to facilitate the existing champion titles. This should make it easier to extend or create other related game content. Champion titles will be saved in a folder called _ChampionTitles_.

### Motivation

The motivation to refactor was two-folder, but mostly related to performance in two ways.
First, every player had a ChampionTitleInfo object with an array of ChamptionTitleInfo. We want to eliminate the need for this information unless a player actually uses it. This should save a considerable amount of memory.

Second, to facilitate the atrophy mechanic, the champion titles would run atrophy post-world save, adding to the time that the server is frozen. Eliminating this post-world save side effect unlocks our ability to further optimize the world save process since there are no direct side effects.


### Bugs fixed

- [X] Fixed titles getting cut off on the paperdoll
- [X] Fixed champion title not displaying overhead (OPL)

### Screenshots
<img width="216" alt="image" src="https://github.com/modernuo/ModernUO/assets/3953314/8916f895-8d68-4fb0-892e-108a0c43be90">
2023-07-27 21:44:06 -07:00
Fabrizio
537526a328
fix: Update LabelTo, SendMessage, etc to Interpolated Strings (#1283) 2022-11-28 19:58:12 -08:00
Kamron Batman
6d00b2caa9
fix: Fixes critical serial dupe bug and deserialization/serialization issues. (#1245) 2022-11-13 15:54:58 -08:00
Kamron Batman
d8cfb6b935
fix: Makes logging more consistent (#1246) 2022-11-13 00:36:23 -08:00
Kamron Batman
6426996f29
chore: Code Cleanup (#1239) 2022-11-10 22:49:25 -08:00
Kamron Batman
4eee508358
fix: Adds logging for possible duplicate objects being added (#1238)
Adds logging for `World.AddEntity<T>` and `World.AddGuild` just in case.
2022-11-10 20:49:51 -08:00
Kamron Batman
712f089f81
fix: Fixes serilog logging (#1237) 2022-11-10 14:47:56 -08:00
Kamron Batman
ebbffc3755
fix: Fixes world load not deleting bad objects (#1212) 2022-10-28 16:58:52 -07:00
Kamron Batman
49e6c6f2d1
fix: Adds AfterSerialize support. Removes BeforeSerialize support. (#1208)
### Changes
* Implements an AfterSerialize method that is executed synchronously.
* Removes `BeforeSerialize` support since it was dangerous in its current implementation.
* Moves PlayerMobile kill/virtual decay to AfterSerialize.
* Adds kill decay to after Deserialize.
2022-10-27 23:27:22 -07:00
Kamron Batman
e1e30998ba
fix: Adds ReadType/Write(Type) and improves type referencing (#1172)
## Changes
* Improves type hashing by introducing xxHash3 (64bit)
* Removes individual `tdb` files in favor of a single `SerializedTypes.db` file. This file is only used to identify a type that is being deserialized, which doesn't exist.
* Adds duplicate type alias detection
* Adds `AssemblyHandler.FindTypeByHash`

View changed files whitespaces: https://github.com/modernuo/ModernUO/pull/1172/files?diff=split&w=1

## SerializedTypes.db
The serialized types file is used to get back the original name of a type in case it no longer exists in code. This can easily be necessary if a class is renamed in code and no `TypeAlias` is provided.

### Format
byte[4] - version
byte[4] - count
--array--
byte[8] - xxHash
byte[1] - flag, 0 - null, 1 - not null
byte[n] - Full class name in UTF8

### Example
<img width="472" alt="SerializedTypes_Example" src="https://user-images.githubusercontent.com/3953314/195255429-31d24293-6bd1-419e-811b-07874dd0f78d.png">

## Benchmarks
Serialized 500 Type fields. The 8192bytes comes from the _ConcurrentQueue_ that would later be used for SerializedTypes.
Note that the queue is never cleared, so it's size grew considerably.
```cs
|               Method |     Mean |    Error |   StdDev | Allocated |
|--------------------- |---------:|---------:|---------:|----------:|
|      BenchmarkXXHash | 18.44 us | 0.278 us | 0.260 us |    8192 B |
| BenchmarkTypeStrings | 25.09 us | 0.292 us | 0.259 us |         - |
```

TODO:
* Add support in the Serialization Generator for `ReadType()` and `Write(Type)`
* Remove `SetTypeRef` from Serialization Generator
2022-10-11 22:17:22 -07:00
Kamron Batman
f268d5d4e2
fix: Cleans up core code (#1187)
**Only one functional change**
* Fixes a bug in LogFactory where `Warning` is being logged as `Information`

Non-functional changes:
* Updates/Fixes copyright headers
* Removes namespace scopes for core files.

View with [whitespace off](https://github.com/modernuo/ModernUO/pull/1187/files?w=1).
2022-10-10 21:47:08 -07:00
Kamron Batman
906ec095b9
fix: Use the latest number for the next serial after world load. (#1171) 2022-09-13 21:25:21 -07:00
Kamron Batman
2ae762904f
fix: Fixes serializing strings and uses memory mapped files for loading (#1133) 2022-08-16 08:59:00 -07:00
Kamron Batman
ec422988f5
fix: Fixes logging of world load (#1131) 2022-07-23 16:14:54 -07:00
Kamron Batman
6e69d25e33
fix: Fixes structured logging (#1043)
- [X] Fixes various bugs in logging.
2022-06-05 01:00:22 -07:00
Kamron Batman
dfdadd3204
fix: Fixes pool of acid and other empty serializations (#1025)
Fixes an issue where 0 byte objects are improperly deserialized. They should not be deserialized at all and instead deleted.
2022-05-18 17:36:23 -07:00
Kamron Batman
fc3d9b926d
fix: Adds Poison to codegen (#1002) 2022-04-18 00:05:44 -07:00
Kamron Batman
a5a5460291
fix: Fixes world save timer issue (#981)
Fixes a major bug where a race condition could cause the timer wheel to have non-deterministic behavior and potentially never finish the save.
2022-03-29 15:08:23 -07:00