Introduces IConnectionFilter and the ConnectionFilters registry: one seam the
accept path consults per inbound socket, so core no longer has to know where a
gate's data comes from. The registry is a plain array walked by an indexed loop,
so the hot path has no enumerator, no closure and no allocation; an interface
dispatch is noise next to the accept syscall. Filters register during the
Configure sweep, cheapest first, and the first denial short-circuits.
A filter that throws on the accept path is unregistered and the connection
fails open. A filter that faults once faults for every subsequent connection,
so leaving it registered would mean an exception and a log line per accept --
exactly the amplification an attacker wants -- and a broken filter must not be
able to deny every connection either.
With that seam in place the whole file blocklist moves to UOContent: it is
policy (which feeds, when to promote, what to report) built on an external file
format with an external producer, and core does not need any of it. Firewall
stays in core -- it is long-standing public API, it is what an admin reaches for
manually, and a shard running without UOContent still has to be able to block an
address -- but it now reaches the accept path through the same registry via a
small adapter, so the two remain separate implementations rather than one
conflated store.
Blocklist policy moves out of bans.json into a UOContent Configuration/
blocklist.json, next to crowdsec.json. bans.json keeps only what core decides:
reportRateLimitTrips and autoBanDuration.
Cleanups found while moving the code:
- BanChannel.Stop() persisted the Firewall. A contribution coordinator has no
business saving an enforcement store; that is now the firewall filter's Stop.
- BlocklistGate.Evaluate took a `whitelisted` flag that was hardcoded false at
its only call site, and no whitelist concept exists anywhere in core. Dropped.
- FileBlocklist was a static holding a snapshot and a promote-guard, which
forced its tests onto the sequential collection and a LoadForTesting reset
hook. It is now an instance, so each test owns its own state and they run in
parallel. BlocklistGate folds into it: the pure decision survives as an
internal Evaluate, which is all the separate type ever provided.
- The promote-guard sweep timer was registered from NetState.Configure, two
files from the guard it swept, and ran even with the blocklist disabled. It
now starts with the filter that owns it.
Core sheds ~570 source and ~340 test lines for ~90 of seam. 1344 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds tools/Export-IpBlocklist.ps1, the producer half of the in-app blocklist
gate. It merges a thin, non-overlapping set of public IP threat feeds into one
de-duplicated, bogon-filtered list and writes the plain-text file the shard
reads via `blocklistFile`. Parsing is done in a compiled Add-Type hot loop
because the anchor feed alone is ~4M lines; the output is streamed so millions
of entries never become millions of strings.
The file is written to a .tmp sibling and swapped with File.Replace, so the
shard never observes a half-written list, and a total feed outage refuses to
overwrite a good list with an empty one. Re-running is idempotent: the script
exits without downloading anything while the list on disk is younger than
-MinInterval (default 2h, the anchor feed's own refresh period), so a
misconfigured scheduler cannot hammer the upstream feeds. Age comes from the
`generated=` header the script writes, falling back to mtime, so no sidecar
state file is needed. -Force overrides.
`blocklistFile` now defaults to Configuration/ip-blocklist.txt instead of being
empty. The gate stays inert while that file is absent, so this is a no-op for
shards that never run the generator, and dropping the file in later is picked
up by the existing poll with no restart.
Two fixes this exposed:
- FileBlocklist used the configured path verbatim despite documenting it as
relative to Core.BaseDirectory, so a relative path resolved against the
process working directory. Relative paths now resolve against BaseDirectory;
absolute paths are honored so several shards can share one generated list.
- A missing file is now the shipped default rather than a misconfiguration, so
boot logs "inert: no blocklist at ..." instead of "loaded 0 entries".
tools/ stays ignored except for this script.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hook the accept gate (NetState) so a rate-limit trip and a blocklist hit each
report to the ban channel and a blocklisted IP is denied; start/stop the ban
channel and FileBlocklist with the server; contribute single-IP manual bans
from the Admin gump and [Firewall command. Rewrites CLAUDE.md rule #3 into a
three-part threading policy (paired with rule #10: background→loop handoff via
volatile snapshot / Core.LoopContext.Post), and stops tracking the local docs/
folder.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enforce a millions-strong external IP blocklist in-app so the OS firewall
never has to hold it (Windows BFE can't). FileBlocklist loads a versioned
file into an immutable SortedRangeIndex snapshot off the game loop (yields to
world saves) and swaps it atomically; the accept path gates against it after
the manual-ban check and, once per suppression window (PromotedGuard),
promotes the hit to CrowdSec (scenario modernuo/blocklist) so the OS bouncer
kernel-drops repeat traffic. The bulk list is produced entirely out-of-process.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a pluggable ban seam: BanChannel fans locally-decided bans out to
IBanReporter sinks (Report/Retract) and the accept path enforces locally.
CrowdSec is a write-only reporter living in UOContent (registered into the
Core seam via BanChannel.Register) — it batches decisions onto a bounded,
coalescing, drop-on-overflow queue and POSTs /v1/alerts with watcher creds,
retry/backoff, and a bounded flush-on-stop. CrowdSec never enforces in-app.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Merge the old Firewall engine, AdminFirewall (parsing/persistence), and the
runtime TTL sweep into one single-threaded store: no locks/version-counter,
main-thread TTL expiry, and persistence to Configuration/firewall.json (with a
one-time firewall.cfg migration). Lookups go through a shared, coalesced
SortedRangeIndex<T> (binary search); IP conversion/parse helpers are collected
in IPAddressUtility. Firewall keeps IFirewallEntry for admin/gump/persistence.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## The bug
#2522 rewrote the outgoing huffman table in `NetworkCompression.cs` and transposed symbol `0x19`'s code from `0x1CE` to `0x12E` (both 9 bits, so the length distribution — and the Kraft sum — stayed valid, which is why nothing obvious tripped).
The real damage is that it broke prefix-freeness. `0x12E` is `100101110`, and symbol `0x0D`'s 8-bit code is `10010111` — a proper prefix of it. The client's decoder walks the tree bit by bit, so it hit a valid leaf at `0x0D` after 8 bits, emitted the wrong byte, and then reframed every subsequent code.
That is exactly what the reporter's capture shows. Server sends `BF 00 0C 00 19 02 00 00 00 01 00 00`; the client's post-decompression stream reads `BF 00 0C 00 0D 55 00 00 01 00 00` — the literal `0D` is the mis-decoded `0x19`, and the packet is now one byte short, so framing desyncs from there on.
## Impact
Any outgoing packet with byte `0x19` anywhere in its body (serials, coordinates, hues, lengths, text) corrupted the stream. Because the desync is in framing rather than a single field, the client silently stops applying server updates while still being able to send — no disconnect, no error.
`StatLockInfo` (`0xBF` subcommand `0x19`) is sent during login, so it reproduces on essentially every connection. This is also #2526: "can only walk a few steps, then the client stops responding" is the same desync, not a VPS sizing problem.
## Fix
One entry, restored to the canonical value:
```diff
- 0x9, 0x191, 0x9, 0x12E, 0x7, 0x03F, ...
+ 0x9, 0x191, 0x9, 0x1CE, 0x7, 0x03F, ...
```
## Validation of the whole table
Rather than eyeball 257 entries, I diffed the current table against **every revision of it in this repo's history** — all 34, back through the renames to the original import. All 34 agree with each other, and `0x19` is the sole disagreement with #2522's rewrite. No other entry has ever changed.
I also validated the table structurally: all 257 lengths in `[2,11]`, every value fits its declared bit-length, Kraft–McMillan sum exactly 1, and no code is a prefix of any other. It passes on all counts now, and the prefix check is what located the bug in the first place.
Both checks were one-off validation scripts, not committed — see below.
## Test
A single known-answer test (`~10ms`) that compresses all 256 symbols and asserts the exact output bytes. The expected bytes were generated from the canonical table, *not* from the implementation, so the test isn't circular. Any single wrong table entry changes the output, so it pins all 256 entries plus the terminal code, and it exercises the encoder end to end.
A round-trip test would **not** catch this class of bug — encoder and decoder built from the same table agree with each other even when the table is wrong. The contract being violated is with the client's hard-coded tree, so the expected bytes have to come from outside the implementation.
The structural prefix-free check and a second `StatLockInfo` vector were deliberately dropped after they'd served their purpose: the table is now verified and effectively frozen, so the structural check was guarding a constant, and the `StatLockInfo` vector is a strict subset of the all-symbols one. What remains covers the risk that's still live — `Compress` is a hand-unrolled bit-packing loop that will get optimized again, and this is the guard against that rewrite silently corrupting the wire format, which is precisely what happened here.
Verified the test fails when the bug is reintroduced and passes when fixed. Full `Server.Tests` suite green: 727 passed.
Fixes#2462
## Summary
Removes the per-object `VirtualHairInfo` heap wrapper for mobile/corpse hair. Hair is now stored **inline** on `Mobile` and `Corpse` as `int _hairItemId` / `int _hairHue` plus a lazily-allocated, **non-serialized** ephemeral `Serial _hairSerial` (in the high virtual-serial range) — and likewise for facial hair. The `VirtualHairInfo` class is deleted, with a **lossless** save migration.
This delivers three things:
1. **Fixes a hair-removal bug.** `Delta(MobileDelta.Hair)` is deferred (it enqueues; `ProcessDeltaQueue` runs later in the tick). The old `HairItemID = 0` setter nulled `_hair` *immediately*, so by the time `ProcessDelta` built the remove packet the equipped virtual serial was already gone — the old `??=` code then re-materialized a **fresh** serial (≠ the equipped one), so clients never removed the right entity, and it left a phantom ItemId-0 object behind. The serial now lives on the entity and **persists across removal**, so remove packets carry the correct serial.
2. **Lightens the entity.** No heap hair object; bald mobiles allocate nothing (the serial is minted lazily only when hair is present). This was the original reason `HairItemID`/`HairHue` exist.
3. **Removes `VirtualHairInfo` entirely**, keeping the high-range virtual serial behavior.
## How
- **Mobile** (manual serialization): inline `_hairItemId/_hairHue/_hairSerial` (+facial); lazy `HairSerial`/`FacialHairSerial`; `ProcessDelta` reads those. Serialization **v36 → v37** — the v30-v37 deserialize is unified, reading the legacy per-hair `VirtualHairInfo` version int only when `version < 37`. Setting item id to 0 clears the hue (matching the old object-nulling) while retaining the serial.
- **Corpse** (codegen serialization): decomposed to `[SerializableField] int _hairItemId/_hairHue` (+facial) + ephemeral serial; **v16 → v17** with `MigrateFrom(V16Content)`.
- **Lossless migration:** the loader validates exact byte length, and the old corpse hair is a presence-bool-gated block, so a tiny **migration-only** `LegacyHairInfo` reader (no runtime role) consumes the legacy `[bool][int ver][int itemId][int hue]` bytes. Frozen `Corpse.v14/v15/v16.json` are retyped to it; `v17.json` describes the new int fields.
- All consumers updated to discrete accessors: `OutgoingMobilePackets`, `CorpsePackets`, corpse subclasses (`MilitiaFighterCorpse`, `SchmendrickApprenticeCorpse`), and the packet test mirrors.
- `VirtualHair.cs` renamed to `OutgoingVirtualHairPackets.cs` (the only type left in it after `VirtualHairInfo` was removed).
## Test Plan
- [x] Full solution build: **0 warnings, 0 errors** (`TreatWarningsAsErrors`).
- [x] `Server.Tests`: **708 passed** (incl. new `RemoveHairUsesEquippedSerial` / `RemoveFacialHairUsesEquippedSerial` proving the serial survives removal + hue clears).
- [x] `UOContent.Tests` corpse/hair: **6 passed** (incl. `CorpseHairMigrationTests` asserting the legacy hair bytes are consumed exactly — the loader's length invariant).
- [x] Generated migration code inspected: V14/V15/V16 readers consume the legacy block byte-for-byte; serial never written to disk.
## Upgrade notes
- Old Mobile (v30–v36) and Corpse (v13–v16) saves load losslessly.
- Minor cosmetic-only change: `SchmendrickApprenticeCorpse` hair/facial-hair RNG draws shift order within each pair (same draw count); irrelevant for a quest NPC corpse.
## Summary
Phase 1 of a multi-phase optimization to eliminate intermediate string allocations between `$"..."` interpolation and the packet text region for ModernUO's player-facing message APIs.
- Adds `[InterpolatedStringHandler]` overloads to every `Send*`/`Public/Local/Private/NonlocalOverheadMessage`/`Say`/`Emote`/`Whisper`/`Yell`/`SendLocalizedMessageTo` API in `OutgoingMessagePackets`, `Mobile`, and `Item`. Each overload is a 3-line shim that forwards `handler.Text` to the existing span-based path then calls `handler.Clear()` to return the rented `STArrayPool<char>` buffer (matches the established `SpanWriter.WriteAscii(ref RawInterpolatedStringHandler)` precedent).
- Converts `string text/args/affix/name` parameters to `ReadOnlySpan<char>` for consistency with the handler path. `lang` intentionally stays `string` (it's never interpolated and the `??= "ENU"` fallback stays cleaner).
- Adds `int charCount` overloads of the three `GetMaxMessage*Length` helpers so stackalloc sizing can avoid the redundant `ROS<char>` round-trip.
- Moves `Mobile` (17 methods) and `Item` (4 methods) message methods into new partial-class files (`Mobile.Messages.cs`, `Item.Messages.cs`) for organization.
No UOContent call sites change in this PR — existing `string`/`ROS<char>` calls compile unchanged via implicit conversion. Phase 2 (intermediate-string audit) and Phase 3 (cleanup PRs) follow.
## Files
- `Projects/Server/Network/Packets/OutgoingMessagePackets.cs` — `string` → `ROS<char>` for text params, `int charCount` length helpers added, class made `partial`
- `Projects/Server/Network/Packets/OutgoingMessagePackets.Interpolated.cs` (new) — 3 `ref RawInterpolatedStringHandler` extension overloads
- `Projects/Server/Mobiles/Mobile.cs` — message methods extracted (-262 lines)
- `Projects/Server/Mobiles/Mobile.Messages.cs` (new, 463 lines) — moved + ROS-converted methods + 25 handler overloads
- `Projects/Server/Items/Item.cs` — message methods extracted (-93 lines)
- `Projects/Server/Items/Item.Messages.cs` (new, 142 lines) — moved + ROS-converted methods + 4 handler overloads
- `Projects/Server.Tests/Tests/Network/Packets/Outgoing/MessagePacketTests.cs` — 3 new regression tests verifying byte-equivalence for the handler overloads
## Summary
Migrates the Old Guild System (pre-AOS guild stones) gumps from the legacy `Gump` class to `DynamicGump`, following the same pattern used for the Quest gump migration in #2416. All concrete gumps now have private constructors gated by static `DisplayTo` entry points (empty-gump rule), and `Singleton => true` is set across the board so reopening a sibling dialog automatically closes the previous one.
**Migrated gumps:**
- `GuildGump` - main guild dialog
- `GuildmasterGump` - guildmaster functions
- `GuildCharterGump` - charter and website display
- `GuildWarGump` - warfare status (kept as player-facing)
- `GuildWarAdminGump` - war menu (retained as player-facing - reachable from `GuildmasterGump`'s WAR button by guildmasters)
- `GuildChangeTypeGump` - Standard/Order/Chaos selection
**Abstract bases:** `GuildListGump` and `GuildMobileListGump` keep their shared list-rendering chrome inside a single concrete `BuildLayout` on the abstract class and expose a `protected abstract void BuildHeader(ref DynamicGumpBuilder builder)` hook for subclasses (replacing the old `Design()` override). This mirrors the abstract-base treatment used for the ML quest base in the quest-gump migration PR.
**Concrete subclasses migrated alongside the abstract bases:**
- `GuildListGump` subclasses: `GuildAcceptWarGump`, `GuildDeclarePeaceGump`, `GuildDeclareWarGump`, `GuildRejectWarGump`, `GuildRescindDeclarationGump`
- `GuildMobileListGump` subclasses: `DeclareFealtyGump`, `GrantGuildTitleGump`, `GuildAdminCandidatesGump`, `GuildCandidatesGump`, `GuildDismissGump`, `GuildRosterGump`
**Cliloc rule:** Every gump bakes per-instance dynamic content (guild names, member names, war declarations, candidate lists), which would defeat `StaticGump<T>` caching. Per the cliloc rule, all are `DynamicGump`.
**External callers updated:** the prompt files (`GuildAbbrvPrompt`, `GuildCharterPrompt`, `GuildDeclareWarPrompt`, `GuildNamePrompt`, `GuildTitlePrompt`, `GuildWebsitePrompt`), `RecruitTarget`, the `Guildstone` item, and the New Guild System `GuildInfoGump`'s Order/Chaos handler all now go through static `DisplayTo` entry points instead of `new XGump(...)`.
## Summary
- **Timer-aware idle sleep**: Exposes `Timer.MillisecondsUntilNextTick()` to calculate remaining ms until the next timer wheel tick (0–8ms). The game loop sleeps for that duration minus a 1ms safety margin, instead of spinning at 100% CPU.
- **I/O completion wakeup**: Replaces `Thread.Sleep` with `NetState.WaitForCompletion()`, which uses platform-native completion notification (RIO `RIONotify` on Windows, `eventfd` on Linux, `kevent` timeout on macOS) to wake immediately when network data arrives during sleep.
- **Always-on**: Removes the debug-only `core.enableIdleCPU` config gate. The sleep is self-regulating — under load, `MillisecondsUntilNextTick` returns 0 so no sleep occurs (zero overhead). On idle, CPU drops from ~100% to ~1%.
- **CPS calculation cleanup**: Replaces the 128-element ring buffer with an EMA (exponential moving average) for `CyclesPerSecond`/`AverageCPS` — fewer allocations, no LINQ `.Average()` call each sample.
- **IORingGroup 1.0.6**: Adds `WaitForCompletion(int timeoutMs)` to the `IIORingGroup` interface with platform implementations:
- **Windows**: `RIONotify` arms the CQ event, `WaitForSingleObject` with timeout
- **Linux**: `eventfd` registered with io_uring, `poll()` with timeout
- **macOS**: `kevent()` with timeout
## Test plan
- [ ] Build succeeds on all platforms (`dotnet build`)
- [ ] Empty server: verify CPU usage drops from ~100% to ~1% idle
- [ ] Loaded server: verify no added latency — `MillisecondsUntilNextTick` returns 0 when timers are firing, sleep is skipped
- [ ] Connect a client during idle — verify connection accepted within one timer tick (~8ms)
- [ ] Verify `[admin` gump shows reasonable CPS values (EMA convergence)
## Summary
Server-side movement throttle that prevents speed hacking while accurately identifying cheaters with detection of lagging connections.
**Key features:**
- Credit buffer (200ms) absorbs timing jitter from legitimate players
- Movement queue handles larger bursts, draining at proper game-tick intervals
- RTT measurement distinguishes network lag from speed hacks
- Queue depth detection catches ACK-throttled speed hacks (going straight)
## How It Works
**Throttle** (prevention): Movements arriving too early either consume credit or get queued. The queue drains at
correct intervals, so speed hackers can't move faster regardless of what they send.
**Detection** (identification): Combines multiple signals to identify cheaters:
| Signal | What it catches |
|--------|-----------------|
| Queue depth ≥4 sustained | ACK-throttled speed hacks (client limits unacked moves to 5) |
| Movement rate >1.05x | Direction-change speed hacks where timing is visible |
| Stable RTT + high queue | Eliminates false positives from laggy players |
**RTT-Aware Logic:**
- Probes only sent to players actively moving (event-driven, not global loop)
- Stable low-latency + problems = suspicious
- Unstable/high-latency + problems = probably just lag, throttle handles it
## Configuration
```json
{
"movementThrottle.maxCredit": 200,
"movementThrottle.softQueueLimit": 6,
"movementThrottle.hardQueueLimit": 10,
"movementThrottle.debugLogging": false
}
```
### Summary
- Bump IORingGroup 1.0.0 → 1.0.1 — fixes a disconnect handling bug in the native ring layer
- Fix ghost NetStates — Dispose() set _running = false before checking it, so the "force immediate disconnect" path
was dead code. Capture wasRunning before clearing it, add [Obsolete] guard, and route internal callers through
DisposeInternal()
- Fix unauthenticated socket cleanup — graceful disconnect on unauthed connections could get stuck with pending sends;
now force-immediate after Disconnect() if DisconnectPending is already set
- Replace ConcurrentQueue<NetState> _disposed with Queue<NetState> — server is single-threaded; moved the field into
the Network partial class where it's consumed
- Move ConnectingSocketIdleLimit into the Network partial class alongside DisconnectUnattachedSockets
- Reset activity timer on receive, not just send — receiving data directly proves liveness instead of relying on the
ping→pong→send round-trip to reset the timer
- Move CheckAllAlive from Timer into Slice — the timer fired before I/O completions were processed, so after server
stalls (world saves), buffered client pings hadn't reset timestamps yet, causing false disconnects. Now runs at the
end of Slice() after all recv completions are handled
- Lower inactivity timeout 90s → 30s, check interval 90s → 5s — clients ping every ~1s, so 30s of silence is ~30
missed pings; worst-case detection drops from ~180s to ~35s
- Simplify CheckAlive — early-return when socket is null or alive; force-kill stuck DisconnectPending sockets
immediately instead of calling Disconnect() again
- Remove unused imports from GameEncryption.cs
> [!IMPORTANT]
> **Breaking Changes**
> - DecodePacket and EncodePacket delegates replaced with IClientEncryption interface
> - NetState.Connection (Socket) replaced with internal RingSocket management
> - NetState.RecvPipe and NetState.SendPipe removed (buffers managed internally)
## Summary
Upgrades the networking stack from PollGroup-based I/O to io_uring, significantly improving I/O performance on Linux.
This also adds native client encryption support for encrypted UO clients.
## Major Changes
io_uring Networking Architecture
- Replaced PollGroup with IORingGroup for async socket I/O operations
- Removed Pipe.cs (mirrored ring buffer) and TcpServer.cs in favor of RingSocketManager
- Added NetState.Network.cs - centralized network infrastructure handling accept, recv, send, and disconnect
completions
- Added SocketHelper.cs - platform-specific socket utilities for raw socket handle operations (getpeername,
getsockname)
- Buffer management now handled by RingSocketManager with configurable slab allocation
### Client Encryption Support
- Added full encryption stack in Network/Encryption/:
- EncryptionConfig.cs - configurable encryption modes (None, Unencrypted, Encrypted, Both)
- EncryptionManager.cs - encryption detection and initialization for login/game packets
- LoginEncryption.cs - handles login packet encryption with version-derived keys
- GameEncryption.cs - handles game server encryption using Twofish
- TwofishEngine.cs - optimized Twofish block cipher implementation
- LoginKeys.cs - encryption key table for client versions
- IClientEncryption.cs - interface for client encryption implementations
### NetState Improvements
- Replaced Socket Connection with RingSocket _socket for managed socket lifecycle
- Changed from GCHandle polling to event-based completion processing
- Disconnect handling now properly waits for pending sends to flush
- Simplified connecting socket management using lazy queue removal
### Configuration
- New settings: network.encryptionMode and network.encryptionDebug
- Encryption mode flags: Unencrypted, Encrypted, or Both
### Dependencies
- Replaced PollGroup NuGet package with IORingGroup
- Linux requires liburing-dev / liburing-devel package
### Test plan
- Verify server starts and accepts connections on Linux with io_uring
- Verify server starts and accepts connections on Windows (fallback to IOCP)
- Test unencrypted client connections (ClassicUO with encryption disabled)
- Test encrypted client connections if available
- Verify graceful disconnect flushes pending data
- Confirm CI builds pass on all target platforms
## Summary
- Adds proper Latin1 encoding support, replacing CP1252 usage throughout the codebase
- Adds specialized, optimized string decoding methods with safe string filtering for each encoding type
- Filters invalid Unicode characters (C0/C1 control codes, non-characters) by removal rather than replacement since
the UO client renders nothing for these characters
- Fixes UTF-16 null terminator position handling to correctly advance by 2 bytes
## Changes
TextEncoding.cs
- Added SearchValues-based invalid byte/char detection for efficient filtering
- Added encoding-specific GetString methods: GetStringAscii, GetStringLatin1, GetStringUtf8, GetStringBigUni,
GetStringLittleUni
- Each method supports a safeString parameter for filtering invalid characters
- Little-endian UTF-16 uses direct memory cast for zero-copy decoding on LE systems
- Invalid characters are removed (not replaced with U+FFFD) since the client renders nothing for them
SpanReader.cs
- Added ReadLatin1() and ReadLatin1Safe() methods
- Rewrote encoding-specific read methods to use optimized TextEncoding.GetString* methods
- Fixed UTF-16 null terminator handling: position now correctly advances by byteLength (2) instead of 1
SpanWriter.cs
- Added WriteLatin1 and WriteLatin1Null methods
## Packet Updates
- Updated all packet code to use Latin1 encoding instead of CP1252
- Affected: account packets, equipment packets, menu packets, message packets, mobile packets, player packets, secure
trade packets, vendor packets, gump packets, book packets, mahjong packets
## Filtering Behavior
Invalid characters filtered in safe mode:
```
┌───────────────┬────────────────────────┐
│ Range │ Description │
├───────────────┼────────────────────────┤
│ 0x00-0x1F │ C0 control codes │
├───────────────┼────────────────────────┤
│ 0x7F │ DEL │
├───────────────┼────────────────────────┤
│ 0x80-0x9F │ C1 control codes │
├───────────────┼────────────────────────┤
│ 0xFFFE-0xFFFF │ Unicode non-characters │
└───────────────┴────────────────────────┘
```
Note: Surrogate pairs (0xD800-0xDFFF) are not filtered because proper validation requires context checking for paired
vs unpaired surrogates. The UO client renders nothing for these anyway.
## Test Plan
- All 631 Server.Tests pass
- Verified client rendering behavior using TestUnicodeGump command (pages 1-5)
- Confirmed U+FFFD, unpaired surrogates, and non-characters all render as blank in client
- Verified Latin1 characters (0xA0-0xFF) display correctly
- Verified C1 control codes (0x80-0x9F) are filtered and don't display
### Summary
* Bumps to .NET 9 with updated dependencies
* Comparing a value type against null is no longer allowed
* CI/CD now uses the version specified in global.json
* Serialization generator updated to .NET 9 with bug fixes, fixes to turkish language, and parallelization
### Summary
- Puts back `EventSink.SocketConnect`.
- Reverts networking change to push the networking to a separate thread.
- Reverts changes to the firewall by removing the firewall queue.
- Fixes listeners not shutting down with the server.
- Fixes race condition causing connections to get stuck even after they are disposed.
> [!NOTE]
> **Developer Note**
> Networking has been reverted back to using the main thread instead of a background thread. This alleviated complexity and the requirement for concurrent queues all over the place.
> [!Important]
> **Developer Note**
> This code change will **completely move gumps out of the core**
### Summary
- Adds `GetGumps()` convenience which exposes methods to Find/Close/Send multiple gumps. This helper is a performance improvement by eliminating the Dictionary<Player, List> lookup for gumps.
### Summary
* Added World.NewVirtual for creating virtual serial numbers
* Reserved range 0x7EEEEEEE to 0x7FFFFFFF for virtual serials
* Hair and Facial hair (for mobiles) now use virtual serials instead of FakeSerial() functions
* Consolidated virtual hair to a single `VirtualHairInfo` class.
Corpse hair and facial hair now persists across save/load and hair and facial hair no longer teleport to newest corpse.
## Summary
- Removes allocation of a `List<ContextMenuEntry>` every time a context menu is created.
- Moves packet/context menu creation logic out of the core
- Fixes tame entry
## BREAKING CHANGE
> [!Important]
> **Developer Note**
> ```cs
> public virtual void GetContextMenuEntries(Mobile from, List<ContextMenuEntry> list)
> ```
> and similar functions changed to
> ```cs
> public virtual void GetContextMenuEntries(Mobile from, ref PooledRefList<ContextMenuEntry> list)
> ```
### Summary
- Fixes various exploits that can crash the shard when the client misbehaves.
- Clients will now be disconnected if they send packets that are marked as out of game only (new flag), while they are in-game.
> [!Note]
> **Developer Note**
> Added an `OutOfGameOnly` which should be used to flag packets as only available out of the game.
> This is the opposite of, yet not the converse to `InGameOnly`.
> [!Warning]
> **Developer Warning**
> The `PacketThrottle` callback return value is now reversed. `true` indicates the connection is _throttled_.
### Summary
- Fixes an issue where connections get stalled forever
- Fixes an issue where the throttler is not working properly
- Removes account attack limiter
- Rewrites IP limiter
- Removes IP restrictions (they weren't used, and not practical)
- Fixes issue where IP limiter was counting before firewall was blocking.
View without whitespace:
https://github.com/modernuo/ModernUO/pull/1796/files?diff=split&w=1
> [!Warning]
> Users on Linux/OSX will need to follow the Readme
> and make sure `libdeflate` is properly installed
> [!Note]
> **Developer Note**
> The API for compression has changed. Use `Deflate.Standard` for the same functionality.
### Summary
* Replaces Zlib with LibDeflate for a 50% performance improvement!
* Adds MacOS 14 to properly test Arm64
> [!CAUTION]
> **BREAKING CHANGE**
> Removed `OrderdHashSet` and `PooledOrderedHashSet` due to bugs.
> [!NOTE]
> **Developer Note**
> The OrderedSet is not a full data structure. It is not particularly efficient. Pull Requests are welcome for a better implementation, especially if it ends up supporting `ISet<T>` and `IReadOnlySet<T>`
## Summary
The ordered hash set was buggy. It's kind of painful to implement, so for now, I added a simple `OrderedSet` to suffice for gumps. Please reach out if this causes disruption!
# New Gump API
We are pleased to release a new API that is faster, allocates nearly zero memory, and still feels very similar to the original API. The API is broken into 3 types of gumps, dynamic, static with placeholders, and static without placeholders.
### Dynamic Gumps
These gumps will inherit `DynamicGump` and are meant for gumps that have a dynamic layout. This includes specifying dynamic arguments to HtmlLocalized entries.
## Static Gumps
Static gumps are those where the function to the build the layout is called only once and cached forever. They can optionally have placeholders. These placeholders allow the developer to specify the string values later, dynamically in a `BuildStrings` method on the gump. If a gump does not have any placeholders, the string entries will also be cached forever.
## Benchmarks
To make sure we were going in the right direction and not wasting time, we took copious benchmarks. Here are the final benchmarks for a really simple gump.
Note:
* The majority of creating a gump is compressing the layout and the strings. Compressing each section takes ~6,000ns (12us total).
```cs
| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------------------- |-------------:|-------------:|-------------:|-------------:|------:|--------:|-------:|----------:|------------:|
| OldGump | 13,308.29 ns | 1,059.695 ns | 1,883.608 ns | 14,330.72 ns | 1.000 | 0.00 | 0.1526 | 2400 B | 1.00 |
| DynamicLayoutGump | 13,357.86 ns | 129.144 ns | 226.185 ns | 13,323.60 ns | 1.029 | 0.17 | - | 48 B | 0.02 |
| StaticLayoutDynamicStringsGump | 6,653.10 ns | 81.815 ns | 143.292 ns | 6,617.45 ns | 0.514 | 0.09 | - | 40 B | 0.02 |
| StaticLayoutGump | 86.33 ns | 0.760 ns | 1.350 ns | 86.07 ns | 0.007 | 0.00 | 0.0020 | 32 B | 0.01 |
```
# Non-Breaking Changes
* All gump components in the core have been moved to `Gumps/Legacy`.
* All legacy gumps will still inherit `Gump`, which now inherits `BaseGump`
# Special Thanks
Thank you to @stefanomerotta for considerable contributions/benchmarking/testing to make this effort a reality! We collectively went through over 10 iterations, but it is finally ready.
### Summary
* Removes old gump packet support
* Removes support for v4 clients
* Removes `Unpack` flag and assumes it is always true.
* Removes StringToBuffer since this is built into .NET now.
> [!Note]
> View the file changes with white space off: https://github.com/modernuo/ModernUO/pull/1739/files?diff=split&w=1
