Enforce a millions-strong external IP blocklist in-app so the OS firewall
never has to hold it (Windows BFE can't). FileBlocklist loads a versioned
file into an immutable SortedRangeIndex snapshot off the game loop (yields to
world saves) and swaps it atomically; the accept path gates against it after
the manual-ban check and, once per suppression window (PromotedGuard),
promotes the hit to CrowdSec (scenario modernuo/blocklist) so the OS bouncer
kernel-drops repeat traffic. The bulk list is produced entirely out-of-process.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a pluggable ban seam: BanChannel fans locally-decided bans out to
IBanReporter sinks (Report/Retract) and the accept path enforces locally.
CrowdSec is a write-only reporter living in UOContent (registered into the
Core seam via BanChannel.Register) — it batches decisions onto a bounded,
coalescing, drop-on-overflow queue and POSTs /v1/alerts with watcher creds,
retry/backoff, and a bounded flush-on-stop. CrowdSec never enforces in-app.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Merge the old Firewall engine, AdminFirewall (parsing/persistence), and the
runtime TTL sweep into one single-threaded store: no locks/version-counter,
main-thread TTL expiry, and persistence to Configuration/firewall.json (with a
one-time firewall.cfg migration). Lookups go through a shared, coalesced
SortedRangeIndex<T> (binary search); IP conversion/parse helpers are collected
in IPAddressUtility. Firewall keeps IFirewallEntry for admin/gump/persistence.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## The bug
#2522 rewrote the outgoing huffman table in `NetworkCompression.cs` and transposed symbol `0x19`'s code from `0x1CE` to `0x12E` (both 9 bits, so the length distribution — and the Kraft sum — stayed valid, which is why nothing obvious tripped).
The real damage is that it broke prefix-freeness. `0x12E` is `100101110`, and symbol `0x0D`'s 8-bit code is `10010111` — a proper prefix of it. The client's decoder walks the tree bit by bit, so it hit a valid leaf at `0x0D` after 8 bits, emitted the wrong byte, and then reframed every subsequent code.
That is exactly what the reporter's capture shows. Server sends `BF 00 0C 00 19 02 00 00 00 01 00 00`; the client's post-decompression stream reads `BF 00 0C 00 0D 55 00 00 01 00 00` — the literal `0D` is the mis-decoded `0x19`, and the packet is now one byte short, so framing desyncs from there on.
## Impact
Any outgoing packet with byte `0x19` anywhere in its body (serials, coordinates, hues, lengths, text) corrupted the stream. Because the desync is in framing rather than a single field, the client silently stops applying server updates while still being able to send — no disconnect, no error.
`StatLockInfo` (`0xBF` subcommand `0x19`) is sent during login, so it reproduces on essentially every connection. This is also #2526: "can only walk a few steps, then the client stops responding" is the same desync, not a VPS sizing problem.
## Fix
One entry, restored to the canonical value:
```diff
- 0x9, 0x191, 0x9, 0x12E, 0x7, 0x03F, ...
+ 0x9, 0x191, 0x9, 0x1CE, 0x7, 0x03F, ...
```
## Validation of the whole table
Rather than eyeball 257 entries, I diffed the current table against **every revision of it in this repo's history** — all 34, back through the renames to the original import. All 34 agree with each other, and `0x19` is the sole disagreement with #2522's rewrite. No other entry has ever changed.
I also validated the table structurally: all 257 lengths in `[2,11]`, every value fits its declared bit-length, Kraft–McMillan sum exactly 1, and no code is a prefix of any other. It passes on all counts now, and the prefix check is what located the bug in the first place.
Both checks were one-off validation scripts, not committed — see below.
## Test
A single known-answer test (`~10ms`) that compresses all 256 symbols and asserts the exact output bytes. The expected bytes were generated from the canonical table, *not* from the implementation, so the test isn't circular. Any single wrong table entry changes the output, so it pins all 256 entries plus the terminal code, and it exercises the encoder end to end.
A round-trip test would **not** catch this class of bug — encoder and decoder built from the same table agree with each other even when the table is wrong. The contract being violated is with the client's hard-coded tree, so the expected bytes have to come from outside the implementation.
The structural prefix-free check and a second `StatLockInfo` vector were deliberately dropped after they'd served their purpose: the table is now verified and effectively frozen, so the structural check was guarding a constant, and the `StatLockInfo` vector is a strict subset of the all-symbols one. What remains covers the risk that's still live — `Compress` is a hand-unrolled bit-packing loop that will get optimized again, and this is the guard against that rewrite silently corrupting the wire format, which is precisely what happened here.
Verified the test fails when the bug is reintroduced and passes when fixed. Full `Server.Tests` suite green: 727 passed.
Fixes#2462
## Summary
Removes the per-object `VirtualHairInfo` heap wrapper for mobile/corpse hair. Hair is now stored **inline** on `Mobile` and `Corpse` as `int _hairItemId` / `int _hairHue` plus a lazily-allocated, **non-serialized** ephemeral `Serial _hairSerial` (in the high virtual-serial range) — and likewise for facial hair. The `VirtualHairInfo` class is deleted, with a **lossless** save migration.
This delivers three things:
1. **Fixes a hair-removal bug.** `Delta(MobileDelta.Hair)` is deferred (it enqueues; `ProcessDeltaQueue` runs later in the tick). The old `HairItemID = 0` setter nulled `_hair` *immediately*, so by the time `ProcessDelta` built the remove packet the equipped virtual serial was already gone — the old `??=` code then re-materialized a **fresh** serial (≠ the equipped one), so clients never removed the right entity, and it left a phantom ItemId-0 object behind. The serial now lives on the entity and **persists across removal**, so remove packets carry the correct serial.
2. **Lightens the entity.** No heap hair object; bald mobiles allocate nothing (the serial is minted lazily only when hair is present). This was the original reason `HairItemID`/`HairHue` exist.
3. **Removes `VirtualHairInfo` entirely**, keeping the high-range virtual serial behavior.
## How
- **Mobile** (manual serialization): inline `_hairItemId/_hairHue/_hairSerial` (+facial); lazy `HairSerial`/`FacialHairSerial`; `ProcessDelta` reads those. Serialization **v36 → v37** — the v30-v37 deserialize is unified, reading the legacy per-hair `VirtualHairInfo` version int only when `version < 37`. Setting item id to 0 clears the hue (matching the old object-nulling) while retaining the serial.
- **Corpse** (codegen serialization): decomposed to `[SerializableField] int _hairItemId/_hairHue` (+facial) + ephemeral serial; **v16 → v17** with `MigrateFrom(V16Content)`.
- **Lossless migration:** the loader validates exact byte length, and the old corpse hair is a presence-bool-gated block, so a tiny **migration-only** `LegacyHairInfo` reader (no runtime role) consumes the legacy `[bool][int ver][int itemId][int hue]` bytes. Frozen `Corpse.v14/v15/v16.json` are retyped to it; `v17.json` describes the new int fields.
- All consumers updated to discrete accessors: `OutgoingMobilePackets`, `CorpsePackets`, corpse subclasses (`MilitiaFighterCorpse`, `SchmendrickApprenticeCorpse`), and the packet test mirrors.
- `VirtualHair.cs` renamed to `OutgoingVirtualHairPackets.cs` (the only type left in it after `VirtualHairInfo` was removed).
## Test Plan
- [x] Full solution build: **0 warnings, 0 errors** (`TreatWarningsAsErrors`).
- [x] `Server.Tests`: **708 passed** (incl. new `RemoveHairUsesEquippedSerial` / `RemoveFacialHairUsesEquippedSerial` proving the serial survives removal + hue clears).
- [x] `UOContent.Tests` corpse/hair: **6 passed** (incl. `CorpseHairMigrationTests` asserting the legacy hair bytes are consumed exactly — the loader's length invariant).
- [x] Generated migration code inspected: V14/V15/V16 readers consume the legacy block byte-for-byte; serial never written to disk.
## Upgrade notes
- Old Mobile (v30–v36) and Corpse (v13–v16) saves load losslessly.
- Minor cosmetic-only change: `SchmendrickApprenticeCorpse` hair/facial-hair RNG draws shift order within each pair (same draw count); irrelevant for a quest NPC corpse.
## Summary
Phase 1 of a multi-phase optimization to eliminate intermediate string allocations between `$"..."` interpolation and the packet text region for ModernUO's player-facing message APIs.
- Adds `[InterpolatedStringHandler]` overloads to every `Send*`/`Public/Local/Private/NonlocalOverheadMessage`/`Say`/`Emote`/`Whisper`/`Yell`/`SendLocalizedMessageTo` API in `OutgoingMessagePackets`, `Mobile`, and `Item`. Each overload is a 3-line shim that forwards `handler.Text` to the existing span-based path then calls `handler.Clear()` to return the rented `STArrayPool<char>` buffer (matches the established `SpanWriter.WriteAscii(ref RawInterpolatedStringHandler)` precedent).
- Converts `string text/args/affix/name` parameters to `ReadOnlySpan<char>` for consistency with the handler path. `lang` intentionally stays `string` (it's never interpolated and the `??= "ENU"` fallback stays cleaner).
- Adds `int charCount` overloads of the three `GetMaxMessage*Length` helpers so stackalloc sizing can avoid the redundant `ROS<char>` round-trip.
- Moves `Mobile` (17 methods) and `Item` (4 methods) message methods into new partial-class files (`Mobile.Messages.cs`, `Item.Messages.cs`) for organization.
No UOContent call sites change in this PR — existing `string`/`ROS<char>` calls compile unchanged via implicit conversion. Phase 2 (intermediate-string audit) and Phase 3 (cleanup PRs) follow.
## Files
- `Projects/Server/Network/Packets/OutgoingMessagePackets.cs` — `string` → `ROS<char>` for text params, `int charCount` length helpers added, class made `partial`
- `Projects/Server/Network/Packets/OutgoingMessagePackets.Interpolated.cs` (new) — 3 `ref RawInterpolatedStringHandler` extension overloads
- `Projects/Server/Mobiles/Mobile.cs` — message methods extracted (-262 lines)
- `Projects/Server/Mobiles/Mobile.Messages.cs` (new, 463 lines) — moved + ROS-converted methods + 25 handler overloads
- `Projects/Server/Items/Item.cs` — message methods extracted (-93 lines)
- `Projects/Server/Items/Item.Messages.cs` (new, 142 lines) — moved + ROS-converted methods + 4 handler overloads
- `Projects/Server.Tests/Tests/Network/Packets/Outgoing/MessagePacketTests.cs` — 3 new regression tests verifying byte-equivalence for the handler overloads
## Summary
Migrates the Old Guild System (pre-AOS guild stones) gumps from the legacy `Gump` class to `DynamicGump`, following the same pattern used for the Quest gump migration in #2416. All concrete gumps now have private constructors gated by static `DisplayTo` entry points (empty-gump rule), and `Singleton => true` is set across the board so reopening a sibling dialog automatically closes the previous one.
**Migrated gumps:**
- `GuildGump` - main guild dialog
- `GuildmasterGump` - guildmaster functions
- `GuildCharterGump` - charter and website display
- `GuildWarGump` - warfare status (kept as player-facing)
- `GuildWarAdminGump` - war menu (retained as player-facing - reachable from `GuildmasterGump`'s WAR button by guildmasters)
- `GuildChangeTypeGump` - Standard/Order/Chaos selection
**Abstract bases:** `GuildListGump` and `GuildMobileListGump` keep their shared list-rendering chrome inside a single concrete `BuildLayout` on the abstract class and expose a `protected abstract void BuildHeader(ref DynamicGumpBuilder builder)` hook for subclasses (replacing the old `Design()` override). This mirrors the abstract-base treatment used for the ML quest base in the quest-gump migration PR.
**Concrete subclasses migrated alongside the abstract bases:**
- `GuildListGump` subclasses: `GuildAcceptWarGump`, `GuildDeclarePeaceGump`, `GuildDeclareWarGump`, `GuildRejectWarGump`, `GuildRescindDeclarationGump`
- `GuildMobileListGump` subclasses: `DeclareFealtyGump`, `GrantGuildTitleGump`, `GuildAdminCandidatesGump`, `GuildCandidatesGump`, `GuildDismissGump`, `GuildRosterGump`
**Cliloc rule:** Every gump bakes per-instance dynamic content (guild names, member names, war declarations, candidate lists), which would defeat `StaticGump<T>` caching. Per the cliloc rule, all are `DynamicGump`.
**External callers updated:** the prompt files (`GuildAbbrvPrompt`, `GuildCharterPrompt`, `GuildDeclareWarPrompt`, `GuildNamePrompt`, `GuildTitlePrompt`, `GuildWebsitePrompt`), `RecruitTarget`, the `Guildstone` item, and the New Guild System `GuildInfoGump`'s Order/Chaos handler all now go through static `DisplayTo` entry points instead of `new XGump(...)`.
## Summary
- **Timer-aware idle sleep**: Exposes `Timer.MillisecondsUntilNextTick()` to calculate remaining ms until the next timer wheel tick (0–8ms). The game loop sleeps for that duration minus a 1ms safety margin, instead of spinning at 100% CPU.
- **I/O completion wakeup**: Replaces `Thread.Sleep` with `NetState.WaitForCompletion()`, which uses platform-native completion notification (RIO `RIONotify` on Windows, `eventfd` on Linux, `kevent` timeout on macOS) to wake immediately when network data arrives during sleep.
- **Always-on**: Removes the debug-only `core.enableIdleCPU` config gate. The sleep is self-regulating — under load, `MillisecondsUntilNextTick` returns 0 so no sleep occurs (zero overhead). On idle, CPU drops from ~100% to ~1%.
- **CPS calculation cleanup**: Replaces the 128-element ring buffer with an EMA (exponential moving average) for `CyclesPerSecond`/`AverageCPS` — fewer allocations, no LINQ `.Average()` call each sample.
- **IORingGroup 1.0.6**: Adds `WaitForCompletion(int timeoutMs)` to the `IIORingGroup` interface with platform implementations:
- **Windows**: `RIONotify` arms the CQ event, `WaitForSingleObject` with timeout
- **Linux**: `eventfd` registered with io_uring, `poll()` with timeout
- **macOS**: `kevent()` with timeout
## Test plan
- [ ] Build succeeds on all platforms (`dotnet build`)
- [ ] Empty server: verify CPU usage drops from ~100% to ~1% idle
- [ ] Loaded server: verify no added latency — `MillisecondsUntilNextTick` returns 0 when timers are firing, sleep is skipped
- [ ] Connect a client during idle — verify connection accepted within one timer tick (~8ms)
- [ ] Verify `[admin` gump shows reasonable CPS values (EMA convergence)
## Summary
Server-side movement throttle that prevents speed hacking while accurately identifying cheaters with detection of lagging connections.
**Key features:**
- Credit buffer (200ms) absorbs timing jitter from legitimate players
- Movement queue handles larger bursts, draining at proper game-tick intervals
- RTT measurement distinguishes network lag from speed hacks
- Queue depth detection catches ACK-throttled speed hacks (going straight)
## How It Works
**Throttle** (prevention): Movements arriving too early either consume credit or get queued. The queue drains at
correct intervals, so speed hackers can't move faster regardless of what they send.
**Detection** (identification): Combines multiple signals to identify cheaters:
| Signal | What it catches |
|--------|-----------------|
| Queue depth ≥4 sustained | ACK-throttled speed hacks (client limits unacked moves to 5) |
| Movement rate >1.05x | Direction-change speed hacks where timing is visible |
| Stable RTT + high queue | Eliminates false positives from laggy players |
**RTT-Aware Logic:**
- Probes only sent to players actively moving (event-driven, not global loop)
- Stable low-latency + problems = suspicious
- Unstable/high-latency + problems = probably just lag, throttle handles it
## Configuration
```json
{
"movementThrottle.maxCredit": 200,
"movementThrottle.softQueueLimit": 6,
"movementThrottle.hardQueueLimit": 10,
"movementThrottle.debugLogging": false
}
```
### Summary
- Bump IORingGroup 1.0.0 → 1.0.1 — fixes a disconnect handling bug in the native ring layer
- Fix ghost NetStates — Dispose() set _running = false before checking it, so the "force immediate disconnect" path
was dead code. Capture wasRunning before clearing it, add [Obsolete] guard, and route internal callers through
DisposeInternal()
- Fix unauthenticated socket cleanup — graceful disconnect on unauthed connections could get stuck with pending sends;
now force-immediate after Disconnect() if DisconnectPending is already set
- Replace ConcurrentQueue<NetState> _disposed with Queue<NetState> — server is single-threaded; moved the field into
the Network partial class where it's consumed
- Move ConnectingSocketIdleLimit into the Network partial class alongside DisconnectUnattachedSockets
- Reset activity timer on receive, not just send — receiving data directly proves liveness instead of relying on the
ping→pong→send round-trip to reset the timer
- Move CheckAllAlive from Timer into Slice — the timer fired before I/O completions were processed, so after server
stalls (world saves), buffered client pings hadn't reset timestamps yet, causing false disconnects. Now runs at the
end of Slice() after all recv completions are handled
- Lower inactivity timeout 90s → 30s, check interval 90s → 5s — clients ping every ~1s, so 30s of silence is ~30
missed pings; worst-case detection drops from ~180s to ~35s
- Simplify CheckAlive — early-return when socket is null or alive; force-kill stuck DisconnectPending sockets
immediately instead of calling Disconnect() again
- Remove unused imports from GameEncryption.cs
> [!IMPORTANT]
> **Breaking Changes**
> - DecodePacket and EncodePacket delegates replaced with IClientEncryption interface
> - NetState.Connection (Socket) replaced with internal RingSocket management
> - NetState.RecvPipe and NetState.SendPipe removed (buffers managed internally)
## Summary
Upgrades the networking stack from PollGroup-based I/O to io_uring, significantly improving I/O performance on Linux.
This also adds native client encryption support for encrypted UO clients.
## Major Changes
io_uring Networking Architecture
- Replaced PollGroup with IORingGroup for async socket I/O operations
- Removed Pipe.cs (mirrored ring buffer) and TcpServer.cs in favor of RingSocketManager
- Added NetState.Network.cs - centralized network infrastructure handling accept, recv, send, and disconnect
completions
- Added SocketHelper.cs - platform-specific socket utilities for raw socket handle operations (getpeername,
getsockname)
- Buffer management now handled by RingSocketManager with configurable slab allocation
### Client Encryption Support
- Added full encryption stack in Network/Encryption/:
- EncryptionConfig.cs - configurable encryption modes (None, Unencrypted, Encrypted, Both)
- EncryptionManager.cs - encryption detection and initialization for login/game packets
- LoginEncryption.cs - handles login packet encryption with version-derived keys
- GameEncryption.cs - handles game server encryption using Twofish
- TwofishEngine.cs - optimized Twofish block cipher implementation
- LoginKeys.cs - encryption key table for client versions
- IClientEncryption.cs - interface for client encryption implementations
### NetState Improvements
- Replaced Socket Connection with RingSocket _socket for managed socket lifecycle
- Changed from GCHandle polling to event-based completion processing
- Disconnect handling now properly waits for pending sends to flush
- Simplified connecting socket management using lazy queue removal
### Configuration
- New settings: network.encryptionMode and network.encryptionDebug
- Encryption mode flags: Unencrypted, Encrypted, or Both
### Dependencies
- Replaced PollGroup NuGet package with IORingGroup
- Linux requires liburing-dev / liburing-devel package
### Test plan
- Verify server starts and accepts connections on Linux with io_uring
- Verify server starts and accepts connections on Windows (fallback to IOCP)
- Test unencrypted client connections (ClassicUO with encryption disabled)
- Test encrypted client connections if available
- Verify graceful disconnect flushes pending data
- Confirm CI builds pass on all target platforms
## Summary
- Adds proper Latin1 encoding support, replacing CP1252 usage throughout the codebase
- Adds specialized, optimized string decoding methods with safe string filtering for each encoding type
- Filters invalid Unicode characters (C0/C1 control codes, non-characters) by removal rather than replacement since
the UO client renders nothing for these characters
- Fixes UTF-16 null terminator position handling to correctly advance by 2 bytes
## Changes
TextEncoding.cs
- Added SearchValues-based invalid byte/char detection for efficient filtering
- Added encoding-specific GetString methods: GetStringAscii, GetStringLatin1, GetStringUtf8, GetStringBigUni,
GetStringLittleUni
- Each method supports a safeString parameter for filtering invalid characters
- Little-endian UTF-16 uses direct memory cast for zero-copy decoding on LE systems
- Invalid characters are removed (not replaced with U+FFFD) since the client renders nothing for them
SpanReader.cs
- Added ReadLatin1() and ReadLatin1Safe() methods
- Rewrote encoding-specific read methods to use optimized TextEncoding.GetString* methods
- Fixed UTF-16 null terminator handling: position now correctly advances by byteLength (2) instead of 1
SpanWriter.cs
- Added WriteLatin1 and WriteLatin1Null methods
## Packet Updates
- Updated all packet code to use Latin1 encoding instead of CP1252
- Affected: account packets, equipment packets, menu packets, message packets, mobile packets, player packets, secure
trade packets, vendor packets, gump packets, book packets, mahjong packets
## Filtering Behavior
Invalid characters filtered in safe mode:
```
┌───────────────┬────────────────────────┐
│ Range │ Description │
├───────────────┼────────────────────────┤
│ 0x00-0x1F │ C0 control codes │
├───────────────┼────────────────────────┤
│ 0x7F │ DEL │
├───────────────┼────────────────────────┤
│ 0x80-0x9F │ C1 control codes │
├───────────────┼────────────────────────┤
│ 0xFFFE-0xFFFF │ Unicode non-characters │
└───────────────┴────────────────────────┘
```
Note: Surrogate pairs (0xD800-0xDFFF) are not filtered because proper validation requires context checking for paired
vs unpaired surrogates. The UO client renders nothing for these anyway.
## Test Plan
- All 631 Server.Tests pass
- Verified client rendering behavior using TestUnicodeGump command (pages 1-5)
- Confirmed U+FFFD, unpaired surrogates, and non-characters all render as blank in client
- Verified Latin1 characters (0xA0-0xFF) display correctly
- Verified C1 control codes (0x80-0x9F) are filtered and don't display
### Summary
* Bumps to .NET 9 with updated dependencies
* Comparing a value type against null is no longer allowed
* CI/CD now uses the version specified in global.json
* Serialization generator updated to .NET 9 with bug fixes, fixes to turkish language, and parallelization
### Summary
- Puts back `EventSink.SocketConnect`.
- Reverts networking change to push the networking to a separate thread.
- Reverts changes to the firewall by removing the firewall queue.
- Fixes listeners not shutting down with the server.
- Fixes race condition causing connections to get stuck even after they are disposed.
> [!NOTE]
> **Developer Note**
> Networking has been reverted back to using the main thread instead of a background thread. This alleviated complexity and the requirement for concurrent queues all over the place.
> [!Important]
> **Developer Note**
> This code change will **completely move gumps out of the core**
### Summary
- Adds `GetGumps()` convenience which exposes methods to Find/Close/Send multiple gumps. This helper is a performance improvement by eliminating the Dictionary<Player, List> lookup for gumps.
### Summary
* Added World.NewVirtual for creating virtual serial numbers
* Reserved range 0x7EEEEEEE to 0x7FFFFFFF for virtual serials
* Hair and Facial hair (for mobiles) now use virtual serials instead of FakeSerial() functions
* Consolidated virtual hair to a single `VirtualHairInfo` class.
Corpse hair and facial hair now persists across save/load and hair and facial hair no longer teleport to newest corpse.
## Summary
- Removes allocation of a `List<ContextMenuEntry>` every time a context menu is created.
- Moves packet/context menu creation logic out of the core
- Fixes tame entry
## BREAKING CHANGE
> [!Important]
> **Developer Note**
> ```cs
> public virtual void GetContextMenuEntries(Mobile from, List<ContextMenuEntry> list)
> ```
> and similar functions changed to
> ```cs
> public virtual void GetContextMenuEntries(Mobile from, ref PooledRefList<ContextMenuEntry> list)
> ```
### Summary
- Fixes various exploits that can crash the shard when the client misbehaves.
- Clients will now be disconnected if they send packets that are marked as out of game only (new flag), while they are in-game.
> [!Note]
> **Developer Note**
> Added an `OutOfGameOnly` which should be used to flag packets as only available out of the game.
> This is the opposite of, yet not the converse to `InGameOnly`.
> [!Warning]
> **Developer Warning**
> The `PacketThrottle` callback return value is now reversed. `true` indicates the connection is _throttled_.
### Summary
- Fixes an issue where connections get stalled forever
- Fixes an issue where the throttler is not working properly
- Removes account attack limiter
- Rewrites IP limiter
- Removes IP restrictions (they weren't used, and not practical)
- Fixes issue where IP limiter was counting before firewall was blocking.
View without whitespace:
https://github.com/modernuo/ModernUO/pull/1796/files?diff=split&w=1
> [!Warning]
> Users on Linux/OSX will need to follow the Readme
> and make sure `libdeflate` is properly installed
> [!Note]
> **Developer Note**
> The API for compression has changed. Use `Deflate.Standard` for the same functionality.
### Summary
* Replaces Zlib with LibDeflate for a 50% performance improvement!
* Adds MacOS 14 to properly test Arm64
> [!CAUTION]
> **BREAKING CHANGE**
> Removed `OrderdHashSet` and `PooledOrderedHashSet` due to bugs.
> [!NOTE]
> **Developer Note**
> The OrderedSet is not a full data structure. It is not particularly efficient. Pull Requests are welcome for a better implementation, especially if it ends up supporting `ISet<T>` and `IReadOnlySet<T>`
## Summary
The ordered hash set was buggy. It's kind of painful to implement, so for now, I added a simple `OrderedSet` to suffice for gumps. Please reach out if this causes disruption!
# New Gump API
We are pleased to release a new API that is faster, allocates nearly zero memory, and still feels very similar to the original API. The API is broken into 3 types of gumps, dynamic, static with placeholders, and static without placeholders.
### Dynamic Gumps
These gumps will inherit `DynamicGump` and are meant for gumps that have a dynamic layout. This includes specifying dynamic arguments to HtmlLocalized entries.
## Static Gumps
Static gumps are those where the function to the build the layout is called only once and cached forever. They can optionally have placeholders. These placeholders allow the developer to specify the string values later, dynamically in a `BuildStrings` method on the gump. If a gump does not have any placeholders, the string entries will also be cached forever.
## Benchmarks
To make sure we were going in the right direction and not wasting time, we took copious benchmarks. Here are the final benchmarks for a really simple gump.
Note:
* The majority of creating a gump is compressing the layout and the strings. Compressing each section takes ~6,000ns (12us total).
```cs
| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------------------- |-------------:|-------------:|-------------:|-------------:|------:|--------:|-------:|----------:|------------:|
| OldGump | 13,308.29 ns | 1,059.695 ns | 1,883.608 ns | 14,330.72 ns | 1.000 | 0.00 | 0.1526 | 2400 B | 1.00 |
| DynamicLayoutGump | 13,357.86 ns | 129.144 ns | 226.185 ns | 13,323.60 ns | 1.029 | 0.17 | - | 48 B | 0.02 |
| StaticLayoutDynamicStringsGump | 6,653.10 ns | 81.815 ns | 143.292 ns | 6,617.45 ns | 0.514 | 0.09 | - | 40 B | 0.02 |
| StaticLayoutGump | 86.33 ns | 0.760 ns | 1.350 ns | 86.07 ns | 0.007 | 0.00 | 0.0020 | 32 B | 0.01 |
```
# Non-Breaking Changes
* All gump components in the core have been moved to `Gumps/Legacy`.
* All legacy gumps will still inherit `Gump`, which now inherits `BaseGump`
# Special Thanks
Thank you to @stefanomerotta for considerable contributions/benchmarking/testing to make this effort a reality! We collectively went through over 10 iterations, but it is finally ready.
### Summary
* Removes old gump packet support
* Removes support for v4 clients
* Removes `Unpack` flag and assumes it is always true.
* Removes StringToBuffer since this is built into .NET now.
> [!Note]
> View the file changes with white space off: https://github.com/modernuo/ModernUO/pull/1739/files?diff=split&w=1
### Summary
- Fixes `[AdvancedSearch` being accessible by players 😱
- Adds `[GenCommands` to generate the same commands html page on https://muo.gg/commands.
- Fixes `[helpinfo` so all commands properly show up!
> [!WARNING]
> ### Developer Warning:
> Commands must now be registered in the `Configure` bootup phase.
> If a command is not registered early enough, it may not be available to systems like [helpinfo
> that cache their information.
> [!NOTE]
> ### Developer Note:
> Various commands related to generating content have been changed to _Developer_ and above access level.
### Screenshots
<img width="673" alt="image" src="https://github.com/modernuo/ModernUO/assets/3953314/b105b5c9-5eb4-4ace-93ff-1bfb31e7132f">
<img width="547" alt="image" src="https://github.com/modernuo/ModernUO/assets/3953314/e97487e8-47a5-4aa7-89cc-9fe3deda584d">
## Breaking Changes
* The Firewall and IP Limiter have been rewritten. Please read the notes carefully!
* `TcpServer.Instances` moved back to `NetState.Instances` - sorry - it was stupid to move it to begin with.
> [!Note]
> Sockets that fail the IP Limiter or Firewall will be immediately and forcibly disconnected.
> This means they will be stuck at "Verifying account..." if it was a real client.
### Summary
- Removes firewall wildcard support.
- Removes `AccessRestrictions`.
- Moves Firewall/IPLimiter to the core.
- Moves `TcpServer` to its own thread.
- Removes the `SocketConnect` and `SocketDisconnect` event sinks.
- Moves `Instances` back to `NetState.Instances`.
- Fixes a long standing bug with bad handling of duplicate listener addresses.
#### Firewall
The firewall has been completely rewritten. There is now an "Admin Firewall" which saves to the config file. Secondarily, there is an internal firewall used exclusively by the TcpServer while processing sockets. The Admin firewall mirrors it's additions/deletions to the internal firewall by adding requests to a queue.
> [!IMPORTANT]
> **Wildcard firewall entries, such as `X`, `*`, `?` are not allowed.**
> **Ranges in between IP classes or sextets are not allowed.**
> **Please make sure to use one of the following:**
> * IP Address - `192.168.1.1`
> * CIDR - `192.168.1.0/24`
> * Range - `192.168.1.1-192.168.1.100`
#### IP Limiter
The IP Limiter has been completely rewritten. The available configurations are:
```json
"ipLimiter.enable": "True",
"ipLimiter.maxConnectionsPerIP": 10,
"ipLimiter.clearConnectionAttemptsDuration": "00:00:00:10",
"ipLimiter.clearThrottledDuration": "00:00:02:00",
```
The IP Limiter is set up to prevent spamming connections from the same IP. Every time an IP connects, it is added to a connection list. After 10 attempts, the IP is added to the throttle list. To keep the system fast, the connection list is entirely wiped every 10 seconds, and the throttle list is entirely wiped every 2 minutes.
