ModernUO/Projects/UOContent/Misc
Kamron Batman c39454137e
feat(network): pluggable connection filters; file blocklist + contribute-first CrowdSec (#2542)
Reshapes IP banning around one idea: **core owns the question, content owns every answer.**

Core gains a single accept-path seam — `IConnectionFilter` — and loses everything that used to implement one. The firewall moves to UOContent, a new file-backed blocklist joins it there, and CrowdSec is repositioned from an in-app enforcer to a contribute-first reporter.

## The seam

```csharp
public interface IConnectionFilter
{
    string Name { get; }
    void Configure();
    void Start(CancellationToken token);
    void Stop();
    bool ShouldDeny(IPAddress address);
}
```

The accept path went from hardcoded branches to one question:

```csharp
else if (ConnectionFilters.ShouldDeny(remoteIP, out var deniedBy))
{
    logger.Debug("{Address} denied by connection filter '{Filter}'", remoteIP, deniedBy);
}
```

Filters register during the Configure sweep. The registry is a plain array walked by an indexed loop — no enumerator, no closure, no allocation — and the first denial short-circuits. An interface dispatch is noise next to the `accept()` syscall, so pluggability costs nothing measurable on the path that has to survive a DDoS.

Whatever a hit implies — persisting, promoting to an OS bouncer, contributing to the ban channel — is the filter's business, not the accept path's.

A filter that throws is **unregistered and the connection fails open**. A filter that faults once faults for every subsequent connection, so leaving it registered means an exception and a log line per accept — exactly the amplification an attacker wants — and a broken filter must not be able to deny everyone either.

This deliberately does **not** reuse `EventSink.InvokeSocketConnect`: that fires later and allocates a `SocketConnectEventArgs` per connection, which is what the accept path avoids for rejected traffic.

## What ships behind it

**`firewall`** (UOContent) — the existing admin-curated set. Collapsed from `Firewall` + `AdminFirewall` + a threaded enforcer into one single-threaded store with **zero concurrency primitives**: the accept path, admin gump, TTL expiry and boot load all run on the game loop. Persists to `Configuration/firewall.json` with automatic migration from the legacy `firewall.cfg`. No behavior change for operators — same namespace, same gump, same commands.

**`blocklist`** (UOContent) — new. Holds a millions-strong list in-app and **demand-pages** hits up to CrowdSec, which promotes them to the OS firewall.

The motivation is concrete: CrowdSec's Windows bouncer cannot load the ~3.9M IPs that 91 community feeds produce, but it handles ~100k fine. So the millions live in-process behind a binary search, and only addresses that *actually connect* get promoted. A `PromotedGuard` suppresses re-reporting an address until the bouncer picks it up.

The list is parsed straight from UTF-8 file bytes with no per-line string allocation, off the game loop, and published as an immutable snapshot swapped through a single `volatile` reference. Reloads yield to world saves.

**`tools/Export-IpBlocklist.ps1`** — the producer. Requires PowerShell 7 and runs on Windows, Linux and macOS; Windows PowerShell 5.1 is refused up front via `#requires`. Merges a thin, non-overlapping feed set into one de-duplicated, bogon-filtered file. Parsing runs in a compiled `Add-Type` hot loop (~1s for ~4M lines instead of minutes). Written to a `.tmp` sibling and swapped with `File.Replace`, so the shard never reads a half-written list, and a total feed outage refuses to overwrite a good list with an empty one. Re-running is idempotent — it exits without downloading anything while the list on disk is younger than `-MinInterval` (default 2h, the anchor feed's own refresh period), so a misconfigured scheduler can't hammer upstream.

## CrowdSec: contribute-first

`IBanReporter` + `BanChannel` fan locally-decided bans out to external systems. `CrowdSecReporter` (UOContent) posts to LAPI `POST /v1/alerts` and retracts via `DELETE /v1/decisions`.

Reporting is **enqueue-only** on the accept path: a bounded, coalescing channel drained off-loop with bounded retry, counted drops on overflow, and a flush on shutdown. Under a DDoS the accept path never does synchronous or lock-contending per-IP work.

### Why not pull decisions from CrowdSec?

The original design streamed decisions into an in-app snapshot and enforced them at the accept gate. That's the wrong layer: by the time the shard sees the connection, the TCP handshake and socket setup are already paid for. `cs-firewall-bouncer` drops the same traffic **at the kernel**, and it's what CrowdSec is built to do. So the shard now contributes what it uniquely knows (rate-limit trips, blocklist hits from real connection attempts) and lets the OS enforce.

The one thing the OS can't do — hold millions of entries on Windows — is exactly what the in-app blocklist covers, and it feeds the same pipeline.

## Threading policy

`CLAUDE.md` rule #3 is rewritten as an explicit three-part policy, with rule #10 restated in tandem:

- Anything touching game state runs **only** on the main loop.
- Heavy work that *needs* game state must be **chunked** across ticks, never threaded.
- Heavy work that does *not* need game state (large-file parse, external I/O) **must** run off-loop **and must yield to world saves**.

Results come back via an immutable snapshot swapped through a single `volatile` reference, or `Core.LoopContext.Post` — never by letting the scheduler decide where heavy work runs. Both new subsystems follow it.

## Shared primitives

`SortedRangeIndex<T> where T : IBinaryInteger<T>` — coalesced disjoint interval arrays plus a binary search. The firewall, the blocklist, and (as of this PR) core's reserved-network tables all use it.

Coalescing is a correctness requirement, not an optimization: multi-feed lists nest CIDRs (`/24` containing a `/32`), and a search that inspects only the rightmost run whose minimum is ≤ the value is sound **only** over disjoint runs. That bug was caught in review and is covered by regression tests.

`IPAddressUtility` collects the allocation-free `IPAddress` ↔ `UInt128` conversions and CIDR parsing that were previously scattered or duplicated.

## Config

| File | Owner | Keys |
|---|---|---|
| `Configuration/bans.json` | core | `reportRateLimitTrips`, `autoBanDuration` |
| `Configuration/blocklist.json` | content | `file`, `reloadInterval`, `reportHits`, `banDuration`, `promoteSuppression` |
| `Configuration/crowdsec.json` | content | `lapiUrl`, `machineId`, `password`, `origin`, `manualBanDuration`, `flushInterval`, `maxQueue` |
| `Configuration/firewall.json` | content | persisted firewall entries (migrated from `firewall.cfg`) |

Everything is inert by default. CrowdSec self-disables without credentials; the blocklist self-disables until its file exists. A shard that changes nothing sees no behavior change.

## Notes for review

- **Core no longer references `Firewall` or `IFirewallEntry` anywhere.** `NetworkUtilities` used to build its reserved-network tables out of `CidrFirewallEntry`, which coupled core to the firewall for something unrelated to banning; those are now a `SortedRangeIndex<UInt128>`, same semantics and public API.
- **`BanChannel.Stop()` no longer persists the firewall** — a contribution coordinator has no business saving an enforcement store. That's the firewall filter's `Stop()`.
- **A dead `whitelisted` parameter was dropped** from the blocklist gate: it was hardcoded `false` at its only call site, and no whitelist concept exists in core.
- **The blocklist filter is an instance, not a static.** The static version forced its tests onto the sequential collection with a reset hook; they now run in parallel.
- `dev-docs/networking-packets.md` documents the seam for content authors, plus a known wart in the `IPAddress` ↔ `UInt128` normalization flagged for a follow-up PR.
- The generator was verified on Linux, macOS and Windows under a temporary CI matrix (since removed). It caught two portability bugs — a Windows-only path separator, and a culture-sensitive duration parse that read `2.5` as `25` on comma-decimal locales and *silently* turned a 2.5h cooldown into 25h — plus a third that made the script unparseable on Windows PowerShell 5.1. The source is ASCII-only for that last reason: `#requires` is only honored once a file parses, so non-ASCII in a BOM-less script produces parse errors instead of the version message.

## Tests

**1344 pass** (782 `Server.Tests`, 562 `UOContent.Tests`). New coverage: filter registry (registration, short-circuit, fault-disable), blocklist parsing/CIDR/coalescing, snapshot reload markers, promote-guard TTL, ban-channel fan-out, CrowdSec alert building/dedup/flush-on-stop, and the generator's output-format contract pinned against the reader.
2026-07-25 11:59:37 -07:00
..
Blocklist feat(network): pluggable connection filters; file blocklist + contribute-first CrowdSec (#2542) 2026-07-25 11:59:37 -07:00
CrowdSec feat(network): pluggable connection filters; file blocklist + contribute-first CrowdSec (#2542) 2026-07-25 11:59:37 -07:00
Firewall feat(network): pluggable connection filters; file blocklist + contribute-first CrowdSec (#2542) 2026-07-25 11:59:37 -07:00
AccountPrompt.cs fix: Fixes console clobbering when prompted to add an owner account (#2133) 2025-02-26 18:43:20 -08:00
Animations.cs fix: Moves many EventSinks out of core. (#1783) 2024-05-21 10:02:30 -07:00
AOS.cs fix(ci): run test projects on CI; remove brittle OPL attribute tests (#2513) 2026-07-02 22:35:37 -07:00
AttackMessage.cs perf(messages): mechanical interpolation cleanups (#2436) 2026-05-03 18:26:49 -07:00
AutoRestart.cs fix: Fixes missing methods. Adds more admin gump world building (#1675) 2024-02-10 20:31:25 -08:00
Broadcasts.cs Fixes exiting/crash handling/ctrl+c (#237) 2020-09-12 00:05:38 -07:00
Cleanup.cs fix: Fixes structured logging (#1043) 2022-06-05 01:00:22 -07:00
ClientVerification.cs feat: Add zero-alloc interpolation handler to ValueStringBuilder, replace all StringBuilder usage (#2387) 2026-03-22 14:23:44 -07:00
CrashGuard.cs chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
DispellableAttribute.cs Formats UO Content (#201) 2020-08-27 18:30:38 -07:00
DispellableFieldAttribute.cs Formats UO Content (#201) 2020-08-27 18:30:38 -07:00
DoorGenerator.cs feat: Adds wall z-offset detection for doorgen (#2025) 2024-12-17 10:40:02 -08:00
Email.cs fix: Fixes spawn entry deserialization (#1693) 2024-03-03 08:45:16 -08:00
Emitter.cs Fixes brace style (#248) 2020-09-13 21:49:46 -07:00
FoodDecay.cs feat: Moves TcpServer to another thread. Rewrites Firewall (#1660) 2024-01-20 14:25:12 -08:00
Geometry.cs chore: Cleans up pattern checks. (#892) 2021-12-24 15:53:59 -08:00
Guild.cs fix: Eliminates double lookup with Contains->Remove (#2539) 2026-07-19 09:26:27 -07:00
HardwareInfo.cs fix: Fixes thread guard and cleans up incoming packet reader (#1641) 2023-12-19 17:04:09 -08:00
InhumanSpeech.cs feat: Replaces params array with params ReadOnlySpan (#2125) 2025-02-13 21:19:02 -08:00
Keywords.cs feat: T2A ping-pong mechanic and consider sins behaviour (#2356) 2026-03-10 23:10:27 -07:00
LanguageStatistics.cs fix: Adds command help, webpage, and fixes issues with other commands (#1669) 2024-02-10 00:19:19 -08:00
LightCycle.cs fix: Fixes buffs don't start/stop properly. Streamlines constructor and Add/Remove buffs. (#2082) 2025-01-24 18:20:23 -08:00
LoginStats.cs feat: Converts OnLogin to a coded generated event (#2070) 2025-01-17 15:21:45 -08:00
Loot.cs feat(throwing): wire SA loot flavor + Valkyrie's Glaive stealable (#2512) 2026-07-02 22:28:55 -07:00
LootPack.cs feat(throwing): wire SA loot flavor + Valkyrie's Glaive stealable (#2512) 2026-07-02 22:28:55 -07:00
MondainsLegacy.cs fix: Consolidates Color/Center html (#1762) 2024-05-07 23:56:32 -07:00
NameList.cs fix: Cleans up murder system (#1428) 2023-07-16 00:04:30 -07:00
NameVerification.cs fix: Fixes detecting consecutive exceptions in name validation (#2192) 2025-05-18 21:02:05 -07:00
Notoriety.cs fix: Eliminates double lookup with Contains->Remove (#2539) 2026-07-19 09:26:27 -07:00
PacketThrottles.cs chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
Paperdoll.cs fix(core): Converts remaining player packets (#374) 2020-12-30 16:09:51 -08:00
Poison.cs feat: Refactor Poison system, implement Darkglow & Parasitic effects (#2385) 2026-03-21 21:27:22 -07:00
PoisonKinds.cs feat: Refactor Poison system, implement Darkglow & Parasitic effects (#2385) 2026-03-21 21:27:22 -07:00
ProfanityProtection.cs fix: Fixes and optimizes NameVerification and ProfanityProtection (#2153) 2025-04-12 02:26:10 -07:00
ProfessionInfo.cs feat: Moves network related events to UOContent using code generation (#1945) 2024-09-06 16:57:10 -07:00
Profile.cs fix: Moves many EventSinks out of core. (#1783) 2024-05-21 10:02:30 -07:00
RaceDefinitions.cs fix: Updates races to properly support gargoyle equipment. (#744) 2021-08-29 00:43:55 -07:00
RegenRates.cs fix: Fixes poison field duration, cleans up skill calculations and poison calculations (#1582) 2023-11-04 12:32:54 -07:00
RenameRequests.cs fix: Fixes and optimizes NameVerification and ProfanityProtection (#2153) 2025-04-12 02:26:10 -07:00
ResourceInfo.cs fix: Codegens BOD rewards. (#1361) 2023-03-06 19:01:14 -08:00
ServerAccess.cs fix: Reverts GetAccount to use IAccount. Adds IAccount to serialization. (#1565) 2023-10-25 19:17:57 -07:00
ServerList.cs feat: Upgrades networking to use io_uring. (#2315) 2026-02-01 16:02:32 -08:00
ShardPoller.cs perf: Migrate Bulletin/Poll/SOS gumps to DynamicGump (#2418) 2026-04-25 20:49:12 -07:00
ShrinkTable.cs chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
StaffInfo.cs fix: Codegens gifts. Unifies staff names. Thank you to the community! (#1766) 2024-05-08 20:44:00 -07:00
StaminaSystem.cs feat: Adds walk/run restriction configs (#2345) 2026-02-28 09:41:40 -08:00
Titles.cs feat: Adjusts fame and karma system with era gates for OSI accuracy (#2389) 2026-04-25 11:19:57 -07:00
ToggleItem.cs fix: Codegens house/vendors (#1767) 2024-05-08 22:45:59 -07:00
TreasureMapProtection.cs chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
ValidationQueue.cs Fixes brace style (#248) 2020-09-13 21:49:46 -07:00
Weather.cs fix: Moves containers/bods/traps/etc to serialization generator (#2310) 2026-01-07 21:55:55 -08:00
WelcomeTimer.cs Removes UOAM vendors. (#241) 2020-09-12 11:56:52 -07:00