feat(network): allowlist false-positive IPs, escalate on behavior (#2556)

## Why

The shard owner, on a Starlink CGNAT address, was blocked by the imported reputation blocklist.

The cause was not CrowdSec. The address was a literal line in `ip-blocklist.txt`, so `BlocklistFilter` denied it at accept and then promoted it — and clearing the CrowdSec decision could not fix it either, because the file entry re-reports within `promoteSuppression` of every reconnect attempt.

This is structural, not a one-off. Reputation feeds list shared consumer address space constantly: on CGNAT one public address fronts many subscribers **at the same time**, so a single abusive customer gets the address listed and everyone else behind it is blocked with them. Where leases rotate, a listing says little about whoever holds the address now. Around 1,000 Starlink addresses sit in the current list.

So exemptions go where they cost nothing, and escalation is driven by what a connection actually does.

## Generator — `tools/Export-IpBlocklist.ps1`

`-AllowlistFile` takes multiple paths, subtracted from the merged set before the output is written. Defaults to every `ip-allowlist*.txt` beside the output, merged into one allow set:

- `ip-allowlist.txt` — operator exemptions, created once and **never rewritten**
- `ip-allowlist-<name>.txt` — a carve-out you built, regenerable and copyable between shards

**Subtraction is range-correct.** An allowlisted address inside a blocked CIDR splits that CIDR around the hole rather than being silently ignored. This also fixes `-ExcludeAnonymizers`, which parsed CIDR entries into `$anonCidr` and then only ever subtracted singles.

**No carve-out ships.** A carve-out names a real network, and which ones a shard should exempt depends on where its players actually are — so publishing one would make that policy call for every shard and put a specific provider's address space in the repo. The script builds them on request instead:

```powershell
.\Export-IpBlocklist.ps1 -AddCarveout starlink -Asn 14593
```

Carve-outs are **discovered, not configured**: every `ip-allowlist*.txt` beside the output is subtracted, by the generator and by the shard, so a file an admin adds needs no config edit and no code change. Each carries an `asn=` marker in its header, which is how `-RefreshCarveouts` rebuilds it without the script keeping a list of anyone's networks; a hand-written allowlist has no marker and is never rewritten.

Prefixes come from **announcements, not ownership records**, because registry data disagrees with what is actually routed and silently caps result sets: ARIN whois returns at most 256 rows and gives per-customer /24s, and `206.83.96.0/19` reads as APNIC in RDAP even though `206.83.96/21` is announced by Starlink.

Editing an allowlist bypasses `-MinInterval`, so a just-added exemption isn't indistinguishable from the allowlist not working. A Starlink carve-out, if you build one, costs **~4,300 IPs + ~144 CIDRs of 4.2M (0.10%)**.

## Allowlists

**`FileAllowlist`** reads the same files the generator subtracts, so an operator entry means "leave this address alone" for real. Subtraction alone only covers being *blocked*; behavioural detections never consult the blocklist, so without this a carve-out was quietly routed around — one scanner behind a shared address was enough to get everyone behind it contributed and firewalled, with nothing in the shard's own config explaining why. Reading the files also means an entry applies on the next reload rather than the next regeneration, which is what matters when someone is complaining now.

**`LoginAllowlist`** is earned by authenticating, with a 90-day TTL because an address that logged in years ago is a stranger. Its own store rather than `Account.LoginIPs`, which has no timestamps and cannot be backfilled. An entry is evidence rather than a licence: 10 suppressed contributions in an hour revokes it, and a fresh login forgives the tally.

Both are consulted **only after the blocklist has already matched**, so a normal accept pays nothing for them and the accept gate stays allowlist-free. `BanExemptions` combines them behind `BanChannel.IsExempt` and suppresses escalation only — every local defence still applies.

Two limits, both deliberate and documented in the class: `LoginAllowlist` **cannot bootstrap** (an entry is only earned by getting in, so it never repairs an existing false positive), and it is weakest on rotating CGNAT. That is why `FileAllowlist` is the fix for those, and why it is manual.

## Behavioural detection

| Reason | Trigger |
|---|---|
| `silent-connect` | Reaped after 5s having sent **zero bytes** |
| `invalid-seed` | Opened with a zero seed |
| `foreign-protocol` | Positively identified as HTTP, TLS or SSH |

**`ForeignProtocol` inverts the test.** Asking "is this a good UO client?" cannot work: `LoginEncryption.ClientDecrypt` is a byte-for-byte stream XOR, so a legitimate client with encryption enabled when the shard expects none sends a structurally perfect connection whose payload is noise. "Speaks HTTP" is safe where "unreadable" is not — however misconfigured a UO client is, it never sends `GET / HTTP/1.1`.

Nothing assumes arrival framing. TCP has no message boundaries, so a rule of the form "these bytes must arrive together" is broken by construction and drops real players on poor links. A prefix match with too few bytes to confirm waits for more. A four-byte seed can legitimately spell `GET ` (the address 71.69.84.32) or `0x16 0x03 0x0?` (22.3.x.x), so confirmation requires the request line to continue in printable ASCII or an actual ClientHello inside a plausible record — a real client's fifth byte is a packet id (`0x80`, `0x91`, `0xEF`), none of them printable, so those collisions fall through.

Everything is keyed on **bytes-received rather than elapsed time**. A connection that sent something and ran out of time is far more likely a slow link than an attack, and banning those produces the worst failure mode available: the player retries, trips the rate limiter, and compounds a bad connection into hours of being firewalled off.

## `AutoDenylist`

A short-lived local hold (15m) on behavioural detections, as `IConnectionFilter` + `IBanReporter` over one store so the engine detection sites never reach into content.

This closes the gap where a flood pays for a socket, buffer and `NetState` slot per connection while waiting for the OS bouncer — the verdicts that matter most are reachable only *after* reading bytes — and it is the entire defence on a shard running no bouncer, which is the default config. Not persisted: a holding pen that survives restarts is a ban without a ban's review.

Cost: one dictionary lookup on a usually-empty dict per accept.

## `BanReasons`

Centralises the reason slugs. `IsBehavioral` is an **opt-in** set, not "everything except manual", so a future reason escalates normally instead of silently inheriting an exemption or entering a local denylist.

This caught a real bug during review: the first cut of the exemption swallowed `manual` admin bans (`Commands.cs`, three sites in `AdminGump`) for any allowlisted address.

## Fixes found in review

- **`BanConfiguration.Settings` was null until `Configure()` ran**, while the reap path dereferences it every `Slice()`. A harness driving `NetState.Slice()` directly hit an NRE that presented as flaky because it depended on whether an earlier test had already called `Configure()` — which is why it failed on some CI platforms and not others. Now starts at the record's defaults, with idempotency tracked by a flag; this also removes the same latent NRE from the pre-existing rate-limit path.
- **`-AllowlistFile` was typed `[string]`** while documented and used as a list, so passing two paths would have collapsed them into one string.

## Layout and docs

Content network code moves out of `Misc/` into `UOContent/Network/`, one concern per folder — `AutoDenylist/`, `Blocklist/`, `CrowdSec/`, `Firewall/`, `LoginAllowlist/`, `Packets/`. **Namespaces are untouched**, so these are pure file moves (git tracks all 16 as renames).

`dev-docs/ip-bans-and-allowlists.md` documents the subsystem, leading with the operator process for unblocking a player — including the three things that look sufficient and are not: deleting the CrowdSec decision alone, editing `ip-blocklist.txt` by hand, and `cscli allowlists` alone. `.gitignore` covers the new config files.

## Testing

Build clean. **Server.Tests 810 passed**, **UOContent.Tests 637 passed**, zero warnings. This branch adds 38 tests; the rest of the delta is main's, since this is rebased on current `main`.

New coverage: TTL boundary and renewal, private-address exclusion, manual-ban-never-exempt, unopted-reason-never-exempt, strike revocation, quiet-window reset, login forgiveness, file-allowlist CIDR coverage, file-allowlist not spending the earned list's strikes, denylist expiry-on-read, cap enforcement, lapsed-entry reclaim, HTTP/TLS/SSH identification, seed-collision fall-through, and encrypted-login-is-not-foreign.

Generator verified end-to-end against live feeds: a clean run ships no carve-out, `-AddCarveout starlink -Asn 14593` fetches and collapses 213 prefixes to 115 ranges in 0.1s over 4.2M entries, `-RefreshCarveouts` rediscovers it by its `asn=` marker, a hand-written allowlist is left untouched, and deleting a carve-out drops it rather than having it rewritten. CIDR splitting verified exhaustively: a single-IP hole in a /24 leaves exactly 255 of 256 addresses blocked.

## Operator note

Existing installs are unaffected until the generator next runs, which creates `ip-allowlist.txt` and nothing else. To unblock someone: add the address to that file and delete any live CrowdSec decision — the existing ban outlives the config change. The shard picks the entry up on its next reload, so re-running the generator is optional.

A shard whose players are on CGNAT (satellite, mobile, or an ISP short on IPv4) will likely also want `-AddCarveout`; see `dev-docs/ip-bans-and-allowlists.md`.

## Also included: a latent CI failure this PR surfaced

`fix(tests): serialize test classes that rent through STArrayPool` touches a property-list test file that has nothing to do with this feature. It is here because it was failing macOS CI, and it is trivially cherry-pickable out if you would rather it went to `main` on its own — **which may be the better call, since it is failing `main` today.**

CI has since gone green with it applied.

`STArrayPool` is single-threaded by design and its bucket cache is a plain `static`, not `[ThreadStatic]`, with a check-then-act initialize in `Return()`:

```csharp
var cacheBuckets = _cacheBuckets ?? InitializeBuckets();
```

Two threads both see null, both initialize, and the loser trips `Debug.Assert(_cacheBuckets is null)`. Anything renting from it has to stay off parallel test threads — which is what the `DisableParallelization` collections are for.

- `ObjectPropertyListReentrancyTests` and `ObjectPropertyListNestedBuildTests` (added in #2555) build property lists, which rent the interpolation buffer, but were not in the sequential collection — unlike `PropertyListInvalidationDuringBuildTests` in the same file. This is a **latent failure already on `main`**; it is timing-dependent, so it shows on some platforms and not others.
- `AutoDenylistTests` (added here) has the same exposure: its cap tests reach `AutoDenylist.Sweep`, which rents a `PooledRefList` without `mt`. The blocklist tests need no marking because `BlocklistSnapshot.Build` asks for the `mt` pool explicitly.

No production change — `STArrayPool` is the right pool on the game loop, where both `Sweep` and the property list actually run.

## Deliberately not included

Waiting for a fragmented four-byte seed at `AwaitingSeed`. It looked like a bug but the disconnect is a deliberate defence: only pre-0xEF clients reach it (0xEF goes through `HandlePacket`, which already waits for its 21 bytes), and waiting converts an instant drop into a full 5s slot hold for a client sending one or two bytes, or a loris dribbling a byte every few seconds. Against a fixed 4096-entry `MaxConnections` table that trades capacity that matters for a fragmentation case a reconnect already fixes.

dev-docs/ip-bans-and-allowlists.md

@@ -0,0 +1,216 @@
+# IP Bans, Blocklists and Allowlists
+
+How a shard decides to refuse a connection, how it contributes bans to an external bouncer, and how an
+operator exempts someone who was caught by mistake.
+
+If you are here because **a player cannot connect**, skip to [Unblocking a player](#unblocking-a-player).
+
+## The shape of it
+
+Two independent questions, deliberately separated:
+
+| Question | Answered by | Effect |
+|---|---|---|
+| Refuse this connection? | `IConnectionFilter` implementations, at accept | The socket is dropped |
+| Tell the outside world about it? | `BanChannel` → `IBanReporter` implementations | CrowdSec, and from there an OS bouncer |
+
+`BanChannel` **never enforces** and filters **never report on each other's behalf**. Enforcement that
+outlives the process belongs to the OS bouncer; the shard only contributes.
+
+### Refusing a connection
+
+Filters are consulted in registration order, first denial wins:
+
+| Filter | Source | Scope |
+|---|---|---|
+| `firewall` | `Configuration/firewall.json`, mutable in-game | Admin-curated, permanent |
+| `blocklist` | `Configuration/ip-blocklist.txt` (millions of entries) | Reputation feeds |
+| `auto-denylist` | In-memory, 15 min | What this shard just caught misbehaving |
+
+### Contributing a ban
+
+`BanChannel.Report(address, ttl, reason)` → `BanExemptions.IsExempt` → if not exempt, fan out to every
+reporter (`crowdsec`, `auto-denylist`).
+
+### Allowlists
+
+Two, with different authority:
+
+| List | Source | Revocable? | Covers |
+|---|---|---|---|
+| `FileAllowlist` | every `ip-allowlist*.txt` | No — an operator said so | Blocking **and** escalation |
+| `LoginAllowlist` | Earned by authenticating, 90-day TTL | Yes — 10 strikes/hour | Blocking **and** escalation |
+
+Both are consulted **only after the blocklist has already matched**, so a normal accept — the one an
+attacker is trying to flood — pays nothing for them. The accept gate itself is deliberately allowlist-free:
+a whitelist there could only turn a deny into an allow at the cost of a lookup on every accept, the
+attacker's included.
+
+## Unblocking a player
+
+### 1. Find out what is actually blocking them
+
+```bash
+# In the reputation blocklist?
+grep -x "203.0.113.42" Distribution/Configuration/ip-blocklist.txt
+
+# A live external decision? (this is what survives a restart)
+cscli decisions list --ip 203.0.113.42
+
+# Admin-curated?
+grep 203.0.113.42 Distribution/Configuration/firewall.json
+```
+
+If none of those match, they may be inside a **CIDR** in the blocklist, or held by the in-memory
+`auto-denylist` — that one is not queryable and expires on its own within 15 minutes.
+
+### 2. Add them to the allowlist
+
+One entry per line in `Distribution/Configuration/ip-allowlist.txt` — a bare address or a CIDR. This file
+is yours; the generator creates it once and never rewrites it.
+
+```
+203.0.113.42        # shard owner, listed via a shared upstream address
+198.51.100.0/24     # a whole range if the ISP rotates within it
+```
+
+The shard reloads within `reloadInterval` (60s default). **No restart, and no need to re-run the
+generator.** From that point the address is neither blocked nor contributed.
+
+### 3. Clear any ban that already exists
+
+A config change cannot retract a ban that has already left the building:
+
+```bash
+cscli decisions delete -i 203.0.113.42
+```
+
+### 4. If it keeps coming back
+
+The shard is no longer contributing them, so a recurring ban is coming from CrowdSec's own sources (the
+community blocklist, another watcher). Allowlist it there too:
+
+```bash
+cscli allowlists create shard-staff -d "Known-good player addresses"
+cscli allowlists add shard-staff 203.0.113.42
+```
+
+### What will NOT work
+
+- **Deleting the CrowdSec decision alone.** If the address is still in `ip-blocklist.txt` and not
+  allowlisted, the next connection re-reports it within `promoteSuppression` (60s).
+- **Editing `ip-blocklist.txt` by hand.** The next generator run rewrites the whole file.
+- **`cscli allowlists` alone.** That is the enforcement layer. The shard's own accept gate sits upstream of
+  it and will still refuse the connection.
+
+## Why entries appear that should not
+
+Reputation feeds list shared consumer address space constantly. On CGNAT one public address fronts many
+subscribers **at the same time**, so a single abusive customer gets the address listed and everyone else
+behind it is blocked with them — and where leases rotate, a listing says little about whoever holds the
+address now. This is near-universal on mobile carriers, satellite (Starlink) and WISPs, and common on
+fixed-line broadband outside North America.
+
+A **carve-out** exempts a whole network. **None ship with ModernUO** — which providers to exempt depends on
+where your players actually are, and a carve-out names a real network, so you build the ones you need:
+
+```powershell
+# Exempt a CGNAT provider whose players keep getting listed
+.\Export-IpBlocklist.ps1 -AddCarveout starlink -Asn 14593
+
+# Later: bring every carve-out up to date with what those networks currently announce
+.\Export-IpBlocklist.ps1 -RefreshCarveouts
+```
+
+That writes `ip-allowlist-starlink.txt` beside the blocklist, and every `ip-allowlist*.txt` there is
+subtracted — both by the generator and by the shard, with no config edit. Starlink costs about 0.1% of the
+list. Blank a file (keep the file) to reputation-block that network again; delete it to drop the carve-out.
+
+Carve-out files carry an `asn=` marker in their header, which is how `-RefreshCarveouts` finds them. A
+hand-written allowlist has no marker and is never rewritten.
+
+**Do you need one?** If players report being blocked and they are on satellite, mobile, or an ISP short on
+IPv4, probably yes. Find the ASN by looking up an address the network hands out on any public BGP lookup.
+
+Prefixes come from **announcements, not ownership records**. Registry data disagrees with what is actually
+routed and silently caps result sets: ARIN whois returns at most 256 rows and gives per-customer /24s, and
+`206.83.96.0/19` reads as APNIC in RDAP even though `206.83.96/21` is announced by Starlink.
+
+## Behavioural detection
+
+Verdicts the shard reaches by watching a connection, rather than by consulting a list:
+
+| Reason | Trigger |
+|---|---|
+| `rate-limit` | Too many connection attempts in the limiter's window |
+| `silent-connect` | Reaped after `ConnectingSocketIdleLimit` (5s) having sent **zero bytes** |
+| `invalid-seed` | Opened with a zero seed, which no real client sends |
+| `foreign-protocol` | Positively identified as HTTP, TLS or SSH |
+
+`BanReasons.IsBehavioral` gates two things: only these may be exempted, and only these enter the
+`auto-denylist`. It is an **opt-in list**, not "everything except `manual`" — a reason added later escalates
+normally rather than silently inheriting an exemption. `manual` is never exempt and never auto-denied.
+
+Escalation is **immediate**, on the first detection: a 15-minute local hold plus a `badConnectDuration`
+(4h) contribution. There is no N-connection threshold; the strike counter governs only revoking a
+`LoginAllowlist` entry.
+
+### What is deliberately NOT detected
+
+**Do not add rules based on arrival framing.** TCP has no message boundaries, so the network, the OS or a
+middlebox can split the opening bytes anywhere regardless of what the client sent. A rule of the form
+"these bytes must arrive together" is broken by construction and drops real players on poor links. This has
+been tried and reverted before.
+
+**Do not treat unreadable payloads as hostile.** A legitimate client with encryption enabled when the shard
+expects none sends a structurally perfect connection whose payload is noise — `LoginEncryption.ClientDecrypt`
+is a byte-for-byte stream XOR, so it preserves length exactly while destroying content. This is why
+detection asks "is this positively some *other* protocol?" rather than "is this a good UO client?": however
+misconfigured a UO client is, it never sends `GET / HTTP/1.1`.
+
+**Timeouts are keyed on bytes-received, not elapsed time.** A connection that sent *something* and ran out
+of time is far more likely a slow link than an attack. Banning those produces the worst failure mode
+available: the player retries, trips the rate limiter, and compounds a bad connection into hours of being
+firewalled off. Shortening the 5s handshake window has been tried and broke real players.
+
+## Known limits
+
+- **An allowlist cannot bootstrap.** A `LoginAllowlist` entry is only earned by getting in, so it can never
+  repair an existing false positive, and it is weakest on rotating CGNAT — a player whose lease moved is a
+  stranger again. `FileAllowlist` is the fix for that, which is why it is manual.
+- **A never-logged-in player on a shared address can still be caught**, for up to `badConnectDuration`, if a
+  co-tenant misbehaves. Accepted: it is 4h and self-healing. The cheapest lever is `badConnectDuration`.
+- **`MaxConnections` (4096) is a hard ceiling.** The accept gate runs *after* the kernel completed the TCP
+  handshake, so a blocklist match saves the socket setup and the `NetState` slot but never the connection
+  itself. Only an upstream L4 proxy or edge scrubbing moves that cost off the shard.
+
+## Configuration
+
+| File | Controls |
+|---|---|
+| `bans.json` | `reportRateLimitTrips`, `autoBanDuration`, `reportBadConnects`, `badConnectDuration` |
+| `blocklist.json` | `file`, `allowlistFiles` (wildcards allowed), `reloadInterval`, `reportHits`, `banDuration`, `promoteSuppression` |
+| `login-allowlist.json` | `enabled`, `file`, `ttl`, `flushInterval`, `escalateAfterStrikes`, `strikeWindow` |
+| `auto-denylist.json` | `enabled`, `duration`, `maxEntries` |
+| `crowdsec.json` | `lapiUrl`, `machineId`, `password`, `origin`, `manualBanDuration`, `flushInterval`, `maxQueue` |
+| `firewall.json` | Admin-curated entries |
+
+A shard fronted by an upstream proxy can disable all of it and register nothing.
+
+## Key files
+
+| File | Role |
+|---|---|
+| `Projects/Server/Network/IConnectionFilter.cs` | Accept-path gate contract |
+| `Projects/Server/Network/ConnectionFilters.cs` | Filter registry + lifecycle |
+| `Projects/Server/Network/ForeignProtocol.cs` | Positive identification of non-UO traffic |
+| `Projects/Server/Network/Bans/BanChannel.cs` | Contribution fan-out + `IsExempt` seam |
+| `Projects/Server/Network/Bans/BanReasons.cs` | Reason slugs + the behavioural opt-in set |
+| `Projects/UOContent/Network/BanExemptions.cs` | Combines both allowlists into one answer |
+| `Projects/UOContent/Network/Blocklist/BlocklistFilter.cs` | File-sourced blocklist filter |
+| `Projects/UOContent/Network/Blocklist/FileAllowlist.cs` | Operator carve-outs, read from the allowlist files |
+| `Projects/UOContent/Network/LoginAllowlist/LoginAllowlist.cs` | Allowlist earned by authenticating |
+| `Projects/UOContent/Network/AutoDenylist/AutoDenylist.cs` | Short-lived local hold |
+| `Projects/UOContent/Network/CrowdSec/CrowdSecReporter.cs` | LAPI contribution sink |
+| `Projects/UOContent/Network/Firewall/Firewall.cs` | Admin-curated firewall set |
+| `tools/Export-IpBlocklist.ps1` | Blocklist generator + allowlist subtraction |