Reshapes IP banning around one idea: **core owns the question, content owns every answer.**
Core gains a single accept-path seam — `IConnectionFilter` — and loses everything that used to implement one. The firewall moves to UOContent, a new file-backed blocklist joins it there, and CrowdSec is repositioned from an in-app enforcer to a contribute-first reporter.
## The seam
```csharp
public interface IConnectionFilter
{
string Name { get; }
void Configure();
void Start(CancellationToken token);
void Stop();
bool ShouldDeny(IPAddress address);
}
```
The accept path went from hardcoded branches to one question:
```csharp
else if (ConnectionFilters.ShouldDeny(remoteIP, out var deniedBy))
{
logger.Debug("{Address} denied by connection filter '{Filter}'", remoteIP, deniedBy);
}
```
Filters register during the Configure sweep. The registry is a plain array walked by an indexed loop — no enumerator, no closure, no allocation — and the first denial short-circuits. An interface dispatch is noise next to the `accept()` syscall, so pluggability costs nothing measurable on the path that has to survive a DDoS.
Whatever a hit implies — persisting, promoting to an OS bouncer, contributing to the ban channel — is the filter's business, not the accept path's.
A filter that throws is **unregistered and the connection fails open**. A filter that faults once faults for every subsequent connection, so leaving it registered means an exception and a log line per accept — exactly the amplification an attacker wants — and a broken filter must not be able to deny everyone either.
This deliberately does **not** reuse `EventSink.InvokeSocketConnect`: that fires later and allocates a `SocketConnectEventArgs` per connection, which is what the accept path avoids for rejected traffic.
## What ships behind it
**`firewall`** (UOContent) — the existing admin-curated set. Collapsed from `Firewall` + `AdminFirewall` + a threaded enforcer into one single-threaded store with **zero concurrency primitives**: the accept path, admin gump, TTL expiry and boot load all run on the game loop. Persists to `Configuration/firewall.json` with automatic migration from the legacy `firewall.cfg`. No behavior change for operators — same namespace, same gump, same commands.
**`blocklist`** (UOContent) — new. Holds a millions-strong list in-app and **demand-pages** hits up to CrowdSec, which promotes them to the OS firewall.
The motivation is concrete: CrowdSec's Windows bouncer cannot load the ~3.9M IPs that 91 community feeds produce, but it handles ~100k fine. So the millions live in-process behind a binary search, and only addresses that *actually connect* get promoted. A `PromotedGuard` suppresses re-reporting an address until the bouncer picks it up.
The list is parsed straight from UTF-8 file bytes with no per-line string allocation, off the game loop, and published as an immutable snapshot swapped through a single `volatile` reference. Reloads yield to world saves.
**`tools/Export-IpBlocklist.ps1`** — the producer. Requires PowerShell 7 and runs on Windows, Linux and macOS; Windows PowerShell 5.1 is refused up front via `#requires`. Merges a thin, non-overlapping feed set into one de-duplicated, bogon-filtered file. Parsing runs in a compiled `Add-Type` hot loop (~1s for ~4M lines instead of minutes). Written to a `.tmp` sibling and swapped with `File.Replace`, so the shard never reads a half-written list, and a total feed outage refuses to overwrite a good list with an empty one. Re-running is idempotent — it exits without downloading anything while the list on disk is younger than `-MinInterval` (default 2h, the anchor feed's own refresh period), so a misconfigured scheduler can't hammer upstream.
## CrowdSec: contribute-first
`IBanReporter` + `BanChannel` fan locally-decided bans out to external systems. `CrowdSecReporter` (UOContent) posts to LAPI `POST /v1/alerts` and retracts via `DELETE /v1/decisions`.
Reporting is **enqueue-only** on the accept path: a bounded, coalescing channel drained off-loop with bounded retry, counted drops on overflow, and a flush on shutdown. Under a DDoS the accept path never does synchronous or lock-contending per-IP work.
### Why not pull decisions from CrowdSec?
The original design streamed decisions into an in-app snapshot and enforced them at the accept gate. That's the wrong layer: by the time the shard sees the connection, the TCP handshake and socket setup are already paid for. `cs-firewall-bouncer` drops the same traffic **at the kernel**, and it's what CrowdSec is built to do. So the shard now contributes what it uniquely knows (rate-limit trips, blocklist hits from real connection attempts) and lets the OS enforce.
The one thing the OS can't do — hold millions of entries on Windows — is exactly what the in-app blocklist covers, and it feeds the same pipeline.
## Threading policy
`CLAUDE.md` rule #3 is rewritten as an explicit three-part policy, with rule #10 restated in tandem:
- Anything touching game state runs **only** on the main loop.
- Heavy work that *needs* game state must be **chunked** across ticks, never threaded.
- Heavy work that does *not* need game state (large-file parse, external I/O) **must** run off-loop **and must yield to world saves**.
Results come back via an immutable snapshot swapped through a single `volatile` reference, or `Core.LoopContext.Post` — never by letting the scheduler decide where heavy work runs. Both new subsystems follow it.
## Shared primitives
`SortedRangeIndex<T> where T : IBinaryInteger<T>` — coalesced disjoint interval arrays plus a binary search. The firewall, the blocklist, and (as of this PR) core's reserved-network tables all use it.
Coalescing is a correctness requirement, not an optimization: multi-feed lists nest CIDRs (`/24` containing a `/32`), and a search that inspects only the rightmost run whose minimum is ≤ the value is sound **only** over disjoint runs. That bug was caught in review and is covered by regression tests.
`IPAddressUtility` collects the allocation-free `IPAddress` ↔ `UInt128` conversions and CIDR parsing that were previously scattered or duplicated.
## Config
| File | Owner | Keys |
|---|---|---|
| `Configuration/bans.json` | core | `reportRateLimitTrips`, `autoBanDuration` |
| `Configuration/blocklist.json` | content | `file`, `reloadInterval`, `reportHits`, `banDuration`, `promoteSuppression` |
| `Configuration/crowdsec.json` | content | `lapiUrl`, `machineId`, `password`, `origin`, `manualBanDuration`, `flushInterval`, `maxQueue` |
| `Configuration/firewall.json` | content | persisted firewall entries (migrated from `firewall.cfg`) |
Everything is inert by default. CrowdSec self-disables without credentials; the blocklist self-disables until its file exists. A shard that changes nothing sees no behavior change.
## Notes for review
- **Core no longer references `Firewall` or `IFirewallEntry` anywhere.** `NetworkUtilities` used to build its reserved-network tables out of `CidrFirewallEntry`, which coupled core to the firewall for something unrelated to banning; those are now a `SortedRangeIndex<UInt128>`, same semantics and public API.
- **`BanChannel.Stop()` no longer persists the firewall** — a contribution coordinator has no business saving an enforcement store. That's the firewall filter's `Stop()`.
- **A dead `whitelisted` parameter was dropped** from the blocklist gate: it was hardcoded `false` at its only call site, and no whitelist concept exists in core.
- **The blocklist filter is an instance, not a static.** The static version forced its tests onto the sequential collection with a reset hook; they now run in parallel.
- `dev-docs/networking-packets.md` documents the seam for content authors, plus a known wart in the `IPAddress` ↔ `UInt128` normalization flagged for a follow-up PR.
- The generator was verified on Linux, macOS and Windows under a temporary CI matrix (since removed). It caught two portability bugs — a Windows-only path separator, and a culture-sensitive duration parse that read `2.5` as `25` on comma-decimal locales and *silently* turned a 2.5h cooldown into 25h — plus a third that made the script unparseable on Windows PowerShell 5.1. The source is ASCII-only for that last reason: `#requires` is only honored once a file parses, so non-ASCII in a BOM-less script produces parse errors instead of the version message.
## Tests
**1344 pass** (782 `Server.Tests`, 562 `UOContent.Tests`). New coverage: filter registry (registration, short-circuit, fault-disable), blocklist parsing/CIDR/coalescing, snapshot reload markers, promote-guard TTL, ban-channel fan-out, CrowdSec alert building/dedup/flush-on-stop, and the generator's output-format contract pinned against the reader.
162 lines
6.8 KiB
C#
162 lines
6.8 KiB
C#
using System;
|
|
using System.IO;
|
|
using System.Reflection;
|
|
using System.Threading;
|
|
using Server.Items;
|
|
using Server.Misc;
|
|
using Server.Movement;
|
|
using Server.PathAlgorithms;
|
|
using Server.Tests.Maps;
|
|
|
|
namespace Server.Tests;
|
|
|
|
/// <summary>
|
|
/// Single, process-wide ModernUO bootstrap for the UOContent test host. Mirrors Server.Tests'
|
|
/// TestServerInitializer in name and shape; kept as a separate (non-shared) copy because this
|
|
/// one loads the UOContent assembly and configures the UOContent-specific systems. Both types
|
|
/// are <c>internal</c> so the shared name stays scoped to each assembly.
|
|
///
|
|
/// ModernUO bootstraps its global singletons (Core, ServerConfiguration, AssemblyHandler,
|
|
/// NetState/io-ring, World, Timer, the serialization workers, and TileData) exactly once per
|
|
/// process. <see cref="World.Load"/> is guarded to run once, and
|
|
/// <see cref="World.ExitSerializationThreads"/> must run once against the live workers. Each
|
|
/// xUnit collection gets its own fixture instance, so this guard makes the bootstrap run a
|
|
/// single time regardless of how many collection fixtures are constructed. The two stateful
|
|
/// collections use <c>[CollectionDefinition(DisableParallelization = true)]</c> so they never
|
|
/// overlap; pure tests still run in parallel.
|
|
/// </summary>
|
|
internal static class TestServerInitializer
|
|
{
|
|
private static bool _initialized;
|
|
private static readonly Lock _lock = new();
|
|
|
|
/// <summary>
|
|
/// True if the UO client tile data was found and loaded. When false (e.g. CI, where the
|
|
/// copyrighted client files are absent), tile/map/multi-dependent tests must skip rather than
|
|
/// fail. Guard such tests with <c>Skip.If(!TestServerInitializer.TileDataLoaded, ...)</c>.
|
|
/// </summary>
|
|
public static bool TileDataLoaded { get; private set; }
|
|
|
|
public static void Initialize()
|
|
{
|
|
lock (_lock)
|
|
{
|
|
if (_initialized)
|
|
{
|
|
return;
|
|
}
|
|
|
|
Core.ApplicationAssembly = Assembly.GetExecutingAssembly();
|
|
Core.LoopContext = new EventLoopContext();
|
|
Core.Expansion = Expansion.EJ;
|
|
|
|
ServerConfiguration.Load(true);
|
|
ServerConfiguration.AssemblyDirectories.Add(Core.BaseDirectory);
|
|
|
|
// Required for the pathfinding tests (real .mul tile data). Harmless for the rest.
|
|
var clientFiles = Environment.GetEnvironmentVariable("MODERNUO_TEST_DATA_DIR")
|
|
?? @"C:\Ultima Online Classic";
|
|
ServerConfiguration.DataDirectories.Add(clientFiles);
|
|
|
|
AssemblyHandler.LoadAssemblies(["Server.dll", "UOContent.dll"]);
|
|
|
|
SkillsInfo.Configure();
|
|
|
|
// Seed the loop clock as Main.cs does before the Configure sweep; otherwise Core.Now is
|
|
// DateTime.MinValue for the whole test host.
|
|
Core._now = DateTime.UtcNow;
|
|
|
|
// Timer wheel must exist before NetState.Configure(), which schedules a recurring
|
|
// sweep via Timer.DelayCall (matches production ordering in Main.cs: Timer.Init runs
|
|
// before AssemblyHandler.Invoke("Configure")).
|
|
Timer.Init(0);
|
|
Server.Network.NetState.Configure();
|
|
TestMapDefinitions.ConfigureTestMapDefinitions();
|
|
|
|
// TileData's static cctor short-circuits when running under xUnit
|
|
// (see Server/TileData.cs:295). Force-load via reflection so LandTable/ItemTable
|
|
// flags are populated before anything that reads TileData (MultiData, MovementImpl,
|
|
// CheckMovement). Without this, TileData.MaxItemValue is 0 at MultiData.Configure()
|
|
// time, causing every MCL tile ID to be masked to 0 and stored as ID=0 in Tiles[x][y].
|
|
// The copyrighted client files are absent on CI; when tiledata.mul is missing we skip
|
|
// the tile/map/multi-dependent bootstrap and leave TileDataLoaded false so those tests
|
|
// skip instead of failing the whole collection from the fixture constructor.
|
|
TileDataLoaded = TryForceLoadTileData();
|
|
|
|
// Production runs every static Configure() via AssemblyHandler.Invoke("Configure");
|
|
// the fixture calls a curated subset, so configure the pathfinding singleton here so
|
|
// BitmapAStarAlgorithm.Instance carries its configured MaxSearchNodes before any test
|
|
// calls Find. ServerConfiguration is already loaded above, so the setting resolves.
|
|
BitmapAStarAlgorithm.Configure();
|
|
|
|
if (TileDataLoaded)
|
|
{
|
|
// Multi component lists (multi.mul / MultiCollection.uop). Production invokes this via
|
|
// AssemblyHandler.Invoke("Configure"); the curated fixture subset must call it so that
|
|
// BaseMulti.Components (MultiData.GetComponents) returns real footprints instead of
|
|
// MultiComponentList.Empty. Required by the Multi pathfinding tests. Depends on the
|
|
// client files, so it only runs when tile data loaded.
|
|
MultiData.Configure();
|
|
}
|
|
|
|
World.Configure();
|
|
RaceDefinitions.Configure();
|
|
MovementImpl.Configure();
|
|
PathFollower.Configure();
|
|
World.Load();
|
|
World.ExitSerializationThreads();
|
|
DecayScheduler.Configure();
|
|
Server.Engines.Spawners.SpawnerJsonSerializer.Configure();
|
|
|
|
if (TileDataLoaded)
|
|
{
|
|
VerifyTrammelTileDataLoaded();
|
|
}
|
|
|
|
_initialized = true;
|
|
}
|
|
}
|
|
|
|
private static bool TryForceLoadTileData()
|
|
{
|
|
var tileDataPath = Core.FindDataFile("tiledata.mul", false);
|
|
if (string.IsNullOrEmpty(tileDataPath) || !File.Exists(tileDataPath))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
var loadMethod = typeof(TileData).GetMethod(
|
|
"Load",
|
|
BindingFlags.Static | BindingFlags.NonPublic
|
|
);
|
|
if (loadMethod == null)
|
|
{
|
|
throw new InvalidOperationException(
|
|
"TileData.Load not found via reflection — engine may have refactored."
|
|
);
|
|
}
|
|
loadMethod.Invoke(null, null);
|
|
return true;
|
|
}
|
|
|
|
private static void VerifyTrammelTileDataLoaded()
|
|
{
|
|
var trammel = Map.Maps[1];
|
|
if (trammel == null)
|
|
{
|
|
throw new InvalidOperationException(
|
|
"Trammel (mapId=1) was not registered. Check TestMapDefinitions."
|
|
);
|
|
}
|
|
|
|
var tile = trammel.Tiles.GetLandTile(1500, 1600);
|
|
if (tile.ID == 0)
|
|
{
|
|
throw new InvalidOperationException(
|
|
$"Trammel tile data did not load — GetLandTile(1500,1600) returned ID 0. " +
|
|
$"Verify Distribution/Data/map1*.mul (or map1LegacyMUL.uop) is present at " +
|
|
$"{Path.Combine(Core.BaseDirectory, "Data")}."
|
|
);
|
|
}
|
|
}
|
|
}
|