feat(network): pluggable connection filters; file blocklist + contribute-first CrowdSec (#2542)
Reshapes IP banning around one idea: **core owns the question, content owns every answer.**
Core gains a single accept-path seam — `IConnectionFilter` — and loses everything that used to implement one. The firewall moves to UOContent, a new file-backed blocklist joins it there, and CrowdSec is repositioned from an in-app enforcer to a contribute-first reporter.
## The seam
```csharp
public interface IConnectionFilter
{
string Name { get; }
void Configure();
void Start(CancellationToken token);
void Stop();
bool ShouldDeny(IPAddress address);
}
```
The accept path went from hardcoded branches to one question:
```csharp
else if (ConnectionFilters.ShouldDeny(remoteIP, out var deniedBy))
{
logger.Debug("{Address} denied by connection filter '{Filter}'", remoteIP, deniedBy);
}
```
Filters register during the Configure sweep. The registry is a plain array walked by an indexed loop — no enumerator, no closure, no allocation — and the first denial short-circuits. An interface dispatch is noise next to the `accept()` syscall, so pluggability costs nothing measurable on the path that has to survive a DDoS.
Whatever a hit implies — persisting, promoting to an OS bouncer, contributing to the ban channel — is the filter's business, not the accept path's.
A filter that throws is **unregistered and the connection fails open**. A filter that faults once faults for every subsequent connection, so leaving it registered means an exception and a log line per accept — exactly the amplification an attacker wants — and a broken filter must not be able to deny everyone either.
This deliberately does **not** reuse `EventSink.InvokeSocketConnect`: that fires later and allocates a `SocketConnectEventArgs` per connection, which is what the accept path avoids for rejected traffic.
## What ships behind it
**`firewall`** (UOContent) — the existing admin-curated set. Collapsed from `Firewall` + `AdminFirewall` + a threaded enforcer into one single-threaded store with **zero concurrency primitives**: the accept path, admin gump, TTL expiry and boot load all run on the game loop. Persists to `Configuration/firewall.json` with automatic migration from the legacy `firewall.cfg`. No behavior change for operators — same namespace, same gump, same commands.
**`blocklist`** (UOContent) — new. Holds a millions-strong list in-app and **demand-pages** hits up to CrowdSec, which promotes them to the OS firewall.
The motivation is concrete: CrowdSec's Windows bouncer cannot load the ~3.9M IPs that 91 community feeds produce, but it handles ~100k fine. So the millions live in-process behind a binary search, and only addresses that *actually connect* get promoted. A `PromotedGuard` suppresses re-reporting an address until the bouncer picks it up.
The list is parsed straight from UTF-8 file bytes with no per-line string allocation, off the game loop, and published as an immutable snapshot swapped through a single `volatile` reference. Reloads yield to world saves.
**`tools/Export-IpBlocklist.ps1`** — the producer. Requires PowerShell 7 and runs on Windows, Linux and macOS; Windows PowerShell 5.1 is refused up front via `#requires`. Merges a thin, non-overlapping feed set into one de-duplicated, bogon-filtered file. Parsing runs in a compiled `Add-Type` hot loop (~1s for ~4M lines instead of minutes). Written to a `.tmp` sibling and swapped with `File.Replace`, so the shard never reads a half-written list, and a total feed outage refuses to overwrite a good list with an empty one. Re-running is idempotent — it exits without downloading anything while the list on disk is younger than `-MinInterval` (default 2h, the anchor feed's own refresh period), so a misconfigured scheduler can't hammer upstream.
## CrowdSec: contribute-first
`IBanReporter` + `BanChannel` fan locally-decided bans out to external systems. `CrowdSecReporter` (UOContent) posts to LAPI `POST /v1/alerts` and retracts via `DELETE /v1/decisions`.
Reporting is **enqueue-only** on the accept path: a bounded, coalescing channel drained off-loop with bounded retry, counted drops on overflow, and a flush on shutdown. Under a DDoS the accept path never does synchronous or lock-contending per-IP work.
### Why not pull decisions from CrowdSec?
The original design streamed decisions into an in-app snapshot and enforced them at the accept gate. That's the wrong layer: by the time the shard sees the connection, the TCP handshake and socket setup are already paid for. `cs-firewall-bouncer` drops the same traffic **at the kernel**, and it's what CrowdSec is built to do. So the shard now contributes what it uniquely knows (rate-limit trips, blocklist hits from real connection attempts) and lets the OS enforce.
The one thing the OS can't do — hold millions of entries on Windows — is exactly what the in-app blocklist covers, and it feeds the same pipeline.
## Threading policy
`CLAUDE.md` rule #3 is rewritten as an explicit three-part policy, with rule #10 restated in tandem:
- Anything touching game state runs **only** on the main loop.
- Heavy work that *needs* game state must be **chunked** across ticks, never threaded.
- Heavy work that does *not* need game state (large-file parse, external I/O) **must** run off-loop **and must yield to world saves**.
Results come back via an immutable snapshot swapped through a single `volatile` reference, or `Core.LoopContext.Post` — never by letting the scheduler decide where heavy work runs. Both new subsystems follow it.
## Shared primitives
`SortedRangeIndex<T> where T : IBinaryInteger<T>` — coalesced disjoint interval arrays plus a binary search. The firewall, the blocklist, and (as of this PR) core's reserved-network tables all use it.
Coalescing is a correctness requirement, not an optimization: multi-feed lists nest CIDRs (`/24` containing a `/32`), and a search that inspects only the rightmost run whose minimum is ≤ the value is sound **only** over disjoint runs. That bug was caught in review and is covered by regression tests.
`IPAddressUtility` collects the allocation-free `IPAddress` ↔ `UInt128` conversions and CIDR parsing that were previously scattered or duplicated.
## Config
| File | Owner | Keys |
|---|---|---|
| `Configuration/bans.json` | core | `reportRateLimitTrips`, `autoBanDuration` |
| `Configuration/blocklist.json` | content | `file`, `reloadInterval`, `reportHits`, `banDuration`, `promoteSuppression` |
| `Configuration/crowdsec.json` | content | `lapiUrl`, `machineId`, `password`, `origin`, `manualBanDuration`, `flushInterval`, `maxQueue` |
| `Configuration/firewall.json` | content | persisted firewall entries (migrated from `firewall.cfg`) |
Everything is inert by default. CrowdSec self-disables without credentials; the blocklist self-disables until its file exists. A shard that changes nothing sees no behavior change.
## Notes for review
- **Core no longer references `Firewall` or `IFirewallEntry` anywhere.** `NetworkUtilities` used to build its reserved-network tables out of `CidrFirewallEntry`, which coupled core to the firewall for something unrelated to banning; those are now a `SortedRangeIndex<UInt128>`, same semantics and public API.
- **`BanChannel.Stop()` no longer persists the firewall** — a contribution coordinator has no business saving an enforcement store. That's the firewall filter's `Stop()`.
- **A dead `whitelisted` parameter was dropped** from the blocklist gate: it was hardcoded `false` at its only call site, and no whitelist concept exists in core.
- **The blocklist filter is an instance, not a static.** The static version forced its tests onto the sequential collection with a reset hook; they now run in parallel.
- `dev-docs/networking-packets.md` documents the seam for content authors, plus a known wart in the `IPAddress` ↔ `UInt128` normalization flagged for a follow-up PR.
- The generator was verified on Linux, macOS and Windows under a temporary CI matrix (since removed). It caught two portability bugs — a Windows-only path separator, and a culture-sensitive duration parse that read `2.5` as `25` on comma-decimal locales and *silently* turned a 2.5h cooldown into 25h — plus a third that made the script unparseable on Windows PowerShell 5.1. The source is ASCII-only for that last reason: `#requires` is only honored once a file parses, so non-ASCII in a BOM-less script produces parse errors instead of the version message.
## Tests
**1344 pass** (782 `Server.Tests`, 562 `UOContent.Tests`). New coverage: filter registry (registration, short-circuit, fault-disable), blocklist parsing/CIDR/coalescing, snapshot reload markers, promote-guard TTL, ban-channel fan-out, CrowdSec alert building/dedup/flush-on-stop, and the generator's output-format contract pinned against the reader.
This commit is contained in:
parent
bec4cfa910
commit
c39454137e
52 changed files with 4655 additions and 547 deletions
12
.gitignore
vendored
12
.gitignore
vendored
|
|
@ -9,7 +9,12 @@
|
||||||
/Distribution/bsdtar
|
/Distribution/bsdtar
|
||||||
/Distribution/Configuration/antimacro.json
|
/Distribution/Configuration/antimacro.json
|
||||||
/Distribution/Configuration/assistants.json
|
/Distribution/Configuration/assistants.json
|
||||||
|
/Distribution/Configuration/bans.json
|
||||||
|
/Distribution/Configuration/blocklist.json
|
||||||
|
/Distribution/Configuration/crowdsec.json
|
||||||
/Distribution/Configuration/expansion.json
|
/Distribution/Configuration/expansion.json
|
||||||
|
/Distribution/Configuration/ip-blocklist.txt
|
||||||
|
/Distribution/Configuration/ip-blocklist.txt.tmp
|
||||||
/Distribution/Configuration/modernuo.json
|
/Distribution/Configuration/modernuo.json
|
||||||
/Distribution/Configuration/email-settings.json
|
/Distribution/Configuration/email-settings.json
|
||||||
/Distribution/Configuration/throttles.json
|
/Distribution/Configuration/throttles.json
|
||||||
|
|
@ -19,6 +24,7 @@
|
||||||
/Distribution/Backups
|
/Distribution/Backups
|
||||||
/Distribution/Saves
|
/Distribution/Saves
|
||||||
/Distribution/docs
|
/Distribution/docs
|
||||||
|
/docs/
|
||||||
/Distribution/temp
|
/Distribution/temp
|
||||||
/Distribution/*.dylib
|
/Distribution/*.dylib
|
||||||
/Distribution/*.so
|
/Distribution/*.so
|
||||||
|
|
@ -46,5 +52,7 @@
|
||||||
/packages/*
|
/packages/*
|
||||||
/Distribution/Configuration/server-access.json
|
/Distribution/Configuration/server-access.json
|
||||||
|
|
||||||
# BuildTool native binaries (downloaded from GitHub Releases)
|
# BuildTool native binaries (downloaded from GitHub Releases).
|
||||||
/tools/
|
# Ignore everything under tools/ except the operator scripts checked in below.
|
||||||
|
/tools/*
|
||||||
|
!/tools/Export-IpBlocklist.ps1
|
||||||
|
|
|
||||||
|
|
@ -12,14 +12,14 @@ Apply these when writing or reviewing `.cs` files under `Projects/`.
|
||||||
|
|
||||||
1. **LINQ** — Tier 1 (zero-cost patterns) free on hot paths; Tier 2 (low overhead) OK on warm paths; Tier 3 (allocating) forbidden on hot paths → `dev-docs/code-standards.md`
|
1. **LINQ** — Tier 1 (zero-cost patterns) free on hot paths; Tier 2 (low overhead) OK on warm paths; Tier 3 (allocating) forbidden on hot paths → `dev-docs/code-standards.md`
|
||||||
2. **No `Console.WriteLine`** — use `LogFactory.GetLogger(typeof(MyClass))` → `logger.Information(...)` (requires `using Server.Logging;`)
|
2. **No `Console.WriteLine`** — use `LogFactory.GetLogger(typeof(MyClass))` → `logger.Information(...)` (requires `using Server.Logging;`)
|
||||||
3. **No concurrency primitives** — no `lock`, `volatile`, `ConcurrentDictionary`, `Mutex`, etc. Server is single-threaded.
|
3. **Threading policy** — game logic runs only on the main loop; **never** touch game state (`World`, mobiles, items, maps, timers) from a background thread. Heavy work that *needs* game state must be **chunked** across ticks, not threaded. Heavy work that does *not* need game state (large-file parse, external I/O) **must** run on a background thread **and must yield to world saves** (defer while `World.Saving`/`WorldState.PendingSave`). Publish results back to the loop as an immutable snapshot swapped via a single `volatile` reference — the only sanctioned `volatile`. No `lock`/`Mutex`/`ConcurrentDictionary` in game logic. Rule #10 covers how background work hands results back to the loop → `dev-docs/threading-model.md`
|
||||||
4. **No `World.Mobiles`/`World.Items` iteration** — use spatial queries: `map.GetMobilesInRange<T>()`, `map.GetItemsInRange<T>()`
|
4. **No `World.Mobiles`/`World.Items` iteration** — use spatial queries: `map.GetMobilesInRange<T>()`, `map.GetItemsInRange<T>()`
|
||||||
5. **Clean up refs in `OnDelete()`/`OnAfterDelete()`** — null out `Item`/`Mobile` references
|
5. **Clean up refs in `OnDelete()`/`OnAfterDelete()`** — null out `Item`/`Mobile` references
|
||||||
6. **Cancel timers in `OnDelete()`/`OnAfterDelete()`** — call `_token.Cancel()` or `_timer?.Stop()`
|
6. **Cancel timers in `OnDelete()`/`OnAfterDelete()`** — call `_token.Cancel()` or `_timer?.Stop()`
|
||||||
7. **`STArrayPool<T>.Shared`** not `ArrayPool<T>.Shared` — single-threaded optimized, no locks
|
7. **`STArrayPool<T>.Shared`** not `ArrayPool<T>.Shared` — single-threaded optimized, no locks
|
||||||
8. **`PooledRefList<T>`** not `new List<T>()` on hot paths — zero GC pressure, stack-allocated ref struct
|
8. **`PooledRefList<T>`** not `new List<T>()` on hot paths — zero GC pressure, stack-allocated ref struct
|
||||||
9. **Serialization** — class must be `partial`, constructor needs `[Constructible]`, `TimerExecutionToken` must NOT have `[SerializableField]`. New classes: use `[SerializationGenerator(version)]` (omit `encoded`). When bumping versions, add `MigrateFrom(VXContent)` (X = previous version). Never modify `Deserialize(reader, version)` for version bumps — that method is only for pre-codegen legacy saves. When migrating from pre-codegen Serialize/Deserialize: pass `false` if old code used `reader.ReadInt()`, bump version +1, and keep old logic as `private void Deserialize(IGenericReader reader, int version)` → `dev-docs/runuo-migration-docs/02-serialization.md`
|
9. **Serialization** — class must be `partial`, constructor needs `[Constructible]`, `TimerExecutionToken` must NOT have `[SerializableField]`. New classes: use `[SerializationGenerator(version)]` (omit `encoded`). When bumping versions, add `MigrateFrom(VXContent)` (X = previous version). Never modify `Deserialize(reader, version)` for version bumps — that method is only for pre-codegen legacy saves. When migrating from pre-codegen Serialize/Deserialize: pass `false` if old code used `reader.ReadInt()`, bump version +1, and keep old logic as `private void Deserialize(IGenericReader reader, int version)` → `dev-docs/runuo-migration-docs/02-serialization.md`
|
||||||
10. **No `Task.Run`/`new Thread()`** in game code — game logic is single-threaded event loop
|
10. **No `Task.Run`/`new Thread()` for game logic** (tandem with rule #3) — game logic is the single-threaded event loop. Backgrounding is allowed only for work that does not itself touch game state (external service calls, large-file parse). When such work must *feed* game logic: run the heavy/I/O part off-loop and `ConfigureAwait(false)` its awaits so a continuation never resumes on the loop and silently foregrounds heavy work; then hand the result back **explicitly** — publish an immutable snapshot swapped via a `volatile` reference (the loop reads it lock-free), or marshal the apply step with `Core.LoopContext.Post(() => …)`. Never touch game state off-thread; never let the scheduler decide where the heavy work runs → `dev-docs/threading-model.md`
|
||||||
11. **Never assume era** — if code uses `Core.AOS`/`Core.SE`/etc., ask which expansion to target
|
11. **Never assume era** — if code uses `Core.AOS`/`Core.SE`/etc., ask which expansion to target
|
||||||
12. **Naming** — `_camelCase` private fields, `PascalCase` properties/methods/classes; don't flag legacy `m_` but use `_` for new code
|
12. **Naming** — `_camelCase` private fields, `PascalCase` properties/methods/classes; don't flag legacy `m_` but use `_` for new code
|
||||||
13. **No empty gumps** — every gump must produce visual elements. An empty gump leaks on client+server (no way to close it). Use static `DisplayTo()` to validate before constructing → `dev-docs/gump-system.md`
|
13. **No empty gumps** — every gump must produce visual elements. An empty gump leaks on client+server (no way to close it). Use static `DisplayTo()` to validate before constructing → `dev-docs/gump-system.md`
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,4 @@
|
||||||
|
using System;
|
||||||
using System.IO;
|
using System.IO;
|
||||||
using System.Reflection;
|
using System.Reflection;
|
||||||
using System.Threading;
|
using System.Threading;
|
||||||
|
|
@ -78,6 +79,15 @@ internal static class TestServerInitializer
|
||||||
Core.LoopContext = new EventLoopContext();
|
Core.LoopContext = new EventLoopContext();
|
||||||
Core.Expansion = Expansion.EJ;
|
Core.Expansion = Expansion.EJ;
|
||||||
|
|
||||||
|
// Seed the loop clock as Main.cs does before the Configure sweep; otherwise Core.Now is
|
||||||
|
// DateTime.MinValue for the whole test host.
|
||||||
|
Core._now = DateTime.UtcNow;
|
||||||
|
|
||||||
|
// Timer wheel must exist before NetState.Configure(), which schedules a recurring
|
||||||
|
// sweep via Timer.DelayCall (matches production ordering in Main.cs: Timer.Init runs
|
||||||
|
// before AssemblyHandler.Invoke("Configure")).
|
||||||
|
Timer.Init(0);
|
||||||
|
|
||||||
// Configure networking (initializes RingSocketManager for tests)
|
// Configure networking (initializes RingSocketManager for tests)
|
||||||
Server.Network.NetState.Configure();
|
Server.Network.NetState.Configure();
|
||||||
|
|
||||||
|
|
@ -87,8 +97,6 @@ internal static class TestServerInitializer
|
||||||
// Configure the world
|
// Configure the world
|
||||||
World.Configure();
|
World.Configure();
|
||||||
|
|
||||||
Timer.Init(0);
|
|
||||||
|
|
||||||
// Load the world
|
// Load the world
|
||||||
World.Load();
|
World.Load();
|
||||||
|
|
||||||
|
|
|
||||||
73
Projects/Server.Tests/Tests/Network/Bans/BanChannelTests.cs
Normal file
73
Projects/Server.Tests/Tests/Network/Bans/BanChannelTests.cs
Normal file
|
|
@ -0,0 +1,73 @@
|
||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.Net;
|
||||||
|
using System.Threading;
|
||||||
|
using Server.Network.Bans;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests.Network.Bans;
|
||||||
|
|
||||||
|
public class BanChannelTests
|
||||||
|
{
|
||||||
|
private sealed class FakeReporter : IBanReporter
|
||||||
|
{
|
||||||
|
public readonly List<(IPAddress ip, TimeSpan ttl, string reason)> Reports = [];
|
||||||
|
public readonly List<IPAddress> Retractions = [];
|
||||||
|
public bool ThrowOnReport;
|
||||||
|
|
||||||
|
public string Name => "fake";
|
||||||
|
public bool CanRetract => true;
|
||||||
|
public void Register() { }
|
||||||
|
public void Start(CancellationToken token) { }
|
||||||
|
public void Stop() { }
|
||||||
|
|
||||||
|
public void Report(IPAddress address, TimeSpan ttl, string reason)
|
||||||
|
{
|
||||||
|
if (ThrowOnReport)
|
||||||
|
{
|
||||||
|
throw new InvalidOperationException("boom");
|
||||||
|
}
|
||||||
|
|
||||||
|
Reports.Add((address, ttl, reason));
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Retract(IPAddress address) => Retractions.Add(address);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Report_FansOutToAllReporters()
|
||||||
|
{
|
||||||
|
var a = new FakeReporter();
|
||||||
|
var b = new FakeReporter();
|
||||||
|
BanChannel.ConfigureForTesting([a, b]);
|
||||||
|
|
||||||
|
BanChannel.Report(IPAddress.Parse("1.2.3.4"), TimeSpan.FromHours(1), "rate-limit");
|
||||||
|
|
||||||
|
Assert.Single(a.Reports);
|
||||||
|
Assert.Single(b.Reports);
|
||||||
|
Assert.Equal("rate-limit", a.Reports[0].reason);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Report_SwallowsReporterException()
|
||||||
|
{
|
||||||
|
var bad = new FakeReporter { ThrowOnReport = true };
|
||||||
|
var good = new FakeReporter();
|
||||||
|
BanChannel.ConfigureForTesting([bad, good]);
|
||||||
|
|
||||||
|
BanChannel.Report(IPAddress.Parse("1.2.3.4"), TimeSpan.FromHours(1), "manual");
|
||||||
|
|
||||||
|
Assert.Single(good.Reports); // the throwing reporter does not block the others
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Retract_ReachesRetractCapableReporters()
|
||||||
|
{
|
||||||
|
var a = new FakeReporter();
|
||||||
|
BanChannel.ConfigureForTesting([a]);
|
||||||
|
|
||||||
|
BanChannel.Retract(IPAddress.Parse("9.9.9.9"));
|
||||||
|
|
||||||
|
Assert.Single(a.Retractions);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,44 @@
|
||||||
|
using System;
|
||||||
|
using System.Text.Json;
|
||||||
|
using Server.Json;
|
||||||
|
using Server.Network.Bans;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests;
|
||||||
|
|
||||||
|
public class BanConfigurationTests
|
||||||
|
{
|
||||||
|
// Locks the JsonConfig casing/converter contract: JsonConfig's options are case-SENSITIVE, so
|
||||||
|
// every settings member must carry an explicit [JsonPropertyName("camelCase")] or it silently
|
||||||
|
// binds nothing. These tests round-trip through the exact options the loader uses.
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void BanSettings_RoundTripsThroughJsonConfig()
|
||||||
|
{
|
||||||
|
var original = new BanSettings
|
||||||
|
{
|
||||||
|
ReportRateLimitTrips = false,
|
||||||
|
AutoBanDuration = TimeSpan.FromHours(2)
|
||||||
|
};
|
||||||
|
|
||||||
|
var json = JsonConfig.Serialize(original);
|
||||||
|
|
||||||
|
Assert.Contains("\"reportRateLimitTrips\"", json);
|
||||||
|
Assert.Contains("\"autoBanDuration\"", json);
|
||||||
|
|
||||||
|
var restored = JsonSerializer.Deserialize<BanSettings>(json, JsonConfig.DefaultOptions);
|
||||||
|
|
||||||
|
Assert.NotNull(restored);
|
||||||
|
Assert.Equal(original.ReportRateLimitTrips, restored.ReportRateLimitTrips);
|
||||||
|
Assert.Equal(original.AutoBanDuration, restored.AutoBanDuration); // TimeSpan survives
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void BanSettings_Defaults_AreReportRateLimitTripsFourHourAutoBan()
|
||||||
|
{
|
||||||
|
var settings = new BanSettings();
|
||||||
|
|
||||||
|
Assert.True(settings.ReportRateLimitTrips);
|
||||||
|
Assert.Equal(TimeSpan.FromHours(4), settings.AutoBanDuration);
|
||||||
|
}
|
||||||
|
}
|
||||||
133
Projects/Server.Tests/Tests/Network/ConnectionFiltersTests.cs
Normal file
133
Projects/Server.Tests/Tests/Network/ConnectionFiltersTests.cs
Normal file
|
|
@ -0,0 +1,133 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: ConnectionFiltersTests.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Net;
|
||||||
|
using System.Threading;
|
||||||
|
using Server.Network;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests.Network;
|
||||||
|
|
||||||
|
[Collection("Sequential Server Tests")]
|
||||||
|
public class ConnectionFiltersTests : IDisposable
|
||||||
|
{
|
||||||
|
public ConnectionFiltersTests() => ConnectionFilters.ResetForTesting();
|
||||||
|
|
||||||
|
public void Dispose() => ConnectionFilters.ResetForTesting();
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void No_filters_denies_nothing()
|
||||||
|
{
|
||||||
|
Assert.False(ConnectionFilters.ShouldDeny(IPAddress.Parse("1.2.3.4"), out var deniedBy));
|
||||||
|
Assert.Null(deniedBy);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Register_is_idempotent_by_name()
|
||||||
|
{
|
||||||
|
ConnectionFilters.Register(new FakeFilter("dupe", deny: false));
|
||||||
|
ConnectionFilters.Register(new FakeFilter("dupe", deny: true));
|
||||||
|
|
||||||
|
// The second registration is ignored, so the deny:true instance never gets consulted.
|
||||||
|
Assert.Single(ConnectionFilters.Filters);
|
||||||
|
Assert.False(ConnectionFilters.ShouldDeny(IPAddress.Parse("1.2.3.4"), out _));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void First_denying_filter_short_circuits_and_is_named()
|
||||||
|
{
|
||||||
|
var first = new FakeFilter("allow-all", deny: false);
|
||||||
|
var second = new FakeFilter("deny-all", deny: true);
|
||||||
|
var third = new FakeFilter("never-reached", deny: true);
|
||||||
|
|
||||||
|
ConnectionFilters.Register(first);
|
||||||
|
ConnectionFilters.Register(second);
|
||||||
|
ConnectionFilters.Register(third);
|
||||||
|
|
||||||
|
Assert.True(ConnectionFilters.ShouldDeny(IPAddress.Parse("1.2.3.4"), out var deniedBy));
|
||||||
|
Assert.Equal("deny-all", deniedBy);
|
||||||
|
Assert.Equal(1, first.Calls);
|
||||||
|
Assert.Equal(1, second.Calls);
|
||||||
|
Assert.Equal(0, third.Calls); // short-circuited
|
||||||
|
}
|
||||||
|
|
||||||
|
// A filter that throws once throws for every subsequent connection, which would turn one bug into an
|
||||||
|
// exception per accept. It must be dropped, and the connection must fail open rather than be denied
|
||||||
|
// by a filter that never actually answered.
|
||||||
|
[Fact]
|
||||||
|
public void Throwing_filter_is_unregistered_and_fails_open()
|
||||||
|
{
|
||||||
|
var bad = new FakeFilter("bad", deny: true, throws: true);
|
||||||
|
var good = new FakeFilter("good", deny: false);
|
||||||
|
|
||||||
|
ConnectionFilters.Register(bad);
|
||||||
|
ConnectionFilters.Register(good);
|
||||||
|
|
||||||
|
Assert.False(ConnectionFilters.ShouldDeny(IPAddress.Parse("1.2.3.4"), out _));
|
||||||
|
Assert.Single(ConnectionFilters.Filters);
|
||||||
|
Assert.Equal("good", ConnectionFilters.Filters[0].Name);
|
||||||
|
|
||||||
|
// Remaining filters still run on the same pass the faulty one was dropped in.
|
||||||
|
Assert.Equal(1, good.Calls);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Register_configures_immediately()
|
||||||
|
{
|
||||||
|
var filter = new FakeFilter("cfg", deny: false);
|
||||||
|
ConnectionFilters.Register(filter);
|
||||||
|
|
||||||
|
Assert.True(filter.Configured);
|
||||||
|
}
|
||||||
|
|
||||||
|
private sealed class FakeFilter : IConnectionFilter
|
||||||
|
{
|
||||||
|
private readonly bool _deny;
|
||||||
|
private readonly bool _throws;
|
||||||
|
|
||||||
|
public FakeFilter(string name, bool deny, bool throws = false)
|
||||||
|
{
|
||||||
|
Name = name;
|
||||||
|
_deny = deny;
|
||||||
|
_throws = throws;
|
||||||
|
}
|
||||||
|
|
||||||
|
public string Name { get; }
|
||||||
|
public int Calls { get; private set; }
|
||||||
|
public bool Configured { get; private set; }
|
||||||
|
|
||||||
|
public void Register() => Configured = true;
|
||||||
|
|
||||||
|
public void Start(CancellationToken token)
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Stop()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
public bool ShouldDeny(IPAddress address)
|
||||||
|
{
|
||||||
|
Calls++;
|
||||||
|
if (_throws)
|
||||||
|
{
|
||||||
|
throw new InvalidOperationException("simulated filter bug");
|
||||||
|
}
|
||||||
|
|
||||||
|
return _deny;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -1,137 +0,0 @@
|
||||||
|
|
||||||
using System.Net;
|
|
||||||
using System.Threading.Tasks;
|
|
||||||
using Server.Network;
|
|
||||||
using Xunit;
|
|
||||||
|
|
||||||
namespace Server.Tests;
|
|
||||||
|
|
||||||
public class FirewallTests
|
|
||||||
{
|
|
||||||
[Fact]
|
|
||||||
public void Firewall_BlocksIPAddress_WhenAdded()
|
|
||||||
{
|
|
||||||
var ip = IPAddress.Parse("192.168.1.1");
|
|
||||||
var entry = new SingleIpFirewallEntry("192.168.1.1");
|
|
||||||
|
|
||||||
Assert.False(Firewall.IsBlocked(ip));
|
|
||||||
|
|
||||||
Firewall.Add(entry);
|
|
||||||
|
|
||||||
Assert.True(Firewall.IsBlocked(ip));
|
|
||||||
}
|
|
||||||
|
|
||||||
[Fact]
|
|
||||||
public void Firewall_DoesNotBlockIPAddress_WhenNotAdded()
|
|
||||||
{
|
|
||||||
var ip = IPAddress.Parse("192.168.1.2");
|
|
||||||
Assert.False(Firewall.IsBlocked(ip));
|
|
||||||
}
|
|
||||||
|
|
||||||
[Fact]
|
|
||||||
public void Firewall_StopsBlockingIPAddress_WhenRemoved()
|
|
||||||
{
|
|
||||||
var ip = IPAddress.Parse("192.168.1.3");
|
|
||||||
var entry = new SingleIpFirewallEntry("192.168.1.3");
|
|
||||||
|
|
||||||
Firewall.Add(entry);
|
|
||||||
Assert.True(Firewall.IsBlocked(ip));
|
|
||||||
|
|
||||||
Firewall.Remove(entry);
|
|
||||||
Assert.False(Firewall.IsBlocked(ip));
|
|
||||||
}
|
|
||||||
|
|
||||||
[Fact]
|
|
||||||
public void Firewall_BlocksIPRange()
|
|
||||||
{
|
|
||||||
var entry = new CidrFirewallEntry(IPAddress.Parse("10.0.0.1"), IPAddress.Parse("10.0.0.5"));
|
|
||||||
|
|
||||||
Firewall.Add(entry);
|
|
||||||
|
|
||||||
Assert.True(Firewall.IsBlocked(IPAddress.Parse("10.0.0.1")));
|
|
||||||
Assert.True(Firewall.IsBlocked(IPAddress.Parse("10.0.0.3")));
|
|
||||||
Assert.True(Firewall.IsBlocked(IPAddress.Parse("10.0.0.5")));
|
|
||||||
|
|
||||||
Assert.False(Firewall.IsBlocked(IPAddress.Parse("10.0.0.6")));
|
|
||||||
}
|
|
||||||
|
|
||||||
[Fact]
|
|
||||||
public void Firewall_CacheInvalidation_WorksOnUpdate()
|
|
||||||
{
|
|
||||||
var ip = IPAddress.Parse("192.168.1.10");
|
|
||||||
var entry = new SingleIpFirewallEntry("192.168.1.10");
|
|
||||||
|
|
||||||
Firewall.Add(entry);
|
|
||||||
Assert.True(Firewall.IsBlocked(ip));
|
|
||||||
|
|
||||||
Firewall.Remove(entry);
|
|
||||||
Assert.False(Firewall.IsBlocked(ip));
|
|
||||||
}
|
|
||||||
|
|
||||||
[Fact]
|
|
||||||
public void Firewall_ReadsFirewallSetCorrectly()
|
|
||||||
{
|
|
||||||
var entry = new SingleIpFirewallEntry("172.16.0.1");
|
|
||||||
Firewall.Add(entry);
|
|
||||||
|
|
||||||
var found = false;
|
|
||||||
Firewall.ReadFirewallSet(set =>
|
|
||||||
{
|
|
||||||
found = set.Contains(entry);
|
|
||||||
});
|
|
||||||
|
|
||||||
Assert.True(found);
|
|
||||||
}
|
|
||||||
|
|
||||||
[Fact]
|
|
||||||
public void Firewall_IsThreadSafe()
|
|
||||||
{
|
|
||||||
var testIps = new IPAddress[256];
|
|
||||||
for (var i = 0; i <= 255; i++)
|
|
||||||
{
|
|
||||||
testIps[i] = IPAddress.Parse($"192.168.0.{i}");
|
|
||||||
}
|
|
||||||
|
|
||||||
var entry = new CidrFirewallEntry(IPAddress.Parse("192.168.0.1"), IPAddress.Parse("192.168.0.255"));
|
|
||||||
Firewall.Add(entry);
|
|
||||||
|
|
||||||
Parallel.ForEach(testIps, ip =>
|
|
||||||
{
|
|
||||||
var shouldBlock = int.Parse(ip.ToString().Split('.')[3]) is > 0;
|
|
||||||
Assert.Equal(shouldBlock, Firewall.IsBlocked(ip));
|
|
||||||
});
|
|
||||||
|
|
||||||
Firewall.Remove(entry);
|
|
||||||
|
|
||||||
Parallel.ForEach(testIps, ip =>
|
|
||||||
{
|
|
||||||
Assert.False(Firewall.IsBlocked(ip));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
[Fact]
|
|
||||||
public void Firewall_DoesNotThrowWhenRemovingNonExistentEntry()
|
|
||||||
{
|
|
||||||
var entry = new SingleIpFirewallEntry("203.0.113.5");
|
|
||||||
Assert.False(Firewall.Remove(entry));
|
|
||||||
}
|
|
||||||
|
|
||||||
[Fact]
|
|
||||||
public void Firewall_CacheHandlesMultipleUpdates()
|
|
||||||
{
|
|
||||||
var ip = IPAddress.Parse("192.168.1.20");
|
|
||||||
var entry = new SingleIpFirewallEntry("192.168.1.20");
|
|
||||||
|
|
||||||
Firewall.Add(entry);
|
|
||||||
Assert.True(Firewall.IsBlocked(ip));
|
|
||||||
|
|
||||||
Firewall.Remove(entry);
|
|
||||||
Assert.False(Firewall.IsBlocked(ip));
|
|
||||||
|
|
||||||
Firewall.Add(entry);
|
|
||||||
Assert.True(Firewall.IsBlocked(ip));
|
|
||||||
|
|
||||||
Firewall.Remove(entry);
|
|
||||||
Assert.False(Firewall.IsBlocked(ip));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
134
Projects/Server/Collections/SortedRangeIndex.cs
Normal file
134
Projects/Server/Collections/SortedRangeIndex.cs
Normal file
|
|
@ -0,0 +1,134 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: SortedRangeIndex.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Numerics;
|
||||||
|
|
||||||
|
namespace Server.Collections;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Immutable, allocation-lean membership index over a set of inclusive integer ranges. Ranges are
|
||||||
|
/// stored as two parallel arrays (<c>_mins</c>/<c>_maxs</c>) sorted by minimum and coalesced into
|
||||||
|
/// disjoint runs, so a single binary search decides membership. Coalescing is required for
|
||||||
|
/// correctness: <see cref="Contains"/> only inspects the rightmost run whose minimum is <= the
|
||||||
|
/// value, which is only sound when the runs never overlap or nest.
|
||||||
|
/// </summary>
|
||||||
|
public sealed class SortedRangeIndex<T> where T : IBinaryInteger<T>
|
||||||
|
{
|
||||||
|
public static readonly SortedRangeIndex<T> Empty = new([], []);
|
||||||
|
|
||||||
|
/// <summary>An inclusive <c>[Min, Max]</c> range. Singles are represented as <c>Min == Max</c>.</summary>
|
||||||
|
public readonly record struct Range(T Min, T Max);
|
||||||
|
|
||||||
|
/// <summary>Orders ranges ascending by minimum; the coalescing pass in <see cref="Build"/> requires this.</summary>
|
||||||
|
public static readonly Comparison<Range> ByMin = static (a, b) => a.Min.CompareTo(b.Min);
|
||||||
|
|
||||||
|
private readonly T[] _mins;
|
||||||
|
private readonly T[] _maxs;
|
||||||
|
|
||||||
|
private SortedRangeIndex(T[] mins, T[] maxs)
|
||||||
|
{
|
||||||
|
_mins = mins;
|
||||||
|
_maxs = maxs;
|
||||||
|
}
|
||||||
|
|
||||||
|
public int Count => _mins.Length;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// True when <paramref name="value"/> falls in any range. Binary-searches for the rightmost run
|
||||||
|
/// whose minimum is <= the value, then tests that value against that run's maximum.
|
||||||
|
/// </summary>
|
||||||
|
public bool Contains(T value)
|
||||||
|
{
|
||||||
|
var lo = 0;
|
||||||
|
var hi = _mins.Length - 1;
|
||||||
|
var found = -1;
|
||||||
|
while (lo <= hi)
|
||||||
|
{
|
||||||
|
var mid = (lo + hi) >> 1;
|
||||||
|
if (_mins[mid] <= value)
|
||||||
|
{
|
||||||
|
found = mid;
|
||||||
|
lo = mid + 1;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
hi = mid - 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return found >= 0 && value <= _maxs[found];
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Builds an index from ranges that are already sorted ascending by <see cref="Range.Min"/> (sort
|
||||||
|
/// the source with <see cref="ByMin"/> first). Overlapping and nested ranges are merged into disjoint
|
||||||
|
/// runs. Two passes over the (pooled) input keep the run count exact so only the two final arrays are
|
||||||
|
/// heap-allocated: pass one counts the runs, pass two fills the exact-size arrays.
|
||||||
|
/// </summary>
|
||||||
|
public static SortedRangeIndex<T> Build(ReadOnlySpan<Range> sortedByMin)
|
||||||
|
{
|
||||||
|
if (sortedByMin.IsEmpty)
|
||||||
|
{
|
||||||
|
return Empty;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pass 1: count the disjoint runs so the final arrays can be sized exactly.
|
||||||
|
var runs = 1;
|
||||||
|
var curMax = sortedByMin[0].Max;
|
||||||
|
for (var i = 1; i < sortedByMin.Length; i++)
|
||||||
|
{
|
||||||
|
var r = sortedByMin[i];
|
||||||
|
if (r.Min <= curMax)
|
||||||
|
{
|
||||||
|
if (r.Max > curMax)
|
||||||
|
{
|
||||||
|
curMax = r.Max;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
runs++;
|
||||||
|
curMax = r.Max;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pass 2: write the coalesced runs into the exact-size final arrays.
|
||||||
|
var mins = new T[runs];
|
||||||
|
var maxs = new T[runs];
|
||||||
|
var w = 0;
|
||||||
|
mins[0] = sortedByMin[0].Min;
|
||||||
|
maxs[0] = sortedByMin[0].Max;
|
||||||
|
for (var i = 1; i < sortedByMin.Length; i++)
|
||||||
|
{
|
||||||
|
var r = sortedByMin[i];
|
||||||
|
if (r.Min <= maxs[w])
|
||||||
|
{
|
||||||
|
if (r.Max > maxs[w])
|
||||||
|
{
|
||||||
|
maxs[w] = r.Max;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
w++;
|
||||||
|
mins[w] = r.Min;
|
||||||
|
maxs[w] = r.Max;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return new SortedRangeIndex<T>(mins, maxs);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
/*************************************************************************
|
/*************************************************************************
|
||||||
* ModernUO *
|
* ModernUO *
|
||||||
* Copyright 2019-2026 - ModernUO Development Team *
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
* Email: hi@modernuo.com *
|
* Email: hi@modernuo.com *
|
||||||
|
|
@ -30,6 +30,7 @@ using Server.Compression;
|
||||||
using Server.Json;
|
using Server.Json;
|
||||||
using Server.Logging;
|
using Server.Logging;
|
||||||
using Server.Network;
|
using Server.Network;
|
||||||
|
using Server.Network.Bans;
|
||||||
using Server.Text;
|
using Server.Text;
|
||||||
|
|
||||||
namespace Server;
|
namespace Server;
|
||||||
|
|
@ -260,7 +261,7 @@ public static class Core
|
||||||
// ignored
|
// ignored
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!close && !Core.Headless)
|
if (!close && !Headless)
|
||||||
{
|
{
|
||||||
Console.WriteLine("This exception is fatal, press return to exit");
|
Console.WriteLine("This exception is fatal, press return to exit");
|
||||||
ConsoleInputHandler.ReadLine();
|
ConsoleInputHandler.ReadLine();
|
||||||
|
|
@ -342,6 +343,8 @@ public static class Core
|
||||||
World.ExitSerializationThreads();
|
World.ExitSerializationThreads();
|
||||||
PingServer.Shutdown();
|
PingServer.Shutdown();
|
||||||
NetState.Shutdown();
|
NetState.Shutdown();
|
||||||
|
BanChannel.Stop();
|
||||||
|
ConnectionFilters.Stop();
|
||||||
|
|
||||||
if (!_crashed)
|
if (!_crashed)
|
||||||
{
|
{
|
||||||
|
|
@ -461,6 +464,8 @@ public static class Core
|
||||||
|
|
||||||
AssemblyHandler.Invoke("Initialize");
|
AssemblyHandler.Invoke("Initialize");
|
||||||
|
|
||||||
|
BanChannel.Start(ClosingTokenSource.Token);
|
||||||
|
ConnectionFilters.Start(ClosingTokenSource.Token);
|
||||||
NetState.Start();
|
NetState.Start();
|
||||||
PingServer.Start();
|
PingServer.Start();
|
||||||
EventSink.InvokeServerStarted();
|
EventSink.InvokeServerStarted();
|
||||||
|
|
|
||||||
143
Projects/Server/Network/Bans/BanChannel.cs
Normal file
143
Projects/Server/Network/Bans/BanChannel.cs
Normal file
|
|
@ -0,0 +1,143 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: BanChannel.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.Net;
|
||||||
|
using System.Threading;
|
||||||
|
using Server.Logging;
|
||||||
|
|
||||||
|
namespace Server.Network.Bans;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Coordinates the configured <see cref="IBanReporter"/> contribution sinks. Enforcement is NOT here —
|
||||||
|
/// the accept path asks <see cref="ConnectionFilters"/>. This channel only fans locally-decided bans out
|
||||||
|
/// to external systems (CrowdSec), which distribute them to OS-level bouncers.
|
||||||
|
/// </summary>
|
||||||
|
public static class BanChannel
|
||||||
|
{
|
||||||
|
private static readonly ILogger logger = LogFactory.GetLogger(typeof(BanChannel));
|
||||||
|
|
||||||
|
private static IBanReporter[] _reporters = [];
|
||||||
|
|
||||||
|
public static IReadOnlyList<IBanReporter> Reporters => _reporters;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Registers a contribution sink from content (inversion of control). Idempotent by
|
||||||
|
/// <see cref="IBanReporter.Name"/>: a second registration of the same name is ignored. Configures the
|
||||||
|
/// reporter immediately so it is ready before <see cref="Start"/>.
|
||||||
|
/// </summary>
|
||||||
|
public static void Register(IBanReporter reporter)
|
||||||
|
{
|
||||||
|
if (reporter == null)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
var reporters = _reporters;
|
||||||
|
for (var i = 0; i < reporters.Length; i++)
|
||||||
|
{
|
||||||
|
if (reporters[i].Name == reporter.Name)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
reporter.Register();
|
||||||
|
|
||||||
|
var updated = new IBanReporter[_reporters.Length + 1];
|
||||||
|
Array.Copy(_reporters, updated, _reporters.Length);
|
||||||
|
updated[^1] = reporter;
|
||||||
|
_reporters = updated;
|
||||||
|
|
||||||
|
logger.Information("Ban channel registered reporter '{Name}'", reporter.Name);
|
||||||
|
}
|
||||||
|
|
||||||
|
internal static void ConfigureForTesting(IBanReporter[] reporters) => _reporters = reporters ?? [];
|
||||||
|
|
||||||
|
public static void Start(CancellationToken token)
|
||||||
|
{
|
||||||
|
var reporters = _reporters;
|
||||||
|
for (var i = 0; i < reporters.Length; i++)
|
||||||
|
{
|
||||||
|
var reporter = reporters[i];
|
||||||
|
try
|
||||||
|
{
|
||||||
|
reporter.Start(token);
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
// A broken contribution path must not crash boot — enforcement is local and unaffected.
|
||||||
|
logger.Error(e, "Ban reporter '{Name}' failed to start; continuing without it", reporter.Name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public static void Stop()
|
||||||
|
{
|
||||||
|
var reporters = _reporters;
|
||||||
|
for (var i = 0; i < reporters.Length; i++)
|
||||||
|
{
|
||||||
|
var reporter = reporters[i];
|
||||||
|
try
|
||||||
|
{
|
||||||
|
reporter.Stop();
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
logger.Warning(e, "Ban reporter '{Name}' threw while stopping", reporter.Name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Fans a locally-decided ban out to every reporter. Non-blocking; never throws.</summary>
|
||||||
|
public static void Report(IPAddress ip, TimeSpan ttl, string reason)
|
||||||
|
{
|
||||||
|
var reporters = _reporters;
|
||||||
|
for (var i = 0; i < reporters.Length; i++)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
reporters[i].Report(ip, ttl, reason);
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
logger.Warning(e, "Ban reporter '{Name}' threw during Report", reporters[i].Name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Fans a retraction (manual unban) out to every retract-capable reporter.</summary>
|
||||||
|
public static void Retract(IPAddress ip)
|
||||||
|
{
|
||||||
|
var reporters = _reporters;
|
||||||
|
for (var i = 0; i < reporters.Length; i++)
|
||||||
|
{
|
||||||
|
if (!reporters[i].CanRetract)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
reporters[i].Retract(ip);
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
logger.Warning(e, "Ban reporter '{Name}' threw during Retract", reporters[i].Name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
76
Projects/Server/Network/Bans/BanConfiguration.cs
Normal file
76
Projects/Server/Network/Bans/BanConfiguration.cs
Normal file
|
|
@ -0,0 +1,76 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: BanConfiguration.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.IO;
|
||||||
|
using System.Text.Json.Serialization;
|
||||||
|
using Server.Json;
|
||||||
|
|
||||||
|
namespace Server.Network.Bans;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Loads the <see cref="BanSettings"/> from <c>Configuration/bans.json</c> (matching the per-feature
|
||||||
|
/// JSON config pattern used by <c>AssistantConfiguration</c>). Loaded once; a missing file writes a
|
||||||
|
/// local-only, fail-open template so operators have something to edit.
|
||||||
|
/// </summary>
|
||||||
|
public static class BanConfiguration
|
||||||
|
{
|
||||||
|
private const string _path = "Configuration/bans.json";
|
||||||
|
|
||||||
|
public static BanSettings Settings { get; private set; }
|
||||||
|
|
||||||
|
public static void Configure()
|
||||||
|
{
|
||||||
|
// Idempotent: a second call must not re-deserialize or overwrite an operator's edits.
|
||||||
|
if (Settings != null)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
var path = Path.Join(Core.BaseDirectory, _path);
|
||||||
|
|
||||||
|
if (File.Exists(path))
|
||||||
|
{
|
||||||
|
Settings = JsonConfig.Deserialize<BanSettings>(path);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
Settings = new BanSettings
|
||||||
|
{
|
||||||
|
ReportRateLimitTrips = true,
|
||||||
|
AutoBanDuration = TimeSpan.FromHours(4)
|
||||||
|
};
|
||||||
|
|
||||||
|
Save();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void Save()
|
||||||
|
{
|
||||||
|
JsonConfig.Serialize(Path.Join(Core.BaseDirectory, _path), Settings);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Ban-channel policy: which reporters receive contributions, and how auto-detections are handled.</summary>
|
||||||
|
public record BanSettings
|
||||||
|
{
|
||||||
|
/// <summary>Whether IP rate-limiter trips are contributed to reporters. They never enter the local firewall set.</summary>
|
||||||
|
[JsonPropertyName("reportRateLimitTrips")]
|
||||||
|
public bool ReportRateLimitTrips { get; set; } = true;
|
||||||
|
|
||||||
|
/// <summary>Duration reported for an auto-detected (rate-limit) ban.</summary>
|
||||||
|
[JsonPropertyName("autoBanDuration")]
|
||||||
|
public TimeSpan AutoBanDuration { get; set; } = TimeSpan.FromHours(4);
|
||||||
|
}
|
||||||
55
Projects/Server/Network/Bans/IBanReporter.cs
Normal file
55
Projects/Server/Network/Bans/IBanReporter.cs
Normal file
|
|
@ -0,0 +1,55 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: IBanReporter.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Net;
|
||||||
|
using System.Threading;
|
||||||
|
|
||||||
|
namespace Server.Network.Bans;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// A contribution sink behind <see cref="BanChannel"/>. Reporters receive locally-decided bans
|
||||||
|
/// (manual admin bans, rate-limit trips, blocklist promotions) and forward them to an external system
|
||||||
|
/// (e.g. CrowdSec), which distributes them to OS-level bouncers. Reporters never answer the accept-path
|
||||||
|
/// membership query — that is an <see cref="IConnectionFilter"/>'s job.
|
||||||
|
/// </summary>
|
||||||
|
public interface IBanReporter
|
||||||
|
{
|
||||||
|
/// <summary>Stable id for logging/config (e.g. <c>crowdsec</c>).</summary>
|
||||||
|
string Name { get; }
|
||||||
|
|
||||||
|
/// <summary>Reads configuration. No network or file I/O here.</summary>
|
||||||
|
void Register();
|
||||||
|
|
||||||
|
/// <summary>Starts background delivery. The token is cancelled on shutdown.</summary>
|
||||||
|
void Start(CancellationToken token);
|
||||||
|
|
||||||
|
/// <summary>Flushes and tears down background delivery.</summary>
|
||||||
|
void Stop();
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Enqueues a ban contribution. MUST be non-blocking and safe on the accept path: it may only
|
||||||
|
/// enqueue (bounded, drop-on-overflow) and never perform synchronous I/O.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="ttl"><see cref="TimeSpan.Zero"/> or negative = use the reporter's default duration.</param>
|
||||||
|
/// <param name="reason">Short slug (<c>manual</c>, <c>rate-limit</c>) used as the scenario suffix.</param>
|
||||||
|
void Report(IPAddress address, TimeSpan ttl, string reason);
|
||||||
|
|
||||||
|
/// <summary>True if this reporter can retract a previously-reported ban.</summary>
|
||||||
|
bool CanRetract { get; }
|
||||||
|
|
||||||
|
/// <summary>Enqueues a retraction (e.g. a manual unban). No-op if unsupported.</summary>
|
||||||
|
void Retract(IPAddress address);
|
||||||
|
}
|
||||||
155
Projects/Server/Network/ConnectionFilters.cs
Normal file
155
Projects/Server/Network/ConnectionFilters.cs
Normal file
|
|
@ -0,0 +1,155 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: ConnectionFilters.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.Net;
|
||||||
|
using System.Threading;
|
||||||
|
using Server.Logging;
|
||||||
|
|
||||||
|
namespace Server.Network;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Registry of the <see cref="IConnectionFilter"/> gates the accept path consults, and their lifecycle.
|
||||||
|
/// Filters are registered during the Configure sweep and the backing store is a plain array, so
|
||||||
|
/// <see cref="ShouldDeny"/> is an indexed loop over a field read — no enumerator, no closure, no
|
||||||
|
/// allocation. The whole accept path runs on the game loop, so no synchronization is needed.
|
||||||
|
/// </summary>
|
||||||
|
public static class ConnectionFilters
|
||||||
|
{
|
||||||
|
private static readonly ILogger logger = LogFactory.GetLogger(typeof(ConnectionFilters));
|
||||||
|
|
||||||
|
private static IConnectionFilter[] _filters = [];
|
||||||
|
|
||||||
|
public static IReadOnlyList<IConnectionFilter> Filters => _filters;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Registers a gate (inversion of control, mirroring <c>BanChannel.Register</c>). Idempotent by
|
||||||
|
/// <see cref="IConnectionFilter.Name"/>. Filters are consulted in registration order, so register
|
||||||
|
/// the cheapest and most selective first — core registers the firewall before content is swept.
|
||||||
|
/// </summary>
|
||||||
|
public static void Register(IConnectionFilter filter)
|
||||||
|
{
|
||||||
|
if (filter == null)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
var filters = _filters;
|
||||||
|
for (var i = 0; i < filters.Length; i++)
|
||||||
|
{
|
||||||
|
if (filters[i].Name == filter.Name)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
filter.Register();
|
||||||
|
|
||||||
|
var updated = new IConnectionFilter[_filters.Length + 1];
|
||||||
|
Array.Copy(_filters, updated, _filters.Length);
|
||||||
|
updated[^1] = filter;
|
||||||
|
_filters = updated;
|
||||||
|
|
||||||
|
logger.Information("Registered connection filter '{Name}'", filter.Name);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// True when any filter denies the connection. Short-circuits on the first denial;
|
||||||
|
/// <paramref name="deniedBy"/> names it for logging.
|
||||||
|
/// </summary>
|
||||||
|
public static bool ShouldDeny(IPAddress address, out string deniedBy)
|
||||||
|
{
|
||||||
|
var filters = _filters;
|
||||||
|
for (var i = 0; i < filters.Length; i++)
|
||||||
|
{
|
||||||
|
// A faulty filter must not take down the accept loop for every connection.
|
||||||
|
try
|
||||||
|
{
|
||||||
|
if (filters[i].ShouldDeny(address))
|
||||||
|
{
|
||||||
|
deniedBy = filters[i].Name;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
Disable(filters[i], e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
deniedBy = null;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Drops a filter that threw on the accept path: one that faults once faults for every subsequent
|
||||||
|
/// connection, costing an exception and a log line per accept. Failing open is deliberate — a broken
|
||||||
|
/// filter must not be able to deny every connection either.
|
||||||
|
/// </summary>
|
||||||
|
private static void Disable(IConnectionFilter filter, Exception e)
|
||||||
|
{
|
||||||
|
logger.Error(e, "Connection filter '{Name}' threw on the accept path; unregistering it", filter.Name);
|
||||||
|
|
||||||
|
var filters = _filters;
|
||||||
|
var updated = new List<IConnectionFilter>(filters.Length);
|
||||||
|
for (var i = 0; i < filters.Length; i++)
|
||||||
|
{
|
||||||
|
if (!ReferenceEquals(filters[i], filter))
|
||||||
|
{
|
||||||
|
updated.Add(filters[i]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
_filters = updated.ToArray();
|
||||||
|
}
|
||||||
|
|
||||||
|
public static void Start(CancellationToken token)
|
||||||
|
{
|
||||||
|
var filters = _filters;
|
||||||
|
for (var i = 0; i < filters.Length; i++)
|
||||||
|
{
|
||||||
|
var filter = filters[i];
|
||||||
|
try
|
||||||
|
{
|
||||||
|
filter.Start(token);
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
// A filter that cannot hydrate must not crash boot; it simply denies nothing.
|
||||||
|
logger.Error(e, "Connection filter '{Name}' failed to start; continuing without it", filter.Name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public static void Stop()
|
||||||
|
{
|
||||||
|
var filters = _filters;
|
||||||
|
for (var i = 0; i < filters.Length; i++)
|
||||||
|
{
|
||||||
|
var filter = filters[i];
|
||||||
|
try
|
||||||
|
{
|
||||||
|
filter.Stop();
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
logger.Warning(e, "Connection filter '{Name}' threw while stopping", filter.Name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
internal static void ResetForTesting() => _filters = [];
|
||||||
|
}
|
||||||
|
|
@ -1,168 +0,0 @@
|
||||||
/*************************************************************************
|
|
||||||
* ModernUO *
|
|
||||||
* Copyright 2019-2026 - ModernUO Development Team *
|
|
||||||
* Email: hi@modernuo.com *
|
|
||||||
* File: Firewall.cs *
|
|
||||||
* *
|
|
||||||
* This program is free software: you can redistribute it and/or modify *
|
|
||||||
* it under the terms of the GNU General Public License as published by *
|
|
||||||
* the Free Software Foundation, either version 3 of the License, or *
|
|
||||||
* (at your option) any later version. *
|
|
||||||
* *
|
|
||||||
* You should have received a copy of the GNU General Public License *
|
|
||||||
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
|
||||||
*************************************************************************/
|
|
||||||
|
|
||||||
using System;
|
|
||||||
using System.Collections.Concurrent;
|
|
||||||
using System.Collections.Generic;
|
|
||||||
using System.Net;
|
|
||||||
using System.Runtime.CompilerServices;
|
|
||||||
using System.Threading;
|
|
||||||
|
|
||||||
namespace Server.Network;
|
|
||||||
|
|
||||||
public static class Firewall
|
|
||||||
{
|
|
||||||
[ThreadStatic]
|
|
||||||
private static InternalValidationEntry _validationEntry;
|
|
||||||
private static readonly ConcurrentDictionary<IPAddress, int> _isBlockedCache = [];
|
|
||||||
private static readonly ReaderWriterLockSlim _firewallLock = new(LockRecursionPolicy.NoRecursion);
|
|
||||||
|
|
||||||
private static int _firewallVersion;
|
|
||||||
private static readonly SortedSet<IFirewallEntry> _firewallSet = [];
|
|
||||||
|
|
||||||
public static int FirewallSetCount => _firewallSet.Count;
|
|
||||||
|
|
||||||
public static void ReadFirewallSet(Action<IReadOnlySet<IFirewallEntry>> callback)
|
|
||||||
{
|
|
||||||
_firewallLock.EnterReadLock();
|
|
||||||
try
|
|
||||||
{
|
|
||||||
callback(_firewallSet);
|
|
||||||
}
|
|
||||||
finally
|
|
||||||
{
|
|
||||||
_firewallLock.ExitReadLock();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
internal static bool IsBlocked(IPAddress address)
|
|
||||||
{
|
|
||||||
if (_isBlockedCache.TryGetValue(address, out var blockVersion) && blockVersion == _firewallVersion)
|
|
||||||
{
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (_validationEntry == null)
|
|
||||||
{
|
|
||||||
_validationEntry = new InternalValidationEntry(address);
|
|
||||||
}
|
|
||||||
else
|
|
||||||
{
|
|
||||||
_validationEntry.Address = address;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (CheckBlocked(_validationEntry))
|
|
||||||
{
|
|
||||||
_isBlockedCache[address] = _firewallVersion;
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
private static bool CheckBlocked(IFirewallEntry validationEntry)
|
|
||||||
{
|
|
||||||
if (_firewallSet.Count == 0)
|
|
||||||
{
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
_firewallLock.EnterReadLock();
|
|
||||||
try
|
|
||||||
{
|
|
||||||
var min = _firewallSet.Min;
|
|
||||||
if (validationEntry.CompareTo(min) < 0)
|
|
||||||
{
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Get all entries that are lower than our validation entry
|
|
||||||
var view = _firewallSet.GetViewBetween(min, validationEntry);
|
|
||||||
|
|
||||||
// Loop backward since there shouldn't be any entries where the Min address is higher than ours
|
|
||||||
foreach (var firewallEntry in view.Reverse())
|
|
||||||
{
|
|
||||||
if (firewallEntry.IsBlocked(validationEntry.MinIpAddress))
|
|
||||||
{
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return view.Max?.IsBlocked(validationEntry.MinIpAddress) == true;
|
|
||||||
}
|
|
||||||
finally
|
|
||||||
{
|
|
||||||
_firewallLock.ExitReadLock();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
[MethodImpl(MethodImplOptions.AggressiveInlining)]
|
|
||||||
public static bool Add(IFirewallEntry firewallEntry)
|
|
||||||
{
|
|
||||||
_firewallLock.EnterWriteLock();
|
|
||||||
try
|
|
||||||
{
|
|
||||||
if (_firewallSet.Add(firewallEntry))
|
|
||||||
{
|
|
||||||
Interlocked.Increment(ref _firewallVersion); // Update version
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
finally
|
|
||||||
{
|
|
||||||
_firewallLock.ExitWriteLock();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
[MethodImpl(MethodImplOptions.AggressiveInlining)]
|
|
||||||
public static bool Remove(IFirewallEntry entry)
|
|
||||||
{
|
|
||||||
if (entry == null)
|
|
||||||
{
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
_firewallLock.EnterWriteLock();
|
|
||||||
try
|
|
||||||
{
|
|
||||||
if (_firewallSet.Remove(entry))
|
|
||||||
{
|
|
||||||
Interlocked.Increment(ref _firewallVersion); // Update version
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
finally
|
|
||||||
{
|
|
||||||
_firewallLock.ExitWriteLock();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private class InternalValidationEntry : BaseFirewallEntry
|
|
||||||
{
|
|
||||||
private UInt128 _address;
|
|
||||||
|
|
||||||
public IPAddress Address
|
|
||||||
{
|
|
||||||
set => _address = value.ToUInt128();
|
|
||||||
}
|
|
||||||
|
|
||||||
public override UInt128 MinIpAddress => _address;
|
|
||||||
public override UInt128 MaxIpAddress => _address;
|
|
||||||
|
|
||||||
public InternalValidationEntry(IPAddress ipAddress) => Address = ipAddress;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
60
Projects/Server/Network/IConnectionFilter.cs
Normal file
60
Projects/Server/Network/IConnectionFilter.cs
Normal file
|
|
@ -0,0 +1,60 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: IConnectionFilter.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System.Net;
|
||||||
|
using System.Threading;
|
||||||
|
|
||||||
|
namespace Server.Network;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// A gate consulted for every inbound connection, before the socket is configured and before any
|
||||||
|
/// per-connection allocation. Implementations decide membership only — the accept path neither knows
|
||||||
|
/// nor cares where a filter's data comes from, so a filter may be a handful of admin-curated entries,
|
||||||
|
/// a millions-strong list hydrated from a file, or a query against something else entirely. Core owns
|
||||||
|
/// the question; content owns every answer (see <c>Firewall</c> and <c>BlocklistFilter</c> in UOContent).
|
||||||
|
/// </summary>
|
||||||
|
/// <remarks>
|
||||||
|
/// <para>
|
||||||
|
/// <see cref="ShouldDeny"/> runs on the game loop once per accepted socket, which is the path that has
|
||||||
|
/// to survive a DDoS. Implementations MUST be allocation-free and O(log n) at worst, MUST NOT perform
|
||||||
|
/// I/O, and MUST NOT block. Anything expensive (parsing, reloading, reporting to an external service)
|
||||||
|
/// belongs off the loop or behind a bounded, non-blocking enqueue.
|
||||||
|
/// </para>
|
||||||
|
/// <para>
|
||||||
|
/// Side effects that a hit implies (contributing to <c>BanChannel</c>, promoting to an OS firewall,
|
||||||
|
/// suppressing duplicate reports) are the filter's own business, not the accept path's. This is why
|
||||||
|
/// <see cref="ShouldDeny"/> returns a bare bool: the accept path asks one question and does one thing.
|
||||||
|
/// </para>
|
||||||
|
/// </remarks>
|
||||||
|
public interface IConnectionFilter
|
||||||
|
{
|
||||||
|
/// <summary>Stable id for logging/config (e.g. <c>firewall</c>, <c>blocklist</c>).</summary>
|
||||||
|
string Name { get; }
|
||||||
|
|
||||||
|
/// <summary>Reads configuration. Called by <see cref="ConnectionFilters.Register"/>. No I/O beyond config.</summary>
|
||||||
|
void Register();
|
||||||
|
|
||||||
|
/// <summary>Starts any background hydration. The token is cancelled on shutdown.</summary>
|
||||||
|
void Start(CancellationToken token);
|
||||||
|
|
||||||
|
/// <summary>Flushes and tears down. Called during shutdown.</summary>
|
||||||
|
void Stop();
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// True to deny the connection. Must be allocation-free and non-blocking; see the remarks on
|
||||||
|
/// <see cref="IConnectionFilter"/>.
|
||||||
|
/// </summary>
|
||||||
|
bool ShouldDeny(IPAddress address);
|
||||||
|
}
|
||||||
|
|
@ -224,10 +224,19 @@ public partial class NetState
|
||||||
if (_ipRateLimiter != null && !_ipRateLimiter.Verify(remoteIP, out var totalAttempts))
|
if (_ipRateLimiter != null && !_ipRateLimiter.Verify(remoteIP, out var totalAttempts))
|
||||||
{
|
{
|
||||||
logger.Debug("{Address} Past IP limit threshold ({TotalAttempts})", remoteIP, totalAttempts);
|
logger.Debug("{Address} Past IP limit threshold ({TotalAttempts})", remoteIP, totalAttempts);
|
||||||
}
|
|
||||||
else if (Firewall.IsBlocked(remoteIP))
|
if (Bans.BanConfiguration.Settings.ReportRateLimitTrips)
|
||||||
{
|
{
|
||||||
logger.Debug("{Address} Firewalled", remoteIP);
|
// Enqueue-only contribution; NOT added to the local firewall set (the limiter already
|
||||||
|
// gates it here and the OS bouncer drops it at the kernel).
|
||||||
|
Bans.BanChannel.Report(remoteIP, Bans.BanConfiguration.Settings.AutoBanDuration, "rate-limit");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else if (ConnectionFilters.ShouldDeny(remoteIP, out var deniedBy))
|
||||||
|
{
|
||||||
|
// Whatever a hit implies (persisting, promoting to an OS bouncer, contributing to the
|
||||||
|
// ban channel) is the filter's own business; the accept path just drops the socket.
|
||||||
|
logger.Debug("{Address} denied by connection filter '{Filter}'", remoteIP, deniedBy);
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
|
|
|
||||||
240
Projects/Server/Utilities/IPAddressUtility.cs
Normal file
240
Projects/Server/Utilities/IPAddressUtility.cs
Normal file
|
|
@ -0,0 +1,240 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: IPAddressUtility.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Buffers.Binary;
|
||||||
|
using System.Net;
|
||||||
|
using System.Net.Sockets;
|
||||||
|
using System.Numerics;
|
||||||
|
|
||||||
|
namespace Server;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Low-level IPAddress conversion and parsing helpers shared by the firewall, ban channel, and
|
||||||
|
/// blocklist. All members are allocation-free (stack buffers only) so they are safe on hot accept
|
||||||
|
/// paths and inside tight parse loops.
|
||||||
|
/// </summary>
|
||||||
|
public static class IPAddressUtility
|
||||||
|
{
|
||||||
|
// Converts an IPAddress to a UInt128 in IPv6 format.
|
||||||
|
// The IsIPv4MappedToIPv6 clause below looks redundant (the BCL only ever sets it on InterNetworkV6),
|
||||||
|
// but it guards the v4 -> UInt128 -> IPAddress round-trip, which can return a mapped v6 address for
|
||||||
|
// what is really a v4 one.
|
||||||
|
//TODO Rework as an explicit "to canonical v6 bits" step that needs no family check
|
||||||
|
// (see dev-docs/networking-packets.md, "IP Address Normalization")
|
||||||
|
public static UInt128 ToUInt128(this IPAddress ip)
|
||||||
|
{
|
||||||
|
if (ip.AddressFamily == AddressFamily.InterNetwork && !ip.IsIPv4MappedToIPv6)
|
||||||
|
{
|
||||||
|
Span<byte> integer = stackalloc byte[4];
|
||||||
|
return !ip.TryWriteBytes(integer, out _)
|
||||||
|
? (UInt128)0
|
||||||
|
: new UInt128(0, 0xFFFF00000000UL | BinaryPrimitives.ReadUInt32BigEndian(integer));
|
||||||
|
}
|
||||||
|
|
||||||
|
Span<byte> bytes = stackalloc byte[16];
|
||||||
|
if (!ip.TryWriteBytes(bytes, out _))
|
||||||
|
{
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
var high = BinaryPrimitives.ReadUInt64BigEndian(bytes[..8]);
|
||||||
|
var low = BinaryPrimitives.ReadUInt64BigEndian(bytes.Slice(8, 8));
|
||||||
|
|
||||||
|
return new UInt128(high, low);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Converts a UInt128 in IPv6 format to an IPAddress
|
||||||
|
public static IPAddress ToIpAddress(this UInt128 value, bool mapToIpv6 = false)
|
||||||
|
{
|
||||||
|
// IPv4 mapped IPv6 address
|
||||||
|
if (!mapToIpv6 && value >= 0xFFFF00000000UL && value <= 0xFFFFFFFFFFFFUL)
|
||||||
|
{
|
||||||
|
var newAddress = IPAddress.HostToNetworkOrder((int)value);
|
||||||
|
return new IPAddress(unchecked((uint)newAddress));
|
||||||
|
}
|
||||||
|
|
||||||
|
Span<byte> bytes = stackalloc byte[16]; // 128 bits for IPv6 address
|
||||||
|
((IBinaryInteger<UInt128>)value).WriteBigEndian(bytes);
|
||||||
|
|
||||||
|
return new IPAddress(bytes);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Parses <c>a.b.c.d/n</c>, <c>::/n</c>, or a bare address (treated as a single-host range) into an
|
||||||
|
/// inclusive <see cref="UInt128"/> range in normalized IPv6 form. A bare IPv4 prefix is widened by 96
|
||||||
|
/// bits so v4 and v6 ranges are directly comparable. Returns false on anything malformed.
|
||||||
|
/// </summary>
|
||||||
|
public static bool TryParseCidrRange(ReadOnlySpan<char> cidr, out UInt128 min, out UInt128 max)
|
||||||
|
{
|
||||||
|
min = default;
|
||||||
|
max = default;
|
||||||
|
|
||||||
|
var slash = cidr.IndexOf('/');
|
||||||
|
if (!IPAddress.TryParse(slash >= 0 ? cidr[..slash] : cidr, out var ip))
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
var isV6 = ip.AddressFamily == AddressFamily.InterNetworkV6;
|
||||||
|
var maxPrefixLength = isV6 ? 128 : 32;
|
||||||
|
int prefixLength;
|
||||||
|
|
||||||
|
if (slash < 0)
|
||||||
|
{
|
||||||
|
prefixLength = maxPrefixLength;
|
||||||
|
}
|
||||||
|
else if (!int.TryParse(cidr[(slash + 1)..], out prefixLength) ||
|
||||||
|
prefixLength < 0 || prefixLength > maxPrefixLength)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!isV6)
|
||||||
|
{
|
||||||
|
prefixLength += 96; // 32 -> 128
|
||||||
|
}
|
||||||
|
|
||||||
|
Span<byte> bytes = stackalloc byte[16];
|
||||||
|
ip.WriteMappedIPv6To(bytes);
|
||||||
|
|
||||||
|
min = Utility.CreateCidrAddress(bytes, prefixLength, false);
|
||||||
|
max = Utility.CreateCidrAddress(bytes, prefixLength, true);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Extracts the big-endian uint of an <see cref="AddressFamily.InterNetwork"/> address.</summary>
|
||||||
|
public static bool TryV4(IPAddress ip, out uint v)
|
||||||
|
{
|
||||||
|
Span<byte> b = stackalloc byte[4];
|
||||||
|
if (ip.TryWriteBytes(b, out var n) && n == 4)
|
||||||
|
{
|
||||||
|
v = ((uint)b[0] << 24) | ((uint)b[1] << 16) | ((uint)b[2] << 8) | b[3];
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
v = 0;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Extracts the embedded v4 uint from a v4-mapped-v6 address directly from the mapped bytes,
|
||||||
|
/// avoiding the allocation of <see cref="IPAddress.MapToIPv4"/>.
|
||||||
|
/// </summary>
|
||||||
|
public static bool TryMappedV4(IPAddress ip, out uint v)
|
||||||
|
{
|
||||||
|
Span<byte> b = stackalloc byte[16];
|
||||||
|
if (ip.TryWriteBytes(b, out var n) && n == 16)
|
||||||
|
{
|
||||||
|
v = ((uint)b[12] << 24) | ((uint)b[13] << 16) | ((uint)b[14] << 8) | b[15];
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
v = 0;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Parses a dotted-quad IPv4 literal into a big-endian uint. Allocation-free, strict.</summary>
|
||||||
|
public static bool TryParseV4(ReadOnlySpan<char> s, out uint v)
|
||||||
|
{
|
||||||
|
v = 0;
|
||||||
|
uint acc = 0;
|
||||||
|
int octet = 0, digits = 0, dots = 0;
|
||||||
|
for (var i = 0; i < s.Length; i++)
|
||||||
|
{
|
||||||
|
var c = s[i];
|
||||||
|
if (c == '.')
|
||||||
|
{
|
||||||
|
if (digits == 0 || octet > 255)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
acc = (acc << 8) | (uint)octet;
|
||||||
|
dots++;
|
||||||
|
octet = 0;
|
||||||
|
digits = 0;
|
||||||
|
}
|
||||||
|
else if (c is >= '0' and <= '9')
|
||||||
|
{
|
||||||
|
octet = octet * 10 + (c - '0');
|
||||||
|
if (++digits > 3)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (dots != 3 || digits == 0 || octet > 255)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
v = (acc << 8) | (uint)octet;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// UTF-8/ASCII byte overload of <see cref="TryParseV4(ReadOnlySpan{char}, out uint)"/>, mirroring its
|
||||||
|
/// validation exactly so the blocklist can parse dotted-quads straight from file bytes with no
|
||||||
|
/// per-line string allocation.
|
||||||
|
/// </summary>
|
||||||
|
public static bool TryParseV4(ReadOnlySpan<byte> s, out uint v)
|
||||||
|
{
|
||||||
|
v = 0;
|
||||||
|
uint acc = 0;
|
||||||
|
int octet = 0, digits = 0, dots = 0;
|
||||||
|
for (var i = 0; i < s.Length; i++)
|
||||||
|
{
|
||||||
|
var c = s[i];
|
||||||
|
if (c == (byte)'.')
|
||||||
|
{
|
||||||
|
if (digits == 0 || octet > 255)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
acc = (acc << 8) | (uint)octet;
|
||||||
|
dots++;
|
||||||
|
octet = 0;
|
||||||
|
digits = 0;
|
||||||
|
}
|
||||||
|
else if (c is >= (byte)'0' and <= (byte)'9')
|
||||||
|
{
|
||||||
|
octet = octet * 10 + (c - '0');
|
||||||
|
if (++digits > 3)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (dots != 3 || digits == 0 || octet > 255)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
v = (acc << 8) | (uint)octet;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -1,6 +1,7 @@
|
||||||
|
using System;
|
||||||
using System.Net;
|
using System.Net;
|
||||||
using System.Net.Sockets;
|
using System.Net.Sockets;
|
||||||
using Server.Network;
|
using Server.Collections;
|
||||||
|
|
||||||
namespace Server;
|
namespace Server;
|
||||||
|
|
||||||
|
|
@ -14,36 +15,42 @@ public static class NetworkUtilities
|
||||||
_ => false
|
_ => false
|
||||||
};
|
};
|
||||||
|
|
||||||
private static readonly IFirewallEntry[] _privateNetworkV4 =
|
// These are constant reserved ranges, not firewall entries -- they only ever answer "is this address
|
||||||
[
|
// in one of these blocks?", which is exactly what SortedRangeIndex is for. Building them through the
|
||||||
new CidrFirewallEntry("127.0.0.1/8"),
|
// firewall entry types was a convenience that made core depend on the firewall for something that has
|
||||||
new CidrFirewallEntry("192.168.0.0/16"),
|
// nothing to do with banning.
|
||||||
new CidrFirewallEntry("10.0.0.0/8"),
|
private static readonly SortedRangeIndex<UInt128> _privateNetworkV4 = BuildIndex(
|
||||||
new CidrFirewallEntry("172.16.0.0/12"),
|
"127.0.0.1/8",
|
||||||
new CidrFirewallEntry("169.254.0.0/16"),
|
"192.168.0.0/16",
|
||||||
new CidrFirewallEntry("100.64.0.0/10")
|
"10.0.0.0/8",
|
||||||
];
|
"172.16.0.0/12",
|
||||||
|
"169.254.0.0/16",
|
||||||
|
"100.64.0.0/10"
|
||||||
|
);
|
||||||
|
|
||||||
private static readonly IFirewallEntry[] _privateNetworkV6 =
|
private static readonly SortedRangeIndex<UInt128> _privateNetworkV6 = BuildIndex(
|
||||||
[
|
"fc00::/7",
|
||||||
new CidrFirewallEntry("fc00::/7"),
|
"fe80::/10"
|
||||||
new CidrFirewallEntry("fe80::/10")
|
);
|
||||||
];
|
|
||||||
|
|
||||||
public static bool IsPrivateNetworkV4(this IPAddress ip)
|
private static SortedRangeIndex<UInt128> BuildIndex(params ReadOnlySpan<string> cidrs)
|
||||||
{
|
{
|
||||||
for (var i = 0; i < _privateNetworkV4.Length; i++)
|
var ranges = new SortedRangeIndex<UInt128>.Range[cidrs.Length];
|
||||||
|
for (var i = 0; i < cidrs.Length; i++)
|
||||||
{
|
{
|
||||||
if (_privateNetworkV4[i].IsBlocked(ip))
|
if (!IPAddressUtility.TryParseCidrRange(cidrs[i], out var min, out var max))
|
||||||
{
|
{
|
||||||
return true;
|
throw new ArgumentException($"Invalid reserved-network CIDR \"{cidrs[i]}\"");
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return false;
|
ranges[i] = new SortedRangeIndex<UInt128>.Range(min, max);
|
||||||
}
|
}
|
||||||
|
|
||||||
public static bool IsPrivateNetworkV6(this IPAddress ip) =>
|
Array.Sort(ranges, SortedRangeIndex<UInt128>.ByMin);
|
||||||
_privateNetworkV6[0].IsBlocked(ip) ||
|
return SortedRangeIndex<UInt128>.Build(ranges);
|
||||||
_privateNetworkV6[1].IsBlocked(ip);
|
}
|
||||||
|
|
||||||
|
public static bool IsPrivateNetworkV4(this IPAddress ip) => _privateNetworkV4.Contains(ip.ToUInt128());
|
||||||
|
|
||||||
|
public static bool IsPrivateNetworkV6(this IPAddress ip) => _privateNetworkV6.Contains(ip.ToUInt128());
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -108,45 +108,6 @@ public static partial class Utility
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Converts an IPAddress to a UInt128 in IPv6 format
|
|
||||||
public static UInt128 ToUInt128(this IPAddress ip)
|
|
||||||
{
|
|
||||||
if (ip.AddressFamily == AddressFamily.InterNetwork && !ip.IsIPv4MappedToIPv6)
|
|
||||||
{
|
|
||||||
Span<byte> integer = stackalloc byte[4];
|
|
||||||
return !ip.TryWriteBytes(integer, out _)
|
|
||||||
? (UInt128)0
|
|
||||||
: new UInt128(0, 0xFFFF00000000UL | BinaryPrimitives.ReadUInt32BigEndian(integer));
|
|
||||||
}
|
|
||||||
|
|
||||||
Span<byte> bytes = stackalloc byte[16];
|
|
||||||
if (!ip.TryWriteBytes(bytes, out _))
|
|
||||||
{
|
|
||||||
return 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
var high = BinaryPrimitives.ReadUInt64BigEndian(bytes[..8]);
|
|
||||||
var low = BinaryPrimitives.ReadUInt64BigEndian(bytes.Slice(8, 8));
|
|
||||||
|
|
||||||
return new UInt128(high, low);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Converts a UInt128 in IPv6 format to an IPAddress
|
|
||||||
public static IPAddress ToIpAddress(this UInt128 value, bool mapToIpv6 = false)
|
|
||||||
{
|
|
||||||
// IPv4 mapped IPv6 address
|
|
||||||
if (!mapToIpv6 && value >= 0xFFFF00000000UL && value <= 0xFFFFFFFFFFFFUL)
|
|
||||||
{
|
|
||||||
var newAddress = IPAddress.HostToNetworkOrder((int)value);
|
|
||||||
return new IPAddress(unchecked((uint)newAddress));
|
|
||||||
}
|
|
||||||
|
|
||||||
Span<byte> bytes = stackalloc byte[16]; // 128 bits for IPv6 address
|
|
||||||
((IBinaryInteger<UInt128>)value).WriteBigEndian(bytes);
|
|
||||||
|
|
||||||
return new IPAddress(bytes);
|
|
||||||
}
|
|
||||||
|
|
||||||
[MethodImpl(MethodImplOptions.AggressiveInlining)]
|
[MethodImpl(MethodImplOptions.AggressiveInlining)]
|
||||||
public static UInt128 CreateCidrAddress(ReadOnlySpan<byte> bytes, int prefixLength, bool isMax)
|
public static UInt128 CreateCidrAddress(ReadOnlySpan<byte> bytes, int prefixLength, bool isMax)
|
||||||
{
|
{
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
using System;
|
using System;
|
||||||
using System.IO;
|
using System.IO;
|
||||||
using System.Reflection;
|
using System.Reflection;
|
||||||
using System.Threading;
|
using System.Threading;
|
||||||
|
|
@ -61,6 +61,15 @@ internal static class TestServerInitializer
|
||||||
AssemblyHandler.LoadAssemblies(["Server.dll", "UOContent.dll"]);
|
AssemblyHandler.LoadAssemblies(["Server.dll", "UOContent.dll"]);
|
||||||
|
|
||||||
SkillsInfo.Configure();
|
SkillsInfo.Configure();
|
||||||
|
|
||||||
|
// Seed the loop clock as Main.cs does before the Configure sweep; otherwise Core.Now is
|
||||||
|
// DateTime.MinValue for the whole test host.
|
||||||
|
Core._now = DateTime.UtcNow;
|
||||||
|
|
||||||
|
// Timer wheel must exist before NetState.Configure(), which schedules a recurring
|
||||||
|
// sweep via Timer.DelayCall (matches production ordering in Main.cs: Timer.Init runs
|
||||||
|
// before AssemblyHandler.Invoke("Configure")).
|
||||||
|
Timer.Init(0);
|
||||||
Server.Network.NetState.Configure();
|
Server.Network.NetState.Configure();
|
||||||
TestMapDefinitions.ConfigureTestMapDefinitions();
|
TestMapDefinitions.ConfigureTestMapDefinitions();
|
||||||
|
|
||||||
|
|
@ -91,7 +100,6 @@ internal static class TestServerInitializer
|
||||||
}
|
}
|
||||||
|
|
||||||
World.Configure();
|
World.Configure();
|
||||||
Timer.Init(0);
|
|
||||||
RaceDefinitions.Configure();
|
RaceDefinitions.Configure();
|
||||||
MovementImpl.Configure();
|
MovementImpl.Configure();
|
||||||
PathFollower.Configure();
|
PathFollower.Configure();
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,65 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: BlocklistConfigurationTests.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Text.Json;
|
||||||
|
using Server.Json;
|
||||||
|
using Server.Network.Bans;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests.Network.Bans.Blocklist;
|
||||||
|
|
||||||
|
public class BlocklistConfigurationTests
|
||||||
|
{
|
||||||
|
// Locks the JsonConfig casing contract: JsonConfig's options are case-SENSITIVE, so every settings
|
||||||
|
// member must carry an explicit [JsonPropertyName("camelCase")] or it silently binds nothing.
|
||||||
|
[Fact]
|
||||||
|
public void BlocklistSettings_RoundTripsThroughJsonConfig()
|
||||||
|
{
|
||||||
|
var original = new BlocklistSettings
|
||||||
|
{
|
||||||
|
File = "D:/shared/ip-blocklist.txt",
|
||||||
|
ReloadInterval = TimeSpan.FromMinutes(5),
|
||||||
|
ReportHits = false,
|
||||||
|
BanDuration = TimeSpan.FromHours(2),
|
||||||
|
PromoteSuppression = TimeSpan.FromSeconds(30)
|
||||||
|
};
|
||||||
|
|
||||||
|
var json = JsonConfig.Serialize(original);
|
||||||
|
|
||||||
|
Assert.Contains("\"file\"", json);
|
||||||
|
Assert.Contains("\"reloadInterval\"", json);
|
||||||
|
Assert.Contains("\"reportHits\"", json);
|
||||||
|
Assert.Contains("\"banDuration\"", json);
|
||||||
|
Assert.Contains("\"promoteSuppression\"", json);
|
||||||
|
|
||||||
|
var restored = JsonSerializer.Deserialize<BlocklistSettings>(json, JsonConfig.DefaultOptions);
|
||||||
|
|
||||||
|
Assert.NotNull(restored);
|
||||||
|
Assert.Equal(original.File, restored.File);
|
||||||
|
Assert.Equal(original.ReloadInterval, restored.ReloadInterval);
|
||||||
|
Assert.Equal(original.ReportHits, restored.ReportHits);
|
||||||
|
Assert.Equal(original.BanDuration, restored.BanDuration);
|
||||||
|
Assert.Equal(original.PromoteSuppression, restored.PromoteSuppression);
|
||||||
|
}
|
||||||
|
|
||||||
|
// The generator (tools/Export-IpBlocklist.ps1) writes to this path by default; if one side moves
|
||||||
|
// without the other, a shard silently enforces nothing.
|
||||||
|
[Fact]
|
||||||
|
public void Default_file_matches_the_generator_output_path()
|
||||||
|
{
|
||||||
|
Assert.Equal("Configuration/ip-blocklist.txt", new BlocklistSettings().File);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,85 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: BlocklistFileTests.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.IO;
|
||||||
|
using System.Net;
|
||||||
|
using Server.Network.Bans;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests.Network.Bans.Blocklist;
|
||||||
|
|
||||||
|
public class BlocklistFileTests
|
||||||
|
{
|
||||||
|
private static string WriteTemp(string content)
|
||||||
|
{
|
||||||
|
var p = Path.Combine(Path.GetTempPath(), "bl-" + Guid.NewGuid().ToString("N") + ".txt");
|
||||||
|
File.WriteAllText(p, content);
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Reads_header_generated_and_count()
|
||||||
|
{
|
||||||
|
var p = WriteTemp("# modernuo-blocklist v1 generated=2026-07-21T09:24:23Z count=2\n1.2.3.4\n5.6.7.0/24\n");
|
||||||
|
Assert.True(BlocklistFile.TryReadHeader(p, out var h));
|
||||||
|
Assert.True(h.Present);
|
||||||
|
Assert.Equal("2026-07-21T09:24:23Z", h.Generated);
|
||||||
|
Assert.Equal(2, h.Count);
|
||||||
|
File.Delete(p);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Missing_file_reports_absent_and_loads_empty()
|
||||||
|
{
|
||||||
|
var p = Path.Combine(Path.GetTempPath(), "does-not-exist-" + Guid.NewGuid().ToString("N"));
|
||||||
|
Assert.False(BlocklistFile.TryReadHeader(p, out var h));
|
||||||
|
Assert.False(h.Present);
|
||||||
|
var snap = BlocklistFile.Load(p, out _, out _);
|
||||||
|
Assert.False(snap.IsBanned(IPAddress.Parse("1.2.3.4")));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pins the exact line tools/Export-IpBlocklist.ps1 emits: the producer adds informational tokens the
|
||||||
|
// reader doesn't know about, and the reload detector breaks silently if generated= stops being read.
|
||||||
|
[Fact]
|
||||||
|
public void Reads_generator_header_with_extra_tokens()
|
||||||
|
{
|
||||||
|
var p = WriteTemp(
|
||||||
|
"# modernuo-blocklist generated=2026-07-25T18:03:11Z count=12442 ipv4=7868 cidr=4574 feeds=2\n" +
|
||||||
|
"2.181.183.77\n223.169.0.0/16\n"
|
||||||
|
);
|
||||||
|
|
||||||
|
Assert.True(BlocklistFile.TryReadHeader(p, out var h));
|
||||||
|
Assert.Equal("2026-07-25T18:03:11Z", h.Generated);
|
||||||
|
Assert.Equal(12442, h.Count);
|
||||||
|
|
||||||
|
var snap = BlocklistFile.Load(p, out var parsed, out var skipped);
|
||||||
|
Assert.Equal(2, parsed);
|
||||||
|
Assert.Equal(0, skipped);
|
||||||
|
Assert.True(snap.IsBanned(IPAddress.Parse("2.181.183.77")));
|
||||||
|
Assert.True(snap.IsBanned(IPAddress.Parse("223.169.4.9")));
|
||||||
|
File.Delete(p);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Load_parses_body()
|
||||||
|
{
|
||||||
|
var p = WriteTemp("# generated=x count=1\n8.8.8.0/24\n");
|
||||||
|
var snap = BlocklistFile.Load(p, out var parsed, out _);
|
||||||
|
Assert.Equal(1, parsed);
|
||||||
|
Assert.True(snap.IsBanned(IPAddress.Parse("8.8.8.8")));
|
||||||
|
File.Delete(p);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,94 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: BlocklistFilterTests.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System.Net;
|
||||||
|
using System.Text;
|
||||||
|
using Server.Network.Bans;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests.Network.Bans.Blocklist;
|
||||||
|
|
||||||
|
// No [Collection] and no static reset hook: the filter is an instance, so each test owns its own
|
||||||
|
// snapshot and promote-guard. That is the point of it no longer being a static class.
|
||||||
|
public class BlocklistFilterTests
|
||||||
|
{
|
||||||
|
private static BlocklistFilter WithList(string list, bool reportHits = true, long suppressionMs = 5000)
|
||||||
|
{
|
||||||
|
var filter = new BlocklistFilter();
|
||||||
|
filter.LoadForTesting(
|
||||||
|
BlocklistSnapshot.Build(Encoding.ASCII.GetBytes(list), out _, out _),
|
||||||
|
reportHits,
|
||||||
|
suppressionMs
|
||||||
|
);
|
||||||
|
|
||||||
|
return filter;
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Listed_address_is_denied_and_reported_once_per_window()
|
||||||
|
{
|
||||||
|
var filter = WithList("1.2.3.4");
|
||||||
|
var ip = IPAddress.Parse("1.2.3.4");
|
||||||
|
|
||||||
|
var deny1 = filter.Evaluate(ip, 1000, out var report1);
|
||||||
|
var deny2 = filter.Evaluate(ip, 1500, out var report2);
|
||||||
|
var deny3 = filter.Evaluate(ip, 1000 + 5001, out var report3);
|
||||||
|
|
||||||
|
Assert.True(deny1);
|
||||||
|
Assert.True(report1);
|
||||||
|
|
||||||
|
Assert.True(deny2);
|
||||||
|
Assert.False(report2); // denied again, but promotion suppressed inside the window
|
||||||
|
|
||||||
|
Assert.True(deny3);
|
||||||
|
Assert.True(report3); // window elapsed, promotion may be retried
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Unlisted_address_passes()
|
||||||
|
{
|
||||||
|
var filter = WithList("1.2.3.4");
|
||||||
|
|
||||||
|
Assert.False(filter.Evaluate(IPAddress.Parse("9.9.9.9"), 1, out var report));
|
||||||
|
Assert.False(report);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Cidr_membership_is_honored()
|
||||||
|
{
|
||||||
|
var filter = WithList("10.20.30.0/24");
|
||||||
|
|
||||||
|
Assert.True(filter.Evaluate(IPAddress.Parse("10.20.30.255"), 1, out _));
|
||||||
|
Assert.False(filter.Evaluate(IPAddress.Parse("10.20.31.0"), 1, out _));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void ReportHits_disabled_still_denies_but_never_promotes()
|
||||||
|
{
|
||||||
|
var filter = WithList("1.2.3.4", reportHits: false);
|
||||||
|
|
||||||
|
Assert.True(filter.Evaluate(IPAddress.Parse("1.2.3.4"), 1, out var report));
|
||||||
|
Assert.False(report);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Unconfigured_filter_denies_nothing()
|
||||||
|
{
|
||||||
|
var filter = new BlocklistFilter();
|
||||||
|
|
||||||
|
Assert.Equal(0, filter.Count);
|
||||||
|
Assert.False(filter.ShouldDeny(IPAddress.Parse("8.8.8.8")));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,98 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: BlocklistSnapshotTests.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System.Net;
|
||||||
|
using Server.Network.Bans;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests.Network.Bans.Blocklist;
|
||||||
|
|
||||||
|
public class BlocklistSnapshotTests
|
||||||
|
{
|
||||||
|
private static BlocklistSnapshot Build(params string[] lines) =>
|
||||||
|
BlocklistSnapshot.Build(System.Text.Encoding.ASCII.GetBytes(string.Join('\n', lines)), out _, out _);
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Single_ip_is_matched()
|
||||||
|
{
|
||||||
|
var s = Build("1.2.3.4");
|
||||||
|
Assert.True(s.IsBanned(IPAddress.Parse("1.2.3.4")));
|
||||||
|
Assert.False(s.IsBanned(IPAddress.Parse("1.2.3.5")));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Cidr_contains_and_excludes_boundaries()
|
||||||
|
{
|
||||||
|
var s = Build("10.0.0.0/24");
|
||||||
|
Assert.True(s.IsBanned(IPAddress.Parse("10.0.0.0")));
|
||||||
|
Assert.True(s.IsBanned(IPAddress.Parse("10.0.0.255")));
|
||||||
|
Assert.False(s.IsBanned(IPAddress.Parse("10.0.1.0")));
|
||||||
|
Assert.False(s.IsBanned(IPAddress.Parse("9.255.255.255")));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Comments_blanks_and_garbage_are_skipped_not_thrown()
|
||||||
|
{
|
||||||
|
var s = BlocklistSnapshot.Build(
|
||||||
|
System.Text.Encoding.ASCII.GetBytes(
|
||||||
|
string.Join('\n', "# header generated=x", "", "not-an-ip", "1.2.3.4", "::1", "5.6.7.0/24")),
|
||||||
|
out var parsed, out var skipped);
|
||||||
|
Assert.Equal(3, parsed); // 1.2.3.4 + ::1 (valid loopback) + 5.6.7.0/24
|
||||||
|
Assert.True(skipped >= 1); // "not-an-ip"; blank/comment lines are silently skipped, not counted
|
||||||
|
Assert.True(s.IsBanned(IPAddress.Parse("5.6.7.200")));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Empty_snapshot_matches_nothing()
|
||||||
|
{
|
||||||
|
Assert.False(BlocklistSnapshot.Empty.IsBanned(IPAddress.Parse("1.2.3.4")));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Ipv6_single_and_cidr_are_matched()
|
||||||
|
{
|
||||||
|
var s = Build("2001:db8::1", "2001:db8:1::/48");
|
||||||
|
Assert.True(s.IsBanned(IPAddress.Parse("2001:db8::1")));
|
||||||
|
Assert.True(s.IsBanned(IPAddress.Parse("2001:db8:1::abcd")));
|
||||||
|
Assert.False(s.IsBanned(IPAddress.Parse("2001:db8:2::1")));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Ipv4_mapped_ipv6_is_normalized_to_v4()
|
||||||
|
{
|
||||||
|
var s = Build("1.2.3.4");
|
||||||
|
Assert.True(s.IsBanned(IPAddress.Parse("::ffff:1.2.3.4"))); // must not bypass the v4 set
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Nested_cidr_intervals_are_coalesced()
|
||||||
|
{
|
||||||
|
// A /32 nested inside a /24: InRange's binary search only inspects the
|
||||||
|
// rightmost interval starting <= ip, so without coalescing an IP inside
|
||||||
|
// the /24 but outside the /32 would land on the /32 and wrongly pass.
|
||||||
|
var s = Build("10.0.0.0/24", "10.0.0.5/32");
|
||||||
|
Assert.True(s.IsBanned(IPAddress.Parse("10.0.0.100"))); // inside /24, outside /32
|
||||||
|
Assert.True(s.IsBanned(IPAddress.Parse("10.0.0.5"))); // the nested /32 itself
|
||||||
|
Assert.False(s.IsBanned(IPAddress.Parse("10.0.1.0"))); // genuinely outside both
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Overlapping_cidr_intervals_are_coalesced()
|
||||||
|
{
|
||||||
|
var s = Build("10.0.0.0/25", "10.0.0.64/25");
|
||||||
|
Assert.True(s.IsBanned(IPAddress.Parse("10.0.0.100"))); // covered by the second /25
|
||||||
|
Assert.False(s.IsBanned(IPAddress.Parse("10.0.0.200"))); // outside both
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,46 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: PromotedGuardTests.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using Server.Network.Bans;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests.Network.Bans.Blocklist;
|
||||||
|
|
||||||
|
public class PromotedGuardTests
|
||||||
|
{
|
||||||
|
[Fact]
|
||||||
|
public void First_mark_true_then_suppressed_until_ttl()
|
||||||
|
{
|
||||||
|
var g = new PromotedGuard();
|
||||||
|
Assert.True(g.TryMark((UInt128)42, 1000, 5000));
|
||||||
|
Assert.False(g.TryMark((UInt128)42, 2000, 5000)); // within TTL
|
||||||
|
Assert.True(g.TryMark((UInt128)42, 6001, 5000)); // expired → re-mark
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Sweep_removes_expired_entries_allowing_remark()
|
||||||
|
{
|
||||||
|
var g = new PromotedGuard();
|
||||||
|
Assert.True(g.TryMark((UInt128)7, 0, 1000));
|
||||||
|
Assert.False(g.TryMark((UInt128)7, 500, 1000)); // still within TTL
|
||||||
|
|
||||||
|
g.Sweep(500); // not yet expired, sweep should not remove it
|
||||||
|
Assert.False(g.TryMark((UInt128)7, 999, 1000));
|
||||||
|
|
||||||
|
g.Sweep(1001); // now expired, sweep removes it
|
||||||
|
Assert.True(g.TryMark((UInt128)7, 1002, 1000)); // fresh mark, not "still marked"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,51 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: CrowdSecAlertClientTests.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System.Net;
|
||||||
|
using Server.Network.Bans.CrowdSec;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests.Network.Bans;
|
||||||
|
|
||||||
|
public class CrowdSecAlertClientTests
|
||||||
|
{
|
||||||
|
[Fact]
|
||||||
|
public void BuildDeleteQuery_EscapesOrigin()
|
||||||
|
{
|
||||||
|
// origin is operator-controlled config (crowdsec.json), so it must be treated as untrusted
|
||||||
|
// input going into the URL, same as any other interpolated value.
|
||||||
|
var query = CrowdSecAlertClient.BuildDeleteQuery("modern uo/test&x=1", IPAddress.Parse("1.2.3.4"));
|
||||||
|
|
||||||
|
Assert.Equal("/v1/decisions?origin=modern%20uo%2Ftest%26x%3D1&ip=1.2.3.4", query);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void BuildDeleteQuery_PlainOrigin_Ipv4()
|
||||||
|
{
|
||||||
|
var query = CrowdSecAlertClient.BuildDeleteQuery("modernuo", IPAddress.Parse("192.168.1.1"));
|
||||||
|
|
||||||
|
Assert.Equal("/v1/decisions?origin=modernuo&ip=192.168.1.1", query);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void BuildDeleteQuery_Ipv6_IsEscaped()
|
||||||
|
{
|
||||||
|
// IPv6 textual form contains ':', which Uri.EscapeDataString percent-encodes like any other
|
||||||
|
// reserved character — confirms the escaping is applied uniformly, not just for IPv4.
|
||||||
|
var query = CrowdSecAlertClient.BuildDeleteQuery("modernuo", IPAddress.Parse("2001:db8::1"));
|
||||||
|
|
||||||
|
Assert.Equal("/v1/decisions?origin=modernuo&ip=2001%3Adb8%3A%3A1", query);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,82 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: CrowdSecConfigurationTests.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Text.Json;
|
||||||
|
using Server.Json;
|
||||||
|
using Server.Network.Bans.CrowdSec;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests.Network.Bans;
|
||||||
|
|
||||||
|
public class CrowdSecConfigurationTests
|
||||||
|
{
|
||||||
|
// Locks the JsonConfig casing/converter contract: JsonConfig's options are case-SENSITIVE, so
|
||||||
|
// every settings member must carry an explicit [JsonPropertyName("camelCase")] or it silently
|
||||||
|
// binds nothing. These tests round-trip through the exact options the loader uses.
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void CrowdSecSettings_RoundTripsThroughJsonConfig()
|
||||||
|
{
|
||||||
|
var original = new CrowdSecSettings
|
||||||
|
{
|
||||||
|
LapiUrl = "http://10.0.0.5:9090",
|
||||||
|
MachineId = "shard",
|
||||||
|
Password = "secret",
|
||||||
|
Origin = "modernuo",
|
||||||
|
ManualBanDuration = TimeSpan.FromHours(168),
|
||||||
|
FlushInterval = TimeSpan.FromSeconds(1),
|
||||||
|
MaxQueue = 10000
|
||||||
|
};
|
||||||
|
|
||||||
|
var json = JsonConfig.Serialize(original);
|
||||||
|
|
||||||
|
// camelCase property names must be present (not PascalCase) or the case-sensitive reader binds nothing.
|
||||||
|
Assert.Contains("\"lapiUrl\"", json);
|
||||||
|
Assert.Contains("\"machineId\"", json);
|
||||||
|
Assert.Contains("\"password\"", json);
|
||||||
|
Assert.Contains("\"origin\"", json);
|
||||||
|
Assert.Contains("\"manualBanDuration\"", json);
|
||||||
|
Assert.Contains("\"flushInterval\"", json);
|
||||||
|
Assert.Contains("\"maxQueue\"", json);
|
||||||
|
|
||||||
|
var restored = JsonSerializer.Deserialize<CrowdSecSettings>(json, JsonConfig.DefaultOptions);
|
||||||
|
|
||||||
|
Assert.NotNull(restored);
|
||||||
|
Assert.Equal(original.LapiUrl, restored.LapiUrl);
|
||||||
|
Assert.Equal(original.MachineId, restored.MachineId);
|
||||||
|
Assert.Equal(original.Password, restored.Password);
|
||||||
|
Assert.Equal(original.Origin, restored.Origin);
|
||||||
|
Assert.Equal(original.ManualBanDuration, restored.ManualBanDuration); // TimeSpan survives
|
||||||
|
Assert.Equal(original.FlushInterval, restored.FlushInterval); // TimeSpan survives
|
||||||
|
Assert.Equal(original.MaxQueue, restored.MaxQueue);
|
||||||
|
Assert.True(restored.ReportingEnabled);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void CrowdSecSettings_Defaults_AreReportingDisabledLocalLoopback()
|
||||||
|
{
|
||||||
|
var settings = new CrowdSecSettings();
|
||||||
|
|
||||||
|
Assert.Equal("http://127.0.0.1:8080", settings.LapiUrl);
|
||||||
|
Assert.Equal("", settings.MachineId);
|
||||||
|
Assert.Equal("", settings.Password);
|
||||||
|
Assert.Equal("modernuo", settings.Origin);
|
||||||
|
Assert.Equal(TimeSpan.FromHours(168), settings.ManualBanDuration);
|
||||||
|
Assert.Equal(TimeSpan.FromSeconds(1), settings.FlushInterval);
|
||||||
|
Assert.Equal(10000, settings.MaxQueue);
|
||||||
|
Assert.False(settings.ReportingEnabled); // empty machineId/password => inert
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,229 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: CrowdSecReporterTests.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.Net;
|
||||||
|
using System.Threading;
|
||||||
|
using System.Threading.Tasks;
|
||||||
|
using Server.Network.Bans.CrowdSec;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests.Network.Bans;
|
||||||
|
|
||||||
|
public class CrowdSecReporterTests
|
||||||
|
{
|
||||||
|
private static CrowdSecSettings Settings() => new()
|
||||||
|
{
|
||||||
|
MachineId = "shard",
|
||||||
|
Password = "secret",
|
||||||
|
Origin = "modernuo",
|
||||||
|
ManualBanDuration = TimeSpan.FromHours(168)
|
||||||
|
};
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void BuildAlerts_DedupsByIp()
|
||||||
|
{
|
||||||
|
var now = DateTime.UnixEpoch;
|
||||||
|
var items = new List<CrowdSecReporter.ReportItem>
|
||||||
|
{
|
||||||
|
new(IPAddress.Parse("1.1.1.1"), TimeSpan.FromHours(1), "rate-limit", false),
|
||||||
|
new(IPAddress.Parse("1.1.1.1"), TimeSpan.FromHours(1), "rate-limit", false),
|
||||||
|
new(IPAddress.Parse("2.2.2.2"), TimeSpan.FromHours(1), "rate-limit", false)
|
||||||
|
};
|
||||||
|
|
||||||
|
var alerts = CrowdSecReporter.BuildAlerts(items, Settings(), now);
|
||||||
|
|
||||||
|
Assert.Equal(2, alerts.Count);
|
||||||
|
Assert.All(alerts, a => Assert.Single(a.Decisions));
|
||||||
|
Assert.Contains(alerts, a => a.Source.Value == "1.1.1.1");
|
||||||
|
Assert.Contains(alerts, a => a.Source.Value == "2.2.2.2");
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void BuildAlerts_ScenarioFromReason_OriginFromSettings()
|
||||||
|
{
|
||||||
|
var alerts = CrowdSecReporter.BuildAlerts(
|
||||||
|
[new(IPAddress.Parse("3.3.3.3"), TimeSpan.FromHours(1), "manual", false)],
|
||||||
|
Settings(),
|
||||||
|
DateTime.UnixEpoch);
|
||||||
|
|
||||||
|
var decision = Assert.Single(alerts).Decisions[0];
|
||||||
|
Assert.Equal("modernuo/manual", alerts[0].Scenario);
|
||||||
|
Assert.Equal("modernuo", decision.Origin);
|
||||||
|
Assert.Equal("ban", decision.Type);
|
||||||
|
Assert.Equal("Ip", decision.Scope);
|
||||||
|
Assert.Equal("3.3.3.3", decision.Value);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void BuildAlerts_ScenarioFromReason_Blocklist()
|
||||||
|
{
|
||||||
|
var alerts = CrowdSecReporter.BuildAlerts(
|
||||||
|
[new(IPAddress.Parse("5.5.5.5"), TimeSpan.FromHours(1), "blocklist", false)],
|
||||||
|
Settings(),
|
||||||
|
DateTime.UnixEpoch);
|
||||||
|
|
||||||
|
var decision = Assert.Single(alerts).Decisions[0];
|
||||||
|
Assert.Equal("modernuo/blocklist", alerts[0].Scenario);
|
||||||
|
Assert.Equal("modernuo/blocklist", decision.Scenario);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void FormatDuration_UsesSeconds_FloorsAtOne()
|
||||||
|
{
|
||||||
|
Assert.Equal("3600s", CrowdSecReporter.FormatDuration(TimeSpan.FromHours(1)));
|
||||||
|
Assert.Equal("1s", CrowdSecReporter.FormatDuration(TimeSpan.Zero));
|
||||||
|
Assert.Equal("1s", CrowdSecReporter.FormatDuration(TimeSpan.FromMilliseconds(10)));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Report_WhenQueueFull_DropsAndCounts()
|
||||||
|
{
|
||||||
|
var reporter = new CrowdSecReporter(new NullAlertClient(), new CrowdSecSettings
|
||||||
|
{
|
||||||
|
MachineId = "shard",
|
||||||
|
Password = "secret",
|
||||||
|
MaxQueue = 2
|
||||||
|
});
|
||||||
|
// Do NOT Start() the drain — so the queue fills and overflows deterministically.
|
||||||
|
|
||||||
|
for (var i = 0; i < 10; i++)
|
||||||
|
{
|
||||||
|
reporter.Report(IPAddress.Parse("4.4.4." + i), TimeSpan.FromHours(1), "rate-limit");
|
||||||
|
}
|
||||||
|
|
||||||
|
Assert.True(reporter.DroppedCount >= 8);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Stop() without a prior Start() drives FlushRemainingOnStop() synchronously (no drain task, no
|
||||||
|
// Task.Delay backoff involved), so this is deterministic — no wall-clock timing dependency.
|
||||||
|
[Fact]
|
||||||
|
public void Stop_FlushesQueuedReports_ViaClient()
|
||||||
|
{
|
||||||
|
var client = new RecordingAlertClient();
|
||||||
|
var reporter = new CrowdSecReporter(client, Settings());
|
||||||
|
|
||||||
|
reporter.Report(IPAddress.Parse("6.6.6.6"), TimeSpan.FromHours(1), "rate-limit");
|
||||||
|
reporter.Stop();
|
||||||
|
|
||||||
|
var posted = Assert.Single(client.Posted);
|
||||||
|
Assert.Equal("6.6.6.6", Assert.Single(posted).Source.Value);
|
||||||
|
Assert.Equal(0, reporter.SendFailureCount);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Stop_FlushesQueuedRetracts_ViaClient()
|
||||||
|
{
|
||||||
|
var client = new RecordingAlertClient();
|
||||||
|
var reporter = new CrowdSecReporter(client, Settings());
|
||||||
|
|
||||||
|
reporter.Retract(IPAddress.Parse("8.8.8.8"));
|
||||||
|
reporter.Stop();
|
||||||
|
|
||||||
|
Assert.Equal(IPAddress.Parse("8.8.8.8"), Assert.Single(client.Deleted));
|
||||||
|
Assert.Equal(0, reporter.SendFailureCount);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Stop_WhenFlushSendFails_CountsSendFailure()
|
||||||
|
{
|
||||||
|
var reporter = new CrowdSecReporter(new ThrowingAlertClient(), Settings());
|
||||||
|
|
||||||
|
reporter.Report(IPAddress.Parse("7.7.7.7"), TimeSpan.FromHours(1), "rate-limit");
|
||||||
|
reporter.Stop();
|
||||||
|
|
||||||
|
Assert.Equal(1, reporter.SendFailureCount);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Stop_WithEmptyQueue_DoesNotInvokeClientOrFail()
|
||||||
|
{
|
||||||
|
var client = new RecordingAlertClient();
|
||||||
|
var reporter = new CrowdSecReporter(client, Settings());
|
||||||
|
|
||||||
|
reporter.Stop();
|
||||||
|
|
||||||
|
Assert.Empty(client.Posted);
|
||||||
|
Assert.Equal(0, reporter.SendFailureCount);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// The drain task must track the loop's lifetime, not just its first await — a ValueTask-returning
|
||||||
|
/// drain loop passed to Task.Run yields a Task<ValueTask> that completes immediately, which makes
|
||||||
|
/// Stop()'s drain-exited handshake a no-op. With an empty queue the loop parks on WaitToReadAsync,
|
||||||
|
/// so a correctly unwrapped task cannot win this race; a slow pool only under-detects.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Start_DrainTaskSpansLoopLifetime_NotJustTheFirstAwait()
|
||||||
|
{
|
||||||
|
var reporter = new CrowdSecReporter(new NullAlertClient(), Settings());
|
||||||
|
using var cts = new CancellationTokenSource();
|
||||||
|
|
||||||
|
reporter.Start(cts.Token);
|
||||||
|
|
||||||
|
var drain = reporter.DrainTaskForTesting;
|
||||||
|
Assert.NotNull(drain);
|
||||||
|
|
||||||
|
var first = await Task.WhenAny(drain, Task.Delay(TimeSpan.FromMilliseconds(500)));
|
||||||
|
Assert.False(ReferenceEquals(first, drain), "drain task completed while the loop was still running");
|
||||||
|
|
||||||
|
reporter.Stop();
|
||||||
|
|
||||||
|
Assert.True(drain.IsCompleted, "Stop() returned before the drain loop exited");
|
||||||
|
}
|
||||||
|
|
||||||
|
private sealed class NullAlertClient : ICrowdSecAlertClient
|
||||||
|
{
|
||||||
|
public ValueTask PostAlertsAsync(IReadOnlyList<CrowdSecAlert> alerts, CancellationToken token) =>
|
||||||
|
ValueTask.CompletedTask;
|
||||||
|
|
||||||
|
public ValueTask DeleteDecisionsAsync(string origin, IPAddress ip, CancellationToken token) =>
|
||||||
|
ValueTask.CompletedTask;
|
||||||
|
|
||||||
|
public void Dispose() { }
|
||||||
|
}
|
||||||
|
|
||||||
|
private sealed class RecordingAlertClient : ICrowdSecAlertClient
|
||||||
|
{
|
||||||
|
public List<IReadOnlyList<CrowdSecAlert>> Posted { get; } = [];
|
||||||
|
public List<IPAddress> Deleted { get; } = [];
|
||||||
|
|
||||||
|
public ValueTask PostAlertsAsync(IReadOnlyList<CrowdSecAlert> alerts, CancellationToken token)
|
||||||
|
{
|
||||||
|
Posted.Add(alerts);
|
||||||
|
return ValueTask.CompletedTask;
|
||||||
|
}
|
||||||
|
|
||||||
|
public ValueTask DeleteDecisionsAsync(string origin, IPAddress ip, CancellationToken token)
|
||||||
|
{
|
||||||
|
Deleted.Add(ip);
|
||||||
|
return ValueTask.CompletedTask;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Dispose() { }
|
||||||
|
}
|
||||||
|
|
||||||
|
private sealed class ThrowingAlertClient : ICrowdSecAlertClient
|
||||||
|
{
|
||||||
|
public ValueTask PostAlertsAsync(IReadOnlyList<CrowdSecAlert> alerts, CancellationToken token) =>
|
||||||
|
throw new InvalidOperationException("simulated LAPI outage");
|
||||||
|
|
||||||
|
public ValueTask DeleteDecisionsAsync(string origin, IPAddress ip, CancellationToken token) =>
|
||||||
|
ValueTask.CompletedTask;
|
||||||
|
|
||||||
|
public void Dispose() { }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,72 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: FirewallPersistenceTests.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Net;
|
||||||
|
using Server.Network;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests.Network.Firewall;
|
||||||
|
|
||||||
|
[Collection("Sequential UOContent Tests")]
|
||||||
|
public class FirewallPersistenceTests
|
||||||
|
{
|
||||||
|
[Fact]
|
||||||
|
public void ToSettings_RoundTrips_PermanentAndTtl()
|
||||||
|
{
|
||||||
|
Server.Network.Firewall.ResetForTesting();
|
||||||
|
Server.Network.Firewall.Add(new SingleIpFirewallEntry(IPAddress.Parse("1.2.3.4")));
|
||||||
|
Server.Network.Firewall.Add(new SingleIpFirewallEntry(IPAddress.Parse("2.2.2.2")), TimeSpan.FromHours(1));
|
||||||
|
|
||||||
|
var settings = Server.Network.Firewall.ToSettings();
|
||||||
|
|
||||||
|
Server.Network.Firewall.ResetForTesting();
|
||||||
|
Server.Network.Firewall.LoadFrom(settings);
|
||||||
|
|
||||||
|
Assert.True(Server.Network.Firewall.IsBlocked(IPAddress.Parse("1.2.3.4")));
|
||||||
|
Assert.True(Server.Network.Firewall.IsBlocked(IPAddress.Parse("2.2.2.2")));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void LoadFrom_SkipsAlreadyExpired()
|
||||||
|
{
|
||||||
|
Server.Network.Firewall.ResetForTesting();
|
||||||
|
var settings = new FirewallSettings
|
||||||
|
{
|
||||||
|
Entries =
|
||||||
|
[
|
||||||
|
// Core.Now, matching the clock LoadFrom compares against.
|
||||||
|
new FirewallEntryRecord { Value = "9.9.9.9", Expires = Core.Now.AddHours(-1) }
|
||||||
|
]
|
||||||
|
};
|
||||||
|
|
||||||
|
Server.Network.Firewall.LoadFrom(settings);
|
||||||
|
|
||||||
|
Assert.False(Server.Network.Firewall.IsBlocked(IPAddress.Parse("9.9.9.9")));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void ToSettings_OmitsExpiryForPermanent()
|
||||||
|
{
|
||||||
|
Server.Network.Firewall.ResetForTesting();
|
||||||
|
Server.Network.Firewall.Add(new SingleIpFirewallEntry(IPAddress.Parse("1.2.3.4")));
|
||||||
|
|
||||||
|
var settings = Server.Network.Firewall.ToSettings();
|
||||||
|
|
||||||
|
Assert.Single(settings.Entries);
|
||||||
|
Assert.Null(settings.Entries[0].Expires);
|
||||||
|
Assert.Equal("1.2.3.4", settings.Entries[0].Value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,89 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: FirewallTests.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Net;
|
||||||
|
using Server.Network;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace Server.Tests.Network.Firewall;
|
||||||
|
|
||||||
|
[Collection("Sequential UOContent Tests")]
|
||||||
|
public class FirewallTests
|
||||||
|
{
|
||||||
|
private static IPAddress Ip(string s) => IPAddress.Parse(s);
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Add_ThenIsBlocked_SingleIp()
|
||||||
|
{
|
||||||
|
Server.Network.Firewall.ResetForTesting();
|
||||||
|
Assert.True(Server.Network.Firewall.Add(new SingleIpFirewallEntry(Ip("1.2.3.4"))));
|
||||||
|
Assert.True(Server.Network.Firewall.IsBlocked(Ip("1.2.3.4")));
|
||||||
|
Assert.False(Server.Network.Firewall.IsBlocked(Ip("1.2.3.5")));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Add_Range_BlocksInside_NotOutside()
|
||||||
|
{
|
||||||
|
Server.Network.Firewall.ResetForTesting();
|
||||||
|
Server.Network.Firewall.Add(new CidrFirewallEntry(Ip("10.0.0.0"), Ip("10.0.0.255")));
|
||||||
|
Assert.True(Server.Network.Firewall.IsBlocked(Ip("10.0.0.7")));
|
||||||
|
Assert.False(Server.Network.Firewall.IsBlocked(Ip("10.0.1.0")));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Remove_Unblocks()
|
||||||
|
{
|
||||||
|
Server.Network.Firewall.ResetForTesting();
|
||||||
|
var entry = new SingleIpFirewallEntry(Ip("1.2.3.4"));
|
||||||
|
Server.Network.Firewall.Add(entry);
|
||||||
|
Assert.True(Server.Network.Firewall.Remove(entry));
|
||||||
|
Assert.False(Server.Network.Firewall.IsBlocked(Ip("1.2.3.4")));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void ExpireEntries_RemovesExpired_KeepsPermanent()
|
||||||
|
{
|
||||||
|
Server.Network.Firewall.ResetForTesting();
|
||||||
|
var permanent = new SingleIpFirewallEntry(Ip("1.1.1.1"));
|
||||||
|
var temporary = new SingleIpFirewallEntry(Ip("2.2.2.2"));
|
||||||
|
Server.Network.Firewall.Add(permanent); // no ttl
|
||||||
|
Server.Network.Firewall.Add(temporary, TimeSpan.FromMilliseconds(50)); // ttl
|
||||||
|
|
||||||
|
Server.Network.Firewall.ExpireEntries(Core.TickCount + 100); // past the ttl
|
||||||
|
|
||||||
|
Assert.True(Server.Network.Firewall.IsBlocked(Ip("1.1.1.1")));
|
||||||
|
Assert.False(Server.Network.Firewall.IsBlocked(Ip("2.2.2.2")));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void ToFirewallEntry_ParsesForms()
|
||||||
|
{
|
||||||
|
Assert.IsType<SingleIpFirewallEntry>(Server.Network.Firewall.ToFirewallEntry("1.2.3.4"));
|
||||||
|
Assert.IsType<CidrFirewallEntry>(Server.Network.Firewall.ToFirewallEntry("10.0.0.0/24"));
|
||||||
|
Assert.IsType<CidrFirewallEntry>(Server.Network.Firewall.ToFirewallEntry("10.0.0.0-10.0.0.255"));
|
||||||
|
Assert.Null(Server.Network.Firewall.ToFirewallEntry("not-an-ip"));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void ReadFirewallSet_SurfacesAddedEntries()
|
||||||
|
{
|
||||||
|
Server.Network.Firewall.ResetForTesting();
|
||||||
|
var entry = new SingleIpFirewallEntry(Ip("1.2.3.4"));
|
||||||
|
Server.Network.Firewall.Add(entry);
|
||||||
|
|
||||||
|
Server.Network.Firewall.ReadFirewallSet(set => Assert.Contains(entry, set));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -9,6 +9,7 @@ using Server.Items;
|
||||||
using Server.Mobiles;
|
using Server.Mobiles;
|
||||||
using Server.Multis;
|
using Server.Multis;
|
||||||
using Server.Network;
|
using Server.Network;
|
||||||
|
using Server.Network.Bans;
|
||||||
using Server.Spells;
|
using Server.Spells;
|
||||||
|
|
||||||
namespace Server.Commands.Generic
|
namespace Server.Commands.Generic
|
||||||
|
|
@ -1154,7 +1155,8 @@ namespace Server.Commands.Generic
|
||||||
|
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
AdminFirewall.Add(state.Address);
|
Firewall.Add(new SingleIpFirewallEntry(state.Address));
|
||||||
|
BanChannel.Report(state.Address, TimeSpan.Zero, "manual");
|
||||||
AddResponse("They have been firewalled.");
|
AddResponse("They have been firewalled.");
|
||||||
}
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex)
|
||||||
|
|
|
||||||
|
|
@ -11,6 +11,7 @@ using Server.Maps;
|
||||||
using Server.Misc;
|
using Server.Misc;
|
||||||
using Server.Multis;
|
using Server.Multis;
|
||||||
using Server.Network;
|
using Server.Network;
|
||||||
|
using Server.Network.Bans;
|
||||||
using Server.Prompts;
|
using Server.Prompts;
|
||||||
using Server.Saves;
|
using Server.Saves;
|
||||||
using Server.Text;
|
using Server.Text;
|
||||||
|
|
@ -1743,7 +1744,8 @@ namespace Server.Gumps
|
||||||
{
|
{
|
||||||
for (var i = 0; i < a.LoginIPs.Length; ++i)
|
for (var i = 0; i < a.LoginIPs.Length; ++i)
|
||||||
{
|
{
|
||||||
AdminFirewall.Add(a.LoginIPs[i]);
|
Firewall.Add(new SingleIpFirewallEntry(a.LoginIPs[i]));
|
||||||
|
BanChannel.Report(a.LoginIPs[i], TimeSpan.Zero, "manual");
|
||||||
}
|
}
|
||||||
|
|
||||||
notice = "All addresses in the list have been firewalled.";
|
notice = "All addresses in the list have been firewalled.";
|
||||||
|
|
@ -1767,7 +1769,13 @@ namespace Server.Gumps
|
||||||
|
|
||||||
if (okay)
|
if (okay)
|
||||||
{
|
{
|
||||||
AdminFirewall.Add(toFirewall);
|
var firewallEntry = Firewall.ToFirewallEntry(toFirewall);
|
||||||
|
Firewall.Add(firewallEntry);
|
||||||
|
|
||||||
|
if (firewallEntry.MinIpAddress == firewallEntry.MaxIpAddress)
|
||||||
|
{
|
||||||
|
BanChannel.Report(firewallEntry.MinIpAddress.ToIpAddress(), TimeSpan.Zero, "manual");
|
||||||
|
}
|
||||||
|
|
||||||
notice = $"{toFirewall} : Added to firewall.";
|
notice = $"{toFirewall} : Added to firewall.";
|
||||||
}
|
}
|
||||||
|
|
@ -3559,7 +3567,7 @@ namespace Server.Gumps
|
||||||
IFirewallEntry firewallEntry;
|
IFirewallEntry firewallEntry;
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
firewallEntry = AdminFirewall.ToFirewallEntry(text);
|
firewallEntry = Firewall.ToFirewallEntry(text);
|
||||||
}
|
}
|
||||||
catch
|
catch
|
||||||
{
|
{
|
||||||
|
|
@ -3581,7 +3589,17 @@ namespace Server.Gumps
|
||||||
$"{from.AccessLevel} {CommandLogging.Format(from)} firewalling {firewallEntry}"
|
$"{from.AccessLevel} {CommandLogging.Format(from)} firewalling {firewallEntry}"
|
||||||
);
|
);
|
||||||
|
|
||||||
AdminFirewall.Add(firewallEntry);
|
Firewall.Add(firewallEntry);
|
||||||
|
|
||||||
|
if (firewallEntry.MinIpAddress == firewallEntry.MaxIpAddress)
|
||||||
|
{
|
||||||
|
BanChannel.Report(
|
||||||
|
firewallEntry.MinIpAddress.ToIpAddress(),
|
||||||
|
TimeSpan.Zero,
|
||||||
|
"manual"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
from.SendGump(
|
from.SendGump(
|
||||||
new AdminGump(
|
new AdminGump(
|
||||||
from,
|
from,
|
||||||
|
|
@ -3620,7 +3638,13 @@ namespace Server.Gumps
|
||||||
$"{from.AccessLevel} {CommandLogging.Format(from)} removing {m_State} from firewall list"
|
$"{from.AccessLevel} {CommandLogging.Format(from)} removing {m_State} from firewall list"
|
||||||
);
|
);
|
||||||
|
|
||||||
AdminFirewall.Remove(m_State);
|
Firewall.Remove(m_State as IFirewallEntry);
|
||||||
|
|
||||||
|
if (m_State is IFirewallEntry fe && fe.MinIpAddress == fe.MaxIpAddress)
|
||||||
|
{
|
||||||
|
BanChannel.Retract(fe.MinIpAddress.ToIpAddress());
|
||||||
|
}
|
||||||
|
|
||||||
from.SendGump(
|
from.SendGump(
|
||||||
new AdminGump(
|
new AdminGump(
|
||||||
from,
|
from,
|
||||||
|
|
|
||||||
|
|
@ -1,139 +0,0 @@
|
||||||
using System;
|
|
||||||
using System.Buffers;
|
|
||||||
using System.IO;
|
|
||||||
using System.Net;
|
|
||||||
using System.Runtime.CompilerServices;
|
|
||||||
using Server.Logging;
|
|
||||||
using Server.Network;
|
|
||||||
|
|
||||||
namespace Server;
|
|
||||||
|
|
||||||
public static class AdminFirewall
|
|
||||||
{
|
|
||||||
private static readonly ILogger logger = LogFactory.GetLogger(typeof(AdminFirewall));
|
|
||||||
|
|
||||||
private const string firewallConfigPath = "firewall.cfg";
|
|
||||||
|
|
||||||
public static void Configure()
|
|
||||||
{
|
|
||||||
if (File.Exists(firewallConfigPath))
|
|
||||||
{
|
|
||||||
var searchValues = SearchValues.Create("*Xx?");
|
|
||||||
|
|
||||||
using var ip = new StreamReader(firewallConfigPath);
|
|
||||||
|
|
||||||
while (ip.ReadLine() is { } line)
|
|
||||||
{
|
|
||||||
line = line.Trim();
|
|
||||||
|
|
||||||
if (line.Length == 0)
|
|
||||||
{
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (line.AsSpan().ContainsAny(searchValues))
|
|
||||||
{
|
|
||||||
logger.Warning("Legacy firewall entry \"{Entry}\" ignored", line);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
Add(ToFirewallEntry(line), false);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
[MethodImpl(MethodImplOptions.AggressiveInlining)]
|
|
||||||
public static IFirewallEntry ToFirewallEntry(object entry)
|
|
||||||
{
|
|
||||||
return entry switch
|
|
||||||
{
|
|
||||||
IFirewallEntry firewallEntry => firewallEntry,
|
|
||||||
IPAddress address => new SingleIpFirewallEntry(address),
|
|
||||||
string s => ToFirewallEntry(s),
|
|
||||||
_ => null
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
public static IFirewallEntry ToFirewallEntry(string entry)
|
|
||||||
{
|
|
||||||
if (entry == null)
|
|
||||||
{
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
|
|
||||||
try
|
|
||||||
{
|
|
||||||
var rangeSeparator = entry.IndexOf('-');
|
|
||||||
if (rangeSeparator > -1)
|
|
||||||
{
|
|
||||||
return new CidrFirewallEntry(
|
|
||||||
IPAddress.Parse(entry.AsSpan(0, rangeSeparator)),
|
|
||||||
IPAddress.Parse(entry.AsSpan(rangeSeparator + 1))
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
// CIDR notation
|
|
||||||
if (entry.IndexOf('/') > -1)
|
|
||||||
{
|
|
||||||
return new CidrFirewallEntry(entry);
|
|
||||||
}
|
|
||||||
|
|
||||||
return new SingleIpFirewallEntry(entry);
|
|
||||||
}
|
|
||||||
catch
|
|
||||||
{
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
public static bool Remove(object obj, bool save = true)
|
|
||||||
{
|
|
||||||
var entry = ToFirewallEntry(obj);
|
|
||||||
|
|
||||||
if (entry == null)
|
|
||||||
{
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (!Firewall.Remove(entry))
|
|
||||||
{
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (save)
|
|
||||||
{
|
|
||||||
Save();
|
|
||||||
}
|
|
||||||
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
public static void Add(object obj) => Add(ToFirewallEntry(obj));
|
|
||||||
|
|
||||||
public static bool Add(IFirewallEntry entry, bool save = true)
|
|
||||||
{
|
|
||||||
if (!Firewall.Add(entry))
|
|
||||||
{
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (save)
|
|
||||||
{
|
|
||||||
Save();
|
|
||||||
}
|
|
||||||
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
public static void Save()
|
|
||||||
{
|
|
||||||
Firewall.ReadFirewallSet(firewallSet =>
|
|
||||||
{
|
|
||||||
using var op = new StreamWriter(firewallConfigPath);
|
|
||||||
foreach (var entry in firewallSet)
|
|
||||||
{
|
|
||||||
op.WriteLine(entry);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
90
Projects/UOContent/Misc/Blocklist/BlocklistConfiguration.cs
Normal file
90
Projects/UOContent/Misc/Blocklist/BlocklistConfiguration.cs
Normal file
|
|
@ -0,0 +1,90 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: BlocklistConfiguration.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.IO;
|
||||||
|
using System.Text.Json.Serialization;
|
||||||
|
using Server.Json;
|
||||||
|
|
||||||
|
namespace Server.Network.Bans;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Loads the <see cref="BlocklistSettings"/> from <c>Configuration/blocklist.json</c> (matching the
|
||||||
|
/// per-feature JSON config pattern used by <c>AssistantConfiguration</c>). Loaded once; a missing file
|
||||||
|
/// writes a template so operators have something to edit.
|
||||||
|
/// </summary>
|
||||||
|
public static class BlocklistConfiguration
|
||||||
|
{
|
||||||
|
private const string _path = "Configuration/blocklist.json";
|
||||||
|
|
||||||
|
public static BlocklistSettings Settings { get; private set; }
|
||||||
|
|
||||||
|
public static void Load()
|
||||||
|
{
|
||||||
|
var path = Path.Join(Core.BaseDirectory, _path);
|
||||||
|
|
||||||
|
if (File.Exists(path))
|
||||||
|
{
|
||||||
|
Settings = JsonConfig.Deserialize<BlocklistSettings>(path);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
Settings = new BlocklistSettings();
|
||||||
|
Save();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void Save()
|
||||||
|
{
|
||||||
|
JsonConfig.Serialize(Path.Join(Core.BaseDirectory, _path), Settings);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Bound configuration for <see cref="BlocklistFilter"/>. The filter is inert unless <see cref="File"/>
|
||||||
|
/// points at a list that actually exists, so the shipped defaults are safe on a shard that never runs
|
||||||
|
/// the generator.
|
||||||
|
/// </summary>
|
||||||
|
public record BlocklistSettings
|
||||||
|
{
|
||||||
|
/// <summary>
|
||||||
|
/// Path to the blocklist. A relative path resolves against <see cref="Core.BaseDirectory"/>; an
|
||||||
|
/// absolute path is used as-is (handy when several shards share one generated list). Set to
|
||||||
|
/// <c>""</c> to disable the gate entirely. Produce the file with <c>tools/Export-IpBlocklist.ps1</c>.
|
||||||
|
/// </summary>
|
||||||
|
[JsonPropertyName("file")]
|
||||||
|
public string File { get; set; } = "Configuration/ip-blocklist.txt";
|
||||||
|
|
||||||
|
/// <summary>How often the file is checked for changes. Reloads only happen when it actually changed.</summary>
|
||||||
|
[JsonPropertyName("reloadInterval")]
|
||||||
|
public TimeSpan ReloadInterval { get; set; } = TimeSpan.FromSeconds(60);
|
||||||
|
|
||||||
|
/// <summary>Whether blocklist hits are contributed to the ban channel (the demand-paging promotion).</summary>
|
||||||
|
[JsonPropertyName("reportHits")]
|
||||||
|
public bool ReportHits { get; set; } = true;
|
||||||
|
|
||||||
|
/// <summary>Duration reported for a blocklist-matched ban.</summary>
|
||||||
|
[JsonPropertyName("banDuration")]
|
||||||
|
public TimeSpan BanDuration { get; set; } = TimeSpan.FromHours(6);
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// How long the accept-path guard suppresses re-reporting a promoted address. This only needs to
|
||||||
|
/// bridge the gap until the OS bouncer picks up the promotion (seconds); after that the kernel drops
|
||||||
|
/// repeat traffic. Decoupled from <see cref="BanDuration"/> so the guard doesn't have to remember
|
||||||
|
/// hours' worth of distinct addresses.
|
||||||
|
/// </summary>
|
||||||
|
[JsonPropertyName("promoteSuppression")]
|
||||||
|
public TimeSpan PromoteSuppression { get; set; } = TimeSpan.FromSeconds(60);
|
||||||
|
}
|
||||||
86
Projects/UOContent/Misc/Blocklist/BlocklistFile.cs
Normal file
86
Projects/UOContent/Misc/Blocklist/BlocklistFile.cs
Normal file
|
|
@ -0,0 +1,86 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: BlocklistFile.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.IO;
|
||||||
|
|
||||||
|
namespace Server.Network.Bans;
|
||||||
|
|
||||||
|
public readonly record struct BlocklistHeader(string Generated, int Count, bool Present);
|
||||||
|
|
||||||
|
/// <summary>Reads the versioned blocklist file: cheap header probe + full snapshot load.</summary>
|
||||||
|
public static class BlocklistFile
|
||||||
|
{
|
||||||
|
public static bool TryReadHeader(string path, out BlocklistHeader header)
|
||||||
|
{
|
||||||
|
header = new BlocklistHeader(null, 0, false);
|
||||||
|
try
|
||||||
|
{
|
||||||
|
if (!File.Exists(path))
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
using var reader = new StreamReader(path);
|
||||||
|
var first = reader.ReadLine();
|
||||||
|
if (first == null)
|
||||||
|
{
|
||||||
|
header = new BlocklistHeader(null, 0, true);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
string generated = null;
|
||||||
|
var count = 0;
|
||||||
|
if (first.StartsWith('#'))
|
||||||
|
{
|
||||||
|
var tokens = first.Split(' ', StringSplitOptions.RemoveEmptyEntries);
|
||||||
|
for (var i = 0; i < tokens.Length; i++)
|
||||||
|
{
|
||||||
|
var tok = tokens[i];
|
||||||
|
if (tok.StartsWith("generated=", StringComparison.Ordinal))
|
||||||
|
{
|
||||||
|
generated = tok["generated=".Length..];
|
||||||
|
}
|
||||||
|
else if (tok.StartsWith("count=", StringComparison.Ordinal))
|
||||||
|
{
|
||||||
|
int.TryParse(tok["count=".Length..], out count);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
header = new BlocklistHeader(generated, count, true);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
return false; // treat as absent; caller keeps last-good / empty
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public static BlocklistSnapshot Load(string path, out int parsed, out int skipped)
|
||||||
|
{
|
||||||
|
parsed = 0;
|
||||||
|
skipped = 0;
|
||||||
|
try
|
||||||
|
{
|
||||||
|
if (!File.Exists(path))
|
||||||
|
{
|
||||||
|
return BlocklistSnapshot.Empty;
|
||||||
|
}
|
||||||
|
return BlocklistSnapshot.Build(File.ReadAllBytes(path), out parsed, out skipped);
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
return BlocklistSnapshot.Empty;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
259
Projects/UOContent/Misc/Blocklist/BlocklistFilter.cs
Normal file
259
Projects/UOContent/Misc/Blocklist/BlocklistFilter.cs
Normal file
|
|
@ -0,0 +1,259 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: BlocklistFilter.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.IO;
|
||||||
|
using System.Net;
|
||||||
|
using System.Threading;
|
||||||
|
using System.Threading.Tasks;
|
||||||
|
using Server.Logging;
|
||||||
|
|
||||||
|
namespace Server.Network.Bans;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Accept-path gate for a large, file-sourced IP blocklist, hydrated from the file a generator
|
||||||
|
/// (<c>tools/Export-IpBlocklist.ps1</c>) writes on a schedule. Holds an immutable snapshot swapped
|
||||||
|
/// atomically by an off-loop reload poll, so accept-path reads are lock-free. Inert when no file is
|
||||||
|
/// configured or present.
|
||||||
|
/// </summary>
|
||||||
|
/// <remarks>
|
||||||
|
/// This is the demand-paging half of the design: an OS firewall cannot hold millions of entries on
|
||||||
|
/// Windows, so the millions live here and only addresses that actually connect are promoted to CrowdSec
|
||||||
|
/// (and from there to the OS firewall) through <see cref="BanChannel"/>. <see cref="PromotedGuard"/>
|
||||||
|
/// keeps a flood of repeat connections from re-reporting the same address before the bouncer picks it up.
|
||||||
|
/// </remarks>
|
||||||
|
public sealed class BlocklistFilter : IConnectionFilter
|
||||||
|
{
|
||||||
|
private static readonly ILogger logger = LogFactory.GetLogger(typeof(BlocklistFilter));
|
||||||
|
|
||||||
|
// Written by the reload poll (off-loop), read by the accept path (game loop): a single volatile
|
||||||
|
// reference swap is the whole synchronization story — readers see the old or the new snapshot, whole.
|
||||||
|
private volatile BlocklistSnapshot _snapshot = BlocklistSnapshot.Empty;
|
||||||
|
|
||||||
|
private readonly PromotedGuard _guard = new();
|
||||||
|
|
||||||
|
private string _path;
|
||||||
|
private TimeSpan _interval;
|
||||||
|
private bool _reportHits;
|
||||||
|
private TimeSpan _banDuration;
|
||||||
|
private long _suppressionMs;
|
||||||
|
private string _lastGenerated;
|
||||||
|
private DateTime _lastWriteUtc;
|
||||||
|
private CancellationTokenSource _cts;
|
||||||
|
|
||||||
|
public string Name => "blocklist";
|
||||||
|
|
||||||
|
public int Count => _snapshot.Count;
|
||||||
|
|
||||||
|
public static void Configure()
|
||||||
|
{
|
||||||
|
ConnectionFilters.Register(new BlocklistFilter());
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Register()
|
||||||
|
{
|
||||||
|
BlocklistConfiguration.Load();
|
||||||
|
var s = BlocklistConfiguration.Settings;
|
||||||
|
|
||||||
|
_path = ResolvePath(s.File);
|
||||||
|
_interval = s.ReloadInterval <= TimeSpan.Zero ? TimeSpan.FromSeconds(60) : s.ReloadInterval;
|
||||||
|
_reportHits = s.ReportHits;
|
||||||
|
_banDuration = s.BanDuration;
|
||||||
|
_suppressionMs = (long)s.PromoteSuppression.TotalMilliseconds;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Resolves the configured path once: a relative path is anchored to <see cref="Core.BaseDirectory"/>
|
||||||
|
/// (never the process working directory, which differs when the shard is launched from elsewhere), an
|
||||||
|
/// absolute path is taken as-is so several shards can share one generated list.
|
||||||
|
/// </summary>
|
||||||
|
private static string ResolvePath(string configured)
|
||||||
|
{
|
||||||
|
if (string.IsNullOrWhiteSpace(configured))
|
||||||
|
{
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
return Path.IsPathRooted(configured) ? configured : Path.Join(Core.BaseDirectory, configured);
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Start(CancellationToken token)
|
||||||
|
{
|
||||||
|
if (_path == null)
|
||||||
|
{
|
||||||
|
logger.Information("Blocklist disabled (\"file\" empty in blocklist.json)");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
_cts = CancellationTokenSource.CreateLinkedTokenSource(token);
|
||||||
|
|
||||||
|
// A missing file is the shipped default, not an error: the gate stays inert until the poll picks
|
||||||
|
// up whatever the generator first writes. No restart needed.
|
||||||
|
if (File.Exists(_path))
|
||||||
|
{
|
||||||
|
Reload(); // synchronous prime; empty on failure (fail-open)
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
logger.Information("Blocklist inert: no list at \"{Path}\"; polling every {Interval}", _path, _interval);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sweep the promote-guard so a distinct-IP flood cannot grow it unbounded.
|
||||||
|
Timer.DelayCall(TimeSpan.FromMinutes(1), TimeSpan.FromMinutes(1), SweepGuard);
|
||||||
|
|
||||||
|
_ = Task.Run(() => PollLoop(_cts.Token), _cts.Token);
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Stop()
|
||||||
|
{
|
||||||
|
_cts?.Cancel();
|
||||||
|
_cts?.Dispose();
|
||||||
|
_cts = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
public bool ShouldDeny(IPAddress address)
|
||||||
|
{
|
||||||
|
if (!Evaluate(address, Core.TickCount, out var shouldReport))
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (shouldReport)
|
||||||
|
{
|
||||||
|
// Demand-page this address up to the OS-level bouncer. Enqueue-only; never blocks the loop.
|
||||||
|
BanChannel.Report(address, _banDuration, "blocklist");
|
||||||
|
}
|
||||||
|
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// The pure decision, split out so the accept-path policy can be tested without a clock or a ban
|
||||||
|
/// channel. <paramref name="shouldReport"/> is true at most once per suppression window.
|
||||||
|
/// </summary>
|
||||||
|
internal bool Evaluate(IPAddress address, long nowTicks, out bool shouldReport)
|
||||||
|
{
|
||||||
|
shouldReport = false;
|
||||||
|
|
||||||
|
if (!_snapshot.IsBanned(address))
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (_reportHits)
|
||||||
|
{
|
||||||
|
shouldReport = _guard.TryMark(address.ToUInt128(), nowTicks, _suppressionMs);
|
||||||
|
}
|
||||||
|
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void SweepGuard() => _guard.Sweep(Core.TickCount);
|
||||||
|
|
||||||
|
// Test hook: inject a snapshot and policy without file I/O.
|
||||||
|
internal void LoadForTesting(BlocklistSnapshot snapshot, bool reportHits = true, long suppressionMs = 60000)
|
||||||
|
{
|
||||||
|
_snapshot = snapshot;
|
||||||
|
_reportHits = reportHits;
|
||||||
|
_suppressionMs = suppressionMs;
|
||||||
|
}
|
||||||
|
|
||||||
|
private async ValueTask PollLoop(CancellationToken token)
|
||||||
|
{
|
||||||
|
while (!token.IsCancellationRequested)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await Task.Delay(_interval, token);
|
||||||
|
}
|
||||||
|
catch (OperationCanceledException)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
if (ChangedSinceLastLoad())
|
||||||
|
{
|
||||||
|
// Parsing millions of lines competes with a save for CPU and page cache, so yield until
|
||||||
|
// the world is written out. See the threading policy in CLAUDE.md (rules #3 and #10).
|
||||||
|
while (World.Saving || World.WorldState == WorldState.PendingSave)
|
||||||
|
{
|
||||||
|
await Task.Delay(TimeSpan.FromSeconds(1), token);
|
||||||
|
}
|
||||||
|
|
||||||
|
Reload();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (OperationCanceledException)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
logger.Warning(e, "Blocklist reload check failed; keeping last snapshot ({Count})", Count);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private bool ChangedSinceLastLoad()
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var info = new FileInfo(_path);
|
||||||
|
if (!info.Exists)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (info.LastWriteTimeUtc == _lastWriteUtc)
|
||||||
|
{
|
||||||
|
return false; // cheapest guard
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
return !BlocklistFile.TryReadHeader(_path, out var h) || h.Generated != _lastGenerated;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void Reload()
|
||||||
|
{
|
||||||
|
// Capture the mtime/header BEFORE Load() so the markers describe the version being parsed, not
|
||||||
|
// one the producer swapped in mid-parse. Stale markers only cost an extra reload next poll;
|
||||||
|
// capturing after could skip a version entirely.
|
||||||
|
var writeUtc = default(DateTime);
|
||||||
|
try
|
||||||
|
{
|
||||||
|
writeUtc = new FileInfo(_path).LastWriteTimeUtc;
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
/* keep default */
|
||||||
|
}
|
||||||
|
|
||||||
|
BlocklistFile.TryReadHeader(_path, out var h);
|
||||||
|
|
||||||
|
var next = BlocklistFile.Load(_path, out var parsed, out var skipped);
|
||||||
|
_snapshot = next; // single volatile swap; readers see old or new whole
|
||||||
|
_lastGenerated = h.Generated;
|
||||||
|
_lastWriteUtc = writeUtc;
|
||||||
|
|
||||||
|
logger.Information("Blocklist loaded {Parsed} entr(ies) ({Count} ranges, {Skipped} skipped) gen={Gen}",
|
||||||
|
parsed, next.Count, skipped, h.Generated);
|
||||||
|
}
|
||||||
|
}
|
||||||
205
Projects/UOContent/Misc/Blocklist/BlocklistSnapshot.cs
Normal file
205
Projects/UOContent/Misc/Blocklist/BlocklistSnapshot.cs
Normal file
|
|
@ -0,0 +1,205 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: BlocklistSnapshot.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Buffers.Text;
|
||||||
|
using System.Net;
|
||||||
|
using System.Net.Sockets;
|
||||||
|
using System.Text;
|
||||||
|
using Server.Collections;
|
||||||
|
|
||||||
|
namespace Server.Network.Bans;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Immutable dual-stack blocklist. Singles and CIDRs are folded into a single sorted, coalesced
|
||||||
|
/// interval index per family: IPv4 as <see cref="uint"/> ranges (lean for the millions-strong common
|
||||||
|
/// case), IPv6 as <see cref="UInt128"/> ranges (empty unless the feed carries v6). Immutable → lock-free reads.
|
||||||
|
/// </summary>
|
||||||
|
public sealed class BlocklistSnapshot
|
||||||
|
{
|
||||||
|
public static readonly BlocklistSnapshot Empty = new(SortedRangeIndex<uint>.Empty, SortedRangeIndex<UInt128>.Empty);
|
||||||
|
|
||||||
|
private readonly SortedRangeIndex<uint> _v4;
|
||||||
|
private readonly SortedRangeIndex<UInt128> _v6;
|
||||||
|
|
||||||
|
public int Count => _v4.Count + _v6.Count;
|
||||||
|
|
||||||
|
private BlocklistSnapshot(SortedRangeIndex<uint> v4, SortedRangeIndex<UInt128> v6)
|
||||||
|
{
|
||||||
|
_v4 = v4;
|
||||||
|
_v6 = v6;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Parses a blocklist directly from its UTF-8/ASCII file bytes — one line at a time, splitting on
|
||||||
|
/// <c>'\n'</c> with no per-line string allocation. IPv4 singles and CIDRs are parsed straight from the
|
||||||
|
/// byte span; IPv6 (the rare path) decodes the single address token and defers to the framework parser.
|
||||||
|
/// Malformed lines increment <paramref name="skipped"/> and never throw. Build-time intermediates use
|
||||||
|
/// the multithreaded pool because this runs off the game loop on the reload/bootstrap thread.
|
||||||
|
/// </summary>
|
||||||
|
public static BlocklistSnapshot Build(ReadOnlySpan<byte> data, out int parsed, out int skipped)
|
||||||
|
{
|
||||||
|
parsed = 0;
|
||||||
|
skipped = 0;
|
||||||
|
|
||||||
|
// Only the two final index arrays (allocated inside SortedRangeIndex.Build) hit the heap; every
|
||||||
|
// build-time buffer here is a pooled ref list. mt: true is required — this runs off the game loop.
|
||||||
|
using var v4 = PooledRefList<SortedRangeIndex<uint>.Range>.Create(mt: true);
|
||||||
|
using var v6 = PooledRefList<SortedRangeIndex<UInt128>.Range>.Create(mt: true);
|
||||||
|
|
||||||
|
var rest = data;
|
||||||
|
while (!rest.IsEmpty)
|
||||||
|
{
|
||||||
|
ReadOnlySpan<byte> line;
|
||||||
|
var nl = rest.IndexOf((byte)'\n');
|
||||||
|
if (nl >= 0)
|
||||||
|
{
|
||||||
|
line = rest[..nl];
|
||||||
|
rest = rest[(nl + 1)..];
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
line = rest;
|
||||||
|
rest = default;
|
||||||
|
}
|
||||||
|
|
||||||
|
line = line[Ascii.Trim(line)];
|
||||||
|
if (line.IsEmpty || line[0] == (byte)'#' || line[0] == (byte)';')
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
var slash = line.IndexOf((byte)'/');
|
||||||
|
var addr = slash >= 0 ? line[..slash] : line;
|
||||||
|
var bitsToken = slash >= 0 ? line[(slash + 1)..] : default;
|
||||||
|
|
||||||
|
if (addr.IndexOf((byte)':') < 0)
|
||||||
|
{
|
||||||
|
// IPv4 single or CIDR — parsed straight from the byte span.
|
||||||
|
if (slash >= 0)
|
||||||
|
{
|
||||||
|
if (IPAddressUtility.TryParseV4(addr, out var ip) &&
|
||||||
|
TryParseBits(bitsToken, out var bits) && bits is >= 0 and <= 32)
|
||||||
|
{
|
||||||
|
var size = bits == 0 ? 0xFFFFFFFFu : (1u << (32 - bits)) - 1;
|
||||||
|
var b = ip & ~size;
|
||||||
|
v4.Add(new SortedRangeIndex<uint>.Range(b, b + size));
|
||||||
|
parsed++;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
skipped++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else if (IPAddressUtility.TryParseV4(addr, out var ip))
|
||||||
|
{
|
||||||
|
v4.Add(new SortedRangeIndex<uint>.Range(ip, ip));
|
||||||
|
parsed++;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
skipped++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else if (TryDecodeV6(addr, out var v))
|
||||||
|
{
|
||||||
|
// IPv6 is rare in these feeds; the single token was decoded and framework-parsed above.
|
||||||
|
if (slash >= 0)
|
||||||
|
{
|
||||||
|
if (TryParseBits(bitsToken, out var bits) && bits is >= 0 and <= 128)
|
||||||
|
{
|
||||||
|
var mask = bits == 0 ? UInt128.Zero : ~((UInt128.One << (128 - bits)) - 1);
|
||||||
|
var b = v & mask;
|
||||||
|
v6.Add(new SortedRangeIndex<UInt128>.Range(b, b | ~mask));
|
||||||
|
parsed++;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
skipped++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
v6.Add(new SortedRangeIndex<UInt128>.Range(v, v));
|
||||||
|
parsed++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
skipped++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
v4.Sort(SortedRangeIndex<uint>.ByMin);
|
||||||
|
v6.Sort(SortedRangeIndex<UInt128>.ByMin);
|
||||||
|
return new BlocklistSnapshot(SortedRangeIndex<uint>.Build(v4.AsSpan()), SortedRangeIndex<UInt128>.Build(v6.AsSpan()));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Decodes a single IPv6 address token from ASCII bytes and validates it via the framework parser.
|
||||||
|
private static bool TryDecodeV6(ReadOnlySpan<byte> addr, out UInt128 v)
|
||||||
|
{
|
||||||
|
v = UInt128.Zero;
|
||||||
|
if (addr.Length > 45)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
Span<char> chars = stackalloc char[addr.Length];
|
||||||
|
for (var i = 0; i < addr.Length; i++)
|
||||||
|
{
|
||||||
|
chars[i] = (char)addr[i];
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!IPAddress.TryParse(chars, out var a) || a.AddressFamily != AddressFamily.InterNetworkV6)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
v = a.ToUInt128();
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static bool TryParseBits(ReadOnlySpan<byte> token, out int bits)
|
||||||
|
{
|
||||||
|
if (Utf8Parser.TryParse(token, out bits, out var consumed) && consumed == token.Length)
|
||||||
|
{
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
bits = 0;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
public bool IsBanned(IPAddress ip)
|
||||||
|
{
|
||||||
|
if (ip.IsIPv4MappedToIPv6)
|
||||||
|
{
|
||||||
|
// v6-encoded v4 must not dodge the v4 set; extract the embedded v4 uint directly.
|
||||||
|
return IPAddressUtility.TryMappedV4(ip, out var mv) && _v4.Contains(mv);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (ip.AddressFamily == AddressFamily.InterNetwork)
|
||||||
|
{
|
||||||
|
return IPAddressUtility.TryV4(ip, out var v) && _v4.Contains(v);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (ip.AddressFamily == AddressFamily.InterNetworkV6)
|
||||||
|
{
|
||||||
|
return _v6.Contains(ip.ToUInt128());
|
||||||
|
}
|
||||||
|
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
55
Projects/UOContent/Misc/Blocklist/PromotedGuard.cs
Normal file
55
Projects/UOContent/Misc/Blocklist/PromotedGuard.cs
Normal file
|
|
@ -0,0 +1,55 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: PromotedGuard.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
|
||||||
|
namespace Server.Network.Bans;
|
||||||
|
|
||||||
|
/// <summary>Suppresses re-reporting the same IP within a TTL. Accept-path thread only.</summary>
|
||||||
|
public sealed class PromotedGuard
|
||||||
|
{
|
||||||
|
private readonly Dictionary<UInt128, long> _expiry = [];
|
||||||
|
|
||||||
|
public bool TryMark(UInt128 ip, long nowTicks, long ttlMs)
|
||||||
|
{
|
||||||
|
if (_expiry.TryGetValue(ip, out var exp) && exp - nowTicks > 0)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
_expiry[ip] = nowTicks + ttlMs;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Sweep(long nowTicks)
|
||||||
|
{
|
||||||
|
if (_expiry.Count == 0)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
using var dead = Collections.PooledRefQueue<UInt128>.Create();
|
||||||
|
foreach (var (ip, exp) in _expiry)
|
||||||
|
{
|
||||||
|
if (exp - nowTicks <= 0)
|
||||||
|
{
|
||||||
|
dead.Enqueue(ip);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
while (dead.Count > 0)
|
||||||
|
{
|
||||||
|
_expiry.Remove(dead.Dequeue());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
65
Projects/UOContent/Misc/CrowdSec/CrowdSecAlert.cs
Normal file
65
Projects/UOContent/Misc/CrowdSec/CrowdSecAlert.cs
Normal file
|
|
@ -0,0 +1,65 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: CrowdSecAlert.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System.Text.Json.Serialization;
|
||||||
|
|
||||||
|
namespace Server.Network.Bans.CrowdSec;
|
||||||
|
|
||||||
|
/// <summary>One CrowdSec alert (<c>POST /v1/alerts</c> takes an array of these).</summary>
|
||||||
|
public sealed class CrowdSecAlert
|
||||||
|
{
|
||||||
|
[JsonPropertyName("scenario")] public string Scenario { get; set; }
|
||||||
|
[JsonPropertyName("message")] public string Message { get; set; }
|
||||||
|
[JsonPropertyName("events_count")] public int EventsCount { get; set; } = 1;
|
||||||
|
[JsonPropertyName("start_at")] public string StartAt { get; set; }
|
||||||
|
[JsonPropertyName("stop_at")] public string StopAt { get; set; }
|
||||||
|
[JsonPropertyName("capacity")] public int Capacity { get; set; }
|
||||||
|
[JsonPropertyName("leakspeed")] public string LeakSpeed { get; set; } = "0s";
|
||||||
|
[JsonPropertyName("simulated")] public bool Simulated { get; set; }
|
||||||
|
[JsonPropertyName("events")] public object[] Events { get; set; } = [];
|
||||||
|
[JsonPropertyName("remediation")] public bool Remediation { get; set; } = true;
|
||||||
|
[JsonPropertyName("source")] public CrowdSecSource Source { get; set; }
|
||||||
|
[JsonPropertyName("decisions")] public CrowdSecDecisionDto[] Decisions { get; set; }
|
||||||
|
}
|
||||||
|
|
||||||
|
public sealed class CrowdSecSource
|
||||||
|
{
|
||||||
|
[JsonPropertyName("scope")] public string Scope { get; set; } = "Ip";
|
||||||
|
[JsonPropertyName("value")] public string Value { get; set; }
|
||||||
|
}
|
||||||
|
|
||||||
|
public sealed class CrowdSecDecisionDto
|
||||||
|
{
|
||||||
|
[JsonPropertyName("origin")] public string Origin { get; set; }
|
||||||
|
[JsonPropertyName("type")] public string Type { get; set; } = "ban";
|
||||||
|
[JsonPropertyName("scope")] public string Scope { get; set; } = "Ip";
|
||||||
|
[JsonPropertyName("value")] public string Value { get; set; }
|
||||||
|
[JsonPropertyName("duration")] public string Duration { get; set; }
|
||||||
|
[JsonPropertyName("scenario")] public string Scenario { get; set; }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Watcher login request/response for <c>POST /v1/watchers/login</c>.</summary>
|
||||||
|
public sealed class CrowdSecLoginRequest
|
||||||
|
{
|
||||||
|
[JsonPropertyName("machine_id")] public string MachineId { get; set; }
|
||||||
|
[JsonPropertyName("password")] public string Password { get; set; }
|
||||||
|
}
|
||||||
|
|
||||||
|
public sealed class CrowdSecLoginResponse
|
||||||
|
{
|
||||||
|
[JsonPropertyName("code")] public int Code { get; set; }
|
||||||
|
[JsonPropertyName("token")] public string Token { get; set; }
|
||||||
|
[JsonPropertyName("expire")] public string Expire { get; set; }
|
||||||
|
}
|
||||||
137
Projects/UOContent/Misc/CrowdSec/CrowdSecAlertClient.cs
Normal file
137
Projects/UOContent/Misc/CrowdSec/CrowdSecAlertClient.cs
Normal file
|
|
@ -0,0 +1,137 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: CrowdSecAlertClient.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.Net;
|
||||||
|
using System.Net.Http;
|
||||||
|
using System.Net.Http.Headers;
|
||||||
|
using System.Net.Http.Json;
|
||||||
|
using System.Text.Json;
|
||||||
|
using System.Threading;
|
||||||
|
using System.Threading.Tasks;
|
||||||
|
|
||||||
|
namespace Server.Network.Bans.CrowdSec;
|
||||||
|
|
||||||
|
/// <summary>Reporter-side LAPI operations, mockable for tests.</summary>
|
||||||
|
public interface ICrowdSecAlertClient : IDisposable
|
||||||
|
{
|
||||||
|
ValueTask PostAlertsAsync(IReadOnlyList<CrowdSecAlert> alerts, CancellationToken token);
|
||||||
|
ValueTask DeleteDecisionsAsync(string origin, IPAddress ip, CancellationToken token);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// CrowdSec LAPI watcher client: authenticates with machine credentials and posts/deletes decisions.
|
||||||
|
/// Holds a JWT refreshed on expiry or a 401.
|
||||||
|
/// </summary>
|
||||||
|
public sealed class CrowdSecAlertClient : ICrowdSecAlertClient
|
||||||
|
{
|
||||||
|
private static readonly JsonSerializerOptions _jsonOptions = new() { PropertyNameCaseInsensitive = true };
|
||||||
|
|
||||||
|
private readonly HttpClient _http;
|
||||||
|
private readonly string _machineId;
|
||||||
|
private readonly string _password;
|
||||||
|
|
||||||
|
private string _token;
|
||||||
|
private DateTime _tokenExpiresUtc = DateTime.MinValue;
|
||||||
|
|
||||||
|
public CrowdSecAlertClient(CrowdSecSettings settings)
|
||||||
|
{
|
||||||
|
var baseUri = new Uri(settings.LapiUrl, UriKind.Absolute); // fails loud on malformed url
|
||||||
|
_http = new HttpClient { BaseAddress = baseUri, Timeout = TimeSpan.FromSeconds(30) };
|
||||||
|
_http.DefaultRequestHeaders.Add("User-Agent", "ModernUO-watcher/1.0");
|
||||||
|
_machineId = settings.MachineId;
|
||||||
|
_password = settings.Password;
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task EnsureAuthAsync(CancellationToken token)
|
||||||
|
{
|
||||||
|
if (_token != null && DateTime.UtcNow < _tokenExpiresUtc - TimeSpan.FromMinutes(1))
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
var request = new CrowdSecLoginRequest { MachineId = _machineId, Password = _password };
|
||||||
|
using var response = await _http.PostAsJsonAsync("/v1/watchers/login", request, _jsonOptions, token)
|
||||||
|
.ConfigureAwait(false);
|
||||||
|
response.EnsureSuccessStatusCode();
|
||||||
|
|
||||||
|
var login = await response.Content.ReadFromJsonAsync<CrowdSecLoginResponse>(_jsonOptions, token)
|
||||||
|
.ConfigureAwait(false);
|
||||||
|
_token = login?.Token ?? throw new InvalidOperationException("CrowdSec login returned no token.");
|
||||||
|
_tokenExpiresUtc = DateTime.TryParse(login.Expire, out var exp) ? exp.ToUniversalTime() : DateTime.UtcNow.AddHours(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
private void Authorize(HttpRequestMessage message) =>
|
||||||
|
message.Headers.Authorization = new AuthenticationHeaderValue("Bearer", _token);
|
||||||
|
|
||||||
|
public async ValueTask PostAlertsAsync(IReadOnlyList<CrowdSecAlert> alerts, CancellationToken token)
|
||||||
|
{
|
||||||
|
if (alerts.Count == 0)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
await SendWithRetryAsync(() =>
|
||||||
|
{
|
||||||
|
var message = new HttpRequestMessage(HttpMethod.Post, "/v1/alerts")
|
||||||
|
{
|
||||||
|
Content = JsonContent.Create(alerts, options: _jsonOptions)
|
||||||
|
};
|
||||||
|
Authorize(message);
|
||||||
|
return message;
|
||||||
|
}, token).ConfigureAwait(false);
|
||||||
|
}
|
||||||
|
|
||||||
|
public async ValueTask DeleteDecisionsAsync(string origin, IPAddress ip, CancellationToken token)
|
||||||
|
{
|
||||||
|
var query = BuildDeleteQuery(origin, ip);
|
||||||
|
await SendWithRetryAsync(() =>
|
||||||
|
{
|
||||||
|
var message = new HttpRequestMessage(HttpMethod.Delete, query);
|
||||||
|
Authorize(message);
|
||||||
|
return message;
|
||||||
|
}, token).ConfigureAwait(false);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Builds the decisions-delete query string. <paramref name="origin"/> is operator-controlled config
|
||||||
|
/// (crowdsec.json), so it must be escaped like any other untrusted value going into a URL.
|
||||||
|
/// </summary>
|
||||||
|
internal static string BuildDeleteQuery(string origin, IPAddress ip) =>
|
||||||
|
$"/v1/decisions?origin={Uri.EscapeDataString(origin)}&ip={Uri.EscapeDataString(ip.ToString())}";
|
||||||
|
|
||||||
|
private async ValueTask SendWithRetryAsync(Func<HttpRequestMessage> build, CancellationToken token)
|
||||||
|
{
|
||||||
|
await EnsureAuthAsync(token).ConfigureAwait(false);
|
||||||
|
|
||||||
|
using var first = build();
|
||||||
|
using var response = await _http.SendAsync(first, token).ConfigureAwait(false);
|
||||||
|
if (response.StatusCode != HttpStatusCode.Unauthorized)
|
||||||
|
{
|
||||||
|
response.EnsureSuccessStatusCode();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Token rejected mid-flight: force a re-login and retry once.
|
||||||
|
_token = null;
|
||||||
|
await EnsureAuthAsync(token).ConfigureAwait(false);
|
||||||
|
using var retry = build();
|
||||||
|
using var retryResponse = await _http.SendAsync(retry, token).ConfigureAwait(false);
|
||||||
|
retryResponse.EnsureSuccessStatusCode();
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Dispose() => _http.Dispose();
|
||||||
|
}
|
||||||
101
Projects/UOContent/Misc/CrowdSec/CrowdSecConfiguration.cs
Normal file
101
Projects/UOContent/Misc/CrowdSec/CrowdSecConfiguration.cs
Normal file
|
|
@ -0,0 +1,101 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: CrowdSecConfiguration.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.IO;
|
||||||
|
using System.Text.Json.Serialization;
|
||||||
|
using Server.Json;
|
||||||
|
|
||||||
|
namespace Server.Network.Bans.CrowdSec;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Loads the <see cref="CrowdSecSettings"/> from <c>Configuration/crowdsec.json</c> (matching the
|
||||||
|
/// per-feature JSON config pattern used by <c>AssistantConfiguration</c>). Loaded once; a missing file
|
||||||
|
/// writes a disabled-by-default template so operators have something to edit.
|
||||||
|
/// </summary>
|
||||||
|
public static class CrowdSecConfiguration
|
||||||
|
{
|
||||||
|
private const string _path = "Configuration/crowdsec.json";
|
||||||
|
|
||||||
|
public static CrowdSecSettings Settings { get; private set; }
|
||||||
|
|
||||||
|
public static void Load()
|
||||||
|
{
|
||||||
|
var path = Path.Join(Core.BaseDirectory, _path);
|
||||||
|
|
||||||
|
if (File.Exists(path))
|
||||||
|
{
|
||||||
|
Settings = JsonConfig.Deserialize<CrowdSecSettings>(path);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
Settings = new CrowdSecSettings
|
||||||
|
{
|
||||||
|
LapiUrl = "http://127.0.0.1:8080",
|
||||||
|
MachineId = "",
|
||||||
|
Password = "",
|
||||||
|
Origin = "modernuo",
|
||||||
|
ManualBanDuration = TimeSpan.FromHours(168),
|
||||||
|
FlushInterval = TimeSpan.FromSeconds(1),
|
||||||
|
MaxQueue = 10000
|
||||||
|
};
|
||||||
|
|
||||||
|
Save();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void Save()
|
||||||
|
{
|
||||||
|
JsonConfig.Serialize(Path.Join(Core.BaseDirectory, _path), Settings);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Bound configuration for <see cref="CrowdSecReporter"/>. Read once at <c>Configure()</c>.
|
||||||
|
/// The reporter is inert unless <see cref="ReportingEnabled"/> is true.
|
||||||
|
/// </summary>
|
||||||
|
public record CrowdSecSettings
|
||||||
|
{
|
||||||
|
/// <summary>LAPI endpoint. Default <c>http://127.0.0.1:8080</c>.</summary>
|
||||||
|
[JsonPropertyName("lapiUrl")]
|
||||||
|
public string LapiUrl { get; set; } = "http://127.0.0.1:8080";
|
||||||
|
|
||||||
|
/// <summary>Watcher machine id from <c>cscli machines add</c>. Empty disables reporting.</summary>
|
||||||
|
[JsonPropertyName("machineId")]
|
||||||
|
public string MachineId { get; set; } = "";
|
||||||
|
|
||||||
|
/// <summary>Watcher password paired with <see cref="MachineId"/>.</summary>
|
||||||
|
[JsonPropertyName("password")]
|
||||||
|
public string Password { get; set; } = "";
|
||||||
|
|
||||||
|
/// <summary>Decision <c>origin</c> stamped on our contributions. Default <c>modernuo</c>.</summary>
|
||||||
|
[JsonPropertyName("origin")]
|
||||||
|
public string Origin { get; set; } = "modernuo";
|
||||||
|
|
||||||
|
/// <summary>Duration for manual admin bans pushed to CrowdSec. Default 168h (renewable). Finite so a missed retract self-heals.</summary>
|
||||||
|
[JsonPropertyName("manualBanDuration")]
|
||||||
|
public TimeSpan ManualBanDuration { get; set; } = TimeSpan.FromHours(168);
|
||||||
|
|
||||||
|
/// <summary>Max time the drain coalesces before flushing a batch. Default 1s.</summary>
|
||||||
|
[JsonPropertyName("flushInterval")]
|
||||||
|
public TimeSpan FlushInterval { get; set; } = TimeSpan.FromSeconds(1);
|
||||||
|
|
||||||
|
/// <summary>Bounded contribution queue capacity; overflow is dropped (counted). Default 10000.</summary>
|
||||||
|
[JsonPropertyName("maxQueue")]
|
||||||
|
public int MaxQueue { get; set; } = 10000;
|
||||||
|
|
||||||
|
[JsonIgnore]
|
||||||
|
public bool ReportingEnabled => !string.IsNullOrWhiteSpace(MachineId) && !string.IsNullOrWhiteSpace(Password);
|
||||||
|
}
|
||||||
399
Projects/UOContent/Misc/CrowdSec/CrowdSecReporter.cs
Normal file
399
Projects/UOContent/Misc/CrowdSec/CrowdSecReporter.cs
Normal file
|
|
@ -0,0 +1,399 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: CrowdSecReporter.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.Net;
|
||||||
|
using System.Threading;
|
||||||
|
using System.Threading.Channels;
|
||||||
|
using System.Threading.Tasks;
|
||||||
|
using Server.Logging;
|
||||||
|
|
||||||
|
namespace Server.Network.Bans.CrowdSec;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Contributes locally-decided bans to CrowdSec via the LAPI alerts API. Non-blocking on the accept
|
||||||
|
/// path: <see cref="Report"/> enqueues onto a bounded, drop-on-overflow channel drained by a single
|
||||||
|
/// background task that coalesces by IP and POSTs batched alerts.
|
||||||
|
/// </summary>
|
||||||
|
public sealed class CrowdSecReporter : IBanReporter
|
||||||
|
{
|
||||||
|
private static readonly ILogger logger = LogFactory.GetLogger(typeof(CrowdSecReporter));
|
||||||
|
|
||||||
|
internal readonly record struct ReportItem(IPAddress Ip, TimeSpan Ttl, string Reason, bool Retract);
|
||||||
|
|
||||||
|
// Bounded retry for transient LAPI failures during a drain send (network blips, 5xx). Distinct from
|
||||||
|
// CrowdSecAlertClient.SendWithRetryAsync's single 401-relogin retry, which is an auth concern.
|
||||||
|
private static readonly TimeSpan[] _retryDelays = [TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(2)];
|
||||||
|
|
||||||
|
private ICrowdSecAlertClient _client;
|
||||||
|
private CrowdSecSettings _settings;
|
||||||
|
private Channel<ReportItem> _queue;
|
||||||
|
private CancellationTokenSource _cts;
|
||||||
|
private Task _drainTask;
|
||||||
|
private int _dropped;
|
||||||
|
private int _sendFailures;
|
||||||
|
|
||||||
|
public CrowdSecReporter()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
// Test/embedding ctor with an injected client + settings.
|
||||||
|
internal CrowdSecReporter(ICrowdSecAlertClient client, CrowdSecSettings settings)
|
||||||
|
{
|
||||||
|
_client = client;
|
||||||
|
_settings = settings;
|
||||||
|
_queue = CreateQueue(settings.MaxQueue);
|
||||||
|
}
|
||||||
|
|
||||||
|
public string Name => "crowdsec";
|
||||||
|
public bool CanRetract => true;
|
||||||
|
public int DroppedCount => _dropped;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Batches ultimately dropped after the bounded transient-retry in <see cref="DrainLoop"/> gave up.
|
||||||
|
/// Distinct from <see cref="DroppedCount"/> (queue-overflow drops on the accept path): this counts
|
||||||
|
/// sustained LAPI outages so operators can see contribution loss instead of it being silent.
|
||||||
|
/// </summary>
|
||||||
|
public int SendFailureCount => _sendFailures;
|
||||||
|
|
||||||
|
/// <summary>The drain loop's task, so tests can assert it stays alive until the loop exits.</summary>
|
||||||
|
internal Task DrainTaskForTesting => _drainTask;
|
||||||
|
|
||||||
|
public static void Configure()
|
||||||
|
{
|
||||||
|
BanChannel.Register(new CrowdSecReporter());
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Register()
|
||||||
|
{
|
||||||
|
CrowdSecConfiguration.Load();
|
||||||
|
_settings ??= CrowdSecConfiguration.Settings;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Start(CancellationToken token)
|
||||||
|
{
|
||||||
|
if (!_settings.ReportingEnabled)
|
||||||
|
{
|
||||||
|
logger.Information("CrowdSec reporter disabled (machineId/password empty in crowdsec.json)");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
_client ??= new CrowdSecAlertClient(_settings);
|
||||||
|
_queue ??= CreateQueue(_settings.MaxQueue);
|
||||||
|
_cts = CancellationTokenSource.CreateLinkedTokenSource(token);
|
||||||
|
_drainTask = Task.Run(() => DrainLoop(_cts.Token), _cts.Token);
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Stop()
|
||||||
|
{
|
||||||
|
_cts?.Cancel();
|
||||||
|
|
||||||
|
// The flush below reads a SingleReader channel, so wait for the drain to actually exit first.
|
||||||
|
var drainExited = true;
|
||||||
|
try
|
||||||
|
{
|
||||||
|
// Wait(timeout) is false only on timeout; a throw means faulted/cancelled, which is still exited.
|
||||||
|
drainExited = _drainTask == null || _drainTask.Wait(TimeSpan.FromSeconds(2));
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
// Ignored: a faulted wait means the drain has completed and released the channel.
|
||||||
|
}
|
||||||
|
|
||||||
|
_cts?.Dispose();
|
||||||
|
_cts = null;
|
||||||
|
|
||||||
|
if (drainExited)
|
||||||
|
{
|
||||||
|
FlushRemainingOnStop();
|
||||||
|
}
|
||||||
|
|
||||||
|
_client?.Dispose();
|
||||||
|
_drainTask = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Best-effort bounded flush of whatever is still queued at shutdown. Blocking is correct here — the
|
||||||
|
/// loop has stopped ticking — but must not happen on the loop thread: <see cref="Stop"/> runs where
|
||||||
|
/// <c>SynchronizationContext.Current</c> is the <c>EventLoopContext</c>, and a captured continuation
|
||||||
|
/// would be posted to a queue nothing pumps any more. <see cref="Task.Run(Func{Task})"/> keeps the
|
||||||
|
/// chain on the pool; the bounded wait caps a wedged send at a few seconds of shutdown.
|
||||||
|
/// </summary>
|
||||||
|
private void FlushRemainingOnStop()
|
||||||
|
{
|
||||||
|
if (_queue == null || _client == null)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
_queue.Writer.TryComplete();
|
||||||
|
|
||||||
|
List<ReportItem> reports = [];
|
||||||
|
List<ReportItem> retracts = [];
|
||||||
|
while (_queue.Reader.TryRead(out var item))
|
||||||
|
{
|
||||||
|
(item.Retract ? retracts : reports).Add(item);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (reports.Count == 0 && retracts.Count == 0)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
if (!Task.Run(() => FlushRemainingOnStopAsync(reports, retracts)).Wait(TimeSpan.FromSeconds(4)))
|
||||||
|
{
|
||||||
|
logger.Warning(
|
||||||
|
"CrowdSec flush-on-stop timed out; {Count} item(s) not contributed",
|
||||||
|
reports.Count + retracts.Count
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
logger.Warning(e, "CrowdSec flush-on-stop failed");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Uses a fresh token, not the drain loop's already-cancelled one, which would fail every send
|
||||||
|
/// immediately. Reports go as one deduped batch; retracts go as individual DELETEs so an admin's
|
||||||
|
/// unban propagates on a clean shutdown. Leftovers self-heal via
|
||||||
|
/// <see cref="CrowdSecSettings.ManualBanDuration"/>.
|
||||||
|
/// </summary>
|
||||||
|
private async Task FlushRemainingOnStopAsync(List<ReportItem> reports, List<ReportItem> retracts)
|
||||||
|
{
|
||||||
|
using var flushCts = new CancellationTokenSource(TimeSpan.FromSeconds(3));
|
||||||
|
|
||||||
|
if (reports.Count > 0)
|
||||||
|
{
|
||||||
|
var alerts = BuildAlerts(reports, _settings, DateTime.UtcNow);
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await _client.PostAlertsAsync(alerts, flushCts.Token).ConfigureAwait(false);
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
logger.Warning(e, "CrowdSec flush-on-stop reports failed");
|
||||||
|
RecordSendFailure(alerts.Count);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
HashSet<string> seen = [];
|
||||||
|
for (var i = 0; i < retracts.Count; i++)
|
||||||
|
{
|
||||||
|
if (flushCts.IsCancellationRequested)
|
||||||
|
{
|
||||||
|
break; // out of budget; the rest self-heal via ManualBanDuration
|
||||||
|
}
|
||||||
|
|
||||||
|
var ip = retracts[i].Ip;
|
||||||
|
if (!seen.Add(ip.ToString()))
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await _client.DeleteDecisionsAsync(_settings.Origin, ip, flushCts.Token).ConfigureAwait(false);
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
logger.Warning(e, "CrowdSec flush-on-stop retract failed for {Address}", ip);
|
||||||
|
RecordSendFailure(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Report(IPAddress address, TimeSpan ttl, string reason) =>
|
||||||
|
Enqueue(new ReportItem(address, ttl, reason, false));
|
||||||
|
|
||||||
|
public void Retract(IPAddress address) =>
|
||||||
|
Enqueue(new ReportItem(address, TimeSpan.Zero, "retract", true));
|
||||||
|
|
||||||
|
private void Enqueue(ReportItem item)
|
||||||
|
{
|
||||||
|
if (_queue == null || !_queue.Writer.TryWrite(item))
|
||||||
|
{
|
||||||
|
Interlocked.Increment(ref _dropped);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// FullMode.Wait (the default) makes TryWrite return false immediately when the channel is full
|
||||||
|
// instead of blocking the caller — exactly the non-blocking drop-on-overflow behavior the accept
|
||||||
|
// path requires. DropWrite would silently discard the new item and always report success, which
|
||||||
|
// would make overflow undetectable.
|
||||||
|
private static Channel<ReportItem> CreateQueue(int capacity) =>
|
||||||
|
Channel.CreateBounded<ReportItem>(new BoundedChannelOptions(Math.Max(1, capacity))
|
||||||
|
{
|
||||||
|
FullMode = BoundedChannelFullMode.Wait,
|
||||||
|
SingleReader = true
|
||||||
|
});
|
||||||
|
|
||||||
|
// Must return Task: Start() passes this to Task.Run, which has no Func<ValueTask> overload, so a
|
||||||
|
// ValueTask would bind to Task.Run<TResult> and yield a Task<ValueTask> that completes at the first
|
||||||
|
// await rather than when the loop exits.
|
||||||
|
private async Task DrainLoop(CancellationToken token)
|
||||||
|
{
|
||||||
|
var reader = _queue.Reader;
|
||||||
|
|
||||||
|
while (!token.IsCancellationRequested)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
if (!await reader.WaitToReadAsync(token).ConfigureAwait(false))
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Coalesce a burst before flushing.
|
||||||
|
await Task.Delay(_settings.FlushInterval, token).ConfigureAwait(false);
|
||||||
|
|
||||||
|
List<ReportItem> reports = [];
|
||||||
|
List<ReportItem> retracts = [];
|
||||||
|
while (reader.TryRead(out var item))
|
||||||
|
{
|
||||||
|
(item.Retract ? retracts : reports).Add(item);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (reports.Count > 0)
|
||||||
|
{
|
||||||
|
var alerts = BuildAlerts(reports, _settings, DateTime.UtcNow);
|
||||||
|
if (!await SendWithBoundedRetryAsync(() => _client.PostAlertsAsync(alerts, token), token)
|
||||||
|
.ConfigureAwait(false))
|
||||||
|
{
|
||||||
|
RecordSendFailure(alerts.Count);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for (var i = 0; i < retracts.Count; i++)
|
||||||
|
{
|
||||||
|
var ip = retracts[i].Ip;
|
||||||
|
if (!await SendWithBoundedRetryAsync(() => _client.DeleteDecisionsAsync(_settings.Origin, ip, token), token)
|
||||||
|
.ConfigureAwait(false))
|
||||||
|
{
|
||||||
|
RecordSendFailure(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (OperationCanceledException)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
// Contribution is auxiliary: log and keep draining. Never crash the shard.
|
||||||
|
logger.Warning(e, "CrowdSec contribution flush failed; dropped this batch");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Sends with up to 3 attempts total (1 initial + 2 retries), backing off 1s then 2s between
|
||||||
|
/// attempts, for transient LAPI failures (network blips, 5xx). Backoff uses <see cref="Task.Delay"/>
|
||||||
|
/// so it never blocks the thread; a cancellation during backoff propagates as
|
||||||
|
/// <see cref="OperationCanceledException"/> so the drain loop exits cleanly. Returns false (never
|
||||||
|
/// throws for a send failure) once attempts are exhausted, so the caller can count the drop and keep
|
||||||
|
/// draining instead of losing the rest of the batch/queue.
|
||||||
|
/// </summary>
|
||||||
|
private static async ValueTask<bool> SendWithBoundedRetryAsync(Func<ValueTask> send, CancellationToken token)
|
||||||
|
{
|
||||||
|
for (var attempt = 0; ; attempt++)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await send().ConfigureAwait(false);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
catch (OperationCanceledException)
|
||||||
|
{
|
||||||
|
throw;
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
if (attempt >= _retryDelays.Length)
|
||||||
|
{
|
||||||
|
logger.Warning(e, "CrowdSec send failed after {Attempts} attempt(s); giving up", attempt + 1);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
var delay = _retryDelays[attempt];
|
||||||
|
logger.Warning(e, "CrowdSec send failed (attempt {Attempt}); retrying in {Delay}", attempt + 1, delay);
|
||||||
|
await Task.Delay(delay, token).ConfigureAwait(false);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void RecordSendFailure(int itemCount)
|
||||||
|
{
|
||||||
|
var total = Interlocked.Increment(ref _sendFailures);
|
||||||
|
logger.Warning(
|
||||||
|
"CrowdSec contribution batch dropped after retries ({Items} item(s)); total dropped batches: {Total}",
|
||||||
|
itemCount,
|
||||||
|
total
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Coalesces items by IP (last write wins) and builds one alert per unique address.</summary>
|
||||||
|
internal static List<CrowdSecAlert> BuildAlerts(IEnumerable<ReportItem> items, CrowdSecSettings settings, DateTime nowUtc)
|
||||||
|
{
|
||||||
|
Dictionary<string, ReportItem> byIp = [];
|
||||||
|
foreach (var item in items)
|
||||||
|
{
|
||||||
|
byIp[item.Ip.ToString()] = item;
|
||||||
|
}
|
||||||
|
|
||||||
|
var timestamp = nowUtc.ToString("yyyy-MM-ddTHH:mm:ss.fffZ");
|
||||||
|
var alerts = new List<CrowdSecAlert>(byIp.Count);
|
||||||
|
|
||||||
|
foreach (var (value, item) in byIp)
|
||||||
|
{
|
||||||
|
var ttl = item.Reason == "manual" || item.Ttl <= TimeSpan.Zero ? settings.ManualBanDuration : item.Ttl;
|
||||||
|
var scenario = $"{settings.Origin}/{item.Reason}";
|
||||||
|
|
||||||
|
alerts.Add(new CrowdSecAlert
|
||||||
|
{
|
||||||
|
Scenario = scenario,
|
||||||
|
Message = $"ModernUO {item.Reason} ban for {value}",
|
||||||
|
StartAt = timestamp,
|
||||||
|
StopAt = timestamp,
|
||||||
|
Source = new CrowdSecSource { Scope = "Ip", Value = value },
|
||||||
|
Decisions =
|
||||||
|
[
|
||||||
|
new CrowdSecDecisionDto
|
||||||
|
{
|
||||||
|
Origin = settings.Origin,
|
||||||
|
Type = "ban",
|
||||||
|
Scope = "Ip",
|
||||||
|
Value = value,
|
||||||
|
Duration = FormatDuration(ttl),
|
||||||
|
Scenario = scenario
|
||||||
|
}
|
||||||
|
]
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
return alerts;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>CrowdSec accepts Go durations; whole seconds are unambiguous and sufficient.</summary>
|
||||||
|
internal static string FormatDuration(TimeSpan ttl)
|
||||||
|
{
|
||||||
|
var seconds = (long)ttl.TotalSeconds;
|
||||||
|
return $"{Math.Max(1, seconds)}s";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -25,8 +25,15 @@ public class CidrFirewallEntry : BaseFirewallEntry
|
||||||
public override UInt128 MaxIpAddress { get; }
|
public override UInt128 MaxIpAddress { get; }
|
||||||
|
|
||||||
public CidrFirewallEntry(string ipAddressOrCidr)
|
public CidrFirewallEntry(string ipAddressOrCidr)
|
||||||
: this(ParseIPAddress(ipAddressOrCidr, out var prefixLength), prefixLength)
|
|
||||||
{
|
{
|
||||||
|
// Core owns the CIDR -> normalized range parse.
|
||||||
|
if (!IPAddressUtility.TryParseCidrRange(ipAddressOrCidr, out var min, out var max))
|
||||||
|
{
|
||||||
|
throw new ArgumentException("Invalid IP address or CIDR.", nameof(ipAddressOrCidr));
|
||||||
|
}
|
||||||
|
|
||||||
|
MinIpAddress = min;
|
||||||
|
MaxIpAddress = max;
|
||||||
}
|
}
|
||||||
|
|
||||||
public CidrFirewallEntry(IPAddress minAddress, IPAddress maxAddress)
|
public CidrFirewallEntry(IPAddress minAddress, IPAddress maxAddress)
|
||||||
|
|
@ -50,26 +57,4 @@ public class CidrFirewallEntry : BaseFirewallEntry
|
||||||
MaxIpAddress = Utility.CreateCidrAddress(bytes, prefixLength, true);
|
MaxIpAddress = Utility.CreateCidrAddress(bytes, prefixLength, true);
|
||||||
}
|
}
|
||||||
|
|
||||||
private static IPAddress ParseIPAddress(ReadOnlySpan<char> ipString, out int prefixLength)
|
|
||||||
{
|
|
||||||
var slashIndex = ipString.IndexOf('/');
|
|
||||||
var ipAddress = IPAddress.Parse(slashIndex > -1 ? ipString[..slashIndex] : ipString);
|
|
||||||
var maxPrefixLength = ipAddress.AddressFamily == AddressFamily.InterNetworkV6 ? 128 : 32;
|
|
||||||
|
|
||||||
if (slashIndex == -1)
|
|
||||||
{
|
|
||||||
prefixLength = maxPrefixLength;
|
|
||||||
}
|
|
||||||
else
|
|
||||||
{
|
|
||||||
var prefixPart = ipString[(slashIndex + 1)..];
|
|
||||||
|
|
||||||
if (!int.TryParse(prefixPart, out prefixLength) || prefixLength < 0 || prefixLength > maxPrefixLength)
|
|
||||||
{
|
|
||||||
throw new ArgumentException("Invalid prefix length.");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return ipAddress;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
413
Projects/UOContent/Misc/Firewall/Firewall.cs
Normal file
413
Projects/UOContent/Misc/Firewall/Firewall.cs
Normal file
|
|
@ -0,0 +1,413 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: Firewall.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Buffers;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.IO;
|
||||||
|
using System.Net;
|
||||||
|
using System.Runtime.CompilerServices;
|
||||||
|
using Server.Collections;
|
||||||
|
using Server.Json;
|
||||||
|
using Server.Logging;
|
||||||
|
|
||||||
|
namespace Server.Network;
|
||||||
|
|
||||||
|
public static class Firewall
|
||||||
|
{
|
||||||
|
// Single-threaded: the accept path, admin gump/command, TTL expiry timer, and boot load all run on
|
||||||
|
// the main game loop. No locks, caches, or version counters are needed. See the ban-channel design doc.
|
||||||
|
// _entries is the authoritative store (gump/persistence/TTL/command all work against it); _index is a
|
||||||
|
// derived, rebuild-on-demand SortedRangeIndex used only for the accept-path IsBlocked lookup, shared
|
||||||
|
// with the same sorted-range binary-search primitive the blocklist uses (see BlocklistSnapshot).
|
||||||
|
private static readonly List<IFirewallEntry> _entries = [];
|
||||||
|
|
||||||
|
// Entries with a TTL: entry -> absolute expiry tick (Core.TickCount). Permanent entries are absent.
|
||||||
|
private static readonly Dictionary<IFirewallEntry, long> _expiring = [];
|
||||||
|
|
||||||
|
private static SortedRangeIndex<UInt128> _index = SortedRangeIndex<UInt128>.Empty;
|
||||||
|
private static bool _indexDirty;
|
||||||
|
|
||||||
|
private static readonly ILogger logger = LogFactory.GetLogger(typeof(Firewall));
|
||||||
|
private const string _path = "Configuration/firewall.json";
|
||||||
|
private const string _legacyPath = "firewall.cfg";
|
||||||
|
private static bool _dirty;
|
||||||
|
private static bool _configured;
|
||||||
|
|
||||||
|
public static int FirewallSetCount => _entries.Count;
|
||||||
|
|
||||||
|
public static void ReadFirewallSet(Action<IReadOnlyCollection<IFirewallEntry>> callback) => callback(_entries);
|
||||||
|
|
||||||
|
public static bool IsBlocked(IPAddress address)
|
||||||
|
{
|
||||||
|
if (_entries.Count == 0)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
EnsureIndex();
|
||||||
|
return _index.Contains(address.ToUInt128());
|
||||||
|
}
|
||||||
|
|
||||||
|
// Rebuilds the derived lookup index from the authoritative _entries list, but only when entries have
|
||||||
|
// changed since the last build. Runs on the main game loop, so the pooled build buffer is single-threaded
|
||||||
|
// (mt: false); only the two final SortedRangeIndex arrays are heap-allocated.
|
||||||
|
private static void EnsureIndex()
|
||||||
|
{
|
||||||
|
if (!_indexDirty)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
using var ranges = PooledRefList<SortedRangeIndex<UInt128>.Range>.Create(_entries.Count, mt: false);
|
||||||
|
for (var i = 0; i < _entries.Count; i++)
|
||||||
|
{
|
||||||
|
var entry = _entries[i];
|
||||||
|
ranges.Add(new SortedRangeIndex<UInt128>.Range(entry.MinIpAddress, entry.MaxIpAddress));
|
||||||
|
}
|
||||||
|
|
||||||
|
ranges.Sort(SortedRangeIndex<UInt128>.ByMin);
|
||||||
|
_index = SortedRangeIndex<UInt128>.Build(ranges.AsSpan());
|
||||||
|
_indexDirty = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
public static bool Add(IFirewallEntry firewallEntry) => Add(firewallEntry, TimeSpan.Zero);
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Adds an entry. <paramref name="ttl"/> <= <see cref="TimeSpan.Zero"/> means permanent. Returns false
|
||||||
|
/// if the entry was already present.
|
||||||
|
/// </summary>
|
||||||
|
// Indexed scan (no closure allocation); firewall lists are small, so O(n) is negligible and this
|
||||||
|
// stays off the hot path (Add/Remove are admin/boot actions, not the accept path).
|
||||||
|
private static int IndexOfEntry(IFirewallEntry entry)
|
||||||
|
{
|
||||||
|
for (var i = 0; i < _entries.Count; i++)
|
||||||
|
{
|
||||||
|
if (_entries[i].CompareTo(entry) == 0)
|
||||||
|
{
|
||||||
|
return i;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
public static bool Add(IFirewallEntry firewallEntry, TimeSpan ttl, bool persist = true)
|
||||||
|
{
|
||||||
|
if (firewallEntry == null || IndexOfEntry(firewallEntry) >= 0)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
_entries.Add(firewallEntry);
|
||||||
|
|
||||||
|
if (ttl > TimeSpan.Zero)
|
||||||
|
{
|
||||||
|
_expiring[firewallEntry] = Core.TickCount + (long)ttl.TotalMilliseconds;
|
||||||
|
}
|
||||||
|
|
||||||
|
_indexDirty = true;
|
||||||
|
|
||||||
|
if (persist)
|
||||||
|
{
|
||||||
|
MarkDirty();
|
||||||
|
}
|
||||||
|
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
public static bool Remove(IFirewallEntry entry)
|
||||||
|
{
|
||||||
|
if (entry == null)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
var index = IndexOfEntry(entry);
|
||||||
|
if (index < 0)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove the stored instance from _expiring (not the passed reference), so a value-equal
|
||||||
|
// entry created elsewhere still clears the TTL bookkeeping.
|
||||||
|
var stored = _entries[index];
|
||||||
|
_entries.RemoveAt(index);
|
||||||
|
_expiring.Remove(stored);
|
||||||
|
_indexDirty = true;
|
||||||
|
MarkDirty();
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Removes every entry whose TTL has elapsed. Called from the main-thread maintenance timer (Task 2).
|
||||||
|
/// </summary>
|
||||||
|
internal static void ExpireEntries(long nowTicks)
|
||||||
|
{
|
||||||
|
if (_expiring.Count == 0)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
List<IFirewallEntry> expired = null;
|
||||||
|
foreach (var (entry, expiresAt) in _expiring)
|
||||||
|
{
|
||||||
|
if (expiresAt - nowTicks <= 0)
|
||||||
|
{
|
||||||
|
(expired ??= []).Add(entry);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (expired == null)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
for (var i = 0; i < expired.Count; i++)
|
||||||
|
{
|
||||||
|
var entry = expired[i];
|
||||||
|
_entries.Remove(entry);
|
||||||
|
_expiring.Remove(entry);
|
||||||
|
}
|
||||||
|
|
||||||
|
_indexDirty = true;
|
||||||
|
MarkDirty();
|
||||||
|
}
|
||||||
|
|
||||||
|
[MethodImpl(MethodImplOptions.AggressiveInlining)]
|
||||||
|
public static IFirewallEntry ToFirewallEntry(object entry) =>
|
||||||
|
entry switch
|
||||||
|
{
|
||||||
|
IFirewallEntry firewallEntry => firewallEntry,
|
||||||
|
IPAddress address => new SingleIpFirewallEntry(address),
|
||||||
|
string s => ToFirewallEntry(s),
|
||||||
|
_ => null
|
||||||
|
};
|
||||||
|
|
||||||
|
public static IFirewallEntry ToFirewallEntry(string entry)
|
||||||
|
{
|
||||||
|
if (entry == null)
|
||||||
|
{
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var rangeSeparator = entry.IndexOf('-');
|
||||||
|
if (rangeSeparator > -1)
|
||||||
|
{
|
||||||
|
return new CidrFirewallEntry(
|
||||||
|
IPAddress.Parse(entry.AsSpan(0, rangeSeparator)),
|
||||||
|
IPAddress.Parse(entry.AsSpan(rangeSeparator + 1))
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (entry.IndexOf('/') > -1)
|
||||||
|
{
|
||||||
|
return new CidrFirewallEntry(entry);
|
||||||
|
}
|
||||||
|
|
||||||
|
return new SingleIpFirewallEntry(entry);
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public static void Configure()
|
||||||
|
{
|
||||||
|
if (_configured)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
_configured = true;
|
||||||
|
|
||||||
|
var path = Path.Join(Core.BaseDirectory, _path);
|
||||||
|
|
||||||
|
if (File.Exists(path))
|
||||||
|
{
|
||||||
|
LoadFrom(JsonConfig.Deserialize<FirewallSettings>(path));
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
var legacyPath = ResolveLegacyCfgPath();
|
||||||
|
if (legacyPath != null)
|
||||||
|
{
|
||||||
|
MigrateLegacyCfg(legacyPath);
|
||||||
|
Save(); // materialize firewall.json; the .cfg is no longer read after this
|
||||||
|
TryMarkLegacyCfgMigrated(legacyPath);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Main-thread maintenance: expire TTLs and flush pending writes. No background thread.
|
||||||
|
Timer.DelayCall(TimeSpan.FromSeconds(30), TimeSpan.FromSeconds(30), Maintenance);
|
||||||
|
|
||||||
|
// Expose the set to the accept path. Everything else (gump, commands, persistence) keeps using
|
||||||
|
// the Firewall API directly; only the per-connection question goes through the filter registry.
|
||||||
|
ConnectionFilters.Register(FirewallConnectionFilter.Instance);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void Maintenance()
|
||||||
|
{
|
||||||
|
ExpireEntries(Core.TickCount);
|
||||||
|
|
||||||
|
if (_dirty)
|
||||||
|
{
|
||||||
|
Save();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void MarkDirty() => _dirty = true;
|
||||||
|
|
||||||
|
internal static void LoadFrom(FirewallSettings settings)
|
||||||
|
{
|
||||||
|
if (settings?.Entries == null)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Core.Now: this runs on the game loop, via the Configure sweep.
|
||||||
|
var now = Core.Now;
|
||||||
|
var records = settings.Entries;
|
||||||
|
for (var i = 0; i < records.Length; i++)
|
||||||
|
{
|
||||||
|
var record = records[i];
|
||||||
|
var entry = ToFirewallEntry(record.Value);
|
||||||
|
if (entry == null)
|
||||||
|
{
|
||||||
|
logger.Warning("Ignoring unparseable firewall entry \"{Entry}\"", record.Value);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
var ttl = TimeSpan.Zero;
|
||||||
|
if (record.Expires is { } expires)
|
||||||
|
{
|
||||||
|
ttl = expires - now;
|
||||||
|
if (ttl <= TimeSpan.Zero)
|
||||||
|
{
|
||||||
|
continue; // already expired
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Add(entry, ttl, persist: false);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
internal static FirewallSettings ToSettings()
|
||||||
|
{
|
||||||
|
// expires is derived below as now + (expiresAtTick - nowTicks), so both operands must come from
|
||||||
|
// the same instant. Core.Now and Core.TickCount are refreshed together each loop iteration; a
|
||||||
|
// fresh DateTime.UtcNow here would bake the loop's lag into every persisted expiry.
|
||||||
|
var now = Core.Now;
|
||||||
|
var nowTicks = Core.TickCount;
|
||||||
|
var list = new List<FirewallEntryRecord>(_entries.Count);
|
||||||
|
|
||||||
|
for (var i = 0; i < _entries.Count; i++)
|
||||||
|
{
|
||||||
|
var entry = _entries[i];
|
||||||
|
DateTime? expires = null;
|
||||||
|
if (_expiring.TryGetValue(entry, out var expiresAtTick))
|
||||||
|
{
|
||||||
|
expires = now.AddMilliseconds(expiresAtTick - nowTicks);
|
||||||
|
}
|
||||||
|
|
||||||
|
list.Add(new FirewallEntryRecord { Value = entry.ToString(), Expires = expires });
|
||||||
|
}
|
||||||
|
|
||||||
|
return new FirewallSettings { Entries = list.ToArray() };
|
||||||
|
}
|
||||||
|
|
||||||
|
public static void Save()
|
||||||
|
{
|
||||||
|
_dirty = false;
|
||||||
|
var path = Path.Join(Core.BaseDirectory, _path);
|
||||||
|
var tmp = $"{path}.tmp";
|
||||||
|
JsonConfig.Serialize(tmp, ToSettings());
|
||||||
|
File.Move(tmp, path, overwrite: true); // atomic swap
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Locates the legacy firewall.cfg to migrate. The modern convention is <see cref="Core.BaseDirectory"/>,
|
||||||
|
/// checked first; the pre-collapse <c>AdminFirewall</c> used a bare relative path (resolved against the
|
||||||
|
/// process's current working directory), which may differ from <see cref="Core.BaseDirectory"/> when the
|
||||||
|
/// shard is launched from elsewhere, so that's checked as a fallback. Returns null if neither exists.
|
||||||
|
/// </summary>
|
||||||
|
private static string ResolveLegacyCfgPath()
|
||||||
|
{
|
||||||
|
var underBaseDirectory = Path.Join(Core.BaseDirectory, _legacyPath);
|
||||||
|
if (File.Exists(underBaseDirectory))
|
||||||
|
{
|
||||||
|
return underBaseDirectory;
|
||||||
|
}
|
||||||
|
|
||||||
|
return File.Exists(_legacyPath) ? _legacyPath : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void MigrateLegacyCfg(string legacyPath)
|
||||||
|
{
|
||||||
|
var searchValues = SearchValues.Create("*Xx?");
|
||||||
|
|
||||||
|
using var reader = new StreamReader(legacyPath);
|
||||||
|
while (reader.ReadLine() is { } line)
|
||||||
|
{
|
||||||
|
line = line.Trim();
|
||||||
|
if (line.Length == 0)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (line.AsSpan().ContainsAny(searchValues))
|
||||||
|
{
|
||||||
|
logger.Warning("Legacy firewall entry \"{Entry}\" ignored during migration", line);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
var entry = ToFirewallEntry(line);
|
||||||
|
if (entry != null)
|
||||||
|
{
|
||||||
|
Add(entry, TimeSpan.Zero, persist: false);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
logger.Information("Migrated {Count} entr(ies) from legacy firewall.cfg to firewall.json", _entries.Count);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Renames the migrated <c>.cfg</c> to <c>firewall.cfg.migrated</c> so it isn't re-scanned on the next
|
||||||
|
/// boot and operators can see it was already migrated. Best-effort: a locked/read-only file must not
|
||||||
|
/// fail startup, since the migration itself (firewall.json) already succeeded.
|
||||||
|
/// </summary>
|
||||||
|
private static void TryMarkLegacyCfgMigrated(string legacyPath)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
File.Move(legacyPath, $"{legacyPath}.migrated", overwrite: true);
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
logger.Warning(e, "Could not rename migrated legacy firewall file \"{Path}\"", legacyPath);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
internal static void ResetForTesting()
|
||||||
|
{
|
||||||
|
_entries.Clear();
|
||||||
|
_expiring.Clear();
|
||||||
|
_index = SortedRangeIndex<UInt128>.Empty;
|
||||||
|
_indexDirty = false;
|
||||||
|
_configured = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
52
Projects/UOContent/Misc/Firewall/FirewallConnectionFilter.cs
Normal file
52
Projects/UOContent/Misc/Firewall/FirewallConnectionFilter.cs
Normal file
|
|
@ -0,0 +1,52 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: FirewallConnectionFilter.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System.Net;
|
||||||
|
using System.Threading;
|
||||||
|
|
||||||
|
namespace Server.Network;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Exposes the admin-curated <see cref="Firewall"/> set to the accept path as an
|
||||||
|
/// <see cref="IConnectionFilter"/>. The firewall keeps its own API (the admin gump and commands mutate
|
||||||
|
/// it directly); this is only the accept-path adapter, since a static class cannot implement an
|
||||||
|
/// interface. It is the cheapest gate to consult — an empty set costs one length compare — so
|
||||||
|
/// <see cref="Firewall.Configure"/> registers it first.
|
||||||
|
/// </summary>
|
||||||
|
internal sealed class FirewallConnectionFilter : IConnectionFilter
|
||||||
|
{
|
||||||
|
public static readonly FirewallConnectionFilter Instance = new();
|
||||||
|
|
||||||
|
private FirewallConnectionFilter()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
public string Name => "firewall";
|
||||||
|
|
||||||
|
// Firewall.Configure() owns loading and registration.
|
||||||
|
public void Register()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
// Nothing to hydrate: the set loads at Configure and is maintained by a main-loop timer.
|
||||||
|
public void Start(CancellationToken token)
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Flushes pending writes on the way down so a TTL expiry or late admin edit is not lost.</summary>
|
||||||
|
public void Stop() => Firewall.Save();
|
||||||
|
|
||||||
|
public bool ShouldDeny(IPAddress address) => Firewall.IsBlocked(address);
|
||||||
|
}
|
||||||
42
Projects/UOContent/Misc/Firewall/FirewallSettings.cs
Normal file
42
Projects/UOContent/Misc/Firewall/FirewallSettings.cs
Normal file
|
|
@ -0,0 +1,42 @@
|
||||||
|
/*************************************************************************
|
||||||
|
* ModernUO *
|
||||||
|
* Copyright 2019-2026 - ModernUO Development Team *
|
||||||
|
* Email: hi@modernuo.com *
|
||||||
|
* File: FirewallSettings.cs *
|
||||||
|
* *
|
||||||
|
* This program is free software: you can redistribute it and/or modify *
|
||||||
|
* it under the terms of the GNU General Public License as published by *
|
||||||
|
* the Free Software Foundation, either version 3 of the License, or *
|
||||||
|
* (at your option) any later version. *
|
||||||
|
* *
|
||||||
|
* You should have received a copy of the GNU General Public License *
|
||||||
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||||
|
*************************************************************************/
|
||||||
|
|
||||||
|
using System;
|
||||||
|
using System.Text.Json.Serialization;
|
||||||
|
|
||||||
|
namespace Server.Network;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Persisted local firewall entries (manual admin bans). Stored at <c>Configuration/firewall.json</c>.
|
||||||
|
/// Auto-detected rate-limit trips are never persisted — they are contributed to CrowdSec, not stored here.
|
||||||
|
/// </summary>
|
||||||
|
public record FirewallSettings
|
||||||
|
{
|
||||||
|
[JsonPropertyName("entries")]
|
||||||
|
public FirewallEntryRecord[] Entries { get; set; } = [];
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// One persisted entry. <see cref="Value"/> is a single IP, a <c>min-max</c> range, or CIDR.
|
||||||
|
/// <see cref="Expires"/> is UTC wall-clock; null means permanent.
|
||||||
|
/// </summary>
|
||||||
|
public record FirewallEntryRecord
|
||||||
|
{
|
||||||
|
[JsonPropertyName("value")]
|
||||||
|
public string Value { get; set; }
|
||||||
|
|
||||||
|
[JsonPropertyName("expires")]
|
||||||
|
public DateTime? Expires { get; set; }
|
||||||
|
}
|
||||||
|
|
@ -473,6 +473,65 @@ ns.SendMovementRej(int sequence, Mobile m);
|
||||||
6. **Big-endian by default** -- only use `WriteLE`/`ReadLE` when the protocol requires it
|
6. **Big-endian by default** -- only use `WriteLE`/`ReadLE` when the protocol requires it
|
||||||
7. **Function pointers** (`&Handler`) for incoming packet registration (no delegate allocation)
|
7. **Function pointers** (`&Handler`) for incoming packet registration (no delegate allocation)
|
||||||
|
|
||||||
|
## Connection Filtering (Accept Path)
|
||||||
|
|
||||||
|
Every inbound socket is checked before it becomes a `NetState`. The check runs on the game loop once
|
||||||
|
per accepted connection -- this is the path that has to survive a DDoS -- so it must be allocation-free
|
||||||
|
and non-blocking.
|
||||||
|
|
||||||
|
Gates plug in through `IConnectionFilter`, registered with `ConnectionFilters.Register()` during the
|
||||||
|
Configure sweep:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public sealed class MyFilter : IConnectionFilter
|
||||||
|
{
|
||||||
|
public string Name => "my-filter";
|
||||||
|
public void Configure() { /* read config, no I/O */ }
|
||||||
|
public void Start(CancellationToken token) { /* background hydration */ }
|
||||||
|
public void Stop() { }
|
||||||
|
public bool ShouldDeny(IPAddress address) => /* allocation-free membership test */;
|
||||||
|
}
|
||||||
|
|
||||||
|
// In a static Configure() so the sweep finds it:
|
||||||
|
ConnectionFilters.Register(new MyFilter());
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- `ShouldDeny` must be **allocation-free**, O(log n) at worst, no I/O, no blocking. Anything expensive
|
||||||
|
(parsing, reloading, contributing to an external service) belongs off the loop or behind a bounded,
|
||||||
|
non-blocking enqueue.
|
||||||
|
- Side effects a hit implies (reporting to `BanChannel`, promoting to an OS firewall, suppressing
|
||||||
|
duplicate reports) are the **filter's** business, not the accept path's.
|
||||||
|
- Filters are consulted in registration order and the first denial short-circuits, so register the
|
||||||
|
cheapest and most selective first. Order affects only how quickly a denial is reached, never whether
|
||||||
|
one happens.
|
||||||
|
- A filter that throws is **unregistered** and the connection fails open. A filter that faults once
|
||||||
|
faults for every connection, so leaving it registered would mean an exception per accept.
|
||||||
|
|
||||||
|
Core owns the question; **every implementation lives in UOContent**. The two that ship are `firewall`
|
||||||
|
(admin-curated, mutable at runtime, persisted to `Configuration/firewall.json`) and `blocklist`
|
||||||
|
(file-sourced, millions of entries, demand-pages hits to CrowdSec). A shard that fronts its server with
|
||||||
|
an upstream proxy or edge scrubbing can drop both and register nothing.
|
||||||
|
|
||||||
|
Do **not** route this kind of check through `EventSink.InvokeSocketConnect` -- that fires later and
|
||||||
|
allocates a `SocketConnectEventArgs` per connection, which is exactly what the accept path avoids for
|
||||||
|
rejected traffic.
|
||||||
|
|
||||||
|
### IP Address Normalization (`IPAddressUtility`)
|
||||||
|
|
||||||
|
Addresses are normalized to `UInt128` in **IPv6 form** so a single comparison/index works for both
|
||||||
|
families. An IPv4 address becomes its v4-mapped-v6 value (`::ffff:a.b.c.d`), which is why round-tripping
|
||||||
|
matters: `IPv4 -> UInt128 -> IPAddress` can come back as `InterNetworkV6` with `IsIPv4MappedToIPv6`
|
||||||
|
set, even though it is "really" a v4 address. Code that switches on `AddressFamily` alone will mis-handle
|
||||||
|
those, so the helpers check both.
|
||||||
|
|
||||||
|
> **Known wart / follow-up:** `ToUInt128` guards with `AddressFamily == InterNetwork && !IsIPv4MappedToIPv6`.
|
||||||
|
> Per BCL semantics `IsIPv4MappedToIPv6` is only ever true for `InterNetworkV6`, so the second clause
|
||||||
|
> reads as redundant -- it is really defending the round-trip described above. The normalization would be
|
||||||
|
> clearer as an explicit "to canonical v6 bits" step that never needs the family check at all. Deliberately
|
||||||
|
> left as-is; to be revisited in a follow-up PR rather than churned mid-feature.
|
||||||
|
|
||||||
## Key File References
|
## Key File References
|
||||||
|
|
||||||
| File | Description |
|
| File | Description |
|
||||||
|
|
@ -492,3 +551,8 @@ ns.SendMovementRej(int sequence, Mobile m);
|
||||||
| `Projects/Server/Network/Packets/OutgoingAccountPackets.cs` | Account packets |
|
| `Projects/Server/Network/Packets/OutgoingAccountPackets.cs` | Account packets |
|
||||||
| `Projects/Server/Network/Packets/OutgoingContainerPackets.cs` | Container packets |
|
| `Projects/Server/Network/Packets/OutgoingContainerPackets.cs` | Container packets |
|
||||||
| `Projects/Server/Network/PacketHandler.cs` | PacketHandler class |
|
| `Projects/Server/Network/PacketHandler.cs` | PacketHandler class |
|
||||||
|
| `Projects/Server/Network/IConnectionFilter.cs` | Accept-path gate contract |
|
||||||
|
| `Projects/Server/Network/ConnectionFilters.cs` | Filter registry + lifecycle |
|
||||||
|
| `Projects/UOContent/Misc/Firewall/Firewall.cs` | Admin-curated firewall set |
|
||||||
|
| `Projects/Server/Utilities/IPAddressUtility.cs` | IPAddress <-> UInt128 normalization, CIDR parsing |
|
||||||
|
| `Projects/UOContent/Misc/Blocklist/BlocklistFilter.cs` | File-sourced blocklist filter |
|
||||||
|
|
|
||||||
543
tools/Export-IpBlocklist.ps1
Normal file
543
tools/Export-IpBlocklist.ps1
Normal file
|
|
@ -0,0 +1,543 @@
|
||||||
|
#requires -Version 7.0
|
||||||
|
<#
|
||||||
|
.SYNOPSIS
|
||||||
|
Downloads a small, non-overlapping set of public IP threat feeds and writes them to a single
|
||||||
|
ModernUO blocklist file -- merged, de-duplicated and bogon-filtered.
|
||||||
|
|
||||||
|
.DESCRIPTION
|
||||||
|
This is the producer half of ModernUO's in-app blocklist gate. It fetches a deliberately THIN feed
|
||||||
|
set, merges every source into one global set, drops duplicates and reserved/bogon addresses, then
|
||||||
|
writes the result to a plain text file that the shard reads via `file` in
|
||||||
|
Configuration/blocklist.json. Nothing is installed and no credentials are needed -- the output is just
|
||||||
|
a text file, so this can run on any machine that can reach the shard's Distribution folder.
|
||||||
|
|
||||||
|
It writes the file the shard's `BlocklistFilter` demand-pages against. IPs that actually connect are
|
||||||
|
promoted to CrowdSec / the OS firewall by the shard; the OS firewall never has to hold millions of
|
||||||
|
entries, which is exactly the scale it cannot handle on Windows.
|
||||||
|
|
||||||
|
Inclusion principle: any category of IP used in OTHER attacks that could plausibly be turned against a
|
||||||
|
game server should be blocked -- compromised hosts, botnets, scanners, spam / DDoS-as-a-service bots,
|
||||||
|
open proxies and Tor relays. That whole surface is already covered by the anchor feed `bitwire-it`,
|
||||||
|
which is itself a 91-source aggregator (it folds in spamhaus, ipsum, firehol-level2, blocklist-de,
|
||||||
|
dshield, emergingthreats, binarydefense, cins-army, bruteforceblocker, greensnow, vxvault, ThreatFox,
|
||||||
|
StopForumSpam/sblam, Tor, open-proxy and C2 lists). So the inclusive posture lives in the base layer,
|
||||||
|
and every one of those standalone feeds is dropped as pure redundancy. Only the feeds bitwire does NOT
|
||||||
|
already carry are kept on top of it:
|
||||||
|
|
||||||
|
bitwire-it 2h-refreshed 91-source aggregate (compromised hosts, botnets, scanners, spam
|
||||||
|
bots, Tor/open-proxy abuse relays, ThreatFox C2) -- the broad base layer.
|
||||||
|
romainmarcoux ~130k fresh attacker IPs bitwire's snapshot lags on (high-churn feed).
|
||||||
|
sentinel-turris ~800 unique honeypot probers (Turris greylist) not in bitwire.
|
||||||
|
firehol-level1 hijacked/reputation NETBLOCKS (spamhaus DROP-style) -- bogon-filtered.
|
||||||
|
|
||||||
|
The only category deliberately held back is commercial VPN exit endpoints, which could block a legit
|
||||||
|
player -- and those are barely present here anyway (bitwire is ~5% of VPN-tunnel lists). If you ever want
|
||||||
|
to protect VPN/Tor players, pass -ExcludeAnonymizers to subtract Tor/open-proxy/VPN IPs from the output.
|
||||||
|
|
||||||
|
OUTPUT FORMAT (must stay in sync with UOContent/Misc/Blocklist/BlocklistFile.cs):
|
||||||
|
Line 1 is a header comment carrying the version markers, e.g.
|
||||||
|
# modernuo-blocklist generated=2026-07-25T18:03:11Z count=3914022 ipv4=3901188 cidr=12834
|
||||||
|
The shard polls `reloadInterval` and reloads when the file mtime AND `generated=` change,
|
||||||
|
so the header is REQUIRED -- without it the shard loads once and never picks up a new file.
|
||||||
|
Every following line is one entry: a bare IPv4/IPv6 address or a CIDR (`1.2.3.0/24`). Blank lines
|
||||||
|
and lines starting with `#` or `;` are ignored. Order does not matter; the shard sorts and
|
||||||
|
coalesces on load. The feeds used here are IPv4-only, but the shard parses IPv6 lines too.
|
||||||
|
|
||||||
|
The file is written to a `.tmp` sibling and swapped into place atomically, so the shard never reads a
|
||||||
|
half-written list -- it either sees the previous version or the new one, whole.
|
||||||
|
|
||||||
|
Performance: bitwire alone is ~4M lines. Parsing/validating/bogon-filtering that in interpreted
|
||||||
|
PowerShell is the slow part (minutes), so the hot loop is compiled once via Add-Type (C#) -- it runs in
|
||||||
|
~1s. Downloads stream with a live Write-Progress bar; every phase prints its own elapsed time so you can
|
||||||
|
see exactly where the wall-clock goes.
|
||||||
|
|
||||||
|
Requires PowerShell 7 (pwsh), which runs on Windows, Linux and macOS -- Windows PowerShell 5.1 is
|
||||||
|
not supported and the script refuses to run there. Schedule it with Task Scheduler, cron, or a
|
||||||
|
systemd timer.
|
||||||
|
|
||||||
|
Every run rewrites the whole file, so an IP that drops off the feeds stops being blocked on the next
|
||||||
|
run -- there is no TTL to tune. Calling it is idempotent: if the list on disk is younger than
|
||||||
|
-MinInterval the script exits without downloading anything, so an over-eager trigger costs nothing
|
||||||
|
upstream. -Force overrides that.
|
||||||
|
|
||||||
|
.PARAMETER DistributionPath
|
||||||
|
Path to the shard's Distribution folder. The blocklist is written to the Configuration/ip-blocklist.txt
|
||||||
|
beneath it, which is the default `file` in blocklist.json. Not needed when the script is run from its
|
||||||
|
place in the repo (tools/), or when -OutFile is given.
|
||||||
|
|
||||||
|
.PARAMETER OutFile
|
||||||
|
Explicit output path, overriding -DistributionPath. Use this if you relocated the blocklist and
|
||||||
|
changed `file` in blocklist.json to match.
|
||||||
|
|
||||||
|
.PARAMETER MinInterval
|
||||||
|
Refuse to re-run while the existing blocklist is younger than this (default 2h), so a misbehaving
|
||||||
|
scheduler, a login script or a stuck retry loop cannot hammer the upstream feeds. The age comes from
|
||||||
|
the `generated=` header of the file already on disk (falling back to its mtime), so it survives across
|
||||||
|
machines and reboots -- there is no separate state file. Nothing is downloaded when the check trips.
|
||||||
|
Accepts `90s`, `45m`, `2h`, `2.5h`, `1d`, or a bare number of hours. Use `0` to disable the check.
|
||||||
|
Match this to how often you actually want fresh data: the anchor feed only refreshes every 2h, so
|
||||||
|
running more often than that costs bandwidth and gains nothing.
|
||||||
|
|
||||||
|
.PARAMETER Force
|
||||||
|
Run regardless of how recently the blocklist was generated (bypasses -MinInterval).
|
||||||
|
|
||||||
|
.PARAMETER Feeds
|
||||||
|
Which feeds to include (by Name). Default: all of them.
|
||||||
|
|
||||||
|
.PARAMETER ExcludeAnonymizers
|
||||||
|
Also download Tor-exit / open-proxy / VPN-tunnel lists and SUBTRACT those IPs from the output. Off by
|
||||||
|
default -- for a game server, Tor/open-proxy relays are attack infrastructure you want to block. Turn
|
||||||
|
this on only if you need to keep VPN/Tor players reachable.
|
||||||
|
|
||||||
|
.PARAMETER DryRun
|
||||||
|
Download + parse + merge + count only. Writes nothing.
|
||||||
|
|
||||||
|
.EXAMPLE
|
||||||
|
.\Export-IpBlocklist.ps1 -DryRun
|
||||||
|
|
||||||
|
.EXAMPLE
|
||||||
|
# Safe to call as often as you like -- it no-ops unless the list is older than 2h.
|
||||||
|
.\Export-IpBlocklist.ps1 -DistributionPath 'C:\Shard\Distribution'
|
||||||
|
|
||||||
|
.EXAMPLE
|
||||||
|
.\Export-IpBlocklist.ps1 -OutFile 'D:\shared\ip-blocklist.txt' -ExcludeAnonymizers
|
||||||
|
|
||||||
|
.EXAMPLE
|
||||||
|
# Regenerate right now, ignoring the cooldown.
|
||||||
|
.\Export-IpBlocklist.ps1 -DistributionPath 'C:\Shard\Distribution' -Force
|
||||||
|
|
||||||
|
.EXAMPLE
|
||||||
|
# Linux/macOS, e.g. from cron:
|
||||||
|
pwsh -File /opt/modernuo/Export-IpBlocklist.ps1 -DistributionPath /opt/modernuo/Distribution
|
||||||
|
|
||||||
|
.NOTES
|
||||||
|
Feeds are aggressive-but-low-FP for a game server (attacker / botnet / compromised / abuse-relay SOURCE
|
||||||
|
IPs). Reserved/bogon space (0/8, 10/8, 127/8, RFC1918, multicast, etc.) is always filtered out -- this
|
||||||
|
matters because firehol-level1 ships bogon netblocks that would otherwise block private/reserved ranges.
|
||||||
|
#>
|
||||||
|
[CmdletBinding()]
|
||||||
|
param(
|
||||||
|
[string] $DistributionPath,
|
||||||
|
[string] $OutFile,
|
||||||
|
[string] $MinInterval = '2h',
|
||||||
|
[string[]] $Feeds,
|
||||||
|
[switch] $ExcludeAnonymizers,
|
||||||
|
[switch] $Force,
|
||||||
|
[switch] $DryRun
|
||||||
|
)
|
||||||
|
|
||||||
|
$ErrorActionPreference = 'Stop'
|
||||||
|
$UA = 'ModernUO-Blocklist-Export'
|
||||||
|
$totalSw = [System.Diagnostics.Stopwatch]::StartNew()
|
||||||
|
|
||||||
|
# Default location under the Distribution folder. Keep in sync with BlocklistSettings.File.
|
||||||
|
# Kept as separate segments (never a literal 'a\b') so Join-Path picks the right separator per OS.
|
||||||
|
$DefaultPathSegments = @('Configuration', 'ip-blocklist.txt')
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
# Resolve the output path. Explicit -OutFile wins; then -DistributionPath; then the in-repo layout
|
||||||
|
# (tools\ sits next to Distribution\) so a checkout works with no arguments at all. The script is meant to
|
||||||
|
# be copied onto the shard host, and there it needs -DistributionPath (or -OutFile).
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
if (-not $OutFile) {
|
||||||
|
if (-not $DistributionPath -and $PSScriptRoot) {
|
||||||
|
$inRepo = Join-Path (Split-Path -Parent $PSScriptRoot) 'Distribution'
|
||||||
|
if (Test-Path -LiteralPath $inRepo -PathType Container) { $DistributionPath = $inRepo }
|
||||||
|
}
|
||||||
|
if (-not $DistributionPath) {
|
||||||
|
throw "Could not locate the shard's Distribution folder. Pass -DistributionPath 'C:\path\to\Distribution' (or -OutFile)."
|
||||||
|
}
|
||||||
|
if (-not (Test-Path -LiteralPath $DistributionPath -PathType Container)) {
|
||||||
|
throw "DistributionPath '$DistributionPath' does not exist."
|
||||||
|
}
|
||||||
|
$OutFile = Join-Path $DistributionPath @DefaultPathSegments
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
# Cooldown gate. Runs BEFORE anything is downloaded: the whole point is that a misconfigured scheduler or a
|
||||||
|
# retry loop cannot spam the upstream feeds. State lives in the output file itself (`generated=` header,
|
||||||
|
# mtime as fallback), so it is correct across reboots, machines and hand-runs with no sidecar state file.
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
function ConvertTo-Duration {
|
||||||
|
param([string]$Text)
|
||||||
|
if ([string]::IsNullOrWhiteSpace($Text)) { return [TimeSpan]::Zero }
|
||||||
|
$t = $Text.Trim().ToLowerInvariant()
|
||||||
|
$unit = $t[$t.Length - 1]
|
||||||
|
$numText = if ($unit -match '[0-9.]') { $t } else { $t.Substring(0, $t.Length - 1) }
|
||||||
|
$n = 0.0
|
||||||
|
# InvariantCulture is not optional here: under a comma-decimal locale (de-DE, fr-FR, ...) the
|
||||||
|
# current-culture parse reads '2.5' as 25 -- it treats '.' as a group separator and SUCCEEDS, so
|
||||||
|
# `-MinInterval 2.5h` would silently become a 25 hour cooldown instead of failing loudly.
|
||||||
|
if (-not [double]::TryParse($numText, [Globalization.NumberStyles]::Float,
|
||||||
|
[Globalization.CultureInfo]::InvariantCulture, [ref]$n)) {
|
||||||
|
throw "Could not parse duration '$Text' (try 90s, 45m, 2h, 2.5h, 1d)."
|
||||||
|
}
|
||||||
|
switch ($unit) {
|
||||||
|
's' { return [TimeSpan]::FromSeconds($n) }
|
||||||
|
'm' { return [TimeSpan]::FromMinutes($n) }
|
||||||
|
'h' { return [TimeSpan]::FromHours($n) }
|
||||||
|
'd' { return [TimeSpan]::FromDays($n) }
|
||||||
|
default { return [TimeSpan]::FromHours($n) } # bare number == hours
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Age of the list already on disk, or $null when there is nothing usable to age.
|
||||||
|
function Get-BlocklistAge {
|
||||||
|
param([string]$Path)
|
||||||
|
if (-not (Test-Path -LiteralPath $Path -PathType Leaf)) { return $null }
|
||||||
|
|
||||||
|
# Prefer the header we wrote: it describes the data, not the file, so copying/restoring the file
|
||||||
|
# cannot make a stale list look fresh (or a fresh one look stale).
|
||||||
|
try {
|
||||||
|
$first = Get-Content -LiteralPath $Path -TotalCount 1 -ErrorAction Stop
|
||||||
|
if ($first -and $first.StartsWith('#')) {
|
||||||
|
foreach ($tok in $first.Split(' ', [StringSplitOptions]::RemoveEmptyEntries)) {
|
||||||
|
if ($tok.StartsWith('generated=', [StringComparison]::Ordinal)) {
|
||||||
|
$stamp = [DateTime]::MinValue
|
||||||
|
$styles = [Globalization.DateTimeStyles]::AdjustToUniversal -bor [Globalization.DateTimeStyles]::AssumeUniversal
|
||||||
|
if ([DateTime]::TryParse($tok.Substring(10), [Globalization.CultureInfo]::InvariantCulture, $styles, [ref]$stamp)) {
|
||||||
|
return @{ Age = ([DateTime]::UtcNow - $stamp); Stamp = $tok.Substring(10); Source = 'header' }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch { }
|
||||||
|
|
||||||
|
# Hand-maintained or truncated file: fall back to the filesystem timestamp.
|
||||||
|
try {
|
||||||
|
$w = (Get-Item -LiteralPath $Path -ErrorAction Stop).LastWriteTimeUtc
|
||||||
|
return @{ Age = ([DateTime]::UtcNow - $w)
|
||||||
|
Stamp = $w.ToString('yyyy-MM-ddTHH:mm:ssZ', [Globalization.CultureInfo]::InvariantCulture)
|
||||||
|
Source = 'mtime' }
|
||||||
|
}
|
||||||
|
catch { return $null }
|
||||||
|
}
|
||||||
|
|
||||||
|
$minAge = ConvertTo-Duration $MinInterval
|
||||||
|
if (-not $Force -and $minAge -gt [TimeSpan]::Zero) {
|
||||||
|
$existing = Get-BlocklistAge -Path $OutFile
|
||||||
|
if ($existing) {
|
||||||
|
# A negative age means the stamp is in the future (clock skew, or a file from another host). Treat it
|
||||||
|
# as fresh: refusing to run is the recoverable failure, hammering the feeds on every tick is not.
|
||||||
|
if ($existing.Age -lt $minAge) {
|
||||||
|
$agoText = if ($existing.Age -lt [TimeSpan]::Zero) { 'in the future -- check the clock' } else { ("{0:N1}h ago" -f $existing.Age.TotalHours) }
|
||||||
|
Write-Host ("Blocklist at {0} was generated {1} ({2}={3}); newer than -MinInterval {4}." -f `
|
||||||
|
$OutFile, $agoText, $existing.Source, $existing.Stamp, $MinInterval)
|
||||||
|
Write-Host "Nothing downloaded. Pass -Force to regenerate now, or lower -MinInterval."
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
# Compiled hot loop. Interpreted PowerShell chokes on bitwire's ~4M lines; this parses + validates + bogon-
|
||||||
|
# filters + de-dupes in one compiled pass, and writes the final file directly (no 4M-element PS pipelines).
|
||||||
|
# Deliberately plain C#: no LINQ, no generics beyond HashSet, nothing that would slow the hot loop.
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
Add-Type -TypeDefinition @'
|
||||||
|
using System;
|
||||||
|
using System.Collections.Generic;
|
||||||
|
using System.IO;
|
||||||
|
using System.Text;
|
||||||
|
|
||||||
|
public static class BlocklistExporter
|
||||||
|
{
|
||||||
|
static bool TryParseIPv4(string s, int start, int len, out uint val)
|
||||||
|
{
|
||||||
|
val = 0;
|
||||||
|
uint acc = 0; int octet = 0, dots = 0, digits = 0, end = start + len;
|
||||||
|
for (int i = start; i < end; i++)
|
||||||
|
{
|
||||||
|
char c = s[i];
|
||||||
|
if (c == '.')
|
||||||
|
{
|
||||||
|
if (digits == 0 || octet > 255) return false;
|
||||||
|
acc = (acc << 8) | (uint)octet; dots++; octet = 0; digits = 0;
|
||||||
|
}
|
||||||
|
else if (c >= '0' && c <= '9')
|
||||||
|
{
|
||||||
|
octet = octet * 10 + (c - '0'); if (++digits > 3) return false;
|
||||||
|
}
|
||||||
|
else return false;
|
||||||
|
}
|
||||||
|
if (dots != 3 || digits == 0 || octet > 255) return false;
|
||||||
|
val = (acc << 8) | (uint)octet;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
static bool IsBogon(uint start, uint end, uint[] bs, uint[] be)
|
||||||
|
{
|
||||||
|
for (int i = 0; i < bs.Length; i++)
|
||||||
|
if (start <= be[i] && end >= bs[i]) return true;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse one feed's text; add bare IPs to `singles`, CIDRs to `cidrs`. Returns count newly added.
|
||||||
|
public static int AddContent(string content, HashSet<uint> singles, HashSet<string> cidrs, uint[] bs, uint[] be)
|
||||||
|
{
|
||||||
|
int added = 0, n = content.Length, i = 0;
|
||||||
|
while (i < n)
|
||||||
|
{
|
||||||
|
int eol = content.IndexOf('\n', i);
|
||||||
|
int lineEnd = (eol < 0) ? n : eol;
|
||||||
|
int a = i, b = lineEnd;
|
||||||
|
while (a < b && (content[a] == ' ' || content[a] == '\t' || content[a] == '\r')) a++;
|
||||||
|
while (b > a && (content[b - 1] == ' ' || content[b - 1] == '\t' || content[b - 1] == '\r')) b--;
|
||||||
|
i = (eol < 0) ? n : eol + 1;
|
||||||
|
if (a >= b) continue;
|
||||||
|
char first = content[a];
|
||||||
|
if (first == '#' || first == ';') continue;
|
||||||
|
|
||||||
|
// Feeds vary: some are bare IPs, some are CSV/whitespace records with the IP first.
|
||||||
|
int t = a;
|
||||||
|
while (t < b)
|
||||||
|
{
|
||||||
|
char c = content[t];
|
||||||
|
if (c == ' ' || c == '\t' || c == ',' || c == ';') break;
|
||||||
|
t++;
|
||||||
|
}
|
||||||
|
int slash = -1;
|
||||||
|
for (int k = a; k < t; k++) { if (content[k] == '/') { slash = k; break; } }
|
||||||
|
|
||||||
|
if (slash >= 0)
|
||||||
|
{
|
||||||
|
uint ip;
|
||||||
|
if (!TryParseIPv4(content, a, slash - a, out ip)) continue;
|
||||||
|
int bits = 0, bd = 0;
|
||||||
|
for (int k = slash + 1; k < t; k++)
|
||||||
|
{
|
||||||
|
char c = content[k];
|
||||||
|
if (c < '0' || c > '9') { bd = -1; break; }
|
||||||
|
bits = bits * 10 + (c - '0'); bd++;
|
||||||
|
}
|
||||||
|
if (bd <= 0 || bits > 32) continue;
|
||||||
|
ulong size = (bits == 0) ? 0xFFFFFFFFUL : ((1UL << (32 - bits)) - 1UL);
|
||||||
|
ulong endAddr = (ulong)ip + size; if (endAddr > 0xFFFFFFFFUL) endAddr = 0xFFFFFFFFUL;
|
||||||
|
if (IsBogon(ip, (uint)endAddr, bs, be)) continue;
|
||||||
|
if (cidrs.Add(content.Substring(a, t - a))) added++;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
uint ip;
|
||||||
|
if (!TryParseIPv4(content, a, t - a, out ip)) continue;
|
||||||
|
if (IsBogon(ip, ip, bs, be)) continue;
|
||||||
|
if (singles.Add(ip)) added++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return added;
|
||||||
|
}
|
||||||
|
|
||||||
|
static int WriteOctet(char[] buf, int pos, uint v)
|
||||||
|
{
|
||||||
|
if (v >= 100) { buf[pos++] = (char)('0' + v / 100); buf[pos++] = (char)('0' + (v / 10) % 10); }
|
||||||
|
else if (v >= 10) { buf[pos++] = (char)('0' + v / 10); }
|
||||||
|
buf[pos++] = (char)('0' + v % 10);
|
||||||
|
return pos;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Writes the whole blocklist in one streamed pass so we never materialize millions of strings.
|
||||||
|
// Header first (the shard's reload detector requires it), then singles, then CIDRs. LF line endings.
|
||||||
|
public static void Write(string path, string header, HashSet<uint> singles, HashSet<string> cidrs)
|
||||||
|
{
|
||||||
|
using (var w = new StreamWriter(path, false, new UTF8Encoding(false), 1 << 20))
|
||||||
|
{
|
||||||
|
w.Write(header); w.Write('\n');
|
||||||
|
|
||||||
|
char[] buf = new char[20];
|
||||||
|
foreach (uint v in singles)
|
||||||
|
{
|
||||||
|
int p = 0;
|
||||||
|
p = WriteOctet(buf, p, (v >> 24) & 255); buf[p++] = '.';
|
||||||
|
p = WriteOctet(buf, p, (v >> 16) & 255); buf[p++] = '.';
|
||||||
|
p = WriteOctet(buf, p, (v >> 8) & 255); buf[p++] = '.';
|
||||||
|
p = WriteOctet(buf, p, v & 255); buf[p++] = '\n';
|
||||||
|
w.Write(buf, 0, p);
|
||||||
|
}
|
||||||
|
|
||||||
|
foreach (string c in cidrs) { w.Write(c); w.Write('\n'); }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
'@
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
# Feed set -- thin, non-overlapping. See .DESCRIPTION for why each is kept and what was dropped as redundant.
|
||||||
|
# romainmarcoux's "full" set is sharded; only aa..ad carry data today (ae.. are empty placeholders).
|
||||||
|
# A 404/empty shard is skipped, so extend this list if upstream grows the shard count.
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
$rmBase = 'https://raw.githubusercontent.com/romainmarcoux/malicious-ip/main/full-300k-'
|
||||||
|
$rmShards = @('aa','ab','ac','ad') | ForEach-Object { $rmBase + $_ + '.txt' }
|
||||||
|
|
||||||
|
$AllFeeds = @(
|
||||||
|
[pscustomobject]@{ Name = 'bitwire-it'; Urls = @('https://raw.githubusercontent.com/bitwire-it/ipblocklist/main/inbound.txt') }
|
||||||
|
[pscustomobject]@{ Name = 'romainmarcoux'; Urls = $rmShards }
|
||||||
|
[pscustomobject]@{ Name = 'sentinel-turris';Urls = @('https://view.sentinel.turris.cz/greylist-data/greylist-latest.csv') }
|
||||||
|
[pscustomobject]@{ Name = 'firehol-level1'; Urls = @('https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level1.netset') }
|
||||||
|
)
|
||||||
|
|
||||||
|
# Anonymizer / relay lists subtracted only when -ExcludeAnonymizers is set (Tor exits, open proxies, VPN tunnels).
|
||||||
|
$AnonFeeds = @(
|
||||||
|
'https://raw.githubusercontent.com/borestad/firehol-mirror/refs/heads/main/tor_exits.ipset'
|
||||||
|
'https://raw.githubusercontent.com/borestad/firehol-mirror/refs/heads/main/sslproxies_7d.ipset'
|
||||||
|
'https://raw.githubusercontent.com/borestad/firehol-mirror/refs/heads/main/socks_proxy_7d.ipset'
|
||||||
|
'https://raw.githubusercontent.com/ShadowWhisperer/IPs/master/Lists/Tunnels'
|
||||||
|
)
|
||||||
|
|
||||||
|
if ($Feeds) {
|
||||||
|
$AllFeeds = $AllFeeds | Where-Object { $Feeds -contains $_.Name }
|
||||||
|
if (-not $AllFeeds) { throw "No feeds matched -Feeds." }
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
# Reserved / bogon ranges -- never valid attacker SOURCE IPs; always filtered. Built once as uint32 arrays.
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
function ConvertTo-IPv4UInt {
|
||||||
|
param([string]$s)
|
||||||
|
$a = $s.Split('.')
|
||||||
|
if ($a.Length -ne 4) { return $null }
|
||||||
|
$v = [uint32]0
|
||||||
|
foreach ($o in $a) {
|
||||||
|
$n = 0
|
||||||
|
if (-not [int]::TryParse($o, [ref]$n) -or $n -lt 0 -or $n -gt 255) { return $null }
|
||||||
|
$v = ($v -shl 8) -bor [uint32]$n
|
||||||
|
}
|
||||||
|
return $v
|
||||||
|
}
|
||||||
|
|
||||||
|
$bogonCidrs = '0.0.0.0/8','10.0.0.0/8','100.64.0.0/10','127.0.0.0/8','169.254.0.0/16','172.16.0.0/12',
|
||||||
|
'192.0.0.0/24','192.0.2.0/24','192.168.0.0/16','198.18.0.0/15','198.51.100.0/24',
|
||||||
|
'203.0.113.0/24','224.0.0.0/3' # 224/3 covers multicast + reserved + 255.255.255.255
|
||||||
|
$bogStart = [System.Collections.Generic.List[uint32]]::new()
|
||||||
|
$bogEnd = [System.Collections.Generic.List[uint32]]::new()
|
||||||
|
foreach ($c in $bogonCidrs) {
|
||||||
|
$p = $c.Split('/'); $base = ConvertTo-IPv4UInt $p[0]; $bits = [int]$p[1]
|
||||||
|
$size = [uint32]([Math]::Pow(2, 32 - $bits))
|
||||||
|
$bogStart.Add($base); $bogEnd.Add([uint32]($base + $size - 1))
|
||||||
|
}
|
||||||
|
$bogStart = $bogStart.ToArray(); $bogEnd = $bogEnd.ToArray()
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
# Streaming download with a live progress bar (Write-Progress) so large feeds show real byte progress.
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
function Get-Url {
|
||||||
|
param([string]$Url, [string]$Label)
|
||||||
|
$req = [System.Net.HttpWebRequest]::Create($Url)
|
||||||
|
$req.UserAgent = $UA; $req.Timeout = 120000; $req.ReadWriteTimeout = 120000
|
||||||
|
$resp = $req.GetResponse()
|
||||||
|
try {
|
||||||
|
$total = $resp.ContentLength
|
||||||
|
$stream = $resp.GetResponseStream()
|
||||||
|
$ms = New-Object System.IO.MemoryStream
|
||||||
|
$buf = New-Object byte[] (1MB)
|
||||||
|
$read = 0; $lastReport = 0
|
||||||
|
while (($n = $stream.Read($buf, 0, $buf.Length)) -gt 0) {
|
||||||
|
$ms.Write($buf, 0, $n); $read += $n
|
||||||
|
if ($read - $lastReport -ge 2MB) {
|
||||||
|
$lastReport = $read
|
||||||
|
if ($total -gt 0) {
|
||||||
|
Write-Progress -Activity ("Downloading {0}" -f $Label) -PercentComplete ([int](100 * $read / $total)) `
|
||||||
|
-Status ("{0:N1} / {1:N1} MB" -f ($read / 1MB), ($total / 1MB))
|
||||||
|
} else {
|
||||||
|
Write-Progress -Activity ("Downloading {0}" -f $Label) -Status ("{0:N1} MB" -f ($read / 1MB))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Write-Progress -Activity ("Downloading {0}" -f $Label) -Completed
|
||||||
|
return [System.Text.Encoding]::UTF8.GetString($ms.ToArray())
|
||||||
|
}
|
||||||
|
finally { $resp.Close() }
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
# Collect every kept feed into ONE global set, timing each phase.
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
$singles = [System.Collections.Generic.HashSet[uint32]]::new()
|
||||||
|
$cidrs = [System.Collections.Generic.HashSet[string]]::new()
|
||||||
|
$feedCount = 0
|
||||||
|
|
||||||
|
foreach ($feed in $AllFeeds) {
|
||||||
|
$before = $singles.Count + $cidrs.Count
|
||||||
|
$ok = $false
|
||||||
|
foreach ($url in $feed.Urls) {
|
||||||
|
try {
|
||||||
|
$dlSw = [System.Diagnostics.Stopwatch]::StartNew()
|
||||||
|
$content = Get-Url -Url $url -Label $feed.Name
|
||||||
|
$dlSw.Stop()
|
||||||
|
$mb = [Math]::Round($content.Length / 1MB, 1)
|
||||||
|
|
||||||
|
$pSw = [System.Diagnostics.Stopwatch]::StartNew()
|
||||||
|
[void][BlocklistExporter]::AddContent($content, $singles, $cidrs, $bogStart, $bogEnd)
|
||||||
|
$pSw.Stop()
|
||||||
|
Write-Host (" [dl {0,6:N1}s / parse {1,5:N1}s] {2}" -f $dlSw.Elapsed.TotalSeconds, $pSw.Elapsed.TotalSeconds, ("{0} ({1} MB)" -f $feed.Name, $mb))
|
||||||
|
$ok = $true
|
||||||
|
}
|
||||||
|
catch { Write-Warning ("{0}: {1} -- skipping shard ({2})" -f $feed.Name, $url, $_.Exception.Message) }
|
||||||
|
}
|
||||||
|
if ($ok) {
|
||||||
|
$feedCount++
|
||||||
|
$delta = ($singles.Count + $cidrs.Count) - $before
|
||||||
|
Write-Host ("{0,-16} +{1,8} new (running total {2} ip / {3} cidr)`n" -f $feed.Name, $delta, $singles.Count, $cidrs.Count)
|
||||||
|
}
|
||||||
|
else { Write-Warning ("{0}: all sources failed -- skipping" -f $feed.Name) }
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
# Optional: subtract Tor / open-proxy / VPN IPs.
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
if ($ExcludeAnonymizers) {
|
||||||
|
$anon = [System.Collections.Generic.HashSet[uint32]]::new()
|
||||||
|
$anonCidr = [System.Collections.Generic.HashSet[string]]::new()
|
||||||
|
foreach ($url in $AnonFeeds) {
|
||||||
|
try { [void][BlocklistExporter]::AddContent((Get-Url -Url $url -Label 'anonymizers'), $anon, $anonCidr, $bogStart, $bogEnd) }
|
||||||
|
catch { Write-Warning ("anonymizer list {0}: {1}" -f $url, $_.Exception.Message) }
|
||||||
|
}
|
||||||
|
$removed = 0
|
||||||
|
foreach ($ip in @($anon)) { if ($singles.Remove($ip)) { $removed++ } }
|
||||||
|
Write-Host ("ExcludeAnonymizers: removed {0} Tor/proxy/VPN single IPs" -f $removed)
|
||||||
|
}
|
||||||
|
|
||||||
|
$total = $singles.Count + $cidrs.Count
|
||||||
|
Write-Host ("Merged {0} feed(s): {1} unique single IPs + {2} unique CIDRs (bogon-filtered) in {3:N1}s." -f `
|
||||||
|
$feedCount, $singles.Count, $cidrs.Count, $totalSw.Elapsed.TotalSeconds)
|
||||||
|
|
||||||
|
if ($DryRun) {
|
||||||
|
Write-Host ("DRY RUN: nothing written (would have written {0} entries to {1})." -f $total, $OutFile)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
# A partial feed outage must not silently shrink the shard's blocklist to nothing; keep the last good file.
|
||||||
|
if ($total -eq 0) { throw "No entries parsed -- refusing to overwrite '$OutFile' with an empty list." }
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
# Write to a .tmp sibling and swap it into place, so the shard (which reads the whole file on a change)
|
||||||
|
# never observes a half-written list. One rename does it whether or not a list is already there.
|
||||||
|
# ---------------------------------------------------------------------------------------------------------
|
||||||
|
$outDir = Split-Path -Parent $OutFile
|
||||||
|
if ($outDir -and -not (Test-Path -LiteralPath $outDir -PathType Container)) {
|
||||||
|
New-Item -ItemType Directory -Path $outDir -Force | Out-Null
|
||||||
|
}
|
||||||
|
|
||||||
|
# InvariantCulture: ':' is the culture-defined time separator in a custom format string, and the
|
||||||
|
# header is a machine-read marker the shard compares verbatim.
|
||||||
|
$generated = [DateTime]::UtcNow.ToString('yyyy-MM-ddTHH:mm:ssZ', [Globalization.CultureInfo]::InvariantCulture)
|
||||||
|
$header = "# modernuo-blocklist generated=$generated count=$total ipv4=$($singles.Count) cidr=$($cidrs.Count) feeds=$feedCount"
|
||||||
|
|
||||||
|
$tmp = $OutFile + '.tmp'
|
||||||
|
$wSw = [System.Diagnostics.Stopwatch]::StartNew()
|
||||||
|
try {
|
||||||
|
[BlocklistExporter]::Write($tmp, $header, $singles, $cidrs)
|
||||||
|
# One atomic rename over the destination on every platform: MoveFileEx REPLACE_EXISTING on
|
||||||
|
# Windows, rename(2) on Linux and macOS.
|
||||||
|
[IO.File]::Move($tmp, $OutFile, $true)
|
||||||
|
}
|
||||||
|
finally {
|
||||||
|
# Never leave a partial .tmp next to a live blocklist for the next run to trip over.
|
||||||
|
if (Test-Path -LiteralPath $tmp -PathType Leaf) { Remove-Item -LiteralPath $tmp -Force -ErrorAction SilentlyContinue }
|
||||||
|
}
|
||||||
|
$wSw.Stop()
|
||||||
|
|
||||||
|
$sizeMb = [Math]::Round((Get-Item -LiteralPath $OutFile).Length / 1MB, 1)
|
||||||
|
Write-Host ("`nWrote {0} entries ({1} MB) to {2} in {3:N1}s (total {4:N1}s). generated={5}" -f `
|
||||||
|
$total, $sizeMb, $OutFile, $wSw.Elapsed.TotalSeconds, $totalSw.Elapsed.TotalSeconds, $generated)
|
||||||
|
Write-Host "The shard picks this up on its next reloadInterval poll; no restart needed."
|
||||||
Loading…
Add table
Add a link
Reference in a new issue