The shard owner, on a Starlink CGNAT address, was blocked by the imported reputation blocklist. The cause was not CrowdSec: the address was a literal line in ip-blocklist.txt, so BlocklistFilter denied it at accept and then promoted it. Clearing the CrowdSec decision could not fix it either, because the file entry re-reports within promoteSuppression of every reconnect. Reputation feeds list shared consumer address space constantly. On CGNAT one public address fronts many subscribers at the same time, so a single abusive customer gets the address listed and everyone else behind it is blocked with them; where leases rotate, a listing says little about whoever holds the address now. So exemptions go where they cost nothing, and escalation is driven by what a connection actually does. GENERATOR (tools/Export-IpBlocklist.ps1) - -AllowlistFile takes multiple paths, subtracted from the merged set before the output is written. Defaults to the operator's own ip-allowlist.txt (created once, never rewritten) plus one generated file per network carve-out. - Subtraction is range-correct: an allowlisted address inside a blocked CIDR splits that CIDR around the hole instead of being ignored. This also fixes -ExcludeAnonymizers, which parsed CIDR entries and then only subtracted singles. - Carve-outs are a table (name, ASN, reason, offline seed) rather than a hardcoded network, so covering another CGNAT provider is one row. -RefreshCarveouts re-fetches from the ASN's current routing announcements and collapses them; -Carveout '' subtracts none. Announcements rather than ownership records, because registry data disagrees with what is routed and caps its result sets. - Editing an allowlist bypasses -MinInterval, so a just-added exemption is not indistinguishable from the allowlist not working. - The shipped starlink carve-out costs ~0.1% of the list. ALLOWLISTS - FileAllowlist reads the same files the generator subtracts, so an operator entry means "leave this address alone" for real. Subtraction alone only covers being BLOCKED; behavioural detections never consult the blocklist, so without this a carve-out was quietly routed around and one scanner behind a shared address was enough to get everyone behind it firewalled. Reading the files also means an entry applies on the next reload rather than the next regeneration. - LoginAllowlist is earned by authenticating, with a 90 day TTL because an address that logged in years ago is a stranger. Its own store rather than Account.LoginIPs, which has no timestamps and cannot be backfilled. An entry is evidence rather than a licence: 10 suppressed contributions in an hour revokes it, and a fresh login forgives the tally. - Both are consulted only AFTER the blocklist has already matched, so a normal accept pays nothing for them and the accept gate stays allowlist-free. - BanExemptions combines them behind BanChannel.IsExempt. Suppresses escalation only; every local defence still applies. BEHAVIOURAL DETECTION - silent-connect (reaped having sent zero bytes) and invalid-seed (a zero seed) are contributed. Both were already disconnected. - ForeignProtocol positively identifies HTTP, TLS and SSH. Asking "is this a good UO client?" cannot work: LoginEncryption.ClientDecrypt is a byte-for-byte stream XOR, so a client with encryption on when the shard expects none sends a structurally perfect connection whose payload is noise. Nothing assumes arrival framing, since TCP has no message boundaries and a rule of the form "these bytes must arrive together" drops real players on poor links. - Keyed on bytes-received rather than elapsed time throughout. A connection that sent something and ran out of time is far more likely a slow link, and banning those makes the player retry, trip the rate limiter, and compound it. - AutoDenylist holds behavioural detections locally for 15 minutes, as an IConnectionFilter plus IBanReporter over one store so engine detection sites never reach into content. Closes the gap where a flood pays for a socket and a NetState per connection while waiting for an OS bouncer, and is the whole defence on a shard running none. Not persisted: a holding pen that survives restarts is a ban without a ban's review. - BanReasons centralises the slugs. IsBehavioral is an opt-in set, not "everything except manual", so a future reason escalates normally instead of silently inheriting an exemption. The first cut of the exemption swallowed manual admin bans; this is why. FIXES - BanConfiguration.Settings was null until Configure() ran while the reap path dereferences it every Slice(), so a harness driving NetState.Slice() directly hit an NRE that looked flaky because it depended on test ordering. - -AllowlistFile was typed [string] while documented and used as a list. LAYOUT AND DOCS Content network code moves out of Misc into UOContent/Network, one concern per folder. Namespaces are untouched, so these are pure file moves. dev-docs/ip-bans-and-allowlists.md documents the subsystem, leading with the operator process for unblocking a player -- including the three things that look sufficient and are not: deleting the CrowdSec decision alone, editing ip-blocklist.txt by hand, and cscli allowlists alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
116 lines
4.8 KiB
C#
116 lines
4.8 KiB
C#
/*************************************************************************
|
|
* ModernUO *
|
|
* Copyright 2019-2026 - ModernUO Development Team *
|
|
* Email: hi@modernuo.com *
|
|
* File: ForeignProtocolTests.cs *
|
|
* *
|
|
* This program is free software: you can redistribute it and/or modify *
|
|
* it under the terms of the GNU General Public License as published by *
|
|
* the Free Software Foundation, either version 3 of the License, or *
|
|
* (at your option) any later version. *
|
|
* *
|
|
* You should have received a copy of the GNU General Public License *
|
|
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
|
*************************************************************************/
|
|
|
|
using System.Text;
|
|
using Server.Network;
|
|
using Xunit;
|
|
|
|
namespace Server.Tests.Network;
|
|
|
|
public class ForeignProtocolTests
|
|
{
|
|
private static ForeignProtocolMatch Identify(byte[] bytes, out ForeignProtocolKind kind) =>
|
|
ForeignProtocol.Identify(bytes, out kind);
|
|
|
|
private static byte[] Ascii(string s) => Encoding.ASCII.GetBytes(s);
|
|
|
|
[Theory]
|
|
[InlineData("GET / HTTP/1.1\r\n")]
|
|
[InlineData("POST /a HTTP/1.1\r\n")]
|
|
[InlineData("HEAD / HTTP/1.0\r\n")]
|
|
[InlineData("OPTIONS * HTTP/1.1\r\n")]
|
|
[InlineData("CONNECT host:443 HTTP/1.1\r\n")]
|
|
[InlineData("DELETE /x HTTP/1.1\r\n")]
|
|
public void Http_requests_are_identified(string request)
|
|
{
|
|
Assert.Equal(ForeignProtocolMatch.Confirmed, Identify(Ascii(request), out var kind));
|
|
Assert.Equal(ForeignProtocolKind.Http, kind);
|
|
}
|
|
|
|
[Fact]
|
|
public void Ssh_banner_is_identified()
|
|
{
|
|
Assert.Equal(ForeignProtocolMatch.Confirmed, Identify(Ascii("SSH-2.0-OpenSSH_9.6"), out var kind));
|
|
Assert.Equal(ForeignProtocolKind.Ssh, kind);
|
|
}
|
|
|
|
[Fact]
|
|
public void Tls_client_hello_is_identified()
|
|
{
|
|
// handshake, TLS 1.2 record, length 0x0100, ClientHello
|
|
byte[] hello = [0x16, 0x03, 0x03, 0x01, 0x00, 0x01, 0x00, 0x00, 0xFC];
|
|
|
|
Assert.Equal(ForeignProtocolMatch.Confirmed, Identify(hello, out var kind));
|
|
Assert.Equal(ForeignProtocolKind.Tls, kind);
|
|
}
|
|
|
|
[Fact]
|
|
public void Tls_prefix_without_a_client_hello_is_not_foreign()
|
|
{
|
|
// A seed can spell 0x16 0x03 0x0? -- the address 22.3.x.x. Byte five is a packet id, not a
|
|
// handshake type, so it must fall through to normal parsing.
|
|
byte[] seedThenLogin = [0x16, 0x03, 0x03, 0x04, 0x00, 0x80, 0x00, 0x00];
|
|
|
|
Assert.Equal(ForeignProtocolMatch.None, Identify(seedThenLogin, out var kind));
|
|
Assert.Equal(ForeignProtocolKind.None, kind);
|
|
}
|
|
|
|
[Fact]
|
|
public void Seed_that_spells_an_http_method_falls_through()
|
|
{
|
|
// Seed 0x47455420 spells "GET ". The next byte is the 0x80 login packet id, not printable, so this
|
|
// is a real client and must not be flagged.
|
|
byte[] seedThenLogin = [(byte)'G', (byte)'E', (byte)'T', (byte)' ', 0x80, 0x00, 0x3A, 0x00];
|
|
|
|
Assert.Equal(ForeignProtocolMatch.None, Identify(seedThenLogin, out _));
|
|
}
|
|
|
|
[Theory]
|
|
[InlineData(0x80)] // login request
|
|
[InlineData(0x91)] // game server login
|
|
[InlineData(0xEF)] // new-style seed packet
|
|
public void Ordinary_uo_openings_are_not_foreign(byte secondPacketId)
|
|
{
|
|
byte[] buffer = [0x7F, 0x00, 0x00, 0x01, secondPacketId, 0x00, 0x00, 0x00];
|
|
|
|
Assert.Equal(ForeignProtocolMatch.None, Identify(buffer, out _));
|
|
}
|
|
|
|
[Fact]
|
|
public void Encrypted_login_is_not_mistaken_for_a_foreign_protocol()
|
|
{
|
|
// A legitimate client with encryption on when the shard expects none. The login cipher is a
|
|
// byte-for-byte XOR, so this is noise of exactly the right length.
|
|
byte[] buffer = [0x7F, 0x00, 0x00, 0x01, 0xC3, 0x9A, 0x04, 0xE1, 0x55, 0xB2];
|
|
|
|
Assert.Equal(ForeignProtocolMatch.None, Identify(buffer, out _));
|
|
}
|
|
|
|
[Fact]
|
|
public void Prefix_match_without_enough_bytes_waits()
|
|
{
|
|
// Framing is never assumed: "GET " split from its request line must wait.
|
|
Assert.Equal(ForeignProtocolMatch.Incomplete, Identify(Ascii("GET "), out _));
|
|
Assert.Equal(ForeignProtocolMatch.Incomplete, Identify(Ascii("GET /"), out _));
|
|
}
|
|
|
|
[Fact]
|
|
public void Too_few_bytes_to_match_a_prefix_is_not_foreign()
|
|
{
|
|
// Under four bytes the caller's own short-read handling applies.
|
|
Assert.Equal(ForeignProtocolMatch.None, Identify(Ascii("GE"), out _));
|
|
Assert.Equal(ForeignProtocolMatch.None, Identify([], out _));
|
|
}
|
|
}
|