warrentyexpired/ModernUO
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

fix(network): stop sending once a disconnect is handed to the socket (#2637)

Browse source 
## Problem

When a client's send buffer fills, the `send buffer exhausted` warning repeats for every packet, every tick, until the NetState is finally disposed. Before the io_uring transport an overflow produced one message plus the disconnect line.

## Root cause

Since #2315, `NetState.Disconnect()` only queues. The NetState keeps running, the Mobile stays attached, and in `Slice()` the queued disconnect becomes `RingSocket.Disconnect()`, which sees buffered or in-flight sends and merely sets `DisconnectPending` while the transport drains. Nothing stopped game logic from writing into that buffer afterwards, so:

- every broadcast to that player still reached `Send()`, hit the full buffer, and re-reported exhaustion (made visible by #2551);
- refills kept `ReadableBytes` above zero, so the graceful drain could never finish, and `DataSent` completions kept pushing the alive check out. A slow-but-acking client could keep a "disconnected" session attached indefinitely.

Two more sources of the same warning surfaced during the analysis: `Dispose()` sets `_running = false` before nulling `Mobile.NetState`, and the setter's bank-close / target-cancel packets then reported `0 writable`; and when the socket takes the immediate-close branch (nothing in flight) `Connected` drops without `DisconnectPending`, leaving a one-tick window that also re-reported.

## Changes

- `CannotSendPackets()` refuses once the socket is `DisconnectPending` or no longer `Connected`. Sends between `Disconnect()` and the `Slice()` handoff are still delivered (kicks with a message, the play-server ack).
- `SendBufferExhausted()` reports once per disconnect and keeps the first reason.
- `Send()` is silent while closing instead of reporting exhaustion for a socket that is going away.
- `_nextAliveCheck` is seeded from a real tick (the zero default suppressed the alive sweep on hosts whose counter starts negative).
- `CancelAllTrades()` had its null guard inverted since 547c2ea0f (#2603) and never cancelled anything.

## Does the gate cut off the graceful flush?

No. The gate only refuses writes made after `Slice()` has handed the disconnect to the socket. Everything committed before that point is drained by the transport, which then sends FIN. This matches the pre-io_uring lifecycle: `Disconnect()` cleared `_running` at once, and the next `Slice()` made its final `Flush()` and then closed the socket in `Dispose()`. In both worlds the send window after `Disconnect()` ends at the next network slice; the old one made a single flush attempt, the new one drains everything buffered.

Checked flows:

- **Login gateway.** `PlayServer` sends the 0x8C ack, and the parser queues `Disconnect()` in the same `HandleReceive` call. Both happen before the handoff, so the ack is delivered and FIN follows. ClassicUO's `HandleRelayServerPacket` disconnects and opens a fresh connection before sending the seed and second login, so the game login is a new NetState. The `LoginServer_ServerSelectAck` "CUO/Orion do not reconnect" fallback (#489, 2021) resets the parser state and returns without parsing or replying, and no further receive is posted once the disconnect is pending, so it sends nothing either way. Orion likewise opens a separate game socket before closing the login one.
- **Login rejections, character create/select/delete errors, duplicate-packet guards.** Each sends its rejection first and calls `Disconnect()` in the same handler.
- **Kicks and bans** (`[kick`, `[ban`, AdminGump, ClientGump, ClientVerification, AssistantHandler, lockdown). The message is sent first; delayed variants fire `Disconnect()` from a timer with nothing sent afterwards.
- **Main loop order.** Timers run before `NetState.Slice()`, packet handlers run inside it before the flush, and `LoopContext` tasks run after it; in every case a send that precedes `Disconnect()` reaches the buffer before the next handoff.

`Send_BeforeDisconnect_IsDeliveredThenPeerSeesEof` and `Send_LargeBeforeDisconnect_IsFullyDrainedThenPeerSeesEof` read the bytes back from the peer socket after the handoff (the latter 192 KB across several send completions, past the loopback kernel buffers) and then wait for the FIN, proving delivery and clean close with the gate in place.

## Tests

Seven new tests in `NetStateDisconnectTests` over real loopback sockets: force-close after the drain deadline, delivery then EOF for small and multi-completion sends before `Disconnect()`, send dropped after the handoff, send dropped after an immediate close, exhaustion reported once with the first reason kept, and trades cancelled on disconnect. `MockAccount` promoted to a shared test helper. Server.Tests 876 passed, UOContent.Tests 1048 passed against the published 1.0.11.

## Drain deadline and IORingGroup 1.0.11

A socket handed a disconnect drains what is buffered and closes once the peer has acknowledged it. Send completions keep `NextActivityCheck` moving, so a slow but acking peer could hold a closing socket open indefinitely. A deadline (`DrainTimeoutMs`, 10 s) is now armed at the handoff, or on first sight of a transport-initiated drain in `CheckAlive`, and force-closes when it passes, independent of the inactivity check.

That force-close is only safe with IORingGroup 1.0.11 (modernuo/IORingGroup#12), which this PR bumps to. Before it, `DisconnectImmediate` released pooled buffers while recv/send operations could still be in flight, a failed send stranded the socket forever, and a recv completion could be delivered after its buffer was released. 1.0.11 retires every outstanding operation before release, aborts on a failed send, and holds buffers until the pass after the `Disconnected` event.
This commit is contained in:
Kamron Batman 2026-09-11 23:18:37 -07:00 • committed by GitHub
parent 309fcfeb27
commit d16166591c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 393 additions and 58 deletions
Download patch file Download diff file Expand all files Collapse all files

71
Projects/Server/Network/NetState/NetState.cs
View file

@ -36,6 +36,7 @@ public partial class NetState : IComparable<NetState>, IValueLinkListNode<NetSta
private const int HuePickerCap = 512;
private const int MenuCap = 512;
private const int PacketPerSecondThreshold = 3000;
internal const long DrainTimeoutMs = 10000; // graceful disconnect gets this long to drain
private static readonly Queue<NetState> _flushPending = new(2048);
private static readonly Queue<NetState> _pendingDisconnects = new(256); // Processed AFTER flush
@ -54,7 +55,9 @@ public partial class NetState : IComparable<NetState>, IValueLinkListNode<NetSta
private bool _disconnectQueued; // Queued for disconnect processing (after flush)
private long[] _packetThrottles;
private long[] _packetCounts;
private string _disconnectReason = string.Empty;
internal string _disconnectReason = string.Empty;
private long _drainDeadline;
private bool _drainDeadlineArmed;
internal ParserState _parserState = ParserState.AwaitingNextPacket;
internal ProtocolState _protocolState = ProtocolState.AwaitingSeed;
@ -294,7 +297,8 @@ public partial class NetState : IComparable<NetState>, IValueLinkListNode<NetSta
for (var i = Trades.Count - 1; i >= 0; --i)
{
if (Trades != null)
// RemoveTrade() nulls the list once empty
if (Trades == null)
{
break;
}
@ -489,6 +493,12 @@ public partial class NetState : IComparable<NetState>, IValueLinkListNode<NetSta
return;
}
// Closing; nothing to report
if (!_running || _socket == null)
{
return;
}
// Never drop silently: the client would stay connected while missing game state.
if (!GetSendBuffer(out var buffer))
{
@ -552,6 +562,12 @@ public partial class NetState : IComparable<NetState>, IValueLinkListNode<NetSta
/// </remarks>
private void SendBufferExhausted(int needed, int writable)
{
// One report per disconnect; the first reason wins
if (_disconnectQueued)
{
return;
}
var sendBuffer = _socket?.SendBuffer;
var unacked = sendBuffer?.InFlightBytes ?? 0;
var capacity = sendBuffer?.PhysicalSize ?? 0;
@ -1053,31 +1069,52 @@ public partial class NetState : IComparable<NetState>, IValueLinkListNode<NetSta
return ParserState.AwaitingNextPacket;
}
// Bounds the graceful drain. Send completions keep NextActivityCheck moving, so a slow peer
// could otherwise hold a closing socket open indefinitely.
internal void ArmDrainDeadline(long curTicks)
{
if (!_drainDeadlineArmed)
{
_drainDeadlineArmed = true;
_drainDeadline = curTicks + DrainTimeoutMs;
}
}
public void CheckAlive(long curTicks)
{
if (_socket == null || NextActivityCheck - curTicks >= 0)
if (_socket == null)
{
return;
}
if (_socket.DisconnectPending)
{
LogInfo("Force disconnecting stuck socket...");
_socketManager.DisconnectImmediate(_socket);
}
else
{
// Authenticated pre-game clients (login screens): send keep-alive instead of disconnecting.
// The 0xBD ClientVersionRequest resets NextActivityCheck via DataSent.
if (_account != null && Mobile == null)
ArmDrainDeadline(curTicks); // transport-initiated drains are first seen here
if (curTicks - _drainDeadline >= 0 || NextActivityCheck - curTicks < 0)
{
this.SendClientVersionRequest();
return;
LogInfo("Force disconnecting stuck socket...");
_socketManager.DisconnectImmediate(_socket);
}
LogInfo("Disconnecting due to inactivity...");
Disconnect("Disconnecting due to inactivity.");
return;
}
if (NextActivityCheck - curTicks >= 0)
{
return;
}
// Authenticated pre-game clients (login screens): send keep-alive instead of disconnecting.
// The 0xBD ClientVersionRequest resets NextActivityCheck via DataSent.
if (_account != null && Mobile == null)
{
this.SendClientVersionRequest();
return;
}
LogInfo("Disconnecting due to inactivity...");
Disconnect("Disconnecting due to inactivity.");
}
public void Trace(ReadOnlySpan<byte> buffer)
@ -1123,8 +1160,8 @@ public partial class NetState : IComparable<NetState>, IValueLinkListNode<NetSta
}
/// <summary>
/// Requests a graceful disconnect. The disconnect is queued and processed after the flush
/// queue in Slice(), ensuring Send() calls made in the same tick are processed first.
/// Requests a graceful disconnect. Processed after the flush queue in Slice(): sends made before
/// that handoff are flushed first, sends after it are dropped (see CannotSendPackets).
/// </summary>
public void Disconnect(string reason)
{