warrentyexpired/ModernUO
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

perf(saves): chunked worker handoff, LPT blob scheduling, heap pre-sizing

Browse source 
Replaces the per-entity round-robin ConcurrentQueue handoff between the game
loop and the serialization workers with pooled 4096-entity chunks published to
a single shared queue. The producer's per-entity cost drops from a synchronized
enqueue to a plain array store, and workers pulling whole chunks load-balance
dynamically: a worker busy with a thick entity simply takes fewer chunks.

Scheduling changes so indivisible multi-megabyte payloads no longer extend the
freeze tail:
- Persistence.SerializeAll pushes systems largest-first (LPT), using the
  previous save's SerializedLength (or the loaded file size on first save).
- Persistence self-payloads and entities whose previous size exceeds 1MB are
  published as dedicated single-entity chunks so they spread across workers
  instead of riding inside one shared chunk.
- Entity SerializedLength is stamped from the index at load so estimates exist
  on the first save after boot.

Worker heaps are pre-sized from the loaded save's .bin sizes (25% headroom) so
the first save doesn't pay copy-on-grow inside the freeze, and the drain loop
uses SpinWait backoff (never Sleep(1)) instead of hammering the queue head
while the producer works. Snapshot writing uses a 1MB FileStream buffer, and
per-worker entity/byte counts are logged at Debug for balance diagnostics.

Measured on a 24-core machine with a synthetic 10M-entity, 1.7GB world
(64/64/32MB system payloads, 24x 2MB thick entities) through the real
pipeline classes: freeze window 740ms -> 122-160ms steady state (~5-6x),
first save 490ms -> 330ms, steady-state allocations converge to zero, and
worker byte loads converge (previous max/min spread ~2x -> ~1.15x for the
small-entity stream).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Kamron Batman 2026-07-13 22:32:51 -07:00
parent 3cb077a79e
commit 230c39851f
No known key found for this signature in database
GPG key ID: 7D81DF26D9A5D94A
7 changed files with 503 additions and 36 deletions
Download patch file Download diff file Expand all files Collapse all files

70
Projects/Server/Serialization/SerializationThreadWorker.cs
View file

@ -14,7 +14,6 @@
*************************************************************************/
using System;
using System.Collections.Concurrent;
using System.Runtime.CompilerServices;
using System.Threading;
@ -27,23 +26,30 @@ public class SerializationThreadWorker
private readonly Thread _thread;
private readonly AutoResetEvent _startEvent; // Main thread tells the thread to start working
private readonly AutoResetEvent _stopEvent; // Main thread waits for the worker finish draining
private readonly SerializationChunkSource _chunkSource;
private readonly int _heapSizeHint;
private bool _pause;
private bool _exit;
private bool _exited;
private byte[] _heap;
private long _entitiesSerialized;
private long _bytesSerialized;
private readonly ConcurrentQueue<IGenericSerializable> _entities;
public SerializationThreadWorker(int index)
public SerializationThreadWorker(int index, SerializationChunkSource chunkSource, int heapSizeHint = 0)
{
_index = index;
_chunkSource = chunkSource;
_heapSizeHint = heapSizeHint;
_startEvent = new AutoResetEvent(false);
_stopEvent = new AutoResetEvent(false);
_entities = new ConcurrentQueue<IGenericSerializable>();
_thread = new Thread(Execute);
_thread.Start(this);
}
// Stats from the most recent save, for diagnosing load balance.
public long EntitiesSerialized => _entitiesSerialized;
public long BytesSerialized => _bytesSerialized;
public void Wake()
{
_startEvent.Set();
@ -68,43 +74,77 @@ public class SerializationThreadWorker
Sleep();
}
public void AllocateHeap() => _heap ??= GC.AllocateUninitializedArray<byte>(MinHeapSize); // 1MB
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public void Push(IGenericSerializable entity) => _entities.Enqueue(entity);
// Sized from the previous world load so the first save doesn't pay copy-on-grow during the freeze.
public void AllocateHeap() =>
_heap ??= GC.AllocateUninitializedArray<byte>(Math.Max(MinHeapSize, _heapSizeHint));
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ReadOnlySpan<byte> GetHeap(int start, int length) => _heap.AsSpan(start, length);
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static void Serialize(IGenericSerializable e, BufferWriter writer, byte threadIndex)
{
e.SerializedThread = threadIndex;
var start = e.SerializedPosition = (int)writer.Position;
e.Serialize(writer);
e.SerializedLength = (int)(writer.Position - start);
}
private static void Execute(object obj)
{
var worker = (SerializationThreadWorker)obj;
var threadIndex = (byte)worker._index;
var queue = worker._entities;
var chunkSource = worker._chunkSource;
var serializedTypes = World.SerializedTypes;
while (worker._startEvent.WaitOne())
{
var writer = new BufferWriter(worker._heap, true, serializedTypes);
var entities = 0L;
var spinner = new SpinWait();
while (true)
{
var pauseRequested = Volatile.Read(ref worker._pause);
if (queue.TryDequeue(out var e))
if (chunkSource.TryTake(out var chunk))
{
e.SerializedThread = threadIndex;
var start = e.SerializedPosition = (int)writer.Position;
e.Serialize(writer);
e.SerializedLength = (int)(writer.Position - start);
spinner.Reset();
if (chunk.Single != null)
{
Serialize(chunk.Single, writer, threadIndex);
entities++;
}
else
{
var buffer = chunk.Buffer;
var count = chunk.Count;
for (var i = 0; i < count; i++)
{
Serialize(buffer[i], writer, threadIndex);
}
entities += count;
chunkSource.Return(buffer, count);
}
}
else if (pauseRequested) // Break when finished
{
break;
}
else
{
// Idle backoff instead of hammering the queue head while the producer works.
// sleep1Threshold: -1 keeps escalation at Yield/Sleep(0) and never Sleep(1),
// avoiding timer-resolution stalls at the end of the drain.
spinner.SpinOnce(-1);
}
}
worker._heap = writer.Buffer;
worker._entitiesSerialized = entities;
worker._bytesSerialized = writer.Position;
writer.Close();