Every ISerializable carried SerializedThread/SerializedPosition/SerializedLength
so WriteSnapshot could gather each entity's bytes from the worker heaps in
dictionary order. The idx records absolute positions, so bin order is free -
the snapshot can be written in worker-heap order instead, and the join inverts:
- Chunks are persistence-homogeneous: SerializeAll declares the owner at each
boundary, publishing the partial chunk on change.
- Workers log segments (owner, slot range, heap start) plus one length per
record as they serialize. Positions are implicit because a worker's writes
are contiguous; identity comes from re-walking the same snapshot slots in
the same order (guaranteed stable - mutations divert to the pending queues
until PostWorldSave), or from an entities log on the fallback path.
- WriteSnapshot routes segments by owner, emits idx entries during the
re-walk, and writes each segment's heap bytes as a single span instead of
one copy per entity, which also speeds up the background write phase.
- Persistence self-payloads keep placement as three private fields on the
~dozens of persistence instances; PushSingle is now typed accordingly.
Net effect: 9 bytes (plus padding) of resident state removed from every item,
mobile, guild, and account on every shard; three interface-property stores
per entity leave the drain hot path (stamping dirtied one cache line per
entity mid-freeze - the lengths log is a single sequential stream); and the
vestigial loader-side length stamp is gone. The transient cost is ~4 bytes
per entity in pooled per-worker logs that are released after each write.
The save format is unchanged (idx v3, same loader); only the write-side
mechanics moved. Adds an end-to-end round-trip test that drives real workers
through the chunk source, snapshots from the segment logs, and reloads.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Measured (10M entities through the real chunk source, both monomorphic and
16-subclass polymorphic populations): a bare `foreach { PushToCache(entity); }`
runs at 2.3ns/entity while the same loop carrying the heavy-entity check
(interface SerializedLength read + branch) runs at 5.3-5.7ns - 2.3x slower.
Type diversity barely matters; the cost is the fatter loop body, confirming
that per-entity logic in the push loop defeats the JIT's tight-loop codegen.
Entities over 1MB are rare in practice - realistically only whole
GenericPersistence self-payloads, which are already published as dedicated
single chunks - so the fallback loop drops the check and rare thick entities
ride inside shared chunks (bounded tail, same behavior as the slot-range fast
path). The now-unused HeavyEntityThreshold constant is removed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the per-entity round-robin ConcurrentQueue handoff between the game
loop and the serialization workers with pooled 4096-entity chunks published to
a single shared queue. The producer's per-entity cost drops from a synchronized
enqueue to a plain array store, and workers pulling whole chunks load-balance
dynamically: a worker busy with a thick entity simply takes fewer chunks.
Scheduling changes so indivisible multi-megabyte payloads no longer extend the
freeze tail:
- Persistence.SerializeAll pushes systems largest-first (LPT), using the
previous save's SerializedLength (or the loaded file size on first save).
- Persistence self-payloads and entities whose previous size exceeds 1MB are
published as dedicated single-entity chunks so they spread across workers
instead of riding inside one shared chunk.
- Entity SerializedLength is stamped from the index at load so estimates exist
on the first save after boot.
Worker heaps are pre-sized from the loaded save's .bin sizes (25% headroom) so
the first save doesn't pay copy-on-grow inside the freeze, and the drain loop
uses SpinWait backoff (never Sleep(1)) instead of hammering the queue head
while the producer works. Snapshot writing uses a 1MB FileStream buffer, and
per-worker entity/byte counts are logged at Debug for balance diagnostics.
Measured on a 24-core machine with a synthetic 10M-entity, 1.7GB world
(64/64/32MB system payloads, 24x 2MB thick entities) through the real
pipeline classes: freeze window 740ms -> 122-160ms steady state (~5-6x),
first save 490ms -> 330ms, steady-state allocations converge to zero, and
worker byte loads converge (previous max/min spread ~2x -> ~1.15x for the
small-entity stream).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
