> [!IMPORTANT]
> **Breaking Changes**
> - DecodePacket and EncodePacket delegates replaced with IClientEncryption interface
> - NetState.Connection (Socket) replaced with internal RingSocket management
> - NetState.RecvPipe and NetState.SendPipe removed (buffers managed internally)
## Summary
Upgrades the networking stack from PollGroup-based I/O to io_uring, significantly improving I/O performance on Linux.
This also adds native client encryption support for encrypted UO clients.
## Major Changes
io_uring Networking Architecture
- Replaced PollGroup with IORingGroup for async socket I/O operations
- Removed Pipe.cs (mirrored ring buffer) and TcpServer.cs in favor of RingSocketManager
- Added NetState.Network.cs - centralized network infrastructure handling accept, recv, send, and disconnect
completions
- Added SocketHelper.cs - platform-specific socket utilities for raw socket handle operations (getpeername,
getsockname)
- Buffer management now handled by RingSocketManager with configurable slab allocation
### Client Encryption Support
- Added full encryption stack in Network/Encryption/:
- EncryptionConfig.cs - configurable encryption modes (None, Unencrypted, Encrypted, Both)
- EncryptionManager.cs - encryption detection and initialization for login/game packets
- LoginEncryption.cs - handles login packet encryption with version-derived keys
- GameEncryption.cs - handles game server encryption using Twofish
- TwofishEngine.cs - optimized Twofish block cipher implementation
- LoginKeys.cs - encryption key table for client versions
- IClientEncryption.cs - interface for client encryption implementations
### NetState Improvements
- Replaced Socket Connection with RingSocket _socket for managed socket lifecycle
- Changed from GCHandle polling to event-based completion processing
- Disconnect handling now properly waits for pending sends to flush
- Simplified connecting socket management using lazy queue removal
### Configuration
- New settings: network.encryptionMode and network.encryptionDebug
- Encryption mode flags: Unencrypted, Encrypted, or Both
### Dependencies
- Replaced PollGroup NuGet package with IORingGroup
- Linux requires liburing-dev / liburing-devel package
### Test plan
- Verify server starts and accepts connections on Linux with io_uring
- Verify server starts and accepts connections on Windows (fallback to IOCP)
- Test unencrypted client connections (ClassicUO with encryption disabled)
- Test encrypted client connections if available
- Verify graceful disconnect flushes pending data
- Confirm CI builds pass on all target platforms
## Summary
- Adds proper Latin1 encoding support, replacing CP1252 usage throughout the codebase
- Adds specialized, optimized string decoding methods with safe string filtering for each encoding type
- Filters invalid Unicode characters (C0/C1 control codes, non-characters) by removal rather than replacement since
the UO client renders nothing for these characters
- Fixes UTF-16 null terminator position handling to correctly advance by 2 bytes
## Changes
TextEncoding.cs
- Added SearchValues-based invalid byte/char detection for efficient filtering
- Added encoding-specific GetString methods: GetStringAscii, GetStringLatin1, GetStringUtf8, GetStringBigUni,
GetStringLittleUni
- Each method supports a safeString parameter for filtering invalid characters
- Little-endian UTF-16 uses direct memory cast for zero-copy decoding on LE systems
- Invalid characters are removed (not replaced with U+FFFD) since the client renders nothing for them
SpanReader.cs
- Added ReadLatin1() and ReadLatin1Safe() methods
- Rewrote encoding-specific read methods to use optimized TextEncoding.GetString* methods
- Fixed UTF-16 null terminator handling: position now correctly advances by byteLength (2) instead of 1
SpanWriter.cs
- Added WriteLatin1 and WriteLatin1Null methods
## Packet Updates
- Updated all packet code to use Latin1 encoding instead of CP1252
- Affected: account packets, equipment packets, menu packets, message packets, mobile packets, player packets, secure
trade packets, vendor packets, gump packets, book packets, mahjong packets
## Filtering Behavior
Invalid characters filtered in safe mode:
```
┌───────────────┬────────────────────────┐
│ Range │ Description │
├───────────────┼────────────────────────┤
│ 0x00-0x1F │ C0 control codes │
├───────────────┼────────────────────────┤
│ 0x7F │ DEL │
├───────────────┼────────────────────────┤
│ 0x80-0x9F │ C1 control codes │
├───────────────┼────────────────────────┤
│ 0xFFFE-0xFFFF │ Unicode non-characters │
└───────────────┴────────────────────────┘
```
Note: Surrogate pairs (0xD800-0xDFFF) are not filtered because proper validation requires context checking for paired
vs unpaired surrogates. The UO client renders nothing for these anyway.
## Test Plan
- All 631 Server.Tests pass
- Verified client rendering behavior using TestUnicodeGump command (pages 1-5)
- Confirmed U+FFFD, unpaired surrogates, and non-characters all render as blank in client
- Verified Latin1 characters (0xA0-0xFF) display correctly
- Verified C1 control codes (0x80-0x9F) are filtered and don't display
### Summary
- Create BitMask256 struct with scalar operations (benchmarks showed AVX2 vectorization provides no benefit for this size)
- Refactor Map.cs to use BitMask256 for full Z range support (-128 to 127)
- Remove SectorSpawnCache struct, use BitMask256 directly in manager
- Update tests to use BitMask256 directly
- Remove unused test assertions for old 64-bit behavior
### Summary
- Adds CanSpawnMobile(x, y, minZ, maxZ, canSwim, cantWalk, out spawnZ) overload for finding spawn surfaces within a Z range
- Adds CanSpawnItem(x, y, minZ, maxZ, out spawnZ) for item spawning with Surface+Impassable support (tables, furniture)
- Uses bitmask optimization inspired by Item.DropToWorld's m_OpenSlots pattern for O(1) surface/blocker checks
- HomeRange spawners now use surface detection to set proper Z bounds
### Key Changes
Map.cs:
- CanSpawnMobile with Z-range finds lowest valid surface for mobiles
- CanSpawnItem with Z-range finds lowest valid surface for items (including tables)
- CanFitItem for point-check item placement on Surface+Impassable tiles
- Bitmask approach eliminates nested loops and stackalloc arrays
Spawners:
- Simplified GetSpawnPosition using new Z-range methods
- HomeRange setter detects surface below spawner for proper Z bounds
- Consistent handling for mobiles and items
### Bug Fixes
- Water tiles (Impassable | Wet) no longer block swimming mobs
- Items can now spawn on tables/furniture (Surface+Impassable)
### Test Plan
- Run dotnet test - 631 tests pass
- Manual testing: multi-story spawning, water mobs, item spawning on tables
- Verify HomeRange spawner movement shifts bounds correctly
### Summary
Adds `XInRangeByDistance` and `XInBoundsByDistance` methods to `Map.cs`:
**Item Distance Enumeration:**
```cs
ItemDistanceEnumerable<Item> GetItemsInRangeByDistance(Point3D p);
ItemDistanceEnumerable<Item> GetItemsInRangeByDistance(Point3D p, int range);
ItemDistanceEnumerable<T> GetItemsInRangeByDistance<T>(Point3D p) where T : Item;
ItemDistanceEnumerable<T> GetItemsInRangeByDistance<T>(Point3D p, int range) where T : Item;
ItemDistanceEnumerable<Item> GetItemsInRangeByDistance(Point2D p);
ItemDistanceEnumerable<Item> GetItemsInRangeByDistance(Point2D p, int range);
ItemDistanceEnumerable<T> GetItemsInRangeByDistance<T>(Point2D p) where T : Item;
ItemDistanceEnumerable<T> GetItemsInRangeByDistance<T>(Point2D p, int range) where T : Item;
ItemDistanceEnumerable<Item> GetItemsInRangeByDistance(int x, int y, int range);
ItemDistanceEnumerable<T> GetItemsInRangeByDistance<T>(int x, int y, int range) where T : Item;
ItemDistanceEnumerable<Item> GetItemsInBoundsByDistance(Rectangle2D bounds, , bool makeBoundsInclusive = false);
ItemDistanceEnumerable<T> GetItemsInBoundsByDistance<T>(Rectangle2D bounds, bool makeBoundsInclusive = false) where T : Item;
```
**Mobile Distance Enumeration:**
```cs
MobileDistanceEnumerable<Mobile> GetMobilesInRangeByDistance(Point3D p);
MobileDistanceEnumerable<Mobile> GetMobilesInRangeByDistance(Point3D p, int range);
MobileDistanceEnumerable<T> GetMobilesInRangeByDistance<T>(Point3D p) where T : Mobile;
MobileDistanceEnumerable<T> GetMobilesInRangeByDistance<T>(Point3D p, int range) where T : Mobile;
MobileDistanceEnumerable<Mobile> GetMobilesInRangeByDistance(Point2D p);
MobileDistanceEnumerable<Mobile> GetMobilesInRangeByDistance(Point2D p, int range);
MobileDistanceEnumerable<T> GetMobilesInRangeByDistance<T>(Point2D p) where T : Mobile;
MobileDistanceEnumerable<T> GetMobilesInRangeByDistance<T>(Point2D p, int range) where T : Mobile;
MobileDistanceEnumerable<Mobile> GetMobilesInRangeByDistance(int x, int y, int range);
MobileDistanceEnumerable<T> GetMobilesInRangeByDistance<T>(int x, int y, int range) where T : Mobile;
MobileDistanceEnumerable<Mobile> GetMobilesInBoundsByDistance(Rectangle2D bounds, bool makeBoundsInclusive = false);
MobileDistanceEnumerable<T> GetMobilesInBoundsByDistance<T>(Rectangle2D bounds, bool makeBoundsInclusive = false) where T : Mobile;
```
**Client Distance Enumeration:**
```cs
ClientDistanceEnumerable GetClientsInRangeByDistance(Point3D p);
ClientDistanceEnumerable GetClientsInRangeByDistance(Point3D p, int range);
ClientDistanceEnumerable GetClientsInRangeByDistance(Point2D p);
ClientDistanceEnumerable GetClientsInRangeByDistance(Point2D p, int range);
ClientDistanceEnumerable GetClientsInRangeByDistance(int x, int y, int range);
ClientDistanceEnumerable GetClientsInBoundsByDistance(Rectangle2D bounds, bool makeBoundsInclusive = false);
```
**Example Usage:**
How to use `minDistance` to terminate early when all subsequent mobiles in the iteration will be at an increasing min distance.
```csharp
var playerLocation = player.Location;
const int maxRange = 100;
const int maxMobiles = 12;
var closestMobiles = new SortedSet<Mobile>(Comparer<Mobile>.Create((x, y) =>
{
var distX = x.GetDistanceToSqrt(playerLocation);
var distY = y.GetDistanceToSqrt(playerLocation);
int result = distX.CompareTo(distY);
if (result == 0)
{
result = (x?.Serial ?? Serial.MinusOne).CompareTo(y?.Serial ?? Serial.MinusOne);
}
return result;
}));
int lastMinDistance = 0;
foreach (var (mobile, minDistance) in map.GetMobilesInRangeByDistance(playerLocation, maxRange))
{
// Stop if we have enough and distance starts increasing
if (closestMobiles.Count >= maxMobiles && minDistance > lastMinDistance)
{
break;
}
closestMobiles.Add(mobile);
lastMinDistance = minDistance;
}
// Results are already ordered by proximity
foreach (var mobile in closestMobiles)
{
var actualDistance = mobile.GetDistanceToSqrt(playerLocation);
Console.WriteLine($"{mobile.Name}: ActualDist={actualDistance:F2}");
}
```
> [!IMPORTANT]
> **Dev Note:** This is an **important** patch as the bug could lead to major issues like:
> * Multis/Players disappearing from view or not being counted during game logic.
> * World processes (e.g., area checks, targeting) failing to detect entities correctly.
> * General stability and correctness concerns for core map functionality.
>
> **Important Breaking Change**: Multis now properly use the map link list. This means modifying a multi while iterating will cause the server to crash. The crash _is expected_. Please modify/fix code accordingly to create a list using `PooledRefQueue` or `PooledRefList` instead of moving/deleting multis while inside the foreach.
### Summary
* Fixes a bug where deleting/moving a multi (boat/house) in some circumstances can use undefined behavior due to unsafe changes to List<BaseMulti>
* Fixes a bug where Multis may not be considered while searching due to a bug causing the sector search to end early.
### Summary
* Bumps to .NET 9 with updated dependencies
* Comparing a value type against null is no longer allowed
* CI/CD now uses the version specified in global.json
* Serialization generator updated to .NET 9 with bug fixes, fixes to turkish language, and parallelization
### Summary
Fixes a few minor issues with timers:
- Timer.Delay and Timer.Interval was not reflecting the actual tick time (aligned to the next 8ms)
- Negative delay values were causing a crash when DateTime.Now - delay was below DateTime.MinValue
- Timer.Next now reflects the correct wall clock tick time based on the adjusted Delay.
- Timer.Next is not assigned when the timer is started. This was important for timers that were created, but started later.
### Summary
- `GenericEntityPersistence` is now a type of `GenericPersistence`. This allows developers to serialize both entities and non-entities in the same system. 🎉
- Each `SerializationThreadWorker` now allocates 1MB of heap for serialization _permanently_. If more memory is needed, that thread will double it's memory, not to exceed increments of 64MB.
- Several bugs with serialization introduced with the pure MMF implementation have been fixed.
- `BinaryFileReader` has been added back. 🎉
- Adds `world.useMultithreadedSaves` to allow disabling threaded saves.
> [!IMPORTANT]
> **Developer Note**
> The split file serialization has been deprecated and is no longer used. We have effectively gone back to the same file writing we had before the pure MMF implementation.
> [!Important]
> **Developer Note**
> This code change will **completely move gumps out of the core**
### Summary
- Adds `GetGumps()` convenience which exposes methods to Find/Close/Send multiple gumps. This helper is a performance improvement by eliminating the Dictionary<Player, List> lookup for gumps.
### Summary
* Added World.NewVirtual for creating virtual serial numbers
* Reserved range 0x7EEEEEEE to 0x7FFFFFFF for virtual serials
* Hair and Facial hair (for mobiles) now use virtual serials instead of FakeSerial() functions
* Consolidated virtual hair to a single `VirtualHairInfo` class.
Corpse hair and facial hair now persists across save/load and hair and facial hair no longer teleport to newest corpse.
## Summary
- Removes allocation of a `List<ContextMenuEntry>` every time a context menu is created.
- Moves packet/context menu creation logic out of the core
- Fixes tame entry
## BREAKING CHANGE
> [!Important]
> **Developer Note**
> ```cs
> public virtual void GetContextMenuEntries(Mobile from, List<ContextMenuEntry> list)
> ```
> and similar functions changed to
> ```cs
> public virtual void GetContextMenuEntries(Mobile from, ref PooledRefList<ContextMenuEntry> list)
> ```
### Summary
Updates the serialization strategy to use `MemoryMappedFile` instead of thick buffers. This has the benefit of being on-par with the current implementation (based on hardware/OS), however won't incur the double-memory issue.
> [!Important]
> **Developer Note**
> The `BinaryFileWriter` and `BinaryFileReader` has been removed in favor of `MemoryMapFileWriter` and `UnmanagedDataReader`
### Summary
- Generalizes the On/Off toggle items concept
- Updates the OnOffGump so it is static
- Standardizes OnOff items so they can be used by staff
### Notes
Decided to not fix#1417 because it is not clear that the clilocs or errors are for that purpose. Can't test this on OSI anyways.
> [!Note]
> **Developer Note**
> Now developers will only need to build/run the Application project instead of everything.
> When adding new projects, make sure to:
> 1. Add a reference to that project in the Application project.
> 2. Add the dll file to the Distribution/Data/assemblies.json file
### Summary
- Adds an application project
- Consolidates process restarts to use `Core.Kill(true)`
- Removes some old messaging, for example processor optimization
- Fixes missing build cleanup
> [!Warning]
> Users on Linux/OSX will need to follow the Readme
> and make sure `libdeflate` is properly installed
> [!Note]
> **Developer Note**
> The API for compression has changed. Use `Deflate.Standard` for the same functionality.
### Summary
* Replaces Zlib with LibDeflate for a 50% performance improvement!
* Adds MacOS 14 to properly test Arm64
> [!CAUTION]
> **BREAKING CHANGE**
> Removed `OrderdHashSet` and `PooledOrderedHashSet` due to bugs.
> [!NOTE]
> **Developer Note**
> The OrderedSet is not a full data structure. It is not particularly efficient. Pull Requests are welcome for a better implementation, especially if it ends up supporting `ISet<T>` and `IReadOnlySet<T>`
## Summary
The ordered hash set was buggy. It's kind of painful to implement, so for now, I added a simple `OrderedSet` to suffice for gumps. Please reach out if this causes disruption!
### Summary
* Unifies reading UOP File indexes
* Adds ArtData file reader to get graphic bounds
* Automatically generates Bounds.bin from art file if missing
* Adds [GenBounds command to regenerate the bounds.bin file. Useful for when the art file changes.
# New Gump API
We are pleased to release a new API that is faster, allocates nearly zero memory, and still feels very similar to the original API. The API is broken into 3 types of gumps, dynamic, static with placeholders, and static without placeholders.
### Dynamic Gumps
These gumps will inherit `DynamicGump` and are meant for gumps that have a dynamic layout. This includes specifying dynamic arguments to HtmlLocalized entries.
## Static Gumps
Static gumps are those where the function to the build the layout is called only once and cached forever. They can optionally have placeholders. These placeholders allow the developer to specify the string values later, dynamically in a `BuildStrings` method on the gump. If a gump does not have any placeholders, the string entries will also be cached forever.
## Benchmarks
To make sure we were going in the right direction and not wasting time, we took copious benchmarks. Here are the final benchmarks for a really simple gump.
Note:
* The majority of creating a gump is compressing the layout and the strings. Compressing each section takes ~6,000ns (12us total).
```cs
| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------------------- |-------------:|-------------:|-------------:|-------------:|------:|--------:|-------:|----------:|------------:|
| OldGump | 13,308.29 ns | 1,059.695 ns | 1,883.608 ns | 14,330.72 ns | 1.000 | 0.00 | 0.1526 | 2400 B | 1.00 |
| DynamicLayoutGump | 13,357.86 ns | 129.144 ns | 226.185 ns | 13,323.60 ns | 1.029 | 0.17 | - | 48 B | 0.02 |
| StaticLayoutDynamicStringsGump | 6,653.10 ns | 81.815 ns | 143.292 ns | 6,617.45 ns | 0.514 | 0.09 | - | 40 B | 0.02 |
| StaticLayoutGump | 86.33 ns | 0.760 ns | 1.350 ns | 86.07 ns | 0.007 | 0.00 | 0.0020 | 32 B | 0.01 |
```
# Non-Breaking Changes
* All gump components in the core have been moved to `Gumps/Legacy`.
* All legacy gumps will still inherit `Gump`, which now inherits `BaseGump`
# Special Thanks
Thank you to @stefanomerotta for considerable contributions/benchmarking/testing to make this effort a reality! We collectively went through over 10 iterations, but it is finally ready.
### Summary
* Removes old gump packet support
* Removes support for v4 clients
* Removes `Unpack` flag and assumes it is always true.
* Removes StringToBuffer since this is built into .NET now.
> [!Note]
> View the file changes with white space off: https://github.com/modernuo/ModernUO/pull/1739/files?diff=split&w=1
