Commit graph

16 commits

Author SHA1 Message Date
Kamron Batman
89a86d81b4
feat: scope container child removal, skip-serialize and linearly delete container children (#2676)
## Summary

Removing an item from a container now reaches only the clients that were sent the item. This PR also adds container-scoped skip serialization, makes container deletion linear, and closes a lift-rejection resync gap. It is the groundwork for the NPC vendor buyback fix (#2677), and it is safe to merge by itself.

### Container child removal reaches only the clients that were sent the item

Adds and updates for an item inside a private container have always gone only to the root mobile, the secure-trade partner, and the container's `Openers` (`ProcessDelta`, the same as RunUO). Removes, though, were broadcast to every client in range. That too is inherited from RunUO; #326 only converted the packets. Two costs:
- **Bandwidth.** One packet per nearby client for every reagent, bandage or arrow stack that runs out, and every container-to-container move.
- **Disclosure.** It sent the serial and timing of items moved in someone else's pack to every nearby client.

Now:
- **`Item.SendRemovePacket`:** a private container child's remove goes only to the root mobile, both secure-trade parties (found by walking up to the trade container, as `ProcessDelta` does), and the container's `Openers`. Each recipient is deduplicated and must be on the same map and in range.
- **`Container.IsChildPublic(Item child)`** (default `IsPublicContainer`) marks children that were sent to every client in range. Their removal still broadcasts. Overrides can only widen: a public container's children always broadcast.
- **`Corpse`** marks its worn gear public on human corpses. `SendInfoTo` shows that gear to everyone who sees the corpse, so bystanders still see looted gear disappear. Loot stays private, and corpses are not made public.
- **`IsPublicContainer` is unchanged** and remains the escape hatch.
- **Unchanged:** items equipped on mobiles, children of non-container items (spellbooks, bulletin boards), and ground items.
- **`Item.Map` no longer sends removes for contained items.** A contained item only changes map along with its parent or root, and that ancestor's own remove already clears the subtree for everyone who knew it. The owner keeps their nested bags across a facet change, as before, and bystanders no longer get a remove per nested item.

**For forks:** a custom container that sends its contents to clients other than the root, trade parties and openers must override `IsChildPublic` or `IsPublicContainer`. Otherwise those clients keep the item until they reopen the container, it leaves their range, or they log out.

- **`Item.MoveToWorld` no longer sends a second remove for an item that had a parent.** It used to detach through the parent's `RemoveItem` (correctly scoped), then broadcast another remove from the old location. Every lift goes through `Internalize()`, so every lift from a pack or container broadcast the item's serial. Ground items are unchanged.
- **Property-only updates** (`ItemDelta.Properties` without `Update`) of a private child now send the object property list revision only to the same recipients, not every client in range. Gump tooltips that reference such an item without the client holding it no longer get the revision nudge; their cached tooltip refreshes when re-requested.

### Lift rejection resyncs only items the requester was sent

A rejected lift re-sent the item's full data to the requester without checking that their client had ever been sent it. `Item.IsSentTo(Mobile)` now applies the same rule as the adds, including `CanSee`:
- **Private container children:** the root mobile, the trade parties, or the openers.
- **Everything else:** visible and in update range.

`Mobile.Lift` still always sends the lift-reject packet, but resyncs the item only when `IsSentTo` holds.

### Container content packets stay within the protocol limit

`SendContainerContent` stack-allocated `items.Count × 20` bytes, which overflowed the stack for a container with hundreds of thousands of items. The `0x3C` length and count fields are 16-bit anyway. It now caps entries so the packet fits (3448 entries, or 3276 with grid lines), and rents a pooled buffer above a small size. Output is byte-identical for normal containers.

### Container-scoped skip serialization

**`Container.SkipsChildSerialization`** (default `false`). When a container returns true:
- its direct children are left out of world saves (the base `Item.SkipSerialization` checks the parent);
- it writes no child list, so a skipped child's serial can never resolve to an unrelated item at load;
- an item that loads naming such a container as its parent is deleted by the existing missing-parent path.

**Known limitation:** `SkipSerialization` is read while the save files are written, after the world unfreezes. An item that enters an opted-in container in that window can be left out of that save. The next save is correct. Moving the decision into the freeze is a follow-up.

### Linear container deletion

`Item.Delete` removes children back to front, but `RemoveItem` searched from the front with `List.Remove`, so deleting a container of *n* items was O(n²). `RemoveItem` now checks the last entry first.
- 100,000 children took 8.0s before and 162ms after.
- Other removals cost the same as before.

**Save format unchanged for every existing type** (the schema generator produces no diff).

## Test plan

- [x] `ContainerChildRemovalTests`:
  - a private child's removal isn't broadcast;
  - openers, the root owner's client and a trade party (who never opened the pouch) still receive it;
  - public containers and per-child `IsChildPublic` overrides still broadcast;
  - an override can't narrow a public container;
  - no duplicate packets when root and opener overlap;
  - deleting the owner sends no remove per child;
  - a facet change sends no remove for nested children to the owner or bystanders;
  - lifting from a backpack or moving a contained item to the ground sends no remove to bystanders;
  - equipped items and ground items keep their broadcast.
- [x] `PrivateChildPropertiesTests`: a property-only update of a private child reaches the owner and openers, not bystanders; public-container children and ground items still broadcast.
- [x] `CorpseChildPublicTests`: human corpse worn gear is public and loot isn't, bones and non-human bodies aren't; end to end, worn gear leaving the corpse reaches a bystander and deleted loot doesn't.
- [x] `LiftRejectResyncTests`:
  - a bystander and an out-of-range requester get no resync;
  - the owner, an opener and a trade party do;
  - an invisible item isn't resynced to a non-staff owner, but is to staff;
  - equipment on a hidden mobile and an invisible ground item aren't resynced.
- [x] `ContainerPacketTests`: large containers are capped at the protocol limit (with and without grid lines) and the pooled-buffer path is byte-identical.
- [x] `ContainerChildSkipTests`, `ContainerBulkRemovalTests`.
- [x] Server.Tests (944) and UOContent.Tests (1203) green on current `main`; schema generator clean.
- [x] Manual in-game pass with three clients:
  - consumption;
  - snooping;
  - co-opened containers;
  - bank;
  - trade with and without opening the pouch;
  - facet change with nested bags;
  - pack animals;
  - player vendors;
  - human corpse looting by drag and by script;
  - every lift-rejection reason.

  Verified with a ClassicUO build instrumented to count:
  - removes for unknown serials;
  - adds dropped for an unknown parent;
  - items learned inside containers the client never opened;
  - lift resyncs of items the client never had.
2026-10-09 22:21:50 -07:00
Kamron Batman
1992c9d1b5
fix: worn gear lifted off a corpse stays loot when put back (#2701)
A corpse's worn-gear list was fixed at creation, and clients learn worn gear only from the 0x89 packet sent on first sight. A piece lifted and dropped back showed as loose loot to everyone watching, but came back "worn" after a relog. Lifting now takes the piece off the list, so every view agrees it is loot.

Programmatic moves (the duel gather-and-re-equip in `DuelContext`) don't go through `OnItemLifted` and still see the original list.
2026-10-09 17:02:36 -07:00
Sergi Rosell
ee4c174e34
test: delete the corpse and sword CorpsePacketTests leave in the world (#2696)
`TestCorpseEquipPacket` and `TestCorpseContainerPacket` build a `Corpse` and a `VikingSword` and never delete them, so both stay in `World.Items` for every test that runs after them in the sequential collection.

They're now deleted in a `finally`, the same way the other tests that create world items clean up. The packet assertions are unchanged.
2026-10-07 22:26:00 -07:00
Kamron Batman
a8acfa31f8
refactor: decompose mobile/corpse hair, delete VirtualHairInfo, fix removal serial (#2462) (#2463)
Fixes #2462

## Summary
Removes the per-object `VirtualHairInfo` heap wrapper for mobile/corpse hair. Hair is now stored **inline** on `Mobile` and `Corpse` as `int _hairItemId` / `int _hairHue` plus a lazily-allocated, **non-serialized** ephemeral `Serial _hairSerial` (in the high virtual-serial range) — and likewise for facial hair. The `VirtualHairInfo` class is deleted, with a **lossless** save migration.

This delivers three things:
1. **Fixes a hair-removal bug.** `Delta(MobileDelta.Hair)` is deferred (it enqueues; `ProcessDeltaQueue` runs later in the tick). The old `HairItemID = 0` setter nulled `_hair` *immediately*, so by the time `ProcessDelta` built the remove packet the equipped virtual serial was already gone — the old `??=` code then re-materialized a **fresh** serial (≠ the equipped one), so clients never removed the right entity, and it left a phantom ItemId-0 object behind. The serial now lives on the entity and **persists across removal**, so remove packets carry the correct serial.
2. **Lightens the entity.** No heap hair object; bald mobiles allocate nothing (the serial is minted lazily only when hair is present). This was the original reason `HairItemID`/`HairHue` exist.
3. **Removes `VirtualHairInfo` entirely**, keeping the high-range virtual serial behavior.

## How
- **Mobile** (manual serialization): inline `_hairItemId/_hairHue/_hairSerial` (+facial); lazy `HairSerial`/`FacialHairSerial`; `ProcessDelta` reads those. Serialization **v36 → v37** — the v30-v37 deserialize is unified, reading the legacy per-hair `VirtualHairInfo` version int only when `version < 37`. Setting item id to 0 clears the hue (matching the old object-nulling) while retaining the serial.
- **Corpse** (codegen serialization): decomposed to `[SerializableField] int _hairItemId/_hairHue` (+facial) + ephemeral serial; **v16 → v17** with `MigrateFrom(V16Content)`.
- **Lossless migration:** the loader validates exact byte length, and the old corpse hair is a presence-bool-gated block, so a tiny **migration-only** `LegacyHairInfo` reader (no runtime role) consumes the legacy `[bool][int ver][int itemId][int hue]` bytes. Frozen `Corpse.v14/v15/v16.json` are retyped to it; `v17.json` describes the new int fields.
- All consumers updated to discrete accessors: `OutgoingMobilePackets`, `CorpsePackets`, corpse subclasses (`MilitiaFighterCorpse`, `SchmendrickApprenticeCorpse`), and the packet test mirrors.
- `VirtualHair.cs` renamed to `OutgoingVirtualHairPackets.cs` (the only type left in it after `VirtualHairInfo` was removed).

## Test Plan
- [x] Full solution build: **0 warnings, 0 errors** (`TreatWarningsAsErrors`).
- [x] `Server.Tests`: **708 passed** (incl. new `RemoveHairUsesEquippedSerial` / `RemoveFacialHairUsesEquippedSerial` proving the serial survives removal + hue clears).
- [x] `UOContent.Tests` corpse/hair: **6 passed** (incl. `CorpseHairMigrationTests` asserting the legacy hair bytes are consumed exactly — the loader's length invariant).
- [x] Generated migration code inspected: V14/V15/V16 readers consume the legacy block byte-for-byte; serial never written to disk.

## Upgrade notes
- Old Mobile (v30–v36) and Corpse (v13–v16) saves load losslessly.
- Minor cosmetic-only change: `SchmendrickApprenticeCorpse` hair/facial-hair RNG draws shift order within each pair (same draw count); irrelevant for a quest NPC corpse.
2026-06-06 14:33:28 -07:00
Kamron Batman
3c0d6cb9d6
feat: Upgrades networking to use io_uring. (#2315)
> [!IMPORTANT]
> **Breaking Changes**
> - DecodePacket and EncodePacket delegates replaced with IClientEncryption interface
> - NetState.Connection (Socket) replaced with internal RingSocket management
> - NetState.RecvPipe and NetState.SendPipe removed (buffers managed internally)

## Summary

Upgrades the networking stack from PollGroup-based I/O to io_uring, significantly improving I/O performance on Linux.
This also adds native client encryption support for encrypted UO clients.

## Major Changes

io_uring Networking Architecture
- Replaced PollGroup with IORingGroup for async socket I/O operations
- Removed Pipe.cs (mirrored ring buffer) and TcpServer.cs in favor of RingSocketManager
- Added NetState.Network.cs - centralized network infrastructure handling accept, recv, send, and disconnect
completions
- Added SocketHelper.cs - platform-specific socket utilities for raw socket handle operations (getpeername,
getsockname)
- Buffer management now handled by RingSocketManager with configurable slab allocation

### Client Encryption Support
- Added full encryption stack in Network/Encryption/:
  - EncryptionConfig.cs - configurable encryption modes (None, Unencrypted, Encrypted, Both)
  - EncryptionManager.cs - encryption detection and initialization for login/game packets
  - LoginEncryption.cs - handles login packet encryption with version-derived keys
  - GameEncryption.cs - handles game server encryption using Twofish
  - TwofishEngine.cs - optimized Twofish block cipher implementation
  - LoginKeys.cs - encryption key table for client versions
  - IClientEncryption.cs - interface for client encryption implementations

### NetState Improvements
- Replaced Socket Connection with RingSocket _socket for managed socket lifecycle
- Changed from GCHandle polling to event-based completion processing
- Disconnect handling now properly waits for pending sends to flush
- Simplified connecting socket management using lazy queue removal

### Configuration
- New settings: network.encryptionMode and network.encryptionDebug
- Encryption mode flags: Unencrypted, Encrypted, or Both

### Dependencies
- Replaced PollGroup NuGet package with IORingGroup
- Linux requires liburing-dev / liburing-devel package

### Test plan

- Verify server starts and accepts connections on Linux with io_uring
- Verify server starts and accepts connections on Windows (fallback to IOCP)
- Test unencrypted client connections (ClassicUO with encryption disabled)
- Test encrypted client connections if available
- Verify graceful disconnect flushes pending data
- Confirm CI builds pass on all target platforms
2026-02-01 16:02:32 -08:00
Kamron Batman
1e4c32c809
fix: Exempts localhost from IPLimiter. Preps testing for io_uring (#2316)
### Summary
* Exempts localhost from IPLimiter
* Updates tests to have proper sequential testing with packets
* Updates tests to clean up NetState
2026-01-20 08:54:15 -08:00
Kamron Batman
7dbfc9d161
fix: Fixes flaky tests, formatting, and sequential testing. (#2168) 2025-04-30 16:42:38 -07:00
mdodkins
91e37fb8d4
fix: Hair and facial hair "teleporting" when mobile dies several times (#1901)
### Summary

* Added World.NewVirtual for creating virtual serial numbers
* Reserved range 0x7EEEEEEE to 0x7FFFFFFF for virtual serials
* Hair and Facial hair (for mobiles) now use virtual serials instead of FakeSerial() functions
* Consolidated virtual hair to a single `VirtualHairInfo` class.

Corpse hair and facial hair now persists across save/load and hair and facial hair no longer teleport to newest corpse.
2024-08-07 20:10:23 -07:00
Kamron Batman
60a3a6f501
feat: Splits Server in Core and Application (#1806)
> [!Note]
> **Developer Note**
> Now developers will only need to build/run the Application project instead of everything.
> When adding new projects, make sure to: 
> 1. Add a reference to that project in the Application project.
> 2. Add the dll file to the Distribution/Data/assemblies.json file

### Summary
- Adds an application project
- Consolidates process restarts to use `Core.Kill(true)`
- Removes some old messaging, for example processor optimization
- Fixes missing build cleanup
2024-05-30 23:34:03 -07:00
Kamron Batman
5b99402666
chore: Cleans up unused imports (#1550) 2023-10-15 11:45:42 -07:00
Kamron Batman
10a69bf754
feat: Adds a memory mirrored ring buffer for networking. (#1533)
## Breaking Changes

Incoming packet registration signature has changed to:
```cs
delegate* void OnReceiveCallback(NetState state, SpanReader reader, int packetLength);

IncomingPackets.Register(int packetID, int length, bool ingame, OnReceiveCallback onReceive);
```

For example, an incoming packet handler signature would now look like this:
```cs
public static void SomeIncomingPacket(NetState state, SpanReader reader, int packetLength)
{
    // Parse the data
}
```

## Summary

Updates the network Pipe class to use a mirrored memory technique. This technique involves mapping the same physical memory to two contiguous virtual memory spaces so the byte buffer appears duplicated. This allows writing to a double-sized array to wrap around without the need for the `CircularBuffer` classes.

In practice this allows us to use `Span<byte>` as if the buffer was a regular array.


### Bug Fixes

- [X] Fixes bad fixed length string parsing
2023-10-09 00:57:53 -07:00
Kamron Batman
0dc4acc164
fix(core): Removes implicit cast between Serial and uint (#728)
* Fixes spellbooks using serial ctor
* Fixes misc items where someone thought they had an amount and it didn't
* Fixes all `Food` types.
2021-08-25 00:27:19 -07:00
Kamron Batman
67625ad249
fix(core): Removes disposable netstates. (#467)
- [X] Removes IDisposable from NetState
- [X] Swaps Disconnect and Dispose methods
2021-02-05 14:33:35 -08:00
Kamron Batman
dc44142d29
fix(tests): Fixes tests failing (#410) 2021-01-14 20:32:56 -08:00
Kamron Batman
f4dd7c8e7f
fix(party): Converts party packets (#401)
- [X] Converts party packets
2021-01-10 14:23:10 -08:00
Kamron Batman
f52e63f320
fix(core): Converts corpse packets (#400)
- [X] Converts corpse packets
- [X] Some cleanup
2021-01-10 09:53:05 -08:00