### Summary
* Upgrades Serialization Generator to v3. This contains numerous bug fixes and a significant performance improvement.
* Bumps other dependencies.
> ⚠️ **Rollback hazard — one-way door once logins are taken.** Serialization is unchanged, so a save
> written by this build still *loads* on the previous one. Its contents do not survive the trip: on
> its first successful login each account is rehashed to `$argon2id$`, and the previous build ships
> Argon2.Bindings 1.19.0, whose `Verify` is gated by the verifier's own configured type and answers
> `false` for an `$argon2id$` hash. **After a shard running this build has accepted logins, do not
> roll back past this commit** — every account that logged in is locked out on the older binary, and
> the only recovery is rolling forward again or resetting passwords by hand. Roll back only from a
> save taken before the first post-deploy login.
Requires [Argon2.Bindings 1.20.0](https://github.com/modernuo/Argon2.Bindings/pull/14), now published.
## What
- Consume `Argon2.Bindings` 1.20.0, which resolves the Argon2 type from the stored PHC string rather than from the verifier's own configuration.
- Default to **Argon2id, m=16384, t=1, p=1** — 8.51 ms against the old Argon2i 8 MiB t=3 at 10.11 ms. Cheaper *and* stronger.
- Rehash on a successful login whenever the stored parameters are stale, not only when the algorithm changes.
- Fix `SetPassword`, which derived the password phrase from the outgoing algorithm while storing it under the incoming one.
## Why
**Verification was gated by the verifier's configured type.** `Verify` passed the instance's own `ArgonType` to native `argon2_verify`, whose `decode_string` rejects a disagreeing `$argon2i$`/`$argon2id$` prefix and returns `DECODING_FAIL` — folded into `false`, the same answer as a wrong password. Switching the default type would have locked out every existing account, and `VerifyAndUpdate` could not have migrated them either: it delegates to the same type-fixed `Verify` and never compared `ArgonType`. Fixed upstream in 1.20.0. The pinned legacy-`$argon2i$` test here fails on 1.19.0 for exactly that reason, which is what makes the package bump load-bearing rather than incidental.
**Changing the defaults would otherwise have reached nobody.** Argon2's PHC string embeds `m`, `t` and `p`, so verification uses the parameters stored with each account, not the configured ones — and verification is the hot path. `CheckPassword` only rehashed when the *algorithm* changed, never when its cost parameters did, so on an established shard the new defaults would have applied to new accounts only. `IPasswordProtection.NeedsRehash` closes that: it defaults to `false`, so PBKDF2 and the `HashAlgorithm` protections are untouched — only Argon2 carries its cost inside the stored value.
**`SetPassword` picked the phrase rule from the wrong algorithm.** SHA1 and SHA2 salt the phrase with the username; Argon2 and PBKDF2 do not. It chose the rule from the *outgoing* algorithm while storing under the *incoming* one, so any algorithm change wrote a credential its own next verify could not reproduce. It now assigns `PasswordAlgorithm` first and derives the phrase from that. Note this ordering is load-bearing and invisible — `UpgradingAlgorithm_DoesNotLockTheAccountOut` is what pins it.
## Cost
Verification is re-derivation, so these are login numbers. A full login calls `CheckPassword` twice — `AccountLogin` (0x80) then `GameLogin` (0x91): **~20 ms before, ~17 ms after**, plus a one-time ~8.5 ms rehash on each account's migrating login.
That cost is still paid on the game loop. Moving hashing off-loop is deliberately **not** in this PR — it needs a pending-auth state in the login handlers, bounding of in-flight hashes, and login rate limiting.
## Why
ModernUO mandated `-dev` packages on production servers for exactly one reason: `DllImport` never
asks for a versioned SONAME, so `libdeflate.so.0` and `libargon2.so.1` sitting in `/usr/lib` went
unfound, and the `-dev` package's unversioned symlink was the only thing making resolution work.
The `-dev` packages ship no library of their own — operators were installing headers and a static
lib on machines that compile nothing.
Fixed in the binding packages (modernuo/LibDeflate.Bindings#4, modernuo/Argon2.Bindings#13), so
this picks them up and stops asking.
```
LibDeflate.Bindings 1.0.3 -> 1.0.4
Argon2.Bindings 1.17.0 -> 1.19.0
```
## zstd is dropped too, on every platform
ZstdNet bundles `libzstd` for `linux-x64`, `linux-arm64`, `osx-x64`, `osx-arm64` and win, and
nothing shells out to the CLI. Verified: the 15 `ManagedArchive` round-trip tests pass in a
container with no `zstd` package installed and `which zstd` empty. Removed from the README, the
macOS `brew install`, and CI — so the macOS runners now prove it rather than us assuming it.
## NativeLibraryChecker asks a different question
It asked *"is package X installed"* via `dpkg -l` / `rpm -q`. That is what forced `-dev`, and no
hardcoded name works for ICU anyway — its apt package is release-specific (`libicu70` on Ubuntu
22.04, `libicu76` on Debian 13). It now asks *"will the loader find this"*: `NativeLibrary.TryLoad`
on the unversioned name, then `libfoo.so.N` descending through the range the runtime accepts.
It deliberately does not consult a package database or `ldconfig -p`. Both answer a different
question than "will `dlopen` succeed" — see the ICU section below for how that bit.
## What was wrong with the ICU check
`libicuuc` was **inherited, not derived**. It came from translating the old package-name check into
a library probe, without establishing which library that should be. Reviewing it turned up three
defects, all of which could report ICU present on a host where the runtime then refuses to start:
- **`libicui18n` was never probed.** The only ICU names in `libSystem.Globalization.Native.so` are
`libicuuc` and `libicui18n`. `libicudata` arrives as a dependency of `libicuuc`, and
`libicuio`/`libicutu`/`libicutest` are never referenced — so that is the complete list, and both
are checked now.
- **No version floor.** The runtime's `MinICUVersion` is 60, but the probe accepted down to
`.so.0`. RHEL/CentOS 7 ships ICU 50, which passed and then aborted at startup.
- **The `ldconfig` fast path bypassed the range.** A cache line for `libicuuc.so.50` still matches a
`libicuuc.so` prefix test, so the floor was unenforceable through it. It also trusts a stale
cache — observed reporting a deleted `libdeflate` as present. Removed in favour of asking the
loader directly, which reads the same cache but answers the real question, and which also deletes
the musl special-case (`ldconfig -p` exits 0 on musl while producing nothing usable).
Worth knowing when this goes wrong in the field: **missing ICU does not throw, it `FailFast`s** —
SIGABRT, exit 134, uncatchable. The process starts cleanly and dies later at whatever line first
touches a culture, so the stack rarely implicates ICU.
## tzdata is a separate prerequisite, and nothing was checking it
The event scheduler resolves configured zone IDs through `TimeZoneInfo`, which reads
`/usr/share/zoneinfo`. It is data rather than a library, so no loader probe finds it, and slim
container images routinely omit it. Without it every lookup except `UTC` throws
`TimeZoneNotFoundException` and `GetSystemTimeZones()` returns 1 entry instead of ~419.
There is no per-zone packaging to opt into — it is ~2 MB for the whole set. The one split that does
exist is a trap rather than an optimization: Debian 12 and Ubuntu 24.04 move the legacy aliases into
`tzdata-legacy`, so plain `tzdata` has `America/New_York` and `EST5EDT` but is **missing
`US/Eastern` and `Asia/Calcutta`**. A shard configured with a legacy alias throws even though tzdata
is installed. Documented, with both fixes.
## Why `InvariantGlobalization` stays false
Dropping ICU entirely by turning on invariant mode looks tempting and is not safe. Because
`Directory.Build.props` also sets `PredefinedCulturesOnly=false`, invariant mode does **not** throw
`CultureNotFoundException` — it silently hands back invariant data. Measured on .NET 10:
| Behaviour | With ICU | Invariant mode |
|---|---|---|
| `new CultureInfo("de-DE")` | real culture | succeeds, returns invariant data |
| de-DE decimal separator | `,` | `.` |
| `1234.5` as de-DE | `1.234,5` | `1,234.5` |
| `string.Compare("a", "B", InvariantCulture)` | `-1` (linguistic) | `31` (ordinal) |
| sort `[b, A, a, B]` | `a, A, b, B` | `A, B, a, b` |
| `FindSystemTimeZoneById("Eastern Standard Time")` on Linux | resolves | `TimeZoneNotFoundException` |
| UTF-8 round-trip of non-ASCII | unaffected | unaffected |
Number parsing and formatting produce wrong values with no error, and culture-sensitive sort order
silently becomes ordinal. Encoding is not the mechanism — UTF-8 round-trips fine either way.
## Documentation
The rationale now lives in `dev-docs/platform-prerequisites.md` rather than in comments, so it is
discoverable without reading the build tool: what each dependency is for, what breaks without it,
per-distro package names, the ICU floor, the `tzdata-legacy` split, and why the check asks the
loader instead of the package manager.
README drops `libicu-dev`. Matching the runtime package by pattern (`'^libicu[0-9]+$'`) is
version-independent without pulling in headers, so **no `-dev` package is required on any supported
distribution** — which was the point of the whole change.
## CI now proves the claim instead of contradicting it
The dnf job already installed runtime packages only. The apt job installed `libicu-dev`, which ships
the unversioned `libicuuc.so` symlink — so every probe succeeded on the first attempt and the
versioned-SONAME fallback this PR depends on was never exercised. Switched to the pattern match,
verified to resolve exactly one package on jammy (70), bookworm (72), noble (74) and trixie (76).
Added an assertion that the unversioned symlinks are absent. Without it the suite silently stops
testing anything the moment a base image starts shipping one. Verified against all eight matrix
distributions — none ship them — and confirmed the step fails as intended when a symlink is planted.
## Audit of every other native entry point
Checked whether anything else has the same hazard. It does not:
| Import | Verdict |
|---|---|
| `ws2_32.dll` — `SocketHelper` | Always present on Windows |
| `libc` — `SocketHelper` | **Verified safe**, see below |
| ZstdNet → `libzstd` | Bundled for every RID |
| IORingGroup | No native library; raw syscalls |
| ICU | Loaded by the .NET runtime itself, which probes versioned suffixes |
`libc` deserved a hard look, because `libc.so` *is* a `libc6-dev` linker script while the real
library is `libc.so.6` — the same shape as the bug being fixed. It is not affected. Measured in a
container with no `libc6-dev`:
```
/usr/lib/x86_64-linux-gnu/libc.so ABSENT
/lib/x86_64-linux-gnu/libc.so.6 present
TryLoad("libc") LOADED <- resolves where "libdeflate" would not
TryLoad("libc.so") not found
getpid() -> DllImport("libc") WORKS
```
Confirmed on Alpine/musl as well. No code in this repo registers a `DllImportResolver`, and nothing
else P/Invokes.
## `--check-prereqs`
New flag. `Program.cs` only ran the SDK check in non-interactive mode — `NativeLibraryChecker` was
reachable only through the Spectre-driven guided flow, so there was no way to verify a deployment
target from a script or a container. It is what made the container verification below possible, and
it prints the exact ICU package for the running release via `apt-cache`.
It renders through the same `PrerequisiteChecker` the guided menu uses, rather than a second
hand-rolled table that could drift from it. Spectre drops ANSI styling on its own when stdout is not
a terminal, so redirected output stays clean; the console width is widened in that case so the
install hints, which are shell commands meant to be copied, do not gain a newline mid-command.
```
╭───────────────────────────╮
│ Checking native libraries │
╰───────────────────────────╯
✔ libicuuc (Found)
✔ libicui18n (Found)
❌ libdeflate (Not found)
❌ tzdata (Not found — every zone except UTC will throw)
⚠️ Install the missing dependencies. The -dev/-devel packages are not required:
sudo apt-get install -y libicu74 libdeflate0 tzdata
```
Exit code carries the machine-readable half: 0 when everything resolves, 1 when anything is missing.
## Verification
Against 1.0.4 and 1.19.0: build plus **810 Server.Tests and 642 UOContent.Tests**, on Windows and
on Linux with **only** `libdeflate0` and `libargon2-1` installed — with the absence of the
unversioned symlink asserted first so the run could not pass for the wrong reason.
`--check-prereqs` verified in containers on Debian and Alpine across every state that matters: all
present, each dependency removed individually, tzdata removed, a deliberately stale `ldconfig`
cache, and ICU downgraded to `.so.50` to confirm the floor rejects it. Package resolution and the
absence of unversioned symlinks checked on all eight CI distributions.
## Summary
Adds two pieces of pathfinding tooling on top of PR #2448's lazy `.swb` infrastructure:
- **`PathfindRecorder`** — admin-toggled JSONL telemetry capture; one record per `BitmapAStarAlgorithm.Find` call. Output format matches the BDN harness corpus, so production traffic can be captured and replayed in benchmarks without an adapter.
- **Public bake helpers on `StepCache`** — `ComputeLiveTileDataHash`, `TryReadTileDataHashFromFile`, `BakeMap`, `ClearResidentChunks`. Lets the benchmark project (and any future bake utility) drive cache fill + persist without exposing internal types.
The companion BDN harness update lives in [ModernUO-Benchmarks#kb/pathfinding-pr4-bench](https://github.com/modernuo/ModernUO-Benchmarks/tree/kb/pathfinding-pr4-bench): porting `Benchmarks/PathfindInGame/` from the `kb/ai_pathfinding` branch to the API shipped in #2446–#2448.
## What's in this PR
### `PathfindRecorder` (`PathfindRecorder.cs`)
- Holds a single `StreamWriter` open while recording; its internal buffer absorbs per-record writes without per-call `File.AppendAllText`.
- Single `bool` check on the hot path; cheap when disabled.
- Disabling flushes + disposes; an IO failure during write also disables the recorder.
- Server config:
- `pathfinding.recorder.enable` — bool, default `false`. Read on boot via `GetOrUpdateSetting`.
- `pathfinding.recorder.path` — default `<basedir>/Data/Pathfinding/recordings/pathfinds.jsonl`.
- Hooked into `BitmapAStarAlgorithm.Find` — runs once per call, does nothing when disabled.
- Admin command: `[PathRecord [on|off|flush|status]` (default `status`).
### Public cache helpers
- `static ulong StepCache.ComputeLiveTileDataHash()` — wraps the file module's hash function for staleness checks.
- `static bool StepCache.TryReadTileDataHashFromFile(string, out ulong)` — peeks at a `.swb` file's hash field (20 bytes).
- `int StepCache.BakeMap(int, string)` — walks every chunk in the map, populates resident set, saves. Offline / fixture use; blocks for many seconds on a full-map walk.
- `void StepCache.ClearResidentChunks()` — drops chunks + zeros counters but keeps lazy readers open. Lets benchmark loops measure "first query after boot" cost across iterations without the lazy-reader reopen overhead.
## Summary
Replaces the basic `publish.cmd`/`publish.sh` scripts with an interactive **BuildTool** — a C# console app using [Spectre.Console](https://spectreconsole.net/) that guides users through publishing, prerequisite checking, and cross-compilation.
### Why
The community found the existing publish scripts unhelpful for newcomers. They worked but didn't walk users through the process, didn't check prerequisites, and provided no feedback when things went wrong.
### What's New
**Interactive BuildTool** (`Projects/BuildTool/`)
- NativeAOT-compiled C# console app with true-color ASCII logo and ModernUO brand gold/silver palette
- Guided publish wizard with step-by-step back navigation (Ctrl+C or menu "Back" to go to previous step)
- Prerequisite checking: .NET SDK version, VC++ Redistributable (Windows), native libraries (Linux/macOS)
- .NET SDK auto-install offer via Microsoft's official install scripts
- Platform detection: Windows 10 vs 11 (build number), macOS codenames, Linux distro + kernel version
- Cross-compilation support: skips native library checks, shows target prerequisites after build
- Non-interactive mode for CI: `--config Release --skip-prereqs`
- Backward-compatible positional args: `publish.cmd release win x64` still works
**Shell Wrappers** (`publish.cmd`, `publish.ps1`, `publish.sh`)
- Try native BuildTool binary first (downloaded from GitHub Releases)
- Fall back to `dotnet run --project Projects/BuildTool` if unavailable
- SDK bootstrapping: offer to install .NET if not found
**CI/CD Updates**
- Build/test workflows target `Projects/Application/Application.csproj` instead of the solution (excludes BuildTool and test projects from publish)
- New `build-tool-release.yml` workflow builds NativeAOT binaries for win-x64, win-arm64, osx-arm64, linux-x64, linux-arm64
- Minimum SDK bumped to 10.0.201 (required for Serialization Generator 2.14.3 / Roslyn 5.3.0)
**Other Changes**
- Solution converted from `.sln` to `.slnx`
- Updated README with interactive mode instructions and deployment guidance
## Screenshots
<img width="320" height="378" alt="image" src="https://github.com/user-attachments/assets/83c057c5-3992-4dbd-99fa-0e3c24ef6428" />
<img width="749" height="554" alt="image" src="https://github.com/user-attachments/assets/e4ee4d6f-71d4-47f9-86b6-8fd1ca3c3e7a" />
### Summary
* Adds better `SaveFlag` support.
* Fixes `SortedSet` support.
* Adds custom comparer support for `SortedSet`.
* Fixes various bugs.
* Adds support for `struct`, `record`, and `generic` classes.
* Adds support for `readonly` fields by skipping serializaton entirely.
* Adds support for interface fields that inherit ISerializable
* Adds tests.
See: 59a92cc49a
### Summary
* Bumps to .NET 9 with updated dependencies
* Comparing a value type against null is no longer allowed
* CI/CD now uses the version specified in global.json
* Serialization generator updated to .NET 9 with bug fixes, fixes to turkish language, and parallelization
### Summary
- Adds [Code Generated Events](https://github.com/modernuo/CodeGeneratedEvents)
- Removes EventSink.PlayerDeath
- Adds `PlayerDeathEvent` and `CreatureDeathEvent` using code generated events
> [!Important]
> **Developer Note**
> Use `[OnEvent(nameof(PlayerMobile.PlayerDeathEvent))]` instead of `EventSink.PlayerDeath` delegate
> Check this commit for examples of how to use this.
### Summary
Updates the serialization strategy to use `MemoryMappedFile` instead of thick buffers. This has the benefit of being on-par with the current implementation (based on hardware/OS), however won't incur the double-memory issue.
> [!Important]
> **Developer Note**
> The `BinaryFileWriter` and `BinaryFileReader` has been removed in favor of `MemoryMapFileWriter` and `UnmanagedDataReader`
> [!Note]
> **Developer Note**
> Now developers will only need to build/run the Application project instead of everything.
> When adding new projects, make sure to:
> 1. Add a reference to that project in the Application project.
> 2. Add the dll file to the Distribution/Data/assemblies.json file
### Summary
- Adds an application project
- Consolidates process restarts to use `Core.Kill(true)`
- Removes some old messaging, for example processor optimization
- Fixes missing build cleanup
> [!Warning]
> Users on Linux/OSX will need to follow the Readme
> and make sure `libdeflate` is properly installed
> [!Note]
> **Developer Note**
> The API for compression has changed. Use `Deflate.Standard` for the same functionality.
### Summary
* Replaces Zlib with LibDeflate for a 50% performance improvement!
* Adds MacOS 14 to properly test Arm64
### Summary
- Fixes issues with non-_ISerializable_ objects having bad serialization (Skill/Stat. Mods)
- Fixes issues with MarkDirty and namespaces: https://github.com/modernuo/SerializationGenerator/issues/29
- Fixes missing dirty tracking for some data structures.
- Fixes cascading MarkDirty for non-serializable types that have owners that are also non-serializable types.
### New Codegenned API
When a data structure (Array, List, Dictionary, HashSet, etc) is source generated for serialization, new methods are added which will handle dirty tracking. Use these instead of the built in methods for that data structure.
_All Data Structures_
```cs
public void Clear<PropertyName>();
```
_Lists and Sets_
```cs
public void AddTo<PropertyName>(V value);
public void RemoveFrom<PropertyName>(V value);
```
_Lists_
```cs
public void InsertInto<PropertyName>(int index, V value);
public void RemoveFrom<PropertyName>At(int index);
```
_Dictionaries_
```cs
public void AddTo<PropertyName>(T key, V value);
public void RemoveFrom<PropertyName>(T key);
public void ReplaceIn<PropertyName>(T key, V value);
```
Over time, they will be cleaned up and more variants added such as `bool RemoveFrom<PropertyName>(T key, out V value)`
