fix: Require only runtime packages on Linux, and check ICU and tzdata the way the runtime does (#2561)
## Why ModernUO mandated `-dev` packages on production servers for exactly one reason: `DllImport` never asks for a versioned SONAME, so `libdeflate.so.0` and `libargon2.so.1` sitting in `/usr/lib` went unfound, and the `-dev` package's unversioned symlink was the only thing making resolution work. The `-dev` packages ship no library of their own — operators were installing headers and a static lib on machines that compile nothing. Fixed in the binding packages (modernuo/LibDeflate.Bindings#4, modernuo/Argon2.Bindings#13), so this picks them up and stops asking. ``` LibDeflate.Bindings 1.0.3 -> 1.0.4 Argon2.Bindings 1.17.0 -> 1.19.0 ``` ## zstd is dropped too, on every platform ZstdNet bundles `libzstd` for `linux-x64`, `linux-arm64`, `osx-x64`, `osx-arm64` and win, and nothing shells out to the CLI. Verified: the 15 `ManagedArchive` round-trip tests pass in a container with no `zstd` package installed and `which zstd` empty. Removed from the README, the macOS `brew install`, and CI — so the macOS runners now prove it rather than us assuming it. ## NativeLibraryChecker asks a different question It asked *"is package X installed"* via `dpkg -l` / `rpm -q`. That is what forced `-dev`, and no hardcoded name works for ICU anyway — its apt package is release-specific (`libicu70` on Ubuntu 22.04, `libicu76` on Debian 13). It now asks *"will the loader find this"*: `NativeLibrary.TryLoad` on the unversioned name, then `libfoo.so.N` descending through the range the runtime accepts. It deliberately does not consult a package database or `ldconfig -p`. Both answer a different question than "will `dlopen` succeed" — see the ICU section below for how that bit. ## What was wrong with the ICU check `libicuuc` was **inherited, not derived**. It came from translating the old package-name check into a library probe, without establishing which library that should be. Reviewing it turned up three defects, all of which could report ICU present on a host where the runtime then refuses to start: - **`libicui18n` was never probed.** The only ICU names in `libSystem.Globalization.Native.so` are `libicuuc` and `libicui18n`. `libicudata` arrives as a dependency of `libicuuc`, and `libicuio`/`libicutu`/`libicutest` are never referenced — so that is the complete list, and both are checked now. - **No version floor.** The runtime's `MinICUVersion` is 60, but the probe accepted down to `.so.0`. RHEL/CentOS 7 ships ICU 50, which passed and then aborted at startup. - **The `ldconfig` fast path bypassed the range.** A cache line for `libicuuc.so.50` still matches a `libicuuc.so` prefix test, so the floor was unenforceable through it. It also trusts a stale cache — observed reporting a deleted `libdeflate` as present. Removed in favour of asking the loader directly, which reads the same cache but answers the real question, and which also deletes the musl special-case (`ldconfig -p` exits 0 on musl while producing nothing usable). Worth knowing when this goes wrong in the field: **missing ICU does not throw, it `FailFast`s** — SIGABRT, exit 134, uncatchable. The process starts cleanly and dies later at whatever line first touches a culture, so the stack rarely implicates ICU. ## tzdata is a separate prerequisite, and nothing was checking it The event scheduler resolves configured zone IDs through `TimeZoneInfo`, which reads `/usr/share/zoneinfo`. It is data rather than a library, so no loader probe finds it, and slim container images routinely omit it. Without it every lookup except `UTC` throws `TimeZoneNotFoundException` and `GetSystemTimeZones()` returns 1 entry instead of ~419. There is no per-zone packaging to opt into — it is ~2 MB for the whole set. The one split that does exist is a trap rather than an optimization: Debian 12 and Ubuntu 24.04 move the legacy aliases into `tzdata-legacy`, so plain `tzdata` has `America/New_York` and `EST5EDT` but is **missing `US/Eastern` and `Asia/Calcutta`**. A shard configured with a legacy alias throws even though tzdata is installed. Documented, with both fixes. ## Why `InvariantGlobalization` stays false Dropping ICU entirely by turning on invariant mode looks tempting and is not safe. Because `Directory.Build.props` also sets `PredefinedCulturesOnly=false`, invariant mode does **not** throw `CultureNotFoundException` — it silently hands back invariant data. Measured on .NET 10: | Behaviour | With ICU | Invariant mode | |---|---|---| | `new CultureInfo("de-DE")` | real culture | succeeds, returns invariant data | | de-DE decimal separator | `,` | `.` | | `1234.5` as de-DE | `1.234,5` | `1,234.5` | | `string.Compare("a", "B", InvariantCulture)` | `-1` (linguistic) | `31` (ordinal) | | sort `[b, A, a, B]` | `a, A, b, B` | `A, B, a, b` | | `FindSystemTimeZoneById("Eastern Standard Time")` on Linux | resolves | `TimeZoneNotFoundException` | | UTF-8 round-trip of non-ASCII | unaffected | unaffected | Number parsing and formatting produce wrong values with no error, and culture-sensitive sort order silently becomes ordinal. Encoding is not the mechanism — UTF-8 round-trips fine either way. ## Documentation The rationale now lives in `dev-docs/platform-prerequisites.md` rather than in comments, so it is discoverable without reading the build tool: what each dependency is for, what breaks without it, per-distro package names, the ICU floor, the `tzdata-legacy` split, and why the check asks the loader instead of the package manager. README drops `libicu-dev`. Matching the runtime package by pattern (`'^libicu[0-9]+$'`) is version-independent without pulling in headers, so **no `-dev` package is required on any supported distribution** — which was the point of the whole change. ## CI now proves the claim instead of contradicting it The dnf job already installed runtime packages only. The apt job installed `libicu-dev`, which ships the unversioned `libicuuc.so` symlink — so every probe succeeded on the first attempt and the versioned-SONAME fallback this PR depends on was never exercised. Switched to the pattern match, verified to resolve exactly one package on jammy (70), bookworm (72), noble (74) and trixie (76). Added an assertion that the unversioned symlinks are absent. Without it the suite silently stops testing anything the moment a base image starts shipping one. Verified against all eight matrix distributions — none ship them — and confirmed the step fails as intended when a symlink is planted. ## Audit of every other native entry point Checked whether anything else has the same hazard. It does not: | Import | Verdict | |---|---| | `ws2_32.dll` — `SocketHelper` | Always present on Windows | | `libc` — `SocketHelper` | **Verified safe**, see below | | ZstdNet → `libzstd` | Bundled for every RID | | IORingGroup | No native library; raw syscalls | | ICU | Loaded by the .NET runtime itself, which probes versioned suffixes | `libc` deserved a hard look, because `libc.so` *is* a `libc6-dev` linker script while the real library is `libc.so.6` — the same shape as the bug being fixed. It is not affected. Measured in a container with no `libc6-dev`: ``` /usr/lib/x86_64-linux-gnu/libc.so ABSENT /lib/x86_64-linux-gnu/libc.so.6 present TryLoad("libc") LOADED <- resolves where "libdeflate" would not TryLoad("libc.so") not found getpid() -> DllImport("libc") WORKS ``` Confirmed on Alpine/musl as well. No code in this repo registers a `DllImportResolver`, and nothing else P/Invokes. ## `--check-prereqs` New flag. `Program.cs` only ran the SDK check in non-interactive mode — `NativeLibraryChecker` was reachable only through the Spectre-driven guided flow, so there was no way to verify a deployment target from a script or a container. It is what made the container verification below possible, and it prints the exact ICU package for the running release via `apt-cache`. It renders through the same `PrerequisiteChecker` the guided menu uses, rather than a second hand-rolled table that could drift from it. Spectre drops ANSI styling on its own when stdout is not a terminal, so redirected output stays clean; the console width is widened in that case so the install hints, which are shell commands meant to be copied, do not gain a newline mid-command. ``` ╭───────────────────────────╮ │ Checking native libraries │ ╰───────────────────────────╯ ✔ libicuuc (Found) ✔ libicui18n (Found) ❌ libdeflate (Not found) ❌ tzdata (Not found — every zone except UTC will throw) ⚠️ Install the missing dependencies. The -dev/-devel packages are not required: sudo apt-get install -y libicu74 libdeflate0 tzdata ``` Exit code carries the machine-readable half: 0 when everything resolves, 1 when anything is missing. ## Verification Against 1.0.4 and 1.19.0: build plus **810 Server.Tests and 642 UOContent.Tests**, on Windows and on Linux with **only** `libdeflate0` and `libargon2-1` installed — with the absence of the unversioned symlink asserted first so the run could not pass for the wrong reason. `--check-prereqs` verified in containers on Debian and Alpine across every state that matters: all present, each dependency removed individually, tzdata removed, a deliberately stale `ldconfig` cache, and ICU downgraded to `.so.50` to confirm the floor rejects it. Package resolution and the absence of unversioned symlinks checked on all eight CI distributions.
This commit is contained in:
Kamron Batman
GitHub
parent
246f077778
commit
23dc6649a0
9 changed files with 388 additions and 98 deletions
30
.github/workflows/build-test.yml
vendored
30
.github/workflows/build-test.yml
vendored
|
|
@ -46,7 +46,7 @@ jobs:
|
||||||
- name: Install Prerequisites
|
- name: Install Prerequisites
|
||||||
run: |
|
run: |
|
||||||
brew update
|
brew update
|
||||||
brew install icu4c libdeflate zstd argon2
|
brew install icu4c libdeflate argon2
|
||||||
- name: Set Library Path
|
- name: Set Library Path
|
||||||
run: echo "DYLD_LIBRARY_PATH=/opt/homebrew/lib:$DYLD_LIBRARY_PATH" >> $GITHUB_ENV
|
run: echo "DYLD_LIBRARY_PATH=/opt/homebrew/lib:$DYLD_LIBRARY_PATH" >> $GITHUB_ENV
|
||||||
- name: Build
|
- name: Build
|
||||||
|
|
@ -124,12 +124,36 @@ jobs:
|
||||||
dnf config-manager --set-enabled crb
|
dnf config-manager --set-enabled crb
|
||||||
dnf install -y epel-release
|
dnf install -y epel-release
|
||||||
if: ${{ matrix.epel }}
|
if: ${{ matrix.epel }}
|
||||||
|
# Runtime packages only, deliberately. Installing the -dev packages here would add the
|
||||||
|
# unversioned .so symlink and mask the very thing the binding packages now probe for, so a
|
||||||
|
# regression in versioned-SONAME resolution would sail through CI.
|
||||||
- name: Install Prerequisites using dnf
|
- name: Install Prerequisites using dnf
|
||||||
run: dnf makecache --refresh && dnf install -y findutils libicu libdeflate-devel zstd libargon2-devel
|
run: dnf makecache --refresh && dnf install -y findutils libicu libdeflate libargon2 tzdata
|
||||||
if: ${{ matrix.packageManager == 'dnf' }}
|
if: ${{ matrix.packageManager == 'dnf' }}
|
||||||
|
# ICU's runtime package carries the ABI version in its name (libicu70 on jammy, libicu76 on
|
||||||
|
# trixie) and has no stable alias, so match it by pattern. libicu-dev was the old way to stay
|
||||||
|
# version-independent, but it drags in the unversioned symlink and defeats the check below.
|
||||||
- name: Install Prerequisites using apt
|
- name: Install Prerequisites using apt
|
||||||
run: apt-get update -y && apt-get install -y curl libicu-dev libdeflate-dev zstd libargon2-dev tzdata
|
run: apt-get update -y && apt-get install -y curl '^libicu[0-9]+$' libdeflate0 libargon2-1 tzdata
|
||||||
if: ${{ matrix.packageManager == 'apt' }}
|
if: ${{ matrix.packageManager == 'apt' }}
|
||||||
|
# Versioned-SONAME resolution is only under test while the unversioned symlink is absent. If a
|
||||||
|
# base image or a package ever starts shipping it, every probe would succeed on the first try
|
||||||
|
# and a regression in the fallback would sail through CI, so fail loudly instead of silently
|
||||||
|
# testing nothing.
|
||||||
|
- name: Assert the unversioned .so symlinks are absent
|
||||||
|
run: |
|
||||||
|
found=""
|
||||||
|
for lib in libicuuc libicui18n libdeflate libargon2; do
|
||||||
|
hit=$(ls /usr/lib/*/"$lib".so /usr/lib64/"$lib".so 2>/dev/null || true)
|
||||||
|
if [ -n "$hit" ]; then
|
||||||
|
found="$found $hit"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [ -n "$found" ]; then
|
||||||
|
echo "::error::Unversioned symlinks present, so CI is no longer exercising versioned SONAME resolution:$found"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "No unversioned symlinks present; versioned SONAME resolution is under test."
|
||||||
- uses: actions/checkout@v7
|
- uses: actions/checkout@v7
|
||||||
with:
|
with:
|
||||||
fetch-depth: 0 # avoid shallow clone so nbgv can do its work.
|
fetch-depth: 0 # avoid shallow clone so nbgv can do its work.
|
||||||
|
|
|
||||||
|
|
@ -46,6 +46,7 @@ Apply these when writing or reviewing `.cs` files under `Projects/`.
|
||||||
| Event system | `dev-docs/events.md` |
|
| Event system | `dev-docs/events.md` |
|
||||||
| Threading model | `dev-docs/threading-model.md` |
|
| Threading model | `dev-docs/threading-model.md` |
|
||||||
| Server lifecycle & bootstrap phases (Configure/ConfigurePrompts/Initialize) | `dev-docs/server-lifecycle.md` |
|
| Server lifecycle & bootstrap phases (Configure/ConfigurePrompts/Initialize) | `dev-docs/server-lifecycle.md` |
|
||||||
|
| Platform prerequisites (ICU, tzdata, native libs per distro) | `dev-docs/platform-prerequisites.md` |
|
||||||
| Configuration system | `dev-docs/configuration.md` |
|
| Configuration system | `dev-docs/configuration.md` |
|
||||||
| Networking & packets | `dev-docs/networking-packets.md` |
|
| Networking & packets | `dev-docs/networking-packets.md` |
|
||||||
| IP bans, blocklists & allowlists (incl. unblocking a player) | `dev-docs/ip-bans-and-allowlists.md` |
|
| IP bans, blocklists & allowlists (incl. unblocking a player) | `dev-docs/ip-bans-and-allowlists.md` |
|
||||||
|
|
|
||||||
|
|
@ -14,4 +14,11 @@ public sealed class BuildOptions
|
||||||
public string? Arch { get; set; }
|
public string? Arch { get; set; }
|
||||||
public bool SkipPrereqs { get; set; }
|
public bool SkipPrereqs { get; set; }
|
||||||
public bool Interactive { get; set; }
|
public bool Interactive { get; set; }
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Report the native library prerequisites and exit. The interactive flow is the only other
|
||||||
|
/// path that runs these checks, so without this there is no way to verify a deployment target
|
||||||
|
/// from a script or a container.
|
||||||
|
/// </summary>
|
||||||
|
public bool CheckPrereqsOnly { get; set; }
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,4 @@
|
||||||
|
using System.Runtime.InteropServices;
|
||||||
using BuildTool.Platform;
|
using BuildTool.Platform;
|
||||||
using BuildTool.Publishing;
|
using BuildTool.Publishing;
|
||||||
|
|
||||||
|
|
@ -31,8 +32,10 @@ public static class NativeLibraryChecker
|
||||||
"Linux",
|
"Linux",
|
||||||
[
|
[
|
||||||
".NET 10 Runtime — https://dotnet.microsoft.com/download/dotnet/10.0",
|
".NET 10 Runtime — https://dotnet.microsoft.com/download/dotnet/10.0",
|
||||||
"Debian/Ubuntu: sudo apt-get install -y libicu-dev libdeflate-dev zstd libargon2-dev",
|
"Debian/Ubuntu: sudo apt-get install -y libdeflate0 libargon2-1 libicuNN tzdata",
|
||||||
"Fedora/RHEL: sudo dnf install -y libicu libdeflate-devel zstd libargon2-devel",
|
" (libicuNN varies by release, e.g. libicu76 — run build-tool --check-prereqs there for the exact name)",
|
||||||
|
" (add tzdata-legacy if the shard is configured with an alias such as US/Eastern)",
|
||||||
|
"Fedora/RHEL: sudo dnf install -y libdeflate libargon2 libicu tzdata",
|
||||||
"CentOS: Also requires epel-release and CRB enabled"
|
"CentOS: Also requires epel-release and CRB enabled"
|
||||||
]
|
]
|
||||||
),
|
),
|
||||||
|
|
@ -176,82 +179,59 @@ public static class NativeLibraryChecker
|
||||||
return results;
|
return results;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Native libraries the server needs from the system on Linux, and the SONAME range to accept
|
||||||
|
/// for each. Rationale and per-distro package names: dev-docs/platform-prerequisites.md.
|
||||||
|
/// </summary>
|
||||||
|
private static readonly (string Name, int MinSoVersion, int MaxSoVersion)[] _linuxLibraries =
|
||||||
|
[
|
||||||
|
("libicuuc", 60, 120),
|
||||||
|
("libicui18n", 60, 120),
|
||||||
|
("libdeflate", 0, 9),
|
||||||
|
("libargon2", 0, 9)
|
||||||
|
];
|
||||||
|
|
||||||
private static List<PrerequisiteResult> CheckLinux(PlatformInfo platform)
|
private static List<PrerequisiteResult> CheckLinux(PlatformInfo platform)
|
||||||
{
|
|
||||||
return platform.PackageManager switch
|
|
||||||
{
|
|
||||||
PackageManager.Apt => CheckLinuxApt(),
|
|
||||||
PackageManager.Dnf => CheckLinuxDnf(platform),
|
|
||||||
_ => CheckLinuxGeneric(platform)
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
private static List<PrerequisiteResult> CheckLinuxApt()
|
|
||||||
{
|
{
|
||||||
var results = new List<PrerequisiteResult>();
|
var results = new List<PrerequisiteResult>();
|
||||||
var packages = new[] { "libicu-dev", "libdeflate-dev", "zstd", "libargon2-dev" };
|
|
||||||
var missing = new List<string>();
|
var missing = new List<string>();
|
||||||
|
|
||||||
foreach (var package in packages)
|
foreach (var (name, minSoVersion, maxSoVersion) in _linuxLibraries)
|
||||||
{
|
{
|
||||||
var result = ProcessRunner.RunCaptured("dpkg", $"-l {package}");
|
var found = CanLoad(name, minSoVersion, maxSoVersion);
|
||||||
var installed = result.Success && result.StandardOutput.Contains("ii");
|
|
||||||
|
|
||||||
if (!installed)
|
if (!found)
|
||||||
{
|
{
|
||||||
missing.Add(package);
|
missing.Add(name);
|
||||||
}
|
}
|
||||||
|
|
||||||
results.Add(new PrerequisiteResult
|
results.Add(new PrerequisiteResult
|
||||||
{
|
{
|
||||||
Name = package,
|
Name = name,
|
||||||
Passed = installed,
|
Passed = found,
|
||||||
Details = installed ? "Installed" : "Not installed"
|
Details = found ? "Found" : "Not found"
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
if (missing.Count > 0)
|
var hasTimeZoneData = HasTimeZoneData();
|
||||||
|
if (!hasTimeZoneData)
|
||||||
{
|
{
|
||||||
|
missing.Add("tzdata");
|
||||||
|
}
|
||||||
|
|
||||||
results.Add(new PrerequisiteResult
|
results.Add(new PrerequisiteResult
|
||||||
{
|
{
|
||||||
Name = "Install all missing",
|
Name = "tzdata",
|
||||||
Passed = false,
|
Passed = hasTimeZoneData,
|
||||||
IsWarning = true,
|
Details = hasTimeZoneData ? "Found" : "Not found — every zone except UTC will throw"
|
||||||
Details = "Run the following command to install all missing dependencies:",
|
|
||||||
InstallCommand = $"sudo apt-get install -y {string.Join(' ', missing)}"
|
|
||||||
});
|
});
|
||||||
}
|
|
||||||
|
|
||||||
|
if (missing.Count == 0)
|
||||||
|
{
|
||||||
return results;
|
return results;
|
||||||
}
|
}
|
||||||
|
|
||||||
private static List<PrerequisiteResult> CheckLinuxDnf(PlatformInfo platform)
|
if (platform.DistroId?.Equals("centos", StringComparison.OrdinalIgnoreCase) == true)
|
||||||
{
|
|
||||||
var results = new List<PrerequisiteResult>();
|
|
||||||
var packages = new[] { "libicu", "libdeflate-devel", "zstd", "libargon2-devel" };
|
|
||||||
var missing = new List<string>();
|
|
||||||
|
|
||||||
foreach (var package in packages)
|
|
||||||
{
|
|
||||||
var result = ProcessRunner.RunCaptured("rpm", $"-q {package}");
|
|
||||||
var installed = result.Success;
|
|
||||||
|
|
||||||
if (!installed)
|
|
||||||
{
|
|
||||||
missing.Add(package);
|
|
||||||
}
|
|
||||||
|
|
||||||
results.Add(new PrerequisiteResult
|
|
||||||
{
|
|
||||||
Name = package,
|
|
||||||
Passed = installed,
|
|
||||||
Details = installed ? "Installed" : "Not installed"
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check if this is CentOS (needs EPEL)
|
|
||||||
var isCentOs = platform.DistroId?.Equals("centos", StringComparison.OrdinalIgnoreCase) == true;
|
|
||||||
if (isCentOs && missing.Count > 0)
|
|
||||||
{
|
{
|
||||||
results.Add(new PrerequisiteResult
|
results.Add(new PrerequisiteResult
|
||||||
{
|
{
|
||||||
|
|
@ -263,48 +243,131 @@ public static class NativeLibraryChecker
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
if (missing.Count > 0)
|
|
||||||
{
|
|
||||||
results.Add(new PrerequisiteResult
|
results.Add(new PrerequisiteResult
|
||||||
{
|
{
|
||||||
Name = "Install all missing",
|
Name = "Install all missing",
|
||||||
Passed = false,
|
Passed = false,
|
||||||
IsWarning = true,
|
IsWarning = true,
|
||||||
Details = "Run the following command to install all missing dependencies:",
|
Details = "Install the missing dependencies. The -dev/-devel packages are not required:",
|
||||||
InstallCommand = $"sudo dnf install -y {string.Join(' ', missing)}"
|
InstallCommand = BuildInstallCommand(platform, missing)
|
||||||
});
|
});
|
||||||
}
|
|
||||||
|
|
||||||
return results;
|
return results;
|
||||||
}
|
}
|
||||||
|
|
||||||
private static List<PrerequisiteResult> CheckLinuxGeneric(PlatformInfo platform)
|
/// <summary>
|
||||||
|
/// tzdata is data, not a library, so no loader probe finds it. Asking the runtime rather than
|
||||||
|
/// stat'ing a path keeps TZDIR honoured, and the count is still accurate under
|
||||||
|
/// InvariantGlobalization, which this tool runs with — only display names degrade there.
|
||||||
|
/// </summary>
|
||||||
|
private static bool HasTimeZoneData()
|
||||||
{
|
{
|
||||||
var results = new List<PrerequisiteResult>();
|
try
|
||||||
|
|
||||||
// Use ldconfig to check for shared libraries
|
|
||||||
var ldResult = ProcessRunner.RunCaptured("ldconfig", "-p");
|
|
||||||
var ldOutput = ldResult.Success ? ldResult.StandardOutput : "";
|
|
||||||
|
|
||||||
var libraries = new Dictionary<string, string>
|
|
||||||
{
|
{
|
||||||
["libicu"] = "libicuuc",
|
return TimeZoneInfo.GetSystemTimeZones().Count > 1;
|
||||||
["libdeflate"] = "libdeflate",
|
}
|
||||||
["zstd"] = "libzstd",
|
catch
|
||||||
["libargon2"] = "libargon2"
|
|
||||||
};
|
|
||||||
|
|
||||||
foreach (var (name, soName) in libraries)
|
|
||||||
{
|
{
|
||||||
var found = ldOutput.Contains(soName, StringComparison.OrdinalIgnoreCase);
|
return false;
|
||||||
results.Add(new PrerequisiteResult
|
}
|
||||||
{
|
|
||||||
Name = name,
|
|
||||||
Passed = found,
|
|
||||||
Details = found ? "Found" : "Not found — install using your package manager"
|
|
||||||
});
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return results;
|
/// <summary>
|
||||||
|
/// Asks the loader directly rather than querying a package database or scanning ldconfig's
|
||||||
|
/// cache, both of which answer a different question and can disagree with what dlopen will do.
|
||||||
|
/// Mirrors the binding packages' own probing: the unversioned name first, then libfoo.so.N
|
||||||
|
/// descending. Bare names go through the full loader search path, so LD_LIBRARY_PATH and
|
||||||
|
/// /etc/ld.so.conf.d still apply.
|
||||||
|
/// </summary>
|
||||||
|
private static bool CanLoad(string library, int minSoVersion, int maxSoVersion)
|
||||||
|
{
|
||||||
|
if (TryLoadAndFree($"{library}.so"))
|
||||||
|
{
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
for (var soVersion = maxSoVersion; soVersion >= minSoVersion; soVersion--)
|
||||||
|
{
|
||||||
|
if (TryLoadAndFree($"{library}.so.{soVersion}"))
|
||||||
|
{
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static bool TryLoadAndFree(string candidate)
|
||||||
|
{
|
||||||
|
if (!NativeLibrary.TryLoad(candidate, out var handle))
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
NativeLibrary.Free(handle);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string BuildInstallCommand(PlatformInfo platform, List<string> missing)
|
||||||
|
{
|
||||||
|
switch (platform.PackageManager)
|
||||||
|
{
|
||||||
|
case PackageManager.Apt:
|
||||||
|
{
|
||||||
|
// Distinct because the two ICU libraries resolve to the same package, and
|
||||||
|
// ResolveAptIcuPackage shells out, so it is memoized rather than called per name.
|
||||||
|
var packages = missing.Select(
|
||||||
|
library => library switch
|
||||||
|
{
|
||||||
|
"libdeflate" => "libdeflate0",
|
||||||
|
"libargon2" => "libargon2-1",
|
||||||
|
"tzdata" => "tzdata",
|
||||||
|
_ => _aptIcuPackage ??= ResolveAptIcuPackage()
|
||||||
|
}
|
||||||
|
).Distinct();
|
||||||
|
|
||||||
|
return $"sudo apt-get install -y {string.Join(' ', packages)}";
|
||||||
|
}
|
||||||
|
case PackageManager.Dnf:
|
||||||
|
{
|
||||||
|
var packages = missing.Select(
|
||||||
|
library => library switch
|
||||||
|
{
|
||||||
|
"libdeflate" => "libdeflate",
|
||||||
|
"libargon2" => "libargon2",
|
||||||
|
"tzdata" => "tzdata",
|
||||||
|
_ => "libicu"
|
||||||
|
}
|
||||||
|
).Distinct();
|
||||||
|
|
||||||
|
return $"sudo dnf install -y {string.Join(' ', packages)}";
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return $"Install your distribution's runtime packages for: {string.Join(", ", missing)}";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string _aptIcuPackage;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// ICU's apt package carries the ABI version in its name and there is no stable alias, so ask
|
||||||
|
/// apt which one this release actually ships instead of printing a name that rots.
|
||||||
|
/// </summary>
|
||||||
|
private static string ResolveAptIcuPackage()
|
||||||
|
{
|
||||||
|
var result = ProcessRunner.RunCaptured("apt-cache", "search --names-only ^libicu[0-9]+$");
|
||||||
|
if (!result.Success)
|
||||||
|
{
|
||||||
|
return "libicu";
|
||||||
|
}
|
||||||
|
|
||||||
|
var best = result.StandardOutput
|
||||||
|
.Split('\n', StringSplitOptions.RemoveEmptyEntries)
|
||||||
|
.Select(line => line.Split(' ', 2)[0].Trim())
|
||||||
|
.Where(name => name.StartsWith("libicu", StringComparison.Ordinal))
|
||||||
|
.OrderBy(name => int.TryParse(name.AsSpan(6), out var version) ? version : 0)
|
||||||
|
.LastOrDefault();
|
||||||
|
|
||||||
|
return best ?? "libicu";
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -4,6 +4,7 @@ using BuildTool.Interactive;
|
||||||
using BuildTool.Platform;
|
using BuildTool.Platform;
|
||||||
using BuildTool.Prerequisites;
|
using BuildTool.Prerequisites;
|
||||||
using BuildTool.Publishing;
|
using BuildTool.Publishing;
|
||||||
|
using Spectre.Console;
|
||||||
|
|
||||||
Console.OutputEncoding = Encoding.UTF8;
|
Console.OutputEncoding = Encoding.UTF8;
|
||||||
|
|
||||||
|
|
@ -36,6 +37,22 @@ options.Os ??= detectedPlatform.OsRid;
|
||||||
options.Arch ??= detectedPlatform.ArchRid;
|
options.Arch ??= detectedPlatform.ArchRid;
|
||||||
var rid = $"{options.Os}-{options.Arch}";
|
var rid = $"{options.Os}-{options.Arch}";
|
||||||
|
|
||||||
|
if (options.CheckPrereqsOnly)
|
||||||
|
{
|
||||||
|
// Same renderer the guided menu uses, so the two cannot drift. Spectre drops ANSI styling by
|
||||||
|
// itself when stdout is not a terminal, which is the case this flag exists for, but it also
|
||||||
|
// falls back to an 80 column width and folds anything longer. The install hints we print are
|
||||||
|
// shell commands — the CentOS one is 95 characters — and a fold puts a newline in the middle of
|
||||||
|
// a command that someone is meant to copy. Widen the profile so they stay on one line.
|
||||||
|
if (Console.IsOutputRedirected)
|
||||||
|
{
|
||||||
|
AnsiConsole.Profile.Width = 200;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Exit code is the machine-readable half: 0 when everything resolves, 1 when anything is missing.
|
||||||
|
return PrerequisiteChecker.CheckNativeLibraries(detectedPlatform, interactive: false) ? 0 : 1;
|
||||||
|
}
|
||||||
|
|
||||||
// Run prerequisite checks unless skipped
|
// Run prerequisite checks unless skipped
|
||||||
if (!options.SkipPrereqs)
|
if (!options.SkipPrereqs)
|
||||||
{
|
{
|
||||||
|
|
@ -108,6 +125,12 @@ static BuildOptions ParseArguments(string[] args)
|
||||||
hasNamedArgs = true;
|
hasNamedArgs = true;
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
case "--check-prereqs":
|
||||||
|
{
|
||||||
|
options.CheckPrereqsOnly = true;
|
||||||
|
hasNamedArgs = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
case "--interactive":
|
case "--interactive":
|
||||||
{
|
{
|
||||||
options.Interactive = true;
|
options.Interactive = true;
|
||||||
|
|
|
||||||
|
|
@ -36,7 +36,7 @@
|
||||||
<ProjectReference Include="..\Logger\Logger.csproj" />
|
<ProjectReference Include="..\Logger\Logger.csproj" />
|
||||||
<PackageReference Include="IORingGroup" Version="1.0.9" />
|
<PackageReference Include="IORingGroup" Version="1.0.9" />
|
||||||
<PackageReference Include="CommunityToolkit.HighPerformance" Version="8.4.2" />
|
<PackageReference Include="CommunityToolkit.HighPerformance" Version="8.4.2" />
|
||||||
<PackageReference Include="LibDeflate.Bindings" Version="1.0.3" />
|
<PackageReference Include="LibDeflate.Bindings" Version="1.0.4" />
|
||||||
<PackageReference Include="System.IO.Hashing" Version="10.0.10" />
|
<PackageReference Include="System.IO.Hashing" Version="10.0.10" />
|
||||||
|
|
||||||
<PackageReference Include="ModernUO.Serialization.Annotations" Version="2.14.2" />
|
<PackageReference Include="ModernUO.Serialization.Annotations" Version="2.14.2" />
|
||||||
|
|
|
||||||
|
|
@ -40,12 +40,12 @@
|
||||||
<ProjectReference Include="..\Server\Server.csproj" Private="false" PrivateAssets="All" IncludeAssets="None">
|
<ProjectReference Include="..\Server\Server.csproj" Private="false" PrivateAssets="All" IncludeAssets="None">
|
||||||
<IncludeInPackage>false</IncludeInPackage>
|
<IncludeInPackage>false</IncludeInPackage>
|
||||||
</ProjectReference>
|
</ProjectReference>
|
||||||
<PackageReference Include="LibDeflate.Bindings" Version="1.0.3" />
|
<PackageReference Include="LibDeflate.Bindings" Version="1.0.4" />
|
||||||
<PackageReference Include="System.IO.Hashing" Version="10.0.10" />
|
<PackageReference Include="System.IO.Hashing" Version="10.0.10" />
|
||||||
<PackageReference Include="MailKit" Version="4.17.0" />
|
<PackageReference Include="MailKit" Version="4.17.0" />
|
||||||
<PackageReference Include="Microsoft.Extensions.FileSystemGlobbing" Version="10.0.10" />
|
<PackageReference Include="Microsoft.Extensions.FileSystemGlobbing" Version="10.0.10" />
|
||||||
<PackageReference Include="CommunityToolkit.HighPerformance" Version="8.4.2" />
|
<PackageReference Include="CommunityToolkit.HighPerformance" Version="8.4.2" />
|
||||||
<PackageReference Include="Argon2.Bindings" Version="1.17.0" />
|
<PackageReference Include="Argon2.Bindings" Version="1.19.0" />
|
||||||
<PackageReference Include="ModernUO.CodeGeneratedEvents.Annotations" Version="1.0.0" />
|
<PackageReference Include="ModernUO.CodeGeneratedEvents.Annotations" Version="1.0.0" />
|
||||||
<PackageReference Include="ModernUO.CodeGeneratedEvents.Generator" Version="1.0.3.2" PrivateAssets="all" />
|
<PackageReference Include="ModernUO.CodeGeneratedEvents.Generator" Version="1.0.3.2" PrivateAssets="all" />
|
||||||
<PackageReference Include="ZstdNet" Version="1.5.7" />
|
<PackageReference Include="ZstdNet" Version="1.5.7" />
|
||||||
|
|
|
||||||
19
README.md
19
README.md
|
|
@ -87,18 +87,31 @@ dnf install -y dnf-plugins-core
|
||||||
dnf config-manager --set-enabled crb
|
dnf config-manager --set-enabled crb
|
||||||
dnf install -y epel-release
|
dnf install -y epel-release
|
||||||
# Prerequisites
|
# Prerequisites
|
||||||
dnf install -y findutils libicu libdeflate-devel zstd libargon2-devel
|
dnf install -y findutils libicu libdeflate libargon2 tzdata
|
||||||
```
|
```
|
||||||
|
|
||||||
### Ubuntu, Debian, etc
|
### Ubuntu, Debian, etc
|
||||||
```shell
|
```shell
|
||||||
apt-get update -y
|
apt-get update -y
|
||||||
apt-get install -y libicu-dev libdeflate-dev zstd libargon2-dev
|
# The ICU runtime package carries the ABI version in its name (libicu74, libicu76, …) and has no
|
||||||
|
# stable alias, so match it by pattern rather than pinning a release-specific name.
|
||||||
|
apt-get install -y '^libicu[0-9]+$' libdeflate0 libargon2-1 tzdata
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Only the runtime libraries are needed — the `-dev`/`-devel` packages are not. Run
|
||||||
|
`./build-tool --check-prereqs` to check the current machine and print the exact packages your
|
||||||
|
release needs.
|
||||||
|
|
||||||
|
`zstd` is not listed because ZstdNet bundles `libzstd` for every platform, and `liburing` is not
|
||||||
|
listed because IORingGroup issues `io_uring` syscalls directly.
|
||||||
|
|
||||||
|
If the shard's configured time zone is a legacy alias such as `US/Eastern`, Debian 12 and Ubuntu
|
||||||
|
24.04 also need `tzdata-legacy`. See [Platform Prerequisites](dev-docs/platform-prerequisites.md)
|
||||||
|
for what each dependency is for and what breaks without it.
|
||||||
|
|
||||||
## OSX Requirements
|
## OSX Requirements
|
||||||
```shell
|
```shell
|
||||||
brew install icu4c libdeflate zstd argon2
|
brew install icu4c libdeflate argon2
|
||||||
```
|
```
|
||||||
|
|
||||||
## Running the Server
|
## Running the Server
|
||||||
|
|
|
||||||
159
dev-docs/platform-prerequisites.md
Normal file
159
dev-docs/platform-prerequisites.md
Normal file
|
|
@ -0,0 +1,159 @@
|
||||||
|
# Platform Prerequisites
|
||||||
|
|
||||||
|
OS-level dependencies ModernUO needs at runtime, why each one is required, and what breaks without
|
||||||
|
it. This page is about software packages, not hardware sizing.
|
||||||
|
|
||||||
|
Run `./build-tool --check-prereqs` from the repository root to check the current machine. It prints
|
||||||
|
the exact install command for the detected distribution.
|
||||||
|
|
||||||
|
## What is required
|
||||||
|
|
||||||
|
| Dependency | Platform | Why |
|
||||||
|
|---|---|---|
|
||||||
|
| .NET 10 Runtime | all | — |
|
||||||
|
| ICU (`libicuuc`, `libicui18n`) | Linux, macOS | The runtime refuses to start without it; see below |
|
||||||
|
| tzdata | Linux | Time zone lookups; see below |
|
||||||
|
| `libdeflate` | all | `LibDeflate.Bindings` |
|
||||||
|
| `libargon2` | all | `Argon2.Bindings` (password hashing) |
|
||||||
|
| VC++ Redistributable v14 | Windows | Native bindings |
|
||||||
|
|
||||||
|
Not required, despite appearances:
|
||||||
|
|
||||||
|
- **zstd** — `ZstdNet` bundles `libzstd` for every RID.
|
||||||
|
- **liburing** — `IORingGroup` issues `io_uring` syscalls directly. It imports only `libc`,
|
||||||
|
`libSystem.dylib`, `kernel32.dll`, `kernelbase.dll` and `ws2_32.dll`.
|
||||||
|
- **`-dev` / `-devel` packages** — see "Runtime packages only" below.
|
||||||
|
|
||||||
|
## Install
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# Debian / Ubuntu (ICU has no stable package alias, so match it by pattern)
|
||||||
|
sudo apt-get install -y '^libicu[0-9]+$' libdeflate0 libargon2-1 tzdata
|
||||||
|
|
||||||
|
# Fedora / RHEL
|
||||||
|
sudo dnf install -y libdeflate libargon2 libicu tzdata
|
||||||
|
|
||||||
|
# Alpine
|
||||||
|
apk add --no-cache libdeflate argon2-libs icu-libs tzdata
|
||||||
|
|
||||||
|
# macOS
|
||||||
|
brew install icu4c libdeflate argon2
|
||||||
|
```
|
||||||
|
|
||||||
|
CentOS additionally needs EPEL and CRB:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sudo dnf install -y epel-release epel-next-release && sudo dnf config-manager --set-enabled crb
|
||||||
|
```
|
||||||
|
|
||||||
|
## Runtime packages only
|
||||||
|
|
||||||
|
Only the runtime packages are needed. The `-dev`/`-devel` packages are **not** required.
|
||||||
|
|
||||||
|
They used to be, because .NET's `DllImport` probing looks for the unversioned `libfoo.so`, and on
|
||||||
|
Linux that bare symlink ships only in the development package. The runtime package ships the
|
||||||
|
versioned SONAME (`libdeflate.so.0`, `libargon2.so.1`). The binding packages now probe the versioned
|
||||||
|
names as well, so the runtime package is sufficient.
|
||||||
|
|
||||||
|
Anything still documenting `libicu-dev` or `libdeflate-dev` as a requirement is out of date.
|
||||||
|
|
||||||
|
## ICU
|
||||||
|
|
||||||
|
`Directory.Build.props` sets `InvariantGlobalization=false`, so ICU is mandatory. Without it the
|
||||||
|
runtime does **not** throw — it `FailFast`s:
|
||||||
|
|
||||||
|
```
|
||||||
|
Couldn't find a valid ICU package installed on the system. Please install libicu (or icu-libs)
|
||||||
|
using your package manager and try again.
|
||||||
|
```
|
||||||
|
|
||||||
|
That is `SIGABRT` (exit 134) and it cannot be caught. Note the process **starts cleanly and aborts
|
||||||
|
later**, at whatever line first touches a culture, so the crash rarely points at the cause.
|
||||||
|
|
||||||
|
### Why invariant mode is not an option
|
||||||
|
|
||||||
|
`InvariantGlobalization=true` would remove the ICU dependency, but it changes behaviour in ways that
|
||||||
|
corrupt data silently. Measured on .NET 10 with the repository's settings:
|
||||||
|
|
||||||
|
| Behaviour | With ICU | Invariant mode |
|
||||||
|
|---|---|---|
|
||||||
|
| `new CultureInfo("de-DE")` | real culture | succeeds, returns invariant data |
|
||||||
|
| de-DE decimal separator | `,` | `.` |
|
||||||
|
| `1234.5` as de-DE | `1.234,5` | `1,234.5` |
|
||||||
|
| `string.Compare("a", "B", InvariantCulture)` | `-1` (linguistic) | `31` (ordinal) |
|
||||||
|
| sort `[b, A, a, B]` | `a, A, b, B` | `A, B, a, b` |
|
||||||
|
| `FindSystemTimeZoneById("Eastern Standard Time")` on Linux | resolves | `TimeZoneNotFoundException` |
|
||||||
|
| UTF-8 round-trip of non-ASCII | unaffected | unaffected |
|
||||||
|
|
||||||
|
The dangerous row is the first. Because `Directory.Build.props` also sets
|
||||||
|
`PredefinedCulturesOnly=false`, constructing a culture in invariant mode **succeeds** instead of
|
||||||
|
throwing `CultureNotFoundException`, and hands back an object populated with invariant data. Number
|
||||||
|
parsing and formatting then produce wrong values with no error, and culture-sensitive sort order
|
||||||
|
silently becomes ordinal.
|
||||||
|
|
||||||
|
Encoding is not affected — UTF-8 round-trips correctly in both modes.
|
||||||
|
|
||||||
|
### Version floor
|
||||||
|
|
||||||
|
The runtime accepts `libicuuc.so.60` and above (`MinICUVersion` in `pal_icushim.c`). The prerequisite
|
||||||
|
checker enforces the same floor, so a host carrying only an older ICU is reported missing rather than
|
||||||
|
passing and then aborting at startup. RHEL/CentOS 7 ships ICU 50 and is affected.
|
||||||
|
|
||||||
|
ICU tracks its own release train, so the SONAME digit varies widely by distribution — `.so.74` on
|
||||||
|
Ubuntu 24.04, `.so.76` on Alpine, `.so.77` on Fedora, `.so.78` on openSUSE. There is no stable
|
||||||
|
package alias on Debian and Ubuntu, which is why the checker resolves the name via `apt-cache`
|
||||||
|
instead of hardcoding one.
|
||||||
|
|
||||||
|
Only `libicuuc` and `libicui18n` are used; those are the two names
|
||||||
|
`libSystem.Globalization.Native.so` loads. `libicudata` arrives as a dependency of `libicuuc`, and
|
||||||
|
`libicuio`/`libicutu`/`libicutest` are never referenced. Every distribution ships all of them in a
|
||||||
|
single package, so installing ICU at all satisfies both.
|
||||||
|
|
||||||
|
## tzdata
|
||||||
|
|
||||||
|
The event scheduler resolves configured zone IDs through `TimeZoneInfo`, which reads
|
||||||
|
`/usr/share/zoneinfo` on Linux. This is separate from ICU: it is data, not a library, so no loader
|
||||||
|
probe finds it, and slim container images routinely omit it.
|
||||||
|
|
||||||
|
Without tzdata every lookup except `UTC` throws:
|
||||||
|
|
||||||
|
```
|
||||||
|
TimeZoneNotFoundException: The time zone ID 'America/New_York' was not found on the local computer.
|
||||||
|
```
|
||||||
|
|
||||||
|
`TimeZoneInfo.GetSystemTimeZones()` returns 1 entry instead of ~419, and `TimeZoneInfo.Local` falls
|
||||||
|
back to UTC.
|
||||||
|
|
||||||
|
### There is no per-zone subset
|
||||||
|
|
||||||
|
Distributions do not package individual zones — it is one `tzdata` package, about 2 MB installed for
|
||||||
|
the full set. Subsetting is not worth pursuing.
|
||||||
|
|
||||||
|
The one split that does exist is **`tzdata-legacy`** on Debian 12 and Ubuntu 24.04, which carries the
|
||||||
|
deprecated aliases. With plain `tzdata` alone:
|
||||||
|
|
||||||
|
| Zone ID | `tzdata` | `tzdata-legacy` |
|
||||||
|
|---|---|---|
|
||||||
|
| `America/New_York` | present | — |
|
||||||
|
| `Europe/Kyiv` | present | — |
|
||||||
|
| `EST5EDT` | present | — |
|
||||||
|
| `US/Eastern` | **missing** | present |
|
||||||
|
| `Asia/Calcutta` | **missing** | present |
|
||||||
|
|
||||||
|
So a shard configured with a legacy alias such as `US/Eastern` throws on a current Debian or Ubuntu
|
||||||
|
even though tzdata is installed. Either install `tzdata-legacy` or switch the configured value to the
|
||||||
|
canonical ID (`America/New_York`, `Asia/Kolkata`).
|
||||||
|
|
||||||
|
`TZDIR` is honoured if the data lives somewhere non-standard.
|
||||||
|
|
||||||
|
## How the check works
|
||||||
|
|
||||||
|
`--check-prereqs` asks the loader directly — `NativeLibrary.TryLoad` on the unversioned name, then
|
||||||
|
`libfoo.so.N` descending through the accepted range.
|
||||||
|
|
||||||
|
It deliberately does not consult a package database or `ldconfig -p`. Both answer a different
|
||||||
|
question than "will `dlopen` succeed":
|
||||||
|
|
||||||
|
- Package queries need a hardcoded name, which does not exist for ICU.
|
||||||
|
- `ldconfig`'s cache can be stale, omits `LD_LIBRARY_PATH`, and carries no version information to
|
||||||
|
enforce the ICU floor against. On musl it exits successfully while producing nothing usable.
|
||||||
Loading…
Add table
Add a link
Reference in a new issue