6d846b11e5
## Problem
`RunEventLoop` span through its body regardless of whether there was anything to do — ~10% of a desktop core for an empty shard, and ~70% of a core on a 3 vCPU VPS. A process that never idles is exactly what burstable vCPU plans throttle, which is how this surfaced: lag spikes that went away when the operator bought more cores. The spin also denied the GC its natural pause points, so memory climbed until a world save forced a collection — alarming in task manager, harmless in practice, and a recurring source of "is my server leaking?" reports.
## Result
Windows desktop, real world of **190,728 items / 33,158 mobiles**, no players, saves and prebake off, three consecutive runs:
| | Legacy spin | Idle sleeping |
|---|---|---|
| **CPU** | 10.42 – 10.50% of one core | **0.78 – 1.00%** |
| **Tick lag** (peak/15s) | 4–10 ms | 5–11 ms |
**~10× less CPU with tick lag unchanged** — the CPU came free rather than being traded for latency. Slower hosts gain proportionally more. Spin mode (`server.eventLoopIdleWaitMs=0`) independently gained **7× the iterations per core** (1.19M → 8.3M cycles/sec) from the ring's AcceptEx rework.
## How
The loop blocks in `NetState.WaitForCompletion` whenever every queue it drains is empty (all the drains are bounded, so leftovers keep it awake). Receive completions, new connections, and cross-thread `LoopContext.Post` (via the ring's sticky `Wake()`) are all in the wait set, so sleeping adds no latency to any of them. Only timer-driven logic sees wheel lag, bounded by the idle wait.
**Health is measured at the only place sleeping can cause harm.** A sleep is bounded by the time to the next wheel turn, so a correctly honoured sleep can never miss a deadline — the only failure mode is the host returning the wait late. That overshoot is measured on every sleep (one extra timestamp read; production's entire accounting cost), and an escalating backoff suspends sleeping when it persists. By construction, server work — saves, heavy staff commands, deep timer callbacks — cannot trip it, so the warning means exactly one thing: *the host is not scheduling the process promptly*, with two known remedies (dedicated CPU, or `=0`). Hosts with no high-resolution wait mechanism at all are detected once at startup and spin instead.
**CPS is removed.** `Core.CyclesPerSecond`/`AverageCPS` measured nothing actionable before and became actively misleading once the loop sleeps (the rate is set by the sleep, not by shard health). The admin gump's Performance page now shows the verdict instead: `Healthy` / `Sleep suspended (host)` / `Spinning (configured)`.
## Configuration
| Setting | Default | Meaning |
|---|---|---|
| `server.eventLoopIdleWaitMs` | `2` | Longest idle block. Measured across 1/2/4/8 ms, 2 is where the trade stops being free. `0` = never sleep: ~98% of a core, zero scheduling overhead — for large shards on dedicated CPU. |
| `server.lateWakeThreshold` | `1` | Idle waits the host may return a full tick late, per second, before sleeping backs off. Raise for jittery hosts; very high disables the backoff. |
## Diagnostics (compiled out by default)
`dotnet build -p:EventLoopProfiling=true` compiles in `EventLoopProfiler` — every hook is `[Conditional("EVENT_LOOP_PROFILING")]`, so normal builds contain zero profiling IL. The profiling build decomposes each second of wall time into **work (per loop phase) / sleep / GC pause / stolen residual**, keeps ~15 minutes of history in a ring buffer, and the `[LoopStats` command prints the last minute and dumps the full history to CSV. `dev-docs/debugging-event-loop.md` is the diagnosis guide (for humans and AI): what production already tells you, when to flip the profiling build, the signature table for host-steal vs deep-processing vs GC vs wake bugs, why dotnet-trace comes last, and the GC/RAM "leak" misconception.
## Verification
- 815 Server.Tests green; both build configurations compile.
- Docker echo harness green on epoll and io_uring (ping-pong mode); kqueue verified manually on an M1 Max.
- A/B measurements and per-change numbers: `measure/event-loop` branch.
## Notes
The full measurement harness and vendored ring sources used to develop this live on the [`measure/event-loop`](https://github.com/modernuo/ModernUO/tree/measure/event-loop) branch, kept for future loop work.
167 lines
10 KiB
Markdown
167 lines
10 KiB
Markdown
<p align="center">
|
||
<a href="https://muo.gg"><img alt="ModernUO - Ultima Online Server Emulator for the modern era!" src="https://cdn.muo.gg/gh/muo-logo.svg" width=128px /></a>
|
||
</p>
|
||
|
||
ModernUO [](https://muo.gg/discord) [](https://muo.gg/reddit/) [](https://muo.gg/twitter)
|
||
=====
|
||
|
||
##### Ultima Online Server Emulator for the modern era!
|
||
[](https://github.com/modernuo/ModernUO/blob/master/LICENSE)
|
||
[](https://github.com/modernuo/ModernUO/stargazers)
|
||
[](https://github.com/modernuo/ModernUO/issues)
|
||
<br/>
|
||
[](https://github.com/modernuo/ModernUO/actions)
|
||
[](https://dev.azure.com/modernuo/modernuo/_build/latest?definitionId=1&branchName=main)
|
||
|
||
## Requirements
|
||
#### Supported Operating Systems
|
||
[](https://www.microsoft.com/en-US/evalcenter/evaluate-windows-server-2022)
|
||
|
||
[](https://www.debian.org/distrib/)
|
||
[](https://ubuntu.com/download/server)
|
||
<br/>
|
||
[](https://alpinelinux.org/downloads/)
|
||
[](https://getfedora.org/en/server/download/)
|
||
[](https://access.redhat.com/downloads)
|
||
[](https://www.centos.org/download/)
|
||
[](https://almalinux.org/get-almalinux/)
|
||
[](https://get.opensuse.org/)
|
||
[](https://www.suse.com/download/sles/)
|
||
[](https://linuxmint.com/download.php)
|
||
[](https://archlinux.org/download/)
|
||
|
||
#### Required Frameworks
|
||
##### All Operating Systems
|
||
[](https://dotnet.microsoft.com/download/dotnet/10.0)
|
||
|
||
##### Windows
|
||
[](https://aka.ms/vc14/vc_redist.x64.exe)
|
||
|
||
#### Hardware
|
||
|
||
| Use | vCPU | RAM | Storage |
|
||
|---|---|---|---|
|
||
| Development / test | 2 **dedicated** | 2 GB | SSD |
|
||
| Small live shard (< 50 concurrent) | 4 dedicated | 4 GB | NVMe |
|
||
| Medium (50–200) | 4–8 | 8 GB | NVMe |
|
||
| Large (200+) | 8+, high clock | 16 GB+ | NVMe |
|
||
|
||
Game logic is single-threaded, so **single-core clock speed matters more than core count**, and
|
||
**dedicated vCPU matters more than either** — burstable or shared plans throttle once credits run
|
||
out, which is the most common cause of unexplained lag spikes. Save size drives RAM more than
|
||
player count does.
|
||
|
||
See [dev-docs/server-requirements.md](dev-docs/server-requirements.md) for the reasoning and tuning
|
||
options.
|
||
|
||
#### Development
|
||
[](https://git-scm.com/downloads)
|
||
[](https://dotnet.microsoft.com/download/dotnet/10.0)
|
||
|
||
#### Supported IDEs
|
||
<p align="left">
|
||
<a href="https://www.jetbrains.com/rider/download"><img height="64" title="Jetbrains Rider 2025.3+" alt="Jetbrains Rider 2025.3+" src="https://cdn.muo.gg/gh/jetbrains-rider.svg"></a>
|
||
<a href="#"><img alt="space" width="32" src="https://cdn.muo.gg/gh/space.png"></a>
|
||
<a href="https://code.visualstudio.com/download"><img height="64" title="VSCode" alt="VSCode" src="https://cdn.muo.gg/gh/vscode.svg"></a>
|
||
<a href="#"><img alt="space" width="32" src="https://cdn.muo.gg/gh/space.png"></a>
|
||
<a href="https://visualstudio.microsoft.com/vs/community/"><img height="64" title="Visual Studio 2026" alt="Visual Studio 2026" src="https://cdn.muo.gg/gh/vs2026.svg"></a>
|
||
</p>
|
||
|
||
## Getting Started
|
||
- Install prerequisite [requirements](https://github.com/modernuo/ModernUO#requirements)
|
||
- Clone this repository (or download the [latest](https://github.com/modernuo/ModernUO/archive/refs/heads/main.zip)):
|
||
- `git clone https://github.com/modernuo/ModernUO.git`
|
||
- Open `ModernUO.sln` to start developing
|
||
|
||
## Building/Publishing
|
||
|
||
#### Interactive Mode (Recommended for new users)
|
||
Run `./publish.cmd` (Windows) or `./publish.sh` (Linux/macOS) with no arguments to launch the guided build tool. It will:
|
||
- Check prerequisites (.NET SDK, native libraries)
|
||
- Walk you through configuration and platform selection
|
||
- Build and publish the server to the `Distribution` directory
|
||
- Show deployment instructions for cross-compiled builds
|
||
|
||
#### Command Line
|
||
```shell
|
||
./publish.cmd [release|debug] [os] [arch]
|
||
```
|
||
- `os` - [Supported operating systems](https://github.com/dotnet/core/blob/main/release-notes/10.0/supported-os.md)
|
||
- `win` - [Windows](https://learn.microsoft.com/en-us/dotnet/core/install/windows)
|
||
- `osx` - [macOS](https://learn.microsoft.com/en-us/dotnet/core/install/macos)
|
||
- `linux` - [Linux](https://learn.microsoft.com/en-us/dotnet/core/install/linux)
|
||
- `arch`
|
||
- `x64` - Intel/AMD 64-bit
|
||
- `arm64` - ARM 64-bit
|
||
|
||
## Linux Prerequisites
|
||
### Fedora, CentOS, RHEL, AlmaLinux, Rocky, etc
|
||
```shell
|
||
dnf upgrade --refresh -y
|
||
# RHEL-family distributions (CentOS Stream, AlmaLinux, Rocky, RHEL) need CRB and EPEL enabled.
|
||
# Fedora skips this block.
|
||
dnf install -y dnf-plugins-core
|
||
dnf config-manager --set-enabled crb
|
||
dnf install -y epel-release
|
||
# Prerequisites
|
||
dnf install -y findutils libicu libdeflate libargon2 tzdata
|
||
```
|
||
|
||
### Ubuntu, Debian, etc
|
||
```shell
|
||
apt-get update -y
|
||
# The ICU runtime package carries the ABI version in its name (libicu74, libicu76, …) and has no
|
||
# stable alias, so match it by pattern rather than pinning a release-specific name.
|
||
apt-get install -y '^libicu[0-9]+$' libdeflate0 libargon2-1 tzdata
|
||
```
|
||
|
||
Only the runtime libraries are needed — the `-dev`/`-devel` packages are not. Run
|
||
`./build-tool --check-prereqs` to check the current machine and print the exact packages your
|
||
release needs.
|
||
|
||
`zstd` is not listed because ZstdNet bundles `libzstd` for every platform, and `liburing` is not
|
||
listed because IORingGroup issues `io_uring` syscalls directly.
|
||
|
||
If the shard's configured time zone is a legacy alias such as `US/Eastern`, Debian 12 and Ubuntu
|
||
24.04 also need `tzdata-legacy`. See [Platform Prerequisites](dev-docs/platform-prerequisites.md)
|
||
for what each dependency is for and what breaks without it.
|
||
|
||
## OSX Requirements
|
||
```shell
|
||
brew install icu4c libdeflate argon2
|
||
```
|
||
|
||
## Running the Server
|
||
- Follow the [publish](https://github.com/modernuo/ModernUO#buildingpublishing) instructions
|
||
- The `Distribution` directory is portable — copy it to your production server for deployment
|
||
- Run `ModernUO.exe` or `dotnet ModernUO.dll` from the `Distribution` directory
|
||
- On first run, the server will prompt you to configure game data file locations
|
||
|
||
## Troubleshooting / FAQ
|
||
- See [FAQ](./FAQ.md)
|
||
|
||
## Want to sponsor?
|
||
Thank you for supporting us! You can find out how by visiting the [sponsors](./SPONSORS.md) page.
|
||
|
||
## Collaborators
|
||
[](https://github.com/kamronbatman)
|
||
[](https://github.com/mark1145)
|
||
|
||
## Thanks
|
||
- RunUO Team & Community
|
||
- [Voxpire](https://github.com/Voxpire), the ServUO Team & Community
|
||
- [Karasho](https://github.com/andreakarasho), [Jaedan](https://github.com/jaedan) and the ClassicUO Community
|
||
|
||
<br/><br/>
|
||
<p align=center>Development Tools & Plugins provided with ♥ by<br/><br/><a href="https://www.jetbrains.com/?from=ModernUO"><img src="https://cdn.muo.gg/gh/jetbrains.svg" height="64px" alt="JetBrains" title="JetBrains" /></a><br/>
|
||
<a href="https://material-theme.com/"><img src="https://github.com/AtomMaterialUI/material-theme-issues/raw/master/logo.svg" width="64px" alt="Material Theme" title="Material Theme"></a></p>
|
||
|
||
## Code Signing Policy
|
||
|
||
Free code signing provided by [SignPath.io](https://about.signpath.io), certificate by [SignPath Foundation](https://signpath.org).
|
||
|
||
This program will not transfer any information to other networked systems unless specifically requested by the user or the person installing or operating it
|
||
|
||
### Teams & Roles
|
||
Approvers & Committers: [Development Team](https://github.com/orgs/modernuo/teams/development-team)
|
||
|