feat(pathfinding): first-boot prompt to pre-bake the .swb map cache (#2475)
## What
On **first boot** (right after map selection), offer to pre-bake the pathfinding `.swb` cache for the selected maps. This removes first-pathfind-after-boot latency and is now cheap — ~18 MB/facet after the v8 format work (the old ~565 MB is gone). The answer persists in `modernuo.json` as **`pathfinding.prebakeMaps`** (default **false**): asked exactly once, and skipped on headless/CI boots (redirected input) where operators can set the flag directly.
## How — a generic startup phase, not pathfinding hardcoded in the engine
The clean-console (pre-Serilog) prompt window is inside the engine startup, but UOContent isn't loaded until after `ServerConfiguration.Load`. So rather than coupling the engine to pathfinding, this adds a generic lifecycle phase:
- **`Main.cs`**: new `AssemblyHandler.Invoke("ConfigurePrompts")` — runs **after** `LoadAssemblies` (so content can participate) but **before** the first `logger.Information` (so console prompts aren't interleaved with the async console sink). The first log line moves below it. Any class can hook in with `public static void ConfigurePrompts()` and self-gate on first-boot state. No `ServerConfiguration` or pathfinding coupling added to the engine.
- **`PathCacheCommands.ConfigurePrompts()`**: the first-boot prompt (interactive-only, flag-absent-only); persists the answer.
- **`PathCacheCommands.Initialize()`** (`Invoke("Initialize")` phase, after the tile matrix loads — which the bake walks): when the flag is set, bakes any map whose `.swb` is **missing or stale** (tile-data fingerprint mismatch, via `StepCache.ComputeLiveFingerprint` / `TryReadFingerprintFromFile`). A fresh cache is a no-op, so only the first boot — or a post-client-update boot — pays the several-minute cost.
## Docs
Fixed the now-stale "~565 MB / ~1.5–2 GB / do not bake by default" section in `dev-docs/pathfinding.md` (it's 17.9 MB for Trammel, tens of MB for all six facets after v8), added a "First-boot pre-bake prompt" section, and added the `pathfinding.prebakeMaps` lever row.
## Verified
- `dotnet build UOContent -c Release` → 0 errors (rebased on #2474).
- Pathfinding/StepCache tests: **90/90 pass**.
- Bootstrap streamlining of the startup phases is intentionally left as a follow-up.
This commit is contained in:
Kamron Batman
GitHub
parent
30fec7da26
commit
2e93201e51
5 changed files with 243 additions and 9 deletions
|
|
@ -44,6 +44,7 @@ Apply these when writing or reviewing `.cs` files under `Projects/`.
|
|||
| Commands & targeting | `dev-docs/commands-targeting.md` |
|
||||
| Event system | `dev-docs/events.md` |
|
||||
| Threading model | `dev-docs/threading-model.md` |
|
||||
| Server lifecycle & bootstrap phases (Configure/ConfigurePrompts/Initialize) | `dev-docs/server-lifecycle.md` |
|
||||
| Configuration system | `dev-docs/configuration.md` |
|
||||
| Networking & packets | `dev-docs/networking-packets.md` |
|
||||
| Region system | `dev-docs/regions.md` |
|
||||
|
|
|
|||
|
|
@ -413,8 +413,6 @@ public static class Core
|
|||
|
||||
ServerConfiguration.Load();
|
||||
|
||||
logger.Information("Running on {Framework}", RuntimeInformation.FrameworkDescription);
|
||||
|
||||
var assemblyPath = Path.Join(BaseDirectory, AssembliesConfiguration);
|
||||
|
||||
// Load UOContent.dll
|
||||
|
|
@ -431,6 +429,14 @@ public static class Core
|
|||
|
||||
AssemblyHandler.LoadAssemblies(assemblyFiles);
|
||||
|
||||
// First-boot interactive setup. Runs after assemblies are loaded (so content can
|
||||
// register prompts) but before any Serilog output, so console prompts are not
|
||||
// interleaved with the async console sink. Handlers self-gate on first-boot state
|
||||
// (e.g. "is my setting already present?").
|
||||
AssemblyHandler.Invoke("ConfigurePrompts");
|
||||
|
||||
logger.Information("Running on {Framework}", RuntimeInformation.FrameworkDescription);
|
||||
|
||||
VerifySerialization();
|
||||
|
||||
_now = DateTime.UtcNow;
|
||||
|
|
|
|||
|
|
@ -1,6 +1,8 @@
|
|||
using System;
|
||||
using System.Diagnostics;
|
||||
using System.IO;
|
||||
using Server.Engines.Pathing.Cache;
|
||||
using Server.Logging;
|
||||
|
||||
namespace Server.Engines.Pathing;
|
||||
|
||||
|
|
@ -19,6 +21,12 @@ namespace Server.Engines.Pathing;
|
|||
/// </summary>
|
||||
public static class PathCacheCommands
|
||||
{
|
||||
private static readonly ILogger logger = LogFactory.GetLogger(typeof(PathCacheCommands));
|
||||
|
||||
// modernuo.json flag: when true, Initialize() bakes any missing/stale .swb at startup.
|
||||
// The first-boot ConfigurePrompts() prompt writes it.
|
||||
private const string PrebakeSetting = "pathfinding.prebakeMaps";
|
||||
|
||||
private static string PathFor(int mapId) =>
|
||||
Path.Combine(Core.BaseDirectory, "Data", "Pathfinding", $"{mapId}.swb");
|
||||
|
||||
|
|
@ -43,6 +51,81 @@ public static class PathCacheCommands
|
|||
AutoLoadAtStartup();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// First-boot prompt, auto-invoked by <c>AssemblyHandler.Invoke("ConfigurePrompts")</c> in
|
||||
/// the startup sequence — after assemblies load (so content can prompt) but before Serilog
|
||||
/// starts, so the console prompt isn't interleaved with async log output. Offers to pre-bake
|
||||
/// the pathfinding <c>.swb</c> cache for the selected maps; the answer persists in
|
||||
/// modernuo.json (<see cref="PrebakeSetting"/>), so it's asked exactly once. Skipped when the
|
||||
/// setting already exists or when input is redirected (headless/CI) — operators can set the
|
||||
/// flag directly. The bake itself happens later in <see cref="Initialize"/>.
|
||||
/// </summary>
|
||||
public static void ConfigurePrompts()
|
||||
{
|
||||
if (ServerConfiguration.GetSetting(PrebakeSetting, (string)null) != null || Console.IsInputRedirected)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
Console.WriteLine();
|
||||
Console.WriteLine("Pre-bake the pathfinding cache for your selected maps now?");
|
||||
Console.WriteLine(" Bakes each map's .swb so there is zero first-pathfind-after-boot latency.");
|
||||
Console.WriteLine(" Takes several minutes and ~tens of MB of disk per facet. You can also do");
|
||||
Console.WriteLine(" this later at runtime with [PathBake.");
|
||||
Console.Write("Pre-bake now? [y/N] ");
|
||||
|
||||
var answer = Console.ReadLine()?.Trim();
|
||||
var prebake = answer?.StartsWith("y", StringComparison.OrdinalIgnoreCase) == true;
|
||||
|
||||
ServerConfiguration.SetSetting(PrebakeSetting, prebake);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Auto-invoked by <c>AssemblyHandler.Invoke("Initialize")</c> after the tile matrix and
|
||||
/// world are loaded. When <see cref="PrebakeSetting"/> is set, bakes any map whose
|
||||
/// <c>.swb</c> is missing or stale (its tile-data fingerprint no longer matches), so the
|
||||
/// first pathfind on each region is already warm. A fresh cache makes this a no-op, so only
|
||||
/// first boot — or a client/map update that changes the fingerprint — pays the cost.
|
||||
/// </summary>
|
||||
public static void Initialize()
|
||||
{
|
||||
if (!ServerConfiguration.GetSetting(PrebakeSetting, false))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
var baked = 0;
|
||||
for (var i = 0; i < Map.Maps.Length; i++)
|
||||
{
|
||||
var map = Map.Maps[i];
|
||||
if (map == null || map == Map.Internal)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
var path = PathFor(map.MapID);
|
||||
var live = StepCache.ComputeLiveFingerprint(map.MapID);
|
||||
if (StepCache.TryReadFingerprintFromFile(path, out var onDisk) && onDisk == live)
|
||||
{
|
||||
continue; // .swb already matches the current tile data
|
||||
}
|
||||
|
||||
logger.Information(
|
||||
"PathBake: pre-baking map {MapId} (pathfinding.prebakeMaps) — this can take several minutes...",
|
||||
map.MapID
|
||||
);
|
||||
StepCache.Instance.BakeMap(map.MapID, path);
|
||||
StepCache.Instance.ClearResidentChunks();
|
||||
baked++;
|
||||
}
|
||||
|
||||
if (baked > 0)
|
||||
{
|
||||
logger.Information("PathBake: pre-bake complete ({Count} map(s) written).", baked);
|
||||
AutoLoadAtStartup(); // (re)open the freshly written files as lazy backing stores
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Open Data/Pathfinding/<mapId>.swb as a lazy backing store for every map.
|
||||
/// Reads only the header + chunk-offset index up front (~16 bytes per chunk);
|
||||
|
|
|
|||
|
|
@ -116,12 +116,32 @@ up front (~16 B/chunk); individual chunks are fetched on demand and remain LRU-c
|
|||
file buys is **zero first-pathfind-after-boot latency** for a region (chunks reload from file
|
||||
instead of being rebuilt by the runtime baker).
|
||||
|
||||
**Disk cost is large — measured, not the stale ~25 MB some older notes claim:** Trammel
|
||||
(`1.swb`) is **~565 MB**. Felucca is comparable; all six facets together are on the order of
|
||||
**~1.5–2 GB**. Baking is therefore a heavy, opt-in operation for serious shards with disk to
|
||||
spare — **do not ship `.swb` files, and do not bake by default.** (If that footprint seems
|
||||
wrong for what it stores, the file format is worth auditing separately — it is far above what
|
||||
the format's design notes projected.)
|
||||
**Disk cost (current, format v8):** the size-reduction roadmap (§ `.swb` size reduction) took
|
||||
Trammel from ~565 MB to **17.9 MB**; the other facets are smaller, so all six together are on the
|
||||
order of **tens of MB** — not the ~1.5–2 GB the uncompressed v2 format cost. Baking is now cheap
|
||||
enough to **offer by prompt at first boot** (below) rather than being a heavy opt-in-only step.
|
||||
Still don't *ship* prebaked `.swb` files (they're tile-data-version-specific and regenerate from
|
||||
the client files anyway).
|
||||
|
||||
### First-boot pre-bake prompt
|
||||
|
||||
On the **first boot** (right after map selection) an interactive prompt offers to pre-bake the
|
||||
`.swb` cache for the selected maps. The answer is stored in `modernuo.json` as
|
||||
**`pathfinding.prebakeMaps`** (default **false**), so it is asked exactly once; headless/CI boots
|
||||
(redirected input) skip the prompt and default to off — operators can set the flag directly.
|
||||
|
||||
When the flag is set, `PathCacheCommands.Initialize()` bakes, at startup, any map whose `.swb` is
|
||||
missing or **stale** (its tile-data fingerprint no longer matches — e.g. after a client/map
|
||||
update). A fresh cache is a no-op, so only the first boot (or a post-update boot) pays the
|
||||
several-minutes cost. Wiring:
|
||||
|
||||
- The prompt runs in a dedicated `AssemblyHandler.Invoke("ConfigurePrompts")` startup phase —
|
||||
after assemblies load (so content can register prompts) but **before Serilog starts**, so the
|
||||
console prompt is not interleaved with the async console sink. Any class can participate by
|
||||
defining `public static void ConfigurePrompts()` and self-gating on first-boot state.
|
||||
- The bake runs in the later `Invoke("Initialize")` phase (after the tile matrix + world load,
|
||||
which the bake walks).
|
||||
- Staleness uses `StepCache.ComputeLiveFingerprint` vs `StepCache.TryReadFingerprintFromFile`.
|
||||
|
||||
## Configuration levers
|
||||
|
||||
|
|
@ -131,6 +151,7 @@ the format's design notes projected.)
|
|||
| `bitmap_pathfinding_cache` feature flag (`ContentFeatureFlags.BitmapPathfindingCache`, `Server.Systems.FeatureFlags`) | `FeatureFlagManager` | `true` | Off → `BitmapAStar` routes straight to the slow path with **no cache probe and no warming memory**. ≈ old FastAStar at ~1×. |
|
||||
| `pathfinding.maxResidentChunks` | `PathCacheCommands.Configure` | 8192 (~40 MB) | LRU cap on resident chunks = the warming-memory ceiling. Lower it (e.g. 512–1024 ≈ 2.5–5 MB) on small shards. |
|
||||
| `pathfinding.maxSearchNodes` | `PathCacheCommands.Configure` → `BitmapAStarAlgorithm.MaxSearchNodes` | 1000 | A* per-Find node-expansion budget. See limits above; ~1000 is the sweet spot. |
|
||||
| `pathfinding.prebakeMaps` | `PathCacheCommands` (first-boot prompt + `Initialize`) | `false` | When set, bakes any missing/stale `.swb` for the selected maps at startup (fingerprint-gated, so a fresh cache is a no-op). Set interactively by the first-boot prompt. |
|
||||
| `PathFollower` `RepathDelay` | `PathFollower.cs` (const) | 2 s | Throttle: a moving goal re-`Find`s at most ~once per 2 s; a stationary reachable goal is pathed once and reused until arrival. Not a setting (compile-time). |
|
||||
|
||||
### Default configuration (recommended)
|
||||
|
|
@ -146,7 +167,7 @@ that care about first-pathfind-after-boot latency and can spend ~1.5–2 GB of d
|
|||
| `bitmap_pathfinding_cache = false` | ≈1× (old FastAStar) | **0** cache RAM | 0 |
|
||||
| Cache on, `maxResidentChunks` low (~512) | ~2–5× on hot regions | ~few MB | 0 |
|
||||
| Cache on, default cap (8192) | 2–5× warm | ~40 MB plateau | 0 |
|
||||
| Cache on + baked `.swb` | + zero first-pathfind-after-boot latency | ~40 MB + index | **~1.5–2 GB** |
|
||||
| Cache on + baked `.swb` | + zero first-pathfind-after-boot latency | ~40 MB + index | **~tens of MB** (v8) |
|
||||
|
||||
The key point for RAM-starved boxes: **disabling the cache is not a regression** — it's the old
|
||||
FastAStar behavior at ~1× with zero warming memory (the slow path does the same per-cell work,
|
||||
|
|
|
|||
123
dev-docs/server-lifecycle.md
Normal file
123
dev-docs/server-lifecycle.md
Normal file
|
|
@ -0,0 +1,123 @@
|
|||
# Server Lifecycle & Bootstrap Phases
|
||||
|
||||
How a ModernUO server starts, the reflection-discovered lifecycle hooks (`ConfigurePrompts`,
|
||||
`Configure`, `Initialize`), the runtime `EventSink` events, and **which hook to use for what**.
|
||||
|
||||
The startup orchestration lives in `Projects/Server/Main.cs` (`Core` entry point). The named
|
||||
phases are dispatched by `AssemblyHandler.Invoke("<Name>")`, which finds every
|
||||
`public static void <Name>()` (parameterless) across `Core.Assembly` **and** all loaded
|
||||
content assemblies and calls them — no registration required.
|
||||
|
||||
## Startup sequence (in order)
|
||||
|
||||
> Don't hardcode line numbers when reasoning about this — refer to the phase/method names; the
|
||||
> ordering is what's stable.
|
||||
|
||||
1. **Console banner + setup** — direct synchronous `Console.*` writes (no logging yet).
|
||||
2. **`ServerConfiguration.Load()`** — reads/creates `modernuo.json`. On **first boot** (file
|
||||
absent) it runs the engine's own interactive console prompts: data directories, listeners,
|
||||
server name, expansion + map selection. **Pre-Serilog** — nothing has logged yet, so the
|
||||
console is clean for prompts. (`Load(mocked: true)` skips all prompts; that's what tests use.)
|
||||
3. **`AssemblyHandler.LoadAssemblies(...)`** — loads `UOContent.dll` (and friends) from
|
||||
`AssemblyDirectories` (default `./Assemblies`). Note: this depends on `AssemblyDirectories`,
|
||||
**not** `DataDirectories`, so it does not need the data-dir prompt to have run.
|
||||
4. **`AssemblyHandler.Invoke("ConfigurePrompts")`** — first-boot interactive prompts contributed
|
||||
by *any* assembly (engine or content). Runs **after** assemblies load (so content can
|
||||
participate) but **before the first Serilog line** (so prompts aren't interleaved with the
|
||||
async console sink). Each handler self-gates on first-boot state.
|
||||
5. **First `logger.Information(...)`** — Serilog goes live. From here on, log via the logger;
|
||||
the console sink is async, so anything you write with `Console.*` after this can interleave
|
||||
with log output.
|
||||
6. **`VerifySerialization()`** → **`Timer.Init(...)`**.
|
||||
7. **`AssemblyHandler.Invoke("Configure")`** — the main configuration phase. World is **not**
|
||||
loaded yet (no entities), but maps are registered.
|
||||
8. **`TileMatrixLoader.LoadTileMatrix()`** → **`RegionJsonSerializer.LoadRegions()`**.
|
||||
9. **`World.Load()`** — deserializes all items/mobiles; fires `EventSink.WorldLoad`.
|
||||
10. **`AssemblyHandler.Invoke("Initialize")`** — post-world phase. World entities **and** the
|
||||
tile matrix are available.
|
||||
11. **`NetState.Start()`** / **`PingServer.Start()`** → **`EventSink.InvokeServerStarted()`** →
|
||||
**`RunEventLoop()`** (the single-threaded game loop begins).
|
||||
|
||||
## The three reflection phases — which to use
|
||||
|
||||
| Phase | Runs | Use it for | Don't |
|
||||
|---|---|---|---|
|
||||
| **`ConfigurePrompts()`** | after assemblies load, **before logging** | one-time **first-boot interactive prompts**; persist the answer to `modernuo.json`; self-gate so it asks once; skip when input is redirected | log (Serilog isn't live — use `Console`); touch World/maps/tile data (not ready) |
|
||||
| **`Configure()`** | post-logging, **pre-World** | command registration, reading settings (`GetOrUpdateSetting`), `EventSink` subscriptions, wiring systems | anything needing loaded **World entities** or the tile matrix |
|
||||
| **`Initialize()`** | **post-World**, post-tile-matrix | work needing a loaded world / tile data: decoration/generation, validation, pre-baking caches | first-boot prompts (too late, and it would clobber logs) |
|
||||
|
||||
All three are `public static void <Name>()`, parameterless, discovered across every loaded
|
||||
assembly. Within a phase, order is controlled by **`[CallPriority(n)]`** (lower runs first;
|
||||
default `50`). **Same-priority order is unspecified**, so never rely on one class's `Configure`
|
||||
running before another's at the same priority — use `EventSink`/explicit calls for ordering.
|
||||
|
||||
## Pre-Serilog vs post-Serilog — why `ConfigurePrompts` exists
|
||||
|
||||
Logging uses an **async** Serilog console sink (`Serilog.Sinks.Async` → `LogFactory`). Once the
|
||||
first `logger.*` call fires (right after the `ConfigurePrompts` phase), log lines are pumped to
|
||||
the console from a background thread and will **interleave** with anything written via
|
||||
`Console.*`. Interactive prompts therefore have to run *before* that point. `ConfigurePrompts`
|
||||
is the **only** reflection phase that runs pre-logging — that is its entire reason to exist.
|
||||
Inside it: use `Console`, never the logger; and guard with `Console.IsInputRedirected` so
|
||||
headless/CI boots don't block on `Console.ReadLine`.
|
||||
|
||||
## Runtime lifecycle events (`EventSink`)
|
||||
|
||||
Subscribe to these from `Configure`/`Initialize` (`EventSink.<Event> += handler`):
|
||||
|
||||
- **`ServerStarted`** — after world load and listeners are up, at loop start.
|
||||
- **`WorldLoad`** / **`WorldSave`** — around persistence (see `WorldEvents`).
|
||||
- **`Shutdown`** — during shutdown.
|
||||
|
||||
## Recipe: add a first-boot prompt
|
||||
|
||||
```csharp
|
||||
public static void ConfigurePrompts()
|
||||
{
|
||||
// Ask once, and only when a human is at the console. The answer persists in modernuo.json.
|
||||
if (ServerConfiguration.GetSetting("my.feature", (string)null) != null || Console.IsInputRedirected)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
Console.Write("Enable my feature? [y/N] ");
|
||||
var yes = Console.ReadLine()?.Trim().StartsWith("y", StringComparison.OrdinalIgnoreCase) == true;
|
||||
ServerConfiguration.SetSetting("my.feature", yes);
|
||||
}
|
||||
```
|
||||
|
||||
If acting on the answer needs a loaded world / tile data, do that in `Initialize()` (read the
|
||||
setting there), not in `ConfigurePrompts`.
|
||||
|
||||
### Canonical example — pathfinding pre-bake
|
||||
|
||||
`Projects/UOContent/Engines/Pathing/PathCacheCommands.cs` is the reference pairing:
|
||||
|
||||
- `ConfigurePrompts()` — first-boot `[y/N]`, stores `pathfinding.prebakeMaps`.
|
||||
- `Initialize()` — when set, bakes any missing/stale `.swb` (needs the tile matrix, so it must
|
||||
be `Initialize`, not `Configure`).
|
||||
|
||||
## Testing note
|
||||
|
||||
Tests do **not** go through `Main`. The test fixtures (`Server.Tests`/`UOContent.Tests`
|
||||
`TestServerInitializer`) call a curated subset of phase methods directly with
|
||||
`ServerConfiguration.Load(mocked: true)`, so console prompts are skipped. Consequence: changes
|
||||
to the **startup ordering in `Main.cs`** (including the prompt phases) are **not** covered by the
|
||||
test suite and need first-boot runtime verification.
|
||||
|
||||
## Planned: unify the engine's first-boot prompts into `ConfigurePrompts`
|
||||
|
||||
Today the engine's own first-boot prompts (data dirs, listeners, server name, expansion + maps)
|
||||
are inline in `ServerConfiguration.Load`, separate from the `ConfigurePrompts` mechanism. They
|
||||
can be unified into the same phase so there's one prompt sequence/wiring:
|
||||
|
||||
- **Feasible because** assembly loading uses `AssemblyDirectories` (default `./Assemblies`), not
|
||||
`DataDirectories` — so assemblies can load *before* the data-dir prompt, letting all prompts
|
||||
move into the post-assembly `ConfigurePrompts` phase.
|
||||
- **`UOClient.Load()`** (client-file discovery via `Core.FindDataFile`) needs `DataDirectories`,
|
||||
so it must move *with* the data-dir prompt into the unified phase.
|
||||
- **`Core.Expansion`** is currently assigned during `Load`; under unification it'd be set during
|
||||
`ConfigurePrompts` — verify nothing between assembly-load and that point depends on it.
|
||||
|
||||
This is an engine-startup restructure the test suite can't cover (see Testing note), so it needs
|
||||
first-boot runtime verification before merging.
|
||||
Loading…
Add table
Add a link
Reference in a new issue