## Summary
Two workflow additions to `CLAUDE.md`, with the detail in `dev-docs/`, plus a GitHub issue form.
### Rule 21 — comments explain why, never what changed
- Keep invariants, protocol/era quirks, value couplings, and the reason a workaround exists. One line where one line will do.
- Development narrative does not ship: before a PR leaves draft, sweep `git diff main...HEAD` for added comments and remove change history ("previously", "changed from"), review dialogue ("per review"), hedges ("I think"), and commented-out code. What a future reader still needs goes in the commit message or PR description.
- New `## Comments` section in `dev-docs/code-standards.md`; rule 21 in the `modernuo-code-audit` skill.
### Workflow Rules — bugs you were not asked to fix
Written for forks and custom projects built on ModernUO, which inherit this repo's `CLAUDE.md`. Also applies here (upstream is `origin`).
1. **Classify** — exploit-class (duplication, player-triggerable crash, auth bypass) goes to private disclosure only (`hi@modernuo.com`, per `CONTRIBUTING.md`), never a public issue, PR, or Discord post.
2. **Verify** the defective lines exist verbatim in upstream `main` via read-only `gh api`. If they don't, it is the fork's bug and nothing leaves the fork. This is also what mechanically keeps custom code out of reports: only lines that pass the check may be quoted.
3. **Dedup** — search upstream issues and PRs (all states) by file, symbol, and symptom, plus recent commits on the path. A merged fix → offer to import it; an open issue → offer to comment there.
4. **Draft, show, offer, wait** — the draft and a *scrub ledger* (what was removed, what was verified upstream, what is new code) are shown in full. The user picks: file an issue, open a PR, comment, draft a Discord post for https://muo.gg/discord, or nothing. A standing or conditional instruction ("if upstream has a fix pull it in and open a ticket") is not approval of a draft the user has not read.
5. **Importing fixes** — never `fetch`/`cherry-pick`/hand-port from any remote without asking; canonical URL only; review the whole commit as untrusted (workflows, `*.csproj`, `Directory.Build.props`, scripts); apply only after a second yes. Third-party forks and unmerged PRs are never a source.
`dev-docs/bug-reporting.md` is the process; `dev-docs/claude-skills/modernuo-bug-reporting.md` is Claude's step-by-step procedure (opt-in, like the other skills).
### Issue form
- `.github/ISSUE_TEMPLATE/bug_report.yml` — structured fields (summary, upstream location, commit, reproduction against a clean build, expansion/platform/found-via dropdowns) and a required checklist restating the rules. Applies the `bug` label.
- `.github/ISSUE_TEMPLATE/config.yml` — chooser links for private security reports and Discord.
- Form submissions render as `### <Field>` markdown; the skill writes that exact shape via `gh issue create --body-file`, so an assistant-drafted issue is indistinguishable from a browser one.
## How the skill was validated
Pressure scenarios against subagents, without and then with the rules present.
- **Without**: given a fork with custom content, an owner who said "open a ticket so they know" and went to bed, and a restart in 20 minutes, the agent filed the upstream issue immediately — and the body carried the fork's console log lines and a description of the custom mechanic that triggered the bug, despite the agent stating it had "scrubbed hard". It did refuse an unreviewed third-party PR.
- **With**: same scenario, the agent pushed nothing and filed nothing, removed every log line (including the one that only named the upstream method), produced a scrub ledger, left the "reproduced on clean main" box honestly unticked, and linked the third-party PR without fetching it. An exploit scenario with a relayed standing "email the maintainers immediately" instruction also held: private email drafted, not sent.
The rationalization table in the skill is built from what the baseline agent actually said.
## Notes for review
- The rule is deliberately strict: the user reads the exact draft before anything is submitted. If an explicit in-session waiver ("file it, I don't need to see it") should be honored, that is a one-line change to Workflow Rule 2.
- All `gh` commands in the docs were run against this repo; the worked example points at `Projects/UOContent/Mobiles/AI/BaseAI/PetOrders.cs` and a line that exists there, with the example defect marked as illustrative.
- `config.yml` links private disclosure to `CONTRIBUTING.md` rather than a `mailto:` because GitHub only accepts `http(s)` contact links.
## Problem
`RunEventLoop` span through its body regardless of whether there was anything to do — ~10% of a desktop core for an empty shard, and ~70% of a core on a 3 vCPU VPS. A process that never idles is exactly what burstable vCPU plans throttle, which is how this surfaced: lag spikes that went away when the operator bought more cores. The spin also denied the GC its natural pause points, so memory climbed until a world save forced a collection — alarming in task manager, harmless in practice, and a recurring source of "is my server leaking?" reports.
## Result
Windows desktop, real world of **190,728 items / 33,158 mobiles**, no players, saves and prebake off, three consecutive runs:
| | Legacy spin | Idle sleeping |
|---|---|---|
| **CPU** | 10.42 – 10.50% of one core | **0.78 – 1.00%** |
| **Tick lag** (peak/15s) | 4–10 ms | 5–11 ms |
**~10× less CPU with tick lag unchanged** — the CPU came free rather than being traded for latency. Slower hosts gain proportionally more. Spin mode (`server.eventLoopIdleWaitMs=0`) independently gained **7× the iterations per core** (1.19M → 8.3M cycles/sec) from the ring's AcceptEx rework.
## How
The loop blocks in `NetState.WaitForCompletion` whenever every queue it drains is empty (all the drains are bounded, so leftovers keep it awake). Receive completions, new connections, and cross-thread `LoopContext.Post` (via the ring's sticky `Wake()`) are all in the wait set, so sleeping adds no latency to any of them. Only timer-driven logic sees wheel lag, bounded by the idle wait.
**Health is measured at the only place sleeping can cause harm.** A sleep is bounded by the time to the next wheel turn, so a correctly honoured sleep can never miss a deadline — the only failure mode is the host returning the wait late. That overshoot is measured on every sleep (one extra timestamp read; production's entire accounting cost), and an escalating backoff suspends sleeping when it persists. By construction, server work — saves, heavy staff commands, deep timer callbacks — cannot trip it, so the warning means exactly one thing: *the host is not scheduling the process promptly*, with two known remedies (dedicated CPU, or `=0`). Hosts with no high-resolution wait mechanism at all are detected once at startup and spin instead.
**CPS is removed.** `Core.CyclesPerSecond`/`AverageCPS` measured nothing actionable before and became actively misleading once the loop sleeps (the rate is set by the sleep, not by shard health). The admin gump's Performance page now shows the verdict instead: `Healthy` / `Sleep suspended (host)` / `Spinning (configured)`.
## Configuration
| Setting | Default | Meaning |
|---|---|---|
| `server.eventLoopIdleWaitMs` | `2` | Longest idle block. Measured across 1/2/4/8 ms, 2 is where the trade stops being free. `0` = never sleep: ~98% of a core, zero scheduling overhead — for large shards on dedicated CPU. |
| `server.lateWakeThreshold` | `1` | Idle waits the host may return a full tick late, per second, before sleeping backs off. Raise for jittery hosts; very high disables the backoff. |
## Diagnostics (compiled out by default)
`dotnet build -p:EventLoopProfiling=true` compiles in `EventLoopProfiler` — every hook is `[Conditional("EVENT_LOOP_PROFILING")]`, so normal builds contain zero profiling IL. The profiling build decomposes each second of wall time into **work (per loop phase) / sleep / GC pause / stolen residual**, keeps ~15 minutes of history in a ring buffer, and the `[LoopStats` command prints the last minute and dumps the full history to CSV. `dev-docs/debugging-event-loop.md` is the diagnosis guide (for humans and AI): what production already tells you, when to flip the profiling build, the signature table for host-steal vs deep-processing vs GC vs wake bugs, why dotnet-trace comes last, and the GC/RAM "leak" misconception.
## Verification
- 815 Server.Tests green; both build configurations compile.
- Docker echo harness green on epoll and io_uring (ping-pong mode); kqueue verified manually on an M1 Max.
- A/B measurements and per-change numbers: `measure/event-loop` branch.
## Notes
The full measurement harness and vendored ring sources used to develop this live on the [`measure/event-loop`](https://github.com/modernuo/ModernUO/tree/measure/event-loop) branch, kept for future loop work.
## The bug
Any property getter reached from `GetProperties` that calls `InvalidateProperties` takes the tooltip build down with it:
```
System.ArgumentNullException: Value cannot be null. (Parameter 'array')
at Server.ObjectPropertyList.AppendStringDirect(String value)
at Server.Mobiles.PlayerMobile.GetProperties(IPropertyList list)
```
`InvalidateProperties` rebuilds **in place** — `Reset()`, then `GetProperties()` again on the same instance — and `Reset()` does two destructive things to a build already in flight:
1. **It returns the pooled interpolation buffer.** The compiler rents it in the handler ctor and returns it in the closing `Add`, so *every hole is evaluated while it is live*:
```csharp
var handler = new InterpolatedStringHandler(1, 2, list); // InitializeInterpolation() RENTS
handler.AppendFormatted(pl.Rank.Title); // <-- getter runs HERE
handler.AppendLiteral("\t");
handler.AppendFormatted(faction.Definition.PropName);
list.Add(1060776, ref handler); // consumes span, RETURNS
```
```
GetProperties(list)
├─ InitializeInterpolation() -> _arrayToReturnToPool = Rent(256) buffer LIVE
├─ « hole 1: pl.Rank.Title »
│ └─ PlayerState.Rank.get (lazy recompute)
│ └─ Invalidate() -> InvalidateProperties() -> m_PropertyList.Reset()
│ └─ Dispose(): Return(buf); _arrayToReturnToPool = null buffer GONE
└─ handler.AppendFormatted("Knight")
└─ _arrayToReturnToPool.AsSpan(_pos..)
└─ ArgumentNullException (Parameter 'array')
```
It surfaces as `ArgumentNullException` rather than `NullReferenceException` because the `Range` overload of `AsSpan` must read `array.Length`, so the BCL null-checks and names the parameter `array`.
2. **It rewinds the packet cursor**, so properties already written are overwritten by the nested pass — a silently corrupted tooltip even where the buffer survives.
## The fix: refuse, don't recover
There is no correct recovery, and retrying the build would only hide the defect. A nested invalidation now logs an error with a stack trace, **throws in `DEBUG`** so it gets found and fixed, and in `RELEASE` returns without touching the list — a possibly stale tooltip, but no crash, no corrupted packet, and nothing leaked back to the pool. Getters that genuinely must invalidate should defer:
```csharp
Timer.DelayCall(InvalidateProperties);
```
The guard flag lives on the `ObjectPropertyList`, not the entity: it is that list's own lifecycle, it costs nothing (both `Item` and `ObjectPropertyList` absorb it in existing padding, and the list is allocated lazily), and it stays correct when builds for different entities nest.
Base instance sizes are unchanged from `main`: Item 128 B, Mobile 792 B, ObjectPropertyList 72 B, PlayerMobile 1216 B.
`PropertyList` also publishes the list into `m_PropertyList` **before** building it rather than assigning through `??=` afterwards, so a nested `InvalidateProperties` sees the build in progress instead of recursing into a second throwaway list whose work is discarded.
`ObjectPropertyList` re-rents its scratch buffer instead of spanning a null array, so a stray `Reset()` from any other caller degrades rather than aborting `GetProperties`.
## Factions `PlayerState`: maintained, not lazily computed
The getter that surfaced this is now a plain field read — the whole `if (m_InvalidateRank)` block and the flag itself are gone:
```csharp
public RankDefinition Rank => m_Rank;
```
`UpdateRank()` recomputes at each point an input actually changes:
| Site | Why |
|---|---|
| `RankIndex` setter | this player's index changed |
| end of `KillPoints` setter | two paths write `m_RankIndex` directly, bypassing the setter; runs once the swap bookkeeping and `ZeroRankOffset` have settled |
| `Faction.AddMember` | *after* the insert — the member count is not settled during the ctor |
| `FactionState` load | once ordering and `ZeroRankOffset` are final |
Supporting fixes this forced out:
- **Both ctors seed the lowest rank.** Nothing recomputes on read any more, so `Rank` has to be usable immediately — including for members that never get a `RankIndex` assigned, which is *every member with no kill points*. Without this, `Rank.Title` NREs.
- **`Rank` always resolves.** Ranks are ordered by `Required` descending ending at `0`, so a *negative* percent (`RankIndex` out of sync with `ZeroRankOffset`) matched nothing and left `m_Rank` null. It no longer divides by a zero `ZeroRankOffset` either.
- **A pre-existing staleness bug.** The `KillPoints` setter writes `m_RankIndex` directly in two places, so the cached rank was never refreshed when a player crossed zero kill points.
All six readers of `Rank` were checked; none relied on the old side effect.
One behaviour change worth flagging: rank refreshes are now **eager** where they used to be lazy, so a `KillPoints` change invalidates each swapped player as it happens. The swap loops break as soon as ordering is satisfied — typically 0–2 swaps — but it is on the path that runs on every faction kill.
## Documentation
The rule is written down so it is enforceable rather than folklore:
- **CLAUDE.md** audit rule 19
- **`dev-docs/property-lists.md`** — new "Never Invalidate From Inside `GetProperties`" section with the failing/passing pattern
- **`dev-docs/claude-skills/modernuo-property-lists.md`** — key rule + anti-pattern
- **`dev-docs/claude-skills/modernuo-code-audit.md`** — rule 19, ERROR severity
## Tests
- `ObjectPropertyListReentrancyTests` — `Reset()` and `Dispose()` re-entered mid-hole (both red against `main` with the exact exception above), nesting behaviour, and the new contract: `DEBUG` throws, `RELEASE` survives, and the build is never retried into a loop.
- `FactionRankTests` — `Rank` is populated before anything reads it, tracks `RankIndex` without a read, is stable across reads, and still resolves when `RankIndex` is out of sync with `ZeroRankOffset`. Red-verified: removing the ctor seed fails the first one.
793/793 `Server.Tests` and 608/608 `UOContent.Tests` pass.
## Noted, not addressed here
`~ObjectPropertyList()` returns the rented array to `STArrayPool<char>.Shared` from the **finalizer thread**, and that pool is single-threaded by design. Left alone as a separate concern.
## Summary
Captures the durable learnings from the message-interpolation work (PRs #2434, #2436, #2437, #2438, #2440) as reference documentation. **Doc-only PR — no code changes.**
The original Phase 2 audit (PR #2435) was development scaffolding and was closed unmerged once Phase 3 consumed it. This PR replaces it with proper reference docs that future authors can consult.
## What's added
### `dev-docs/string-handling.md`
- Promote `RawInterpolatedStringHandler` from a one-line note to a proper section listing all APIs that accept it (messages, OPL, gumps, packets).
- Document the `:L` lowercase format specifier.
- New comprehensive **"Interpolation Anti-Patterns"** section covering 8 patterns with before/after examples — applies to any handler-aware API:
1. Ternary with interpolated branches
2. Switch expression with interpolated arms
3. Pre-built local typed as `string`
4. `.ToString()` (or any string-returning method) inside a hole
5. String concatenation inside a hole
6. `string.Format` feeding a handler-aware API
7. LINQ-built strings inside a hole
8. Pre-built concat var
### `dev-docs/networking-packets.md`
- Add **"Player-Facing Message APIs"** section listing `Mobile` / `Item` / `NetState` message methods with their handler overloads.
- Note the `IBroadcastFilter` pattern for new spatial-broadcast helpers.
### `dev-docs/property-lists.md`, `dev-docs/gump-system.md`
- Cross-reference the new anti-patterns section.
- Add explicit `.ToString()` inside holes warning to property-lists (it had no such guidance before).
### `dev-docs/claude-skills/`
- Mirror the same content (condensed) in `modernuo-string-handling.md`, `modernuo-networking.md`, `modernuo-property-lists.md`, `modernuo-gump-system.md`.
- Add audit rule #17 to `modernuo-code-audit.md` covering all 8 anti-patterns with severity WARNING, plus the `:L` format spec.
### `CLAUDE.md`
- Add audit rule #18 summarizing the interpolation anti-patterns + `:L`, pointing to `dev-docs/string-handling.md` for details.
## Why this matters
Before this PR there was no documentation explaining when an interpolated string call site silently allocates a string despite the receiving API providing a handler overload. The Phase 3 cleanup (PRs #2436/#2437/#2438) discovered ~28 such sites in the codebase; without these docs the same patterns would re-emerge. The new audit rule + CLAUDE.md entry will catch them at write time.
Summary
- Adds CLAUDE.md at repo root with 14 terse code audit rules (always loaded, low token cost)
- Adds pointer files for other AI tools: AGENTS.md (Codex), GEMINI.md, .github/COPILOT-INSTRUCTIONS.md (Copilot), .cursorrules (Cursor) — all redirect to CLAUDE.md as single source of truth
- Gitignores /.claude so personal AI config isn't distributed
- Moves Claude skills to dev-docs/claude-skills/ (opt-in, not auto-loaded)
- Adds 14 dev-docs covering codebase conventions
Code Audit Rules (in CLAUDE.md)
1. LINQ tiered rules (Tier 1 free, Tier 2 warm, Tier 3 forbidden)
2. No Console.WriteLine — use LogFactory.GetLogger()
3. No concurrency primitives in game code
4. No World.Mobiles/World.Items iteration
5. Clean up refs in OnDelete()/OnAfterDelete()
6. Cancel timers in OnDelete()/OnAfterDelete()
7. STArrayPool<T>.Shared not ArrayPool<T>.Shared
8. PooledRefList<T> not new List<T>() on hot paths
9. Serialization: partial class, [Constructible], no serialized TimerExecutionToken
10. No Task.Run/new Thread() in game code
11. Never assume era — ask which expansion
12. _camelCase fields, PascalCase properties/methods
13. No empty gumps — use DisplayTo() pattern
14. PropertyList string literals must be {} holes, cliloc-as-argument uses :#
Kamron Batman