Commit graph

13 commits

Author SHA1 Message Date
Kamron Batman
459674ce3b
fix(commands): parse and edit IPoint2D/IPoint3D properties (#2646)
## Symptom

`[set TargetLocation (x, y)` (quoted or not) answers **"That is not properly formatted."**, and in `[props` the `>` next to `TargetLocation` does nothing when the value is null — which is its normal idle state (`BaseAI.cs:650` clears it).

This looked like a `Point3D` parsing regression from #2624/#2625, but `Point3D`/`Point2D`-typed properties (`Location`, etc.) were never affected. The only `[CommandProperty]` in the tree declared as an **interface** is `BaseCreature.TargetLocation : IPoint2D` (`BaseCreature.cs:1111`), and both code paths only knew the structs. `git log -S"IPoint"` over the parser and gump files hits nothing but the initial import — the gap is inherited from RunUO, not recent.

## Root cause

- **`[set`** — `Types.TryParse` has no branch for `IPoint2D`/`IPoint3D`. An interface has no static `Parse`, so `GetParseMethod` returns null and the value falls into `Convert.ChangeType("(x, y)", typeof(IPoint2D))`, which throws → "not properly formatted".
- **Props gump** — `PropsGump` routes on `obj?.GetType() ?? prop.PropertyType` (since #2180). With a null value the type is `IPoint2D`; `Point2D.IsAssignableFrom(IPoint2D)` is false, no branch matches, and the click is inert. It only worked when the slot already held a `Point2D`, because the runtime type is then the struct.

## Fix

- `Types.TryParse`: `IPoint3D`/`IPoint2D` targets resolve to the concrete struct — `Point3D` first, then `Point2D` for an `IPoint2D` target (a 3-tuple is a valid `IPoint2D`). `(-null-)` still clears; the existing null branch runs first.
- `PropsGump`: the interface types route to `SetPoint3DGump`/`SetPoint2DGump`. The entity branch stays ahead of them — `TargetLocation` legitimately holds a Mobile too (`ShepherdsCrook.cs:148`, herding toward the shepherd), and that case still opens `SetObjectGump`.
- `SetPoint2DGump`/`SetPoint3DGump`: seed the text entries from `value is IPoint2D/IPoint3D` rather than a hard cast, so a `Point3D` sitting in an `IPoint2D` slot cannot `InvalidCast`.

## Not covered

`[set TargetLocation 0x40001234` (assigning a mobile by serial) still reports "not properly formatted" — the entity branch keys on the *target* type being `IEntity`, which `IPoint2D` isn't. Real state, but niche; left out to keep this to the reported symptom.

## Testing

Five cases in `InterfacePointParseTests`, watched fail before the change (three returned the error string; two pin existing behaviour that must survive): tuple → `Point3D` for both interfaces, pair → `Point2D`, pair rejected for `IPoint3D`, `(-null-)` clears.

`dotnet build` 0 warnings. **1059 UOContent** and **891 Server** tests pass, 0 failures. The gump routing is a one-line branch with no automated test — needs an in-game check: `[props` a creature with a null `TargetLocation`, press `>`, expect the Point2D editor.
2026-09-14 21:20:22 -07:00
Kamron Batman
c9831b9680
fix: TextDefinition was uneditable in the props gump, and where parsed constants its own way (#2624)
## Pressing `>` on a TextDefinition did nothing useful

`#1217` moved `TextDefinition` into the Server project for serialization support and added `[PropertyObject]` along the way. `PropsGump` checks that attribute **before** its parsable fallback, so from that commit on the button either drilled into a read-only `Number`/`String` page (both get-only since `#1221`, so no `>` buttons — a dead end) or, when the value was null, re-sent the same page and looked inert.

Worth being clear that `#765` — which replaced the hand-maintained type list with a generic `IsParsable` branch — was **not** the regression. At that commit `TextDefinition` was `[Parsable]` only, fell through to the new branch, and the editor worked. Only the later attribute hijacked the routing.

`TextDefinition` is now caught ahead of the `[PropertyObject]` branch and routed to `SetGump`, which has had `TextDefinition`-specific code at line 35 all along.

## `0` and `"0"` could not be told apart

`Commands.Split` strips quotes before any parser runs, so this was never fixable in `TextDefinition.Parse` alone — the information is gone two layers earlier. The markers now travel inside the value:

| Input | Result |
|---|---|
| `1060847` | cliloc (unchanged) |
| `#1060847` | cliloc, explicit — the form `ToString()` already writes |
| `@"1060847"` | the literal string |
| `hello` | string (unchanged) |
| `(-null-)` | null (unchanged) |

`GetValue()` quotes a string that would not survive the trip back. That check is a real round trip through the parser rather than a pattern match, so it cannot drift from it. `Types.TryParse` decodes the same escape for plain strings — which is what `[get` has always written for the literal `"null"` without `[set` ever reading it back.

**Saved data is untouched.** Serialization reads a discriminated flag plus int/string (`IGenericReader.cs:175`) and the JSON converter switches on token type; neither calls `Parse`, so a stored string `"1060847"` stays a string and spawner JSON is unchanged. The new syntax stays in the text and command layer, where it reaches `[set`, `[add`, spawner props, the props gump and Advanced Search.

## `where` still parsed its constants its own way

`#2625` carried the hand-rolled constant parser across to `PropertyExpressions.Parse` unchanged. It looks for a static `Parse` overload on the property type and gives up if there is none — and `Type` and `IEntity` have neither:

```
where Subject Kind  = Static     ->  Unable to convert string "Static" into type 'System.Type'.
where Subject Owner = 0x1        ->  Unable to convert string "0x1" into type 'Server.Mobile'.
```

Both resolve fine for every other command. Routing through `Types.TryParse` picks up type-name lookup, entity resolution by serial, and the `@"..."` literal convention, so a value that works in one place now works everywhere. It also deletes the duplicate parser (−56 lines in `PropertyExpressions`).

Two things are load-bearing and preserved. `where` has always spelled a null constant as a bare `null`, where `[set` uses `(-null-)`; the shared parser reads a bare `null` as the text, so the null case is handled ahead of it and `= null` keeps meaning null. And nullable targets still unwrap first, so `where <int? prop> = 5` is unaffected.

Bare and hex integers, enums, bools, strings, `Map`, and TextDefinition's `#` / `@"..."` markers all resolve exactly as before — 13 of the 15 tests in `WhereConstantParsingTests` pin that and passed before the change as well as after.

## Documentation

The generic command system had no documentation anywhere in the repo: scopes, `where` and its operators, dot notation, `order by` / `distinct` / `limit`, the value and quoting syntax, `[interface` and `[batch` were all learn-by-reading-the-source. `dev-docs/generic-commands.md` covers them, linked from `CLAUDE.md` and `commands-targeting.md`, and points at <https://muo.gg/commands> for the per-command list rather than duplicating it.

Two behaviours it records that were news to me while writing it:

- `Contained` honours conditions on the normal command path but never sets `SupportsConditionals`, and `[batch` is the only place that reads the flag — so the same condition works typed directly and is refused under batch.
- `Multi`, `Single`, `Self` and `Serial` do not parse modifiers at all, so a `where` clause there is passed to the command as ordinary arguments rather than rejected.

Comments in the changed code were trimmed to what is not evident from the code itself; the rationale they carried is either in the doc now or in the commit that introduced it.

## Behavior changes worth a reviewer's attention

Two things go beyond strict bug-fixing, both deliberate:

- **Hex now parses into a TextDefinition.** Consolidating the three `Parse` overloads onto one span codec means `[set Message 0x102CE7` is cliloc 1060847 rather than the string. Previously only the 1-arg `Parse(string)` did hex and nothing called it. This makes the props gump's own `1060847 (0x102CE7)` display typeable.
- **`@"..."` decodes generally for strings**, not only the exact token `@"null"`. So `[set Name @"hello"` sets `hello`. A half-working escape seemed more surprising than a general one, and the `where` path already decoded it — but narrowing it back is a one-line change if preferred.

## Testing

40 tests, each written first and watched fail for the right reason. They cover the props gump routing (populated and null), cliloc/string/quoted parsing and `GetValue` round trips, `where` constant resolution for Type- and entity-valued properties, and the bare-`null` vs `@"null"` distinction that must not drift.

One sort case is added that #2625 left uncovered: ordering a value-typed chain whose intermediate is null, which reads as `default(int)` rather than throwing. Two other tests from the pre-rebase branch were dropped as duplicates of `ConditionalCompilerEdgeTests`.

Rebased onto `535a09899`. `dotnet build` clean, 0 warnings. **902 UOContent** and **869 Server** tests pass, 0 failures.

## Known gaps

- `Nullable<T>` comparisons work via #2625's lifted operators; nothing here changes that.
- Spawner **Params** are split on plain spaces (`BaseSpawner.cs:1035`) rather than through `Commands.Split`, so a constructor argument still cannot contain one. Untouched here — a tokenizer issue, not a TextDefinition one.
- `Types.ParseStringNumericParamTypes` is now unreferenced. Left in place because it is public and shards may use it.
2026-09-10 19:36:30 -07:00
Kamron Batman
a7e65aab01
perf(login): run password hashing on a parked worker thread (#2566)
## Why

An Argon2 verify is **~8.9 ms of frozen world per login attempt** — more than half a 16 ms frame. Failed attempts cost exactly the same as successful ones, by design, so a credential-stuffing flood is a full-cost stall per packet without needing valid credentials. `SetPassword` derives a hash too, so `[password`, the admin gump and account creation each pay the same.

## What the measurement says

Off-loading does not delete the cost, it relocates it. Three things stay on the loop:

| Component | Measured |
|---|---:|
| Inline verify (today) | **8.92 ms** |
| Dispatch to the worker | 210 ns |
| Drain the continuation off `LoopContext` | 13 ns |
| Loop's own work slowed by shared-L3 eviction | **0.05 – 5.44 ms** |

Net gain **3.5 – 8.9 ms** of on-loop time per login. Harness in `ModernUO-Benchmarks` (`Benchmarks/Argon2OffLoop/`): it models the loop as a dependent-load pointer chase swept across working-set sizes, which is an upper bound on cache-latency sensitivity, and copies `EventLoopContext` so the hand-off cost is the real one.

Two results shaped the design:

- **The contention tax peaks in the middle of the working-set range**, not at the top — 5.44 ms at 8 MiB (a quarter of this chip's L3), but 0.76 ms at 30 MiB and 0.10 ms at 256 KiB. A tiny hot set has nothing in L3 to lose; a huge one is already DRAM-bound.
- **Per-login tax falls as concurrency rises** (5.44 → 2.56 → 1.60 ms at 1/2/4 hashers) while *total* loop damage rises. Contention is shared, not additive, so a login rush is not the disaster case — a single login is.

## Why exactly one worker

It is load-bearing three times over, which is also why it must not quietly become a pool:

- **Cost bound.** Off-loop loses to inline only if a hash steals ~82% of the loop's throughput. One hasher contending for one core leaves the loop ~50%. **A single background hasher cannot cost the loop more than the inline verify under any scheduling regime**, which is what lets the measurement hold on hardware we cannot inspect — AMD, VPS, oversubscribed VM. Four hashers drop the loop to ~20% and break it.
- **Memory.** Exactly one hashing arena is live at a time whatever the login volume.
- **Ordering.** Writes apply in dispatch order *only* because a single thread drains FIFO. A second worker would need ordering reintroduced; `WritesApplyInDispatchOrder` fails if that happens.

Throughput is ~110 verifies/sec. Only loop time matters, not login latency, so head-of-line blocking during a rush costs nothing.

## Making every protection safe off-thread

The worker was initially Argon2-only. That was the right call for the wrong reason — it was blamed on Argon2's salt RNG, which is a stateless syscall wrapper and was never a problem. The real blockers were elsewhere, and both are fixed at the source:

| Protection | Was | Now |
|---|---|---|
| MD5/SHA1/SHA2 | shared `HashAlgorithm.ComputeHash`, which carries the running digest across `HashCore`/`HashFinal` through process-wide singletons | static `HashData` into a `stackalloc` span — no state, no allocation, identical bytes |
| PBKDF2 | `Utility.RandomMinMax` → shared `System.Random`, thread-unsafe *and* game state | `RandomNumberGenerator.GetInt32`, matching the salt beside it |
| Argon2 | already safe (`Verify` is static + stackalloc) | unchanged, singleton reused |

Literal digests are pinned in a test **before** the change and still pass after it. These are compared as strings against every account database, so any casing or encoding drift would lock out every SHA and MD5 account at once.

With all three safe, the worker no longer knows which algorithm it runs and the dispatch conditions collapse to "is off-loop available".

## Correctness

- **Phrase derivation** moves to `AccountSecurity.DerivePhrase`, so verification (stored algorithm's rule) and rehash (target algorithm's rule) cannot disagree. Deriving with the wrong one is the shape of the lockout fixed in #2562.
- **Liveness** is checked at dequeue *and* at apply — a connection can drop while queued or while the result sits in the loop queue. A job with no connection attached, such as an admin password change, runs regardless.
- **Queue overflow rejects** a login rather than verifying inline; steering work back onto the loop is what a flood wants. A password change instead falls back to hashing inline, because unlike a login it must not be dropped.
- **Shutdown and crash** both just stop the thread, and pending jobs are dropped. No save is initiated once shutdown begins — saving is the operator's choice up front, via the admin gump's save/no-save variants, and `WaitForWriteCompletion` honours one already in flight — so a write applied during teardown would reach no disk. The crash path needs its own subscription because `HandleClosed` skips `InvokeShutdown` when crashed.

## Bounding

`MaxPending` is 4096 — a backstop, not a flood defense. `SentFirstPacket` holds a connection to one pending verify and the engine caps connections at 4096, so the queue is already bounded by construction and this can only trip if that invariant breaks. A cap low enough to blunt an attack would reject real players first; during a mass reconnect they *are* the queue. Flood defense belongs at the connection layer.

The real DoS improvement is elsewhere: today every attempt stalls the world, and after this a flood occupies one core while the loop keeps ticking.

## Gate

Release builds on 4+ cores. Below that there is no spare core to move work to, so off-loading buys nothing by construction; `DEBUG` is excluded because dev boxes and test shards have few logins. Both modes call the same code — the gate only chooses where it runs.

## Engine change

One property, `AccountLoginEventArgs.Deferred`, so a subscriber can say "no verdict yet". `EventSink.AccountLogin` is `Action<...>` with no continuation, and the packet handler replies in the same call. Approved separately since it touches `Projects/Server/`.

## Docs

`dev-docs/threading-model.md` and the threading skill gain a vetted-workers section. The forbidden-patterns table bans `new Thread`, `ConcurrentQueue<T>`, `Interlocked` and `volatile` in `UOContent`, and its exceptions covered only `Projects/Server/` — the existing Advanced Search fan-out already sat outside it. The new section leads with proving the need (measure on-loop time, not wall-clock; gate on core count; record the measurement), keeps game logic on the loop via chunking, and documents the hand-off protocol in both directions.

## Testing

698 UOContent tests, 810 Server tests, Release build clean.

Covered: verify and rehash outcomes, phrase rules for SHA1/SHA2 vs Argon2, stored-format stability for MD5/SHA1/SHA2, jobs with no connection attached, and dispatch ordering through the real queue. The liveness and ordering guards are mutation-verified.
2026-08-09 00:13:34 -07:00
Kamron Batman
1e97ed50f6
fix: Harden Advanced Search: crash-safety, autosave, correct results & worker fixes (#2543)
## Summary

Hardens the **Advanced Search** engine (`Projects/UOContent/Engines/Advanced Search/`) — the GM entity finder that fans searches across background worker threads. A code review surfaced 14 defects (A–N), including a shard-crasher reachable from a single admin typo and a path that silently disables autosave for the rest of the shard's uptime. Each behavioral fix ships with a test.

Full `UOContent.Tests` suite: **530/530 green** (21 new AdvancedSearch tests).

## Fixes

### Crash / data-loss
- **A — Shard crash on a malformed Property Test.** `AdvancedSearchThreadWorker.Execute` had no `try/catch` and the worker `Thread` is foreground, so a parse throw (`Hits>abc`, `Layer=onehanded` — `Enum.Parse` was case-sensitive, `Hits>1@` — empty sub-expression indexing) terminated the process. Now: `ParseValue`/`CompareValues` use `TryParse`/`Enum.TryParse(ignoreCase)` and return no-match instead of throwing; the per-entity filter is wrapped in `try/catch` (logs + skips); empty expressions are guarded.
- **C — Overlapping searches corrupt state + brick autosave.** `_threadWorkers`/`_threadId` were `static` but `DoSearch` is an instance method; a second search (double-click / two admins) stomped shared worker state and could leave a drain waiting forever on the shared `AutoResetEvent`, so `AutoSave.SavesEnabled` was never restored. Now: an `Interlocked` re-entrancy guard rejects concurrent searches.
- **G — Autosave restore not guaranteed.** The restore lived only in the success callback. Now it's in a `finally` (plus an outer `catch` covering the synchronous setup and a `catch` on the drain body), so autosave + the guard are always released.

### Wrong results
- **D — `@`/`|` operator precedence.** `a@b|c` evaluated as `a && (b || c)` instead of `(a && b) || c`. OR now binds looser than AND (`AdvancedSearchUtilities.EvaluateBoolean`, unit-tested).
- **E — Descending sort, partial last page rendered blank** (the index decreased in descending mode and the `break` early-out killed the loop). Now a bounded `VisibleCount`-driven loop renders the last page in both directions.
- **F — Deleted entities** were not skipped (ghost rows). Now `DoEntitySearch` skips `entity.Deleted`.
- **N — Reference-type comparisons** threw (`Comparer<T>.Default.Compare` on non-`IComparable`) and compared references to a string. Now equality is by value and ordering is guarded to `IComparable` (no throw).

### Worker perf / hardening
- **H** busy-spin → `Thread.Yield()` in the drain; **I** `GetProperties()` cached per `Type`; **J** `HandleValidInternal` moved behind the cheap map/range/region filters; **K** worker threads are `IsBackground` + `Exit()` tolerates an already-terminated worker; **L** `_filter == null` guard; **M** consistent `Volatile` access on `_pause`/`_exit`.

### Documented
- **B** — the residual worker/event-loop read race is documented on `AdvancedSearchThreadWorker`: workers read live entity state concurrently with the loop, so value-type reads may be stale-but-safe and getter exceptions are swallowed; fully eliminating it would require snapshotting entity fields on the main thread (deferred).

## Notes
- New test-only seams (`TryBeginSearch`/`EndSearch`/`IsSearchInProgress`/`VisibleCount`/`TryParseValue`/`EvaluateBoolean`) are `internal` via the existing `InternalsVisibleTo("UOContent.Tests")`.
- Dead `public ParseValue<T>` removed.
- `ConcurrentDictionary` for the reflection cache is intentional — these workers are genuinely parallel.
2026-07-21 07:51:06 -07:00
Kamron Batman
ebaf104935
chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
Kamron Batman
6a0fcd62c1
fix: Fixes various mobile packets and setting serials (#1618)
### Summary
- Removes old death packet that isn't used. Doubtful this causes issues with clients that are v4+.
- Removes duplicate incoming packets. Again, probably to fix some old client issues, doubtful it affects clients v4+.
- Fixes setting serials and entities in props/commands. Note: Disabled setting `Parent` since the new sector code has issues. We shouldn't rely on it anyway!
- Reverts a recent change to healthbars that should not have been made. Oops!
2023-11-26 00:42:15 -08:00
mdodkins
e680bdb4b5
fix: Adds boolean support for Server.TryParse (#1605) 2023-11-21 12:21:59 -08:00
Kamron Batman
5fd0e94b7d
fix: Fixes negative numbers in commands (#1343) 2023-02-17 00:34:24 -08:00
Kamron Batman
03fb36c869
fix: Adds ISpanParsable and fixes command conditionals (#1241)
* Adds `ISpanParsable<T>`
* Removes `[Parsable]`
* Fixes querying by serial, body, and a few others.
* Adds `Parse` to `Rectangle3D`
* Fixes AutoArchive NPE


Closes #1209
2022-11-12 00:26:02 -08:00
Kamron Batman
c2d8f7d913
fix: Fixes string set in properties (#779) 2021-09-12 12:03:30 -07:00
Kamron Batman
04b5a6c609
fix: Makes guid settable. Cleans up type conversions (#765)
* Marks GUID a parsable type
* Cleans up type conversions
2021-09-05 01:53:00 -07:00
Kamron Batman
bf20800382
fix: Removes BodyValue and fixes modifying Body and BodyMod in props gump. (#743)
* Removes BodyValue
* Removes BodyPicker gump
* Fixes setting Body and BodyMod in props gump
* Props will now look for an `implicit cast` in order to find a way to allow you to modify values with the props gump.
* Fixed `[self set body 0x190` so it works too.
2021-08-28 22:56:00 -07:00
Kamron Batman
dd2933a9ef
fix(cleanup): Consolidate the duplicate code in Properties, PropsGump, and PropsConfig (#540) 2021-04-14 22:58:56 -07:00