Commit graph

117 commits

Author SHA1 Message Date
Kamron Batman
6d846b11e5
perf: Sleep the event loop when idle. Fixes networking micro-stalls. Adds event loop instrumentation. (#2559)
## Problem

`RunEventLoop` span through its body regardless of whether there was anything to do — ~10% of a desktop core for an empty shard, and ~70% of a core on a 3 vCPU VPS. A process that never idles is exactly what burstable vCPU plans throttle, which is how this surfaced: lag spikes that went away when the operator bought more cores. The spin also denied the GC its natural pause points, so memory climbed until a world save forced a collection — alarming in task manager, harmless in practice, and a recurring source of "is my server leaking?" reports.

## Result

Windows desktop, real world of **190,728 items / 33,158 mobiles**, no players, saves and prebake off, three consecutive runs:

| | Legacy spin | Idle sleeping |
|---|---|---|
| **CPU** | 10.42 – 10.50% of one core | **0.78 – 1.00%** |
| **Tick lag** (peak/15s) | 4–10 ms | 5–11 ms |

**~10× less CPU with tick lag unchanged** — the CPU came free rather than being traded for latency. Slower hosts gain proportionally more. Spin mode (`server.eventLoopIdleWaitMs=0`) independently gained **7× the iterations per core** (1.19M → 8.3M cycles/sec) from the ring's AcceptEx rework.

## How

The loop blocks in `NetState.WaitForCompletion` whenever every queue it drains is empty (all the drains are bounded, so leftovers keep it awake). Receive completions, new connections, and cross-thread `LoopContext.Post` (via the ring's sticky `Wake()`) are all in the wait set, so sleeping adds no latency to any of them. Only timer-driven logic sees wheel lag, bounded by the idle wait.

**Health is measured at the only place sleeping can cause harm.** A sleep is bounded by the time to the next wheel turn, so a correctly honoured sleep can never miss a deadline — the only failure mode is the host returning the wait late. That overshoot is measured on every sleep (one extra timestamp read; production's entire accounting cost), and an escalating backoff suspends sleeping when it persists. By construction, server work — saves, heavy staff commands, deep timer callbacks — cannot trip it, so the warning means exactly one thing: *the host is not scheduling the process promptly*, with two known remedies (dedicated CPU, or `=0`). Hosts with no high-resolution wait mechanism at all are detected once at startup and spin instead.

**CPS is removed.** `Core.CyclesPerSecond`/`AverageCPS` measured nothing actionable before and became actively misleading once the loop sleeps (the rate is set by the sleep, not by shard health). The admin gump's Performance page now shows the verdict instead: `Healthy` / `Sleep suspended (host)` / `Spinning (configured)`.

## Configuration

| Setting | Default | Meaning |
|---|---|---|
| `server.eventLoopIdleWaitMs` | `2` | Longest idle block. Measured across 1/2/4/8 ms, 2 is where the trade stops being free. `0` = never sleep: ~98% of a core, zero scheduling overhead — for large shards on dedicated CPU. |
| `server.lateWakeThreshold` | `1` | Idle waits the host may return a full tick late, per second, before sleeping backs off. Raise for jittery hosts; very high disables the backoff. |

## Diagnostics (compiled out by default)

`dotnet build -p:EventLoopProfiling=true` compiles in `EventLoopProfiler` — every hook is `[Conditional("EVENT_LOOP_PROFILING")]`, so normal builds contain zero profiling IL. The profiling build decomposes each second of wall time into **work (per loop phase) / sleep / GC pause / stolen residual**, keeps ~15 minutes of history in a ring buffer, and the `[LoopStats` command prints the last minute and dumps the full history to CSV. `dev-docs/debugging-event-loop.md` is the diagnosis guide (for humans and AI): what production already tells you, when to flip the profiling build, the signature table for host-steal vs deep-processing vs GC vs wake bugs, why dotnet-trace comes last, and the GC/RAM "leak" misconception.

## Verification

- 815 Server.Tests green; both build configurations compile.
- Docker echo harness green on epoll and io_uring (ping-pong mode); kqueue verified manually on an M1 Max.
- A/B measurements and per-change numbers: `measure/event-loop` branch.

## Notes

The full measurement harness and vendored ring sources used to develop this live on the [`measure/event-loop`](https://github.com/modernuo/ModernUO/tree/measure/event-loop) branch, kept for future loop work.
2026-08-09 13:24:59 -07:00
Kamron Batman
c39454137e
feat(network): pluggable connection filters; file blocklist + contribute-first CrowdSec (#2542)
Reshapes IP banning around one idea: **core owns the question, content owns every answer.**

Core gains a single accept-path seam — `IConnectionFilter` — and loses everything that used to implement one. The firewall moves to UOContent, a new file-backed blocklist joins it there, and CrowdSec is repositioned from an in-app enforcer to a contribute-first reporter.

## The seam

```csharp
public interface IConnectionFilter
{
    string Name { get; }
    void Configure();
    void Start(CancellationToken token);
    void Stop();
    bool ShouldDeny(IPAddress address);
}
```

The accept path went from hardcoded branches to one question:

```csharp
else if (ConnectionFilters.ShouldDeny(remoteIP, out var deniedBy))
{
    logger.Debug("{Address} denied by connection filter '{Filter}'", remoteIP, deniedBy);
}
```

Filters register during the Configure sweep. The registry is a plain array walked by an indexed loop — no enumerator, no closure, no allocation — and the first denial short-circuits. An interface dispatch is noise next to the `accept()` syscall, so pluggability costs nothing measurable on the path that has to survive a DDoS.

Whatever a hit implies — persisting, promoting to an OS bouncer, contributing to the ban channel — is the filter's business, not the accept path's.

A filter that throws is **unregistered and the connection fails open**. A filter that faults once faults for every subsequent connection, so leaving it registered means an exception and a log line per accept — exactly the amplification an attacker wants — and a broken filter must not be able to deny everyone either.

This deliberately does **not** reuse `EventSink.InvokeSocketConnect`: that fires later and allocates a `SocketConnectEventArgs` per connection, which is what the accept path avoids for rejected traffic.

## What ships behind it

**`firewall`** (UOContent) — the existing admin-curated set. Collapsed from `Firewall` + `AdminFirewall` + a threaded enforcer into one single-threaded store with **zero concurrency primitives**: the accept path, admin gump, TTL expiry and boot load all run on the game loop. Persists to `Configuration/firewall.json` with automatic migration from the legacy `firewall.cfg`. No behavior change for operators — same namespace, same gump, same commands.

**`blocklist`** (UOContent) — new. Holds a millions-strong list in-app and **demand-pages** hits up to CrowdSec, which promotes them to the OS firewall.

The motivation is concrete: CrowdSec's Windows bouncer cannot load the ~3.9M IPs that 91 community feeds produce, but it handles ~100k fine. So the millions live in-process behind a binary search, and only addresses that *actually connect* get promoted. A `PromotedGuard` suppresses re-reporting an address until the bouncer picks it up.

The list is parsed straight from UTF-8 file bytes with no per-line string allocation, off the game loop, and published as an immutable snapshot swapped through a single `volatile` reference. Reloads yield to world saves.

**`tools/Export-IpBlocklist.ps1`** — the producer. Requires PowerShell 7 and runs on Windows, Linux and macOS; Windows PowerShell 5.1 is refused up front via `#requires`. Merges a thin, non-overlapping feed set into one de-duplicated, bogon-filtered file. Parsing runs in a compiled `Add-Type` hot loop (~1s for ~4M lines instead of minutes). Written to a `.tmp` sibling and swapped with `File.Replace`, so the shard never reads a half-written list, and a total feed outage refuses to overwrite a good list with an empty one. Re-running is idempotent — it exits without downloading anything while the list on disk is younger than `-MinInterval` (default 2h, the anchor feed's own refresh period), so a misconfigured scheduler can't hammer upstream.

## CrowdSec: contribute-first

`IBanReporter` + `BanChannel` fan locally-decided bans out to external systems. `CrowdSecReporter` (UOContent) posts to LAPI `POST /v1/alerts` and retracts via `DELETE /v1/decisions`.

Reporting is **enqueue-only** on the accept path: a bounded, coalescing channel drained off-loop with bounded retry, counted drops on overflow, and a flush on shutdown. Under a DDoS the accept path never does synchronous or lock-contending per-IP work.

### Why not pull decisions from CrowdSec?

The original design streamed decisions into an in-app snapshot and enforced them at the accept gate. That's the wrong layer: by the time the shard sees the connection, the TCP handshake and socket setup are already paid for. `cs-firewall-bouncer` drops the same traffic **at the kernel**, and it's what CrowdSec is built to do. So the shard now contributes what it uniquely knows (rate-limit trips, blocklist hits from real connection attempts) and lets the OS enforce.

The one thing the OS can't do — hold millions of entries on Windows — is exactly what the in-app blocklist covers, and it feeds the same pipeline.

## Threading policy

`CLAUDE.md` rule #3 is rewritten as an explicit three-part policy, with rule #10 restated in tandem:

- Anything touching game state runs **only** on the main loop.
- Heavy work that *needs* game state must be **chunked** across ticks, never threaded.
- Heavy work that does *not* need game state (large-file parse, external I/O) **must** run off-loop **and must yield to world saves**.

Results come back via an immutable snapshot swapped through a single `volatile` reference, or `Core.LoopContext.Post` — never by letting the scheduler decide where heavy work runs. Both new subsystems follow it.

## Shared primitives

`SortedRangeIndex<T> where T : IBinaryInteger<T>` — coalesced disjoint interval arrays plus a binary search. The firewall, the blocklist, and (as of this PR) core's reserved-network tables all use it.

Coalescing is a correctness requirement, not an optimization: multi-feed lists nest CIDRs (`/24` containing a `/32`), and a search that inspects only the rightmost run whose minimum is ≤ the value is sound **only** over disjoint runs. That bug was caught in review and is covered by regression tests.

`IPAddressUtility` collects the allocation-free `IPAddress` ↔ `UInt128` conversions and CIDR parsing that were previously scattered or duplicated.

## Config

| File | Owner | Keys |
|---|---|---|
| `Configuration/bans.json` | core | `reportRateLimitTrips`, `autoBanDuration` |
| `Configuration/blocklist.json` | content | `file`, `reloadInterval`, `reportHits`, `banDuration`, `promoteSuppression` |
| `Configuration/crowdsec.json` | content | `lapiUrl`, `machineId`, `password`, `origin`, `manualBanDuration`, `flushInterval`, `maxQueue` |
| `Configuration/firewall.json` | content | persisted firewall entries (migrated from `firewall.cfg`) |

Everything is inert by default. CrowdSec self-disables without credentials; the blocklist self-disables until its file exists. A shard that changes nothing sees no behavior change.

## Notes for review

- **Core no longer references `Firewall` or `IFirewallEntry` anywhere.** `NetworkUtilities` used to build its reserved-network tables out of `CidrFirewallEntry`, which coupled core to the firewall for something unrelated to banning; those are now a `SortedRangeIndex<UInt128>`, same semantics and public API.
- **`BanChannel.Stop()` no longer persists the firewall** — a contribution coordinator has no business saving an enforcement store. That's the firewall filter's `Stop()`.
- **A dead `whitelisted` parameter was dropped** from the blocklist gate: it was hardcoded `false` at its only call site, and no whitelist concept exists in core.
- **The blocklist filter is an instance, not a static.** The static version forced its tests onto the sequential collection with a reset hook; they now run in parallel.
- `dev-docs/networking-packets.md` documents the seam for content authors, plus a known wart in the `IPAddress` ↔ `UInt128` normalization flagged for a follow-up PR.
- The generator was verified on Linux, macOS and Windows under a temporary CI matrix (since removed). It caught two portability bugs — a Windows-only path separator, and a culture-sensitive duration parse that read `2.5` as `25` on comma-decimal locales and *silently* turned a 2.5h cooldown into 25h — plus a third that made the script unparseable on Windows PowerShell 5.1. The source is ASCII-only for that last reason: `#requires` is only honored once a file parses, so non-ASCII in a BOM-less script produces parse errors instead of the version message.

## Tests

**1344 pass** (782 `Server.Tests`, 562 `UOContent.Tests`). New coverage: filter registry (registration, short-circuit, fault-disable), blocklist parsing/CIDR/coalescing, snapshot reload markers, promote-guard TTL, ban-channel fan-out, CrowdSec alert building/dedup/flush-on-stop, and the generator's output-format contract pinned against the reader.
2026-07-25 11:59:37 -07:00
Kamron Batman
7434ed7ee1
fix(console): stop headless servers from pegging a CPU core (#2535)
## Problem

On headless Linux deployments (systemd service, Docker without a TTY, `nohup`), the ModernUO process pegs a full CPU core even when idle. It does not reproduce on Windows because that runs with an interactive console.

## Root cause

`ConsoleInputHandler` runs a background thread (named "Console Input Handler") that loops on `Console.ReadLine()`. When stdin is **not** an interactive terminal, `Console.ReadLine()` returns `null` at end-of-stream **immediately** on every call, so the loop `continue`s in a tight spin — one core at 100%.

Reproduced in a container running the actual distribution: the "Console Input Handler" thread sat at ~90% CPU on a headless boot; with a blocking stdin it dropped to idle.

## Fix

1. **Detect headless once at startup:** `Core.Headless = Console.IsInputRedirected`.
2. **Extract a testable `ConsoleInputPump`** that owns the input stream: per line read, it *atomically* (under one lock) either delivers the line to a waiting prompt or dispatches a console command, and it **ends on EOF instead of spinning**. Cleanup runs unconditionally in a `finally`, so a pending prompt is always released (never hangs). Replaces the old `async void` loop and the fragile `_expectUserInput` / two-`AutoResetEvent` / `_input` handshake.
3. **`ConsoleInputHandler` becomes a thin headless-aware facade** over the pump. Headless: the reader thread never starts (`Console input disabled (headless: stdin is not a TTY).`), and `ReadLine()` throws a fatal `HeadlessConsoleInputException`.
4. **Data-gating and first-boot prompts** (deserialization "delete bad types? y/n", save-conflict, config/expansion setup) now route through `ConsoleInputHandler.ReadLine()`, so a headless server crashes fatal with a clear message instead of reading `null` (previously an NRE or a silent wrong branch).

Design decision (model b): headless servers are expected to be supplied with configuration/save data (including the owner account); interactive prompts when headless are fatal by design.

## Testing

- New `ConsoleInputPumpTests` (5 tests): EOF ends the loop without spinning; command dispatch; a pending prompt receives the next line; EOF while a prompt is pending completes it with `null` (no hang); a throwing command lookup does not hang a pending prompt. The tests synchronize on real pump state (no `Thread.Sleep`), so they are deterministic on slow CI.
- Full `Server.Tests`: no new failures introduced.

## End-to-end verification (Docker, real distribution)

| | Console Input Handler thread | Container CPU |
|---|---|---|
| Before fix (headless boot) | ~90% | ~199% (2 cores) |
| After fix (headless boot) | **not started** | **~11%** |

After the fix, a headless boot logs `Console input disabled (headless: stdin is not a TTY).`, loads the world normally, and idles instead of spinning.
2026-07-16 18:52:43 -07:00
Kamron Batman
2e93201e51
feat(pathfinding): first-boot prompt to pre-bake the .swb map cache (#2475)
## What

On **first boot** (right after map selection), offer to pre-bake the pathfinding `.swb` cache for the selected maps. This removes first-pathfind-after-boot latency and is now cheap — ~18 MB/facet after the v8 format work (the old ~565 MB is gone). The answer persists in `modernuo.json` as **`pathfinding.prebakeMaps`** (default **false**): asked exactly once, and skipped on headless/CI boots (redirected input) where operators can set the flag directly.

## How — a generic startup phase, not pathfinding hardcoded in the engine

The clean-console (pre-Serilog) prompt window is inside the engine startup, but UOContent isn't loaded until after `ServerConfiguration.Load`. So rather than coupling the engine to pathfinding, this adds a generic lifecycle phase:

- **`Main.cs`**: new `AssemblyHandler.Invoke("ConfigurePrompts")` — runs **after** `LoadAssemblies` (so content can participate) but **before** the first `logger.Information` (so console prompts aren't interleaved with the async console sink). The first log line moves below it. Any class can hook in with `public static void ConfigurePrompts()` and self-gate on first-boot state. No `ServerConfiguration` or pathfinding coupling added to the engine.
- **`PathCacheCommands.ConfigurePrompts()`**: the first-boot prompt (interactive-only, flag-absent-only); persists the answer.
- **`PathCacheCommands.Initialize()`** (`Invoke("Initialize")` phase, after the tile matrix loads — which the bake walks): when the flag is set, bakes any map whose `.swb` is **missing or stale** (tile-data fingerprint mismatch, via `StepCache.ComputeLiveFingerprint` / `TryReadFingerprintFromFile`). A fresh cache is a no-op, so only the first boot — or a post-client-update boot — pays the several-minute cost.

## Docs

Fixed the now-stale "~565 MB / ~1.5–2 GB / do not bake by default" section in `dev-docs/pathfinding.md` (it's 17.9 MB for Trammel, tens of MB for all six facets after v8), added a "First-boot pre-bake prompt" section, and added the `pathfinding.prebakeMaps` lever row.

## Verified

- `dotnet build UOContent -c Release` → 0 errors (rebased on #2474).
- Pathfinding/StepCache tests: **90/90 pass**.
- Bootstrap streamlining of the startup phases is intentionally left as a follow-up.
2026-06-07 16:30:33 -07:00
Kamron Batman
e3cba66284
feat: Adds dynamic thread idle to address CPU usage (#2370)
## Summary
- **Timer-aware idle sleep**: Exposes `Timer.MillisecondsUntilNextTick()` to calculate remaining ms until the next timer wheel tick (0–8ms). The game loop sleeps for that duration minus a 1ms safety margin, instead of spinning at 100% CPU.
- **I/O completion wakeup**: Replaces `Thread.Sleep` with `NetState.WaitForCompletion()`, which uses platform-native completion notification (RIO `RIONotify` on Windows, `eventfd` on Linux, `kevent` timeout on macOS) to wake immediately when network data arrives during sleep.
- **Always-on**: Removes the debug-only `core.enableIdleCPU` config gate. The sleep is self-regulating — under load, `MillisecondsUntilNextTick` returns 0 so no sleep occurs (zero overhead). On idle, CPU drops from ~100% to ~1%.
- **CPS calculation cleanup**: Replaces the 128-element ring buffer with an EMA (exponential moving average) for `CyclesPerSecond`/`AverageCPS` — fewer allocations, no LINQ `.Average()` call each sample.
- **IORingGroup 1.0.6**: Adds `WaitForCompletion(int timeoutMs)` to the `IIORingGroup` interface with platform implementations:
  - **Windows**: `RIONotify` arms the CQ event, `WaitForSingleObject` with timeout
  - **Linux**: `eventfd` registered with io_uring, `poll()` with timeout
  - **macOS**: `kevent()` with timeout

## Test plan
- [ ] Build succeeds on all platforms (`dotnet build`)
- [ ] Empty server: verify CPU usage drops from ~100% to ~1% idle
- [ ] Loaded server: verify no added latency — `MillisecondsUntilNextTick` returns 0 when timers are firing, sleep is skipped
- [ ] Connect a client during idle — verify connection accepted within one timer tick (~8ms)
- [ ] Verify `[admin` gump shows reasonable CPS values (EMA convergence)
2026-03-13 23:53:49 -07:00
Kamron Batman
04d438239d
feat: Adds robust speed hack detection and movement throttling (#2266)
## Summary

   Server-side movement throttle that prevents speed hacking while accurately identifying cheaters with detection of lagging connections.

   **Key features:**
   - Credit buffer (200ms) absorbs timing jitter from legitimate players
   - Movement queue handles larger bursts, draining at proper game-tick intervals
   - RTT measurement distinguishes network lag from speed hacks
   - Queue depth detection catches ACK-throttled speed hacks (going straight)

   ## How It Works

   **Throttle** (prevention): Movements arriving too early either consume credit or get queued. The queue drains at
   correct intervals, so speed hackers can't move faster regardless of what they send.

   **Detection** (identification): Combines multiple signals to identify cheaters:
   | Signal | What it catches |
   |--------|-----------------|
   | Queue depth ≥4 sustained | ACK-throttled speed hacks (client limits unacked moves to 5) |
   | Movement rate >1.05x | Direction-change speed hacks where timing is visible |
   | Stable RTT + high queue | Eliminates false positives from laggy players |

   **RTT-Aware Logic:**
   - Probes only sent to players actively moving (event-driven, not global loop)
   - Stable low-latency + problems = suspicious
   - Unstable/high-latency + problems = probably just lag, throttle handles it

   ## Configuration

   ```json
   {
     "movementThrottle.maxCredit": 200,
     "movementThrottle.softQueueLimit": 6,
     "movementThrottle.hardQueueLimit": 10,
     "movementThrottle.debugLogging": false
   }
   ```
2026-03-07 11:44:37 -08:00
Kamron Batman
e1e1a7c640
fix: Bumps deps. Updates copyrights (#2353) 2026-03-05 19:36:54 -08:00
Kamron Batman
3c0d6cb9d6
feat: Upgrades networking to use io_uring. (#2315)
> [!IMPORTANT]
> **Breaking Changes**
> - DecodePacket and EncodePacket delegates replaced with IClientEncryption interface
> - NetState.Connection (Socket) replaced with internal RingSocket management
> - NetState.RecvPipe and NetState.SendPipe removed (buffers managed internally)

## Summary

Upgrades the networking stack from PollGroup-based I/O to io_uring, significantly improving I/O performance on Linux.
This also adds native client encryption support for encrypted UO clients.

## Major Changes

io_uring Networking Architecture
- Replaced PollGroup with IORingGroup for async socket I/O operations
- Removed Pipe.cs (mirrored ring buffer) and TcpServer.cs in favor of RingSocketManager
- Added NetState.Network.cs - centralized network infrastructure handling accept, recv, send, and disconnect
completions
- Added SocketHelper.cs - platform-specific socket utilities for raw socket handle operations (getpeername,
getsockname)
- Buffer management now handled by RingSocketManager with configurable slab allocation

### Client Encryption Support
- Added full encryption stack in Network/Encryption/:
  - EncryptionConfig.cs - configurable encryption modes (None, Unencrypted, Encrypted, Both)
  - EncryptionManager.cs - encryption detection and initialization for login/game packets
  - LoginEncryption.cs - handles login packet encryption with version-derived keys
  - GameEncryption.cs - handles game server encryption using Twofish
  - TwofishEngine.cs - optimized Twofish block cipher implementation
  - LoginKeys.cs - encryption key table for client versions
  - IClientEncryption.cs - interface for client encryption implementations

### NetState Improvements
- Replaced Socket Connection with RingSocket _socket for managed socket lifecycle
- Changed from GCHandle polling to event-based completion processing
- Disconnect handling now properly waits for pending sends to flush
- Simplified connecting socket management using lazy queue removal

### Configuration
- New settings: network.encryptionMode and network.encryptionDebug
- Encryption mode flags: Unencrypted, Encrypted, or Both

### Dependencies
- Replaced PollGroup NuGet package with IORingGroup
- Linux requires liburing-dev / liburing-devel package

### Test plan

- Verify server starts and accepts connections on Linux with io_uring
- Verify server starts and accepts connections on Windows (fallback to IOCP)
- Test unencrypted client connections (ClassicUO with encryption disabled)
- Test encrypted client connections if available
- Verify graceful disconnect flushes pending data
- Confirm CI builds pass on all target platforms
2026-02-01 16:02:32 -08:00
Kamron Batman
89f620218d
fix: Fixes idleCPU override (#2308) 2026-01-04 17:21:08 -08:00
Kamron Batman
ebaf104935
chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
Kamron Batman
78feea86b4
feat: Adds scheduler with wallclock timer (#2163)
### Summary

* Adds an event scheduler.
* Adds conveniences for hourly, daily, weekly, biweekly, monthly, ordinal monthly, and yearly recurrences

Example:

```cs
var tz = TimeZoneInfo.FindSystemTimeZoneById("America/New_York");

// Specify the time of the day, and the day of the week you want it to occur. Make sure it is translated into Utc.
// The next occurrence will be _after_ the specified date/time.
var scheduledEvent = EventScheduler.WeeklyAt(new DateTime(2025, 04, 26, 17, 00, 00), StartEvent, tz);

void StartEvent()
{
    World.Broadcast(0x30, false, "The event has started!");
}

Console.WriteLine("Event starts on {0}", scheduledEvent.NextOccurrence);
```

In this example, on _Saturday, May 3rd, 2025 @ 5pm ET_, the message "The event has started!" will be broadcasted.
2025-04-26 22:27:08 -07:00
Kamron Batman
fdf8c5cf23
fix: Fixes critical bug in TickCount calculation. (#2151)
### Summary

On some operating systems (like hosted Linux VMs), the `TimeStamp.GetTimeStamp()` CPU tick count will wrap around. This is generally not an issue, except for legacy reasons the TickCount is returned in milliseconds instead of ticks. This means when the values wrap around, they are already divided by the CPU Frequency (usually 1million) and then converted to milliseconds. That means the delta between the tick count before and after wrapping is off by a magnitude of (Frequency / 1000).

Example:

TimeStamp A = 9223372036654775807
Some time has passed:
TimeStamp B = -9223372036654775809

The raw delta is 400_000_000 (400ms) when you do `unchecked(A - B)`.
If we do the calculation AFTER converting it to milliseconds, then:

TickCount A = 9223372036654
TickCount B = -9223372036654

The raw delta is -18446744073308 instead of 400_000_000.

To fix this the calculation was changed so `long` -> `ulong`, then divided, then converted back to `long`, effectively bypassing wrap-around issue.

The new TickCount values in our example become:

TickCount A = 9223372037054
TickCount B = 9223372036654

The delta is 400 (in milliseconds). 🎉
2025-04-09 16:46:53 -07:00
Kamron Batman
e64a632998
fix: Moves snapshot request synchronously (#2105) 2025-02-02 13:07:28 -08:00
Kamron Batman
717a1a062e
fix: Fixes race condition with world save snapshot request (#2104)
### Summary

* Fixes a race condition where the snapshot path isn't between the request snapshot being set on a background thread, and the main loop consuming that flag.


Closes #2102
2025-02-02 12:05:40 -08:00
Kamron Batman
b475e17c92
fix: Forces InvariantCulture on server start for DefaultThreadCurrentCulture (#1988)
Co-authored-by: Stefano Merotta <97297186+stefanomerotta@users.noreply.github.com>
2024-11-01 21:21:10 -07:00
Kamron Batman
e0fcde885c
fix: Fixes networking issues (#1958)
### Summary
- Puts back `EventSink.SocketConnect`.
- Reverts networking change to push the networking to a separate thread.
- Reverts changes to the firewall by removing the firewall queue.
- Fixes listeners not shutting down with the server.
- Fixes race condition causing connections to get stuck even after they are disposed.

> [!NOTE]
> **Developer Note**
> Networking has been reverted back to using the main thread instead of a background thread. This alleviated complexity and the requirement for concurrent queues all over the place.
2024-09-19 16:54:49 -07:00
Kamron Batman
8639ed2210
fix: Fixes bad password with SHA1/SHA2 (#1952) 2024-09-12 17:20:44 -07:00
Kamron Batman
257825dc23
fix: Removes invalidation of tickcount and now in core (#1949) 2024-09-11 22:51:02 -07:00
Kamron Batman
2e6ddcd31a
fix: Removes profiling. Streamlines core tick count. (#1948)
### Summary
- Removes `Profiling` - Recommend using Visual Studio Performance Profiler,  [dotnet-trace](https://learn.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-trace) or [JetBrains dotTrace](https://www.jetbrains.com/profiler/)
- Cleans up the core tick count, sampling, etc.
2024-09-11 00:55:18 -07:00
Kamron Batman
8ec203f387
feat: Updates serialization to use MMF (considerable memory savings) (#1841)
### Summary
Updates the serialization strategy to use `MemoryMappedFile` instead of thick buffers. This has the benefit of being on-par with the current implementation (based on hardware/OS), however won't incur the double-memory issue.

> [!Important]
> **Developer Note**
> The `BinaryFileWriter` and `BinaryFileReader` has been removed in favor of `MemoryMapFileWriter` and `UnmanagedDataReader`
2024-06-23 10:51:20 -07:00
Kamron Batman
1f8ab0e783
fix: Console Input is disabled on Linux when no standard input is attached. (#1809) 2024-06-06 21:20:22 -07:00
Kamron Batman
a8d3d2773e
fix: Fixes libdeflate threading issue (#1813) 2024-06-02 15:02:47 -07:00
Kamron Batman
60a3a6f501
feat: Splits Server in Core and Application (#1806)
> [!Note]
> **Developer Note**
> Now developers will only need to build/run the Application project instead of everything.
> When adding new projects, make sure to: 
> 1. Add a reference to that project in the Application project.
> 2. Add the dll file to the Distribution/Data/assemblies.json file

### Summary
- Adds an application project
- Consolidates process restarts to use `Core.Kill(true)`
- Removes some old messaging, for example processor optimization
- Fixes missing build cleanup
2024-05-30 23:34:03 -07:00
Kamron Batman
912eaab460
fix: Fixes unresponsive console input (#1720)
### Summary
Fixes an issue where the call priority of `Initialize` functions causes a dead lock because the console input handler was not initialized.
2024-04-05 15:07:43 -07:00
Kamron Batman
183f6fa4ac
fix: Fixes stuck connections (#1708) 2024-04-03 08:14:41 -07:00
Kamron Batman
4cd668ef61
feat: Moves TcpServer to another thread. Rewrites Firewall (#1660)
## Breaking Changes
* The Firewall and IP Limiter have been rewritten. Please read the notes carefully!
* `TcpServer.Instances` moved back to `NetState.Instances` - sorry - it was stupid to move it to begin with.

> [!Note]
> Sockets that fail the IP Limiter or Firewall will be immediately and forcibly disconnected.
> This means they will be stuck at "Verifying account..." if it was a real client.

### Summary
- Removes firewall wildcard support.
- Removes `AccessRestrictions`.
- Moves Firewall/IPLimiter to the core.
- Moves `TcpServer` to its own thread.
- Removes the `SocketConnect` and `SocketDisconnect` event sinks.
- Moves `Instances` back to `NetState.Instances`.
- Fixes a long standing bug with bad handling of duplicate listener addresses.

#### Firewall
The firewall has been completely rewritten. There is now an "Admin Firewall" which saves to the config file. Secondarily, there is an internal firewall used exclusively by the TcpServer while processing sockets. The Admin firewall mirrors it's additions/deletions to the internal firewall by adding requests to a queue.

> [!IMPORTANT]  
> **Wildcard firewall entries, such as `X`, `*`, `?` are not allowed.**
> **Ranges in between IP classes or sextets are not allowed.**
> **Please make sure to use one of the following:**
> * IP Address - `192.168.1.1`
> * CIDR - `192.168.1.0/24`
> * Range - `192.168.1.1-192.168.1.100`

#### IP Limiter
The IP Limiter has been completely rewritten. The available configurations are:
```json
"ipLimiter.enable": "True",
"ipLimiter.maxConnectionsPerIP": 10,
"ipLimiter.clearConnectionAttemptsDuration": "00:00:00:10",
"ipLimiter.clearThrottledDuration": "00:00:02:00",
```

The IP Limiter is set up to prevent spamming connections from the same IP. Every time an IP connects, it is added to a connection list. After 10 attempts, the IP is added to the throttle list. To keep the system fast, the connection list is entirely wiped every 10 seconds, and the throttle list is entirely wiped every 2 minutes.
2024-01-20 14:25:12 -08:00
Kamron Batman
823023275b
fix: Stops trying to perform snapshots when the world is not saving (#1658) 2024-01-15 08:29:54 -08:00
Gnai
25a75fdf24
fix: Update Main.cs to accept utf8 on console (#1639) 2023-12-16 00:33:56 -08:00
Kamron Batman
a93a363c5f
fix: Fixes shutdown+save (#1614) 2023-11-25 10:11:24 -08:00
Kamron Batman
469b89370d
fix: Adds back Average CPS to Admin Gump (#1573) 2023-10-30 18:05:32 -07:00
Kamron Batman
629a5008c3
fix: Fixes hanging tests (#1535) 2023-10-09 00:22:59 -07:00
Kamron Batman
1f0acddc46
fix: Fixes serialization threading, moves world save to end of loop, and eliminates Parallel.ForEach. (#1530) 2023-10-03 00:42:49 -07:00
Kamron Batman
60a6796903
feat: Adds ping listener on port 12000 for server list (#1485) 2023-09-03 09:29:27 -07:00
Kamron Batman
8389bfacfe
chore: Updates copyright (#1448) 2023-08-09 09:09:26 -07:00
mdodkins
a9a2a89908
feat: Customize expansion and set maps on first boot (#1425) 2023-07-31 20:46:49 -07:00
Kamron Batman
51ecee6caa
fix: Overhauls champion titles & codegen champion system (#1430)
## MAJOR CHANGE

Added a champion title system to facilitate the existing champion titles. This should make it easier to extend or create other related game content. Champion titles will be saved in a folder called _ChampionTitles_.

### Motivation

The motivation to refactor was two-folder, but mostly related to performance in two ways.
First, every player had a ChampionTitleInfo object with an array of ChamptionTitleInfo. We want to eliminate the need for this information unless a player actually uses it. This should save a considerable amount of memory.

Second, to facilitate the atrophy mechanic, the champion titles would run atrophy post-world save, adding to the time that the server is frozen. Eliminating this post-world save side effect unlocks our ability to further optimize the world save process since there are no direct side effects.


### Bugs fixed

- [X] Fixed titles getting cut off on the paperdoll
- [X] Fixed champion title not displaying overhead (OPL)

### Screenshots
<img width="216" alt="image" src="https://github.com/modernuo/ModernUO/assets/3953314/8916f895-8d68-4fb0-892e-108a0c43be90">
2023-07-27 21:44:06 -07:00
mdodkins
d366e02251
fix: Fixes missing ordinal string comparisons arguments. (#1431)
---------

Co-authored-by: Kamron Batman <3953314+kamronbatman@users.noreply.github.com>
2023-07-23 20:12:38 -07:00
Kamron Batman
dda55889f6
fix: Fixes tick count calculcations (#1366) 2023-03-09 21:00:17 -08:00
Kamron Batman
18c4459e6b
fix: Fixes region priority and resolution. (#1254)
- [X] Fixes world location JSON deserializer.
- [X] Fixes region resolver and priorities.
- [X] Removes DynamicJson from regions.
- [X] Adds missing region music.
2022-11-21 10:38:20 -08:00
Kamron Batman
d8cfb6b935
fix: Makes logging more consistent (#1246) 2022-11-13 00:36:23 -08:00
Kamron Batman
49e6c6f2d1
fix: Adds AfterSerialize support. Removes BeforeSerialize support. (#1208)
### Changes
* Implements an AfterSerialize method that is executed synchronously.
* Removes `BeforeSerialize` support since it was dangerous in its current implementation.
* Moves PlayerMobile kill/virtual decay to AfterSerialize.
* Adds kill decay to after Deserialize.
2022-10-27 23:27:22 -07:00
Kamron Batman
f268d5d4e2
fix: Cleans up core code (#1187)
**Only one functional change**
* Fixes a bug in LogFactory where `Warning` is being logged as `Information`

Non-functional changes:
* Updates/Fixes copyright headers
* Removes namespace scopes for core files.

View with [whitespace off](https://github.com/modernuo/ModernUO/pull/1187/files?w=1).
2022-10-10 21:47:08 -07:00
Kamron Batman
12404ba6f6
fix: Fixes linux not enumerating directories, and wrong spelling for migration files (#1049) 2022-06-12 09:59:23 -07:00
Kamron Batman
3b7e3c2fb7
fix: Fixes ValueStringBuilder empty ctor issue (#1035) 2022-05-27 17:49:36 -07:00
Stefano Merotta
fe225003c6
fix: Adds multithread optional flag to ValueStringBuilder (#1033)
Added multithread optional flag to ValueStringBuilder for multithread contexts.
Enabled new flag for VerifyType method.

Resolves #1032
2022-05-27 10:25:30 -07:00
Kamron Batman
be7da3a3dc
feat: Adds cliloc support and fixes valuestringbuilder ctor (#1013)
- [X] Adds cliloc support using the following API:
```cs
public static class Localization
{
    string GetText(int number);
    string GetText(int number, string lang);
    string Format(int number, params object[] args);
    string Format(int number, string lang, params object[] args);

    string Format(int number, $"{arg1}{arg2}{arg3}");
    string Format(int number, lang, $"{arg1}{arg2}{arg3}");

    bool TryGetLocalization(int number, out LocalizationEntry entry);
    bool TryGetLocalization(int number, string lang, out LocalizationEntry entry);
}

public class LocalizationEntry
{
    string Language { get; }
    string Number { get; }
    string Text { get; }
    string?[] TextSlices { get; } // Used for string building
    string StringFormatter { get; }

    string Format(params object[] args);
    string Format($"{arg1}{arg2}{arg3}");
}
```

- [X] Fixes a bug with ValueStringBuilder and default initialization size
- [X] Optimizes ValueStringBuilder to use `ISpanFormattable`
- [X] Adds `Append<T>(T value);` support ValueStringBuilder
- [X] Adds `Append($"");` string interpolation support to ValueStringBuider
2022-05-08 21:24:32 -07:00
Kamron Batman
54d728a322
fix: Updates serialization to use v2.0 (#998)
- [X] Deletes serialization annotations
- [X] Updates to ModernUO.Serialization.Annotations nuget
- [X] Updates serializer to v2.0
- [X] Changes all `Serializable()` to `SerializationGenerator()`
- [X] Updates to schema generator v2.0
2022-04-17 08:02:15 -07:00
Kamron Batman
910549db84
fix: Fixes loading multi when client is missing (#996)
* Changes reading multi.mul/multi.idx so it is loaded entirely during bootup.
* Fixes finding files in Linux.
* Fixes handling HS format.
* Removes verdata support.

ModernUO will still have issues if there is a multi.mul that is pre-HS loaded when it detects a client that is 7.0.9+.
2022-04-14 16:52:45 -07:00
Kamron Batman
bfbe2a6512
fix: Updates professions & new player gargish equipment (#819)
* Now reads Prof.txt to get the professions
* The templated professions are based on profession `TrueName`
* Adds missing starting equipment
* Implements gargish starting equipment
* Consolidates some of the code
* Moves TC stuff to the TC file.
* Fixes some starting equipment that was wrong.

Note:
* Implementing the original T2A professions is possible on the client side, but requires moving the old clilocs to the current clilocs file. In the current file the entries are blank.
* Using the original LBR professions (they match AOS but without necromancy/chivalry) is kind of confusing because the profession indexes are ripped out of the UOTD/UOR ones and have non-sequential indexes. This will probably cause confusion and people will get no templated items when they select the wrong profession.
2021-10-16 10:48:58 -07:00
Kamron Batman
a845f8a1c0
feat: Networking v3 using epoll/kqueue (#813)
* Replaces networking internals with epoll/kqueue
2021-10-03 18:04:53 -07:00