Commit graph

16 commits

Author SHA1 Message Date
Kamron Batman
3b4e1137f6
feat: Implements new robust/pluggable backup/archive system. (#2388)
## Summary

Overhauls the backup/archive system to address stability, robustness, fault tolerance, performance, and pluggability.

### Problems solved
- **Critical bug**: `Interlocked.CompareExchange` arguments were reversed — the concurrent archive guard never actually prevented concurrent archives
- **Brittle compression**: Relied on spawning external `zstd.exe`/`tar.exe`/`bsdtar.exe` processes with platform-specific workarounds (Windows two-step hack, auto-downloading bsdtar from libarchive.org)
- **No fault tolerance**: Partial archives possible if process killed mid-write; no recovery mechanism; originals deleted before verification
- **No extensibility**: No way to add remote backup destinations (S3, rsync) without modifying core code
- **Silent failures**: Bare `catch` blocks swallowed exceptions with no diagnostics

### What changed

**Cross-platform streaming compression** — Replaced external process spawning with `System.Formats.Tar` (built-in .NET) + `ZstdNet` (native libzstd P/Invoke). Single-pass streaming: `TarWriter → CompressionStream → FileStream`. No intermediate `.tar` file, no platform-specific code paths.

**Archive journal** — JSON-based operation journal (`Archives/.archive-journal.json`) tracks state machine: `Started → Archived → Distributed → Completed` (or `Failed`). On startup, recovers interrupted operations (cleans temp files, completes pruning).

**Verify before delete** — Archives are verified (entry count check) before deleting source backups. Temp-file-then-atomic-rename pattern prevents partial archives.

**Retry logic** — File operations (moves, deletes) retry with linear backoff for transient I/O failures (antivirus locks, file copy operations).

**Plugin architecture** — `IArchiveDestination` interface in `Projects/Server/` enables external plugins (loaded via `assemblies.json`) to receive completed archives. `ArchiveDestinationRegistry` tracks destinations. Conservative pruning: source backups preserved if any destination with retention fails.

**Configurable** — Retention counts (hourly/daily/monthly), compression level, retry settings, backup max age all configurable via `ServerConfiguration`.

### New admin commands
- `[ArchiveStatus` — Shows journal state, destinations, next scheduled archive times
- `[ArchiveNow` — Forces immediate rollup regardless of schedule

### Files

| Change | File |
|--------|------|
| New | `Server/Saves/ArchiveEnums.cs`, `IArchiveDestination.cs`, `ArchiveDestinationRegistry.cs`, `ArchiveEventArgs.cs` |
| New | `Server/Events/ArchiveEvents.cs` (partial EventSink) |
| New | `UOContent/Compression/ManagedArchive.cs` |
| New | `UOContent/World Saves/ArchiveJournal.cs`, `LocalArchiveDestination.cs` |
| Rewrite | `UOContent/World Saves/AutoArchive.cs` |
| Modified | `UOContent/World Saves/SaveCommands.cs`, `Server/Utilities/PathUtility.cs`, `UOContent.csproj` |
| Deleted | `UOContent/Compression/TarArchive.cs`, `ZstdArchive.cs` |
| Tests | 29 new tests (ManagedArchive, ArchiveJournal, AutoArchiveHelpers) |

### Backward compatibility
- Existing `.tar.zst` archives are fully readable by the new managed code
- Config keys preserved; new keys use sensible defaults
- `autoArchive.compressionFormat` setting removed (Zstd only)
- Legacy `bsdtar/` directory logged as removable on startup

### Plugin example (external repo)
```csharp
public static class S3Plugin
{
    public static void Configure()
    {
        ArchiveDestinationRegistry.Register(new S3Destination("my-bucket", "us-east-1"));
    }
}
```

## Test plan
- [x] 29 new unit tests covering archive create/extract/count, journal state machine, recovery, destinations
- [x] All 969 tests pass (671 Server + 298 UOContent)
- [x] Manual: run server, trigger save, verify backup created and archive rolled up
- [x] Manual: kill server mid-archive, restart, verify journal recovery
- [x] Manual: test with large save files (~500MB+) to benchmark streaming vs old approach
2026-03-22 19:51:20 -07:00
Kamron Batman
61e41df00c
feat: Add zero-alloc interpolation handler to ValueStringBuilder, replace all StringBuilder usage (#2387)
## Summary

- **Add a self-referencing `InterpolationHandler` to `ValueStringBuilder`** that writes directly into the builder's buffer — zero intermediate allocation, works with `stackalloc`-backed builders
- **Replace all `System.Text.StringBuilder` usage** across the codebase with `ValueStringBuilder`
- **Convert `ValueStringBuilder.Create()` to `stackalloc`** at 10 sites where output length is provably bounded
- **Convert manual `Dispose()` to `using var`** where possible, and hoist loop-scoped builders outside loops with `Reset()`
- **Convert verbose `Append()` chains to `Append($"...")`** interpolation for readability
- **Add comprehensive documentation** for string handling patterns

## InterpolationHandler Design

`ValueStringBuilder` is a `ref struct`, which creates challenges for C#'s interpolated string handler pattern:

- **`ref` fields to ref structs are not allowed** (CS9050)
- **`[InterpolatedStringHandlerArgument("")]` passes struct receivers by value**, not by ref
- **`ISelfInterpolatedStringHandler` requires boxing** ref structs into interface fields

**Solution: Copy-and-reconcile pattern.** The handler receives a value copy of the builder. The copy shares the same underlying `char` buffer (`Span` points to the same `stackalloc`/pooled memory), so writes go to the original buffer. `Append()` reconciles by `this = handler._builder`, updating `_length` and any buffer references changed by `Grow()`.

This is safe because:
- The game loop is single-threaded — no concurrent access between handler construction and reconciliation
- If `Grow()` occurs in the copy, the original's stale buffer isn't accessed until `Append()` replaces it
- `Dispose()` correctly returns the reconciled buffer to the pool

## Changes by Category

### ValueStringBuilder (`Projects/Server/Buffers/ValueStringBuilder.cs`)
- Added nested `InterpolationHandler` ref struct with copy-and-reconcile pattern
- Added `Append([InterpolatedStringHandlerArgument("")] scoped ref InterpolationHandler)` method
- Removed `RawInterpolatedStringHandler` overloads (new handler replaces them)
- All `AppendFormatted` overloads delegate to existing `Append` methods (no code duplication)
- Alignment support via direct private field access (nested type privilege)

### StringBuilder → ValueStringBuilder (15 files)
Replaced all `new StringBuilder()` with `ValueStringBuilder.Create()` or `stackalloc`:
- ConPVP games: KingOfTheHill, DoubleDom, CTF, BombingRun, TourneyMatch
- ConPVP infrastructure: Tournament, Participant, TourneyParticipant
- ConPVP gumps: ArenaGump, TournamentBracketGump, AcceptTeamGump, ConfirmSignupGump
- Commands: Handlers, Logging, Add
- Other: TownCrier, SpeechLogGump, TestCenter

Key patterns:
- `sb = new StringBuilder()` reassignment → `sb.Reset()`
- `sb.AppendFormat("{0:N0}", value)` → `sb.Append($"{value:N0}")`
- `sb.Append(x).Append(y)` chains → separate statements (VSB returns void)

### Create() → stackalloc (10 files)
Converted heap-allocated builders to stackalloc where output is bounded:
- ClientVersion (32), MapSelection (160), HouseRaffleStone (48)
- HolySense (96), UnholySense (96), ClientVerification (192)
- AcceptTeamGump (64), ConfirmSignupGump (64)
- BaseWeapon (160), BaseArmor (128)

### Loop optimizations (2 files)
Hoisted `ValueStringBuilder` creation outside loops with `Reset()` per iteration:
- TourneyMatch.cs: `using var` inside for loop → stackalloc before loop
- ArenaGump.cs: `Create()` + `Dispose()` per iteration → stackalloc before loop

### Append chain → interpolation (5 files)
Converted multi-line `Append()` chains to `Append($"...")`:
- BountyMessage.cs: title switch (6 cases), paragraph (15→1 Append), description lines, closing
- AcceptTeamGump, ConfirmSignupGump, TournamentBracketGump: tournament type strings
- AdminGump: comment/tag formatting in loops

### Documentation
- `dev-docs/string-handling.md`: Full reference — construction, interpolation, disposal, decision guide
- `dev-docs/claude-skills/modernuo-string-handling.md`: Claude skill with quick reference
- `CLAUDE.md`: Added rule 17 (no StringBuilder), dev-docs table entry, skills table entry
- `dev-docs/code-standards.md`: Updated memory management section

## Test Plan

- [x] `dotnet build` — 0 errors, 0 warnings
- [x] `dotnet test` — 940/940 tests pass
- [x] 28 ValueStringBuilder tests covering all reconciliation scenarios:
  - Stackalloc no-grow, stackalloc with grow (→pool transition)
  - Heap no-grow, heap with grow, heap double grow
  - Pre-existing content with and without grow
  - Sequential multiple `Append($"...")` calls
  - Mixed plain + interpolated Append
  - Empty interpolation, literal-only, format specifiers
  - Null string holes, ISpanFormattable types
  - Dispose after stackalloc→pool grow
2026-03-22 14:23:44 -07:00
Kamron Batman
ebaf104935
chore: Use var everywhere (#2294) 2025-12-27 16:47:28 -08:00
Kamron Batman
3244704ea2
fix: Fixes directory copying (#1668) 2024-02-09 23:25:20 -08:00
mdodkins
a9a2a89908
feat: Customize expansion and set maps on first boot (#1425) 2023-07-31 20:46:49 -07:00
Kamron Batman
f24c6a08dd
fix: Fixes string interpolation in value string builder (#1339) 2023-02-13 23:09:50 -08:00
Kamron Batman
3b7e3c2fb7
fix: Fixes ValueStringBuilder empty ctor issue (#1035) 2022-05-27 17:49:36 -07:00
Kamron Batman
5df7bf6a75
fix: Removes duplicative spell checks (#849) 2021-11-14 18:45:16 -08:00
Kamron Batman
c31bf20d0e
feat: Updates to .NET 6 (#843)
* Fixes an issue with moving directories across volumes
* Removes usages of WebClient
* Removes usages of Cryptographic Providers

Note: Even though .NET 6 introduces Xoshiro RNG, there is no way to control the seed. I'll do some reconciliation of Xoshiro so it functions closer to the built in one. For the most part, it has parity though.
Benchmarks show there is nothing odd about the implementations, they are within 1ns of each other.
2021-11-13 13:38:01 -08:00
nullptr-w8
7361c9158b
fix: Fixes extracting zstd and deserializing old account version (#828) 2021-10-12 00:04:40 -07:00
Kamron Batman
fe0abc4a8f
fix: Cleans up archiving and fixes wrong relative path used (#825) 2021-10-09 14:36:46 -07:00
Kamron Batman
a55e271a69
fix: Cleans up file system paths and archiving (#815)
* Makes EnsureDirectory properly work for relative and absolute paths
* Adds a `PathUtility.GetFullPath` which returns full paths for relative paths to `Core.BaseDirectory`. If the path is absolute, it will return as-is.
* Moves EnsureDirectory to `PathUtility`. So `ScriptsHandler.EnsureDirectory` and `AssemblyHandler.EnsureDirectory` are now `PathUtility.EnsureDirectory`
* Fixes crash guard so that it copies accounts properly.
* Changes world save and auto archive to use a random folder name inside of the temp folder.
2021-10-05 08:54:44 -07:00
Kamron Batman
a845f8a1c0
feat: Networking v3 using epoll/kqueue (#813)
* Replaces networking internals with epoll/kqueue
2021-10-03 18:04:53 -07:00
Kamron Batman
8eaa859332
fix: Fixes code genning armor, clothing, and BODs (#810)
* Fixes code genning clothing
* Fixes code genning armor
* Fixes code genning BODs
2021-09-28 22:09:34 -07:00
Kamron Batman
fa5eafdaff
fix: Adds Created, LastSerialized, and BeforeSerialize for all entities. (#775)
* Adds `BeforeSerialized` for entities to handle cleanup.
* Adds `Created` and `LastSerialized` fields to all entities. While this is a big bloat, this will be necessary for identifying dangling references to other invalid entities.
* Changes formula for determining a valid reference to be _not null, not deleted, and reference's created date must be at or before the entities last serialized date_.
* Adds versioning to idx file and serializes `Created` and `LastSerialized`.
* Fixes Save Stats and also disables it by default.
2021-09-26 01:09:45 -07:00
Kamron Batman
3f6a87483b
feat: Adds auto archiving (#794)
## Adds Auto Archiving
Backups are archived once an hour, day, and month. Archives older than 60 days are pruned automatically.

_Note: Automatic pruning is off by default_

### Archive compression format
The following formats are supported:
* Zstd (The fastest with best compression ratio)
* GZip
* Zip
* None (Tar)

_Note: By default archives use [zstandard](http://facebook.github.io/zstd/) format.
The archive format can be changed in modernuo.json `autoArchive.compressionFormat`_

### Restoring world from archive
Move the archive to the Saves folder (tar.zst file). On startup the server will extract the file and restore the latest save. See pictures below.

### Manually extracting an archives
#### Windows
* Use the latest version of [7-zip w/ ZStandard](https://github.com/mcmilk/7-Zip-zstd/releases/latest)
  1. Extract the `.tar.zst` file.
  2. Extract the `.tar` file. (Yes you have to do it in two steps)
* On Windows 10 you can use the command line. `zstd.exe` is in the Assemblies folder after building ModernUO.
  1. `tar --use-compress-program "Distribution\Assemblies\zstd.exe -d" -xvf "Archives\Hourly\archivefile.tar.zst" -C "path to where you want to extract it"`
#### Mac
* Install [Keka](https://www.keka.io)
  1. Drop the .tar.zst onto the keka interface.
#### Linux
  1. Install zstd from a package manager
  2. Run `tar -I zstd -xvf "Archives\Hourly\archivefile.tar.zst" -C "path to where you want to extract it"`

### Other changes
* Changes Autosave to occur at the same time no matter when the server is booted.
* Adds `[SaveFrequency <delay> [warning]`command to set save frequency and warning frequency in-game.
* Adds support for time zones that are configurable. The system timezone can also be manually configured. Check `TimeZoneHandler.cs` for details.

<img width="257" alt="Screen Shot 2021-09-22 at 11 23 18 PM" src="https://user-images.githubusercontent.com/3953314/134464929-a5bf3cd8-2ef0-4476-a9ad-71d0816a19ac.png">
<img width="241" alt="Screen Shot 2021-09-22 at 11 23 39 PM" src="https://user-images.githubusercontent.com/3953314/134464945-5f6f96dc-3d1e-434d-8256-5c6b3705e786.png">
<img width="836" alt="Screen Shot 2021-09-22 at 11 34 39 PM" src="https://user-images.githubusercontent.com/3953314/134464957-625e57f2-ef47-4ca1-a0a7-cd40c9b6539c.png">
<img width="631" alt="Screen Shot 2021-09-22 at 11 34 50 PM" src="https://user-images.githubusercontent.com/3953314/134464969-b2d65c0d-f8f5-497a-a832-5d805e8e0b22.png">
2021-09-25 17:22:05 -07:00