Commit graph

5 commits

Author SHA1 Message Date
Kamron Batman
3b4e1137f6
feat: Implements new robust/pluggable backup/archive system. (#2388)
## Summary

Overhauls the backup/archive system to address stability, robustness, fault tolerance, performance, and pluggability.

### Problems solved
- **Critical bug**: `Interlocked.CompareExchange` arguments were reversed — the concurrent archive guard never actually prevented concurrent archives
- **Brittle compression**: Relied on spawning external `zstd.exe`/`tar.exe`/`bsdtar.exe` processes with platform-specific workarounds (Windows two-step hack, auto-downloading bsdtar from libarchive.org)
- **No fault tolerance**: Partial archives possible if process killed mid-write; no recovery mechanism; originals deleted before verification
- **No extensibility**: No way to add remote backup destinations (S3, rsync) without modifying core code
- **Silent failures**: Bare `catch` blocks swallowed exceptions with no diagnostics

### What changed

**Cross-platform streaming compression** — Replaced external process spawning with `System.Formats.Tar` (built-in .NET) + `ZstdNet` (native libzstd P/Invoke). Single-pass streaming: `TarWriter → CompressionStream → FileStream`. No intermediate `.tar` file, no platform-specific code paths.

**Archive journal** — JSON-based operation journal (`Archives/.archive-journal.json`) tracks state machine: `Started → Archived → Distributed → Completed` (or `Failed`). On startup, recovers interrupted operations (cleans temp files, completes pruning).

**Verify before delete** — Archives are verified (entry count check) before deleting source backups. Temp-file-then-atomic-rename pattern prevents partial archives.

**Retry logic** — File operations (moves, deletes) retry with linear backoff for transient I/O failures (antivirus locks, file copy operations).

**Plugin architecture** — `IArchiveDestination` interface in `Projects/Server/` enables external plugins (loaded via `assemblies.json`) to receive completed archives. `ArchiveDestinationRegistry` tracks destinations. Conservative pruning: source backups preserved if any destination with retention fails.

**Configurable** — Retention counts (hourly/daily/monthly), compression level, retry settings, backup max age all configurable via `ServerConfiguration`.

### New admin commands
- `[ArchiveStatus` — Shows journal state, destinations, next scheduled archive times
- `[ArchiveNow` — Forces immediate rollup regardless of schedule

### Files

| Change | File |
|--------|------|
| New | `Server/Saves/ArchiveEnums.cs`, `IArchiveDestination.cs`, `ArchiveDestinationRegistry.cs`, `ArchiveEventArgs.cs` |
| New | `Server/Events/ArchiveEvents.cs` (partial EventSink) |
| New | `UOContent/Compression/ManagedArchive.cs` |
| New | `UOContent/World Saves/ArchiveJournal.cs`, `LocalArchiveDestination.cs` |
| Rewrite | `UOContent/World Saves/AutoArchive.cs` |
| Modified | `UOContent/World Saves/SaveCommands.cs`, `Server/Utilities/PathUtility.cs`, `UOContent.csproj` |
| Deleted | `UOContent/Compression/TarArchive.cs`, `ZstdArchive.cs` |
| Tests | 29 new tests (ManagedArchive, ArchiveJournal, AutoArchiveHelpers) |

### Backward compatibility
- Existing `.tar.zst` archives are fully readable by the new managed code
- Config keys preserved; new keys use sensible defaults
- `autoArchive.compressionFormat` setting removed (Zstd only)
- Legacy `bsdtar/` directory logged as removable on startup

### Plugin example (external repo)
```csharp
public static class S3Plugin
{
    public static void Configure()
    {
        ArchiveDestinationRegistry.Register(new S3Destination("my-bucket", "us-east-1"));
    }
}
```

## Test plan
- [x] 29 new unit tests covering archive create/extract/count, journal state machine, recovery, destinations
- [x] All 969 tests pass (671 Server + 298 UOContent)
- [x] Manual: run server, trigger save, verify backup created and archive rolled up
- [x] Manual: kill server mid-archive, restart, verify journal recovery
- [x] Manual: test with large save files (~500MB+) to benchmark streaming vs old approach
2026-03-22 19:51:20 -07:00
Kamron Batman
f4a87a8629
fix: Fixes adding items with ambiguous type lookup (#2307)
Fix [add command failing with ambiguous type names + refactor for performance

### Problem

[add blight would fail with "No type with that name was found" because multiple types contain "blight" (e.g., Server.Items.Blight, Server.Ethics.Evil.Blight, Server.Items.BlightGrippedLongbow, Server.Items.QuiverOfBlight). The old code only succeeded when exactly one type matched the search regardless of constructability and inheriting Mobi les/Items.

### Solution

Exact match takes priority: If a type's name exactly equals the search string (case-insensitive), use it directly. Otherwise, show the AddGump with all partial matches.

- [add blight → Creates Blight (exact name match)
- [add bligh → Shows gump with Blight, BlightGrippedLongbow, etc.

### Refactoring

- CommandEventArgs context: Added GetContext<T>/SetContext<T> to pass resolved type through the command chain without method signature changes
- Removed TrySetupTarget duplication: Validation now happens only in ValidateArgs, eliminating redundant code paths
- Split type matching:
  - ExactMatch(string) → Returns Type for exact name match (used by [add)
  - MatchEmptyCtor(string) → Returns ConstructorInfo[] for gump display (empty-callable constructors only)

### Memory & Performance Improvements

| Optimization                  | Benefit                                                                                      |
|-------------------------------|----------------------------------------------------------------------------------------------|
| _mobileItemTypes cache        | Filters Mobile/Item types once per assembly, reused on all subsequent searches               |
| ReadOnlySpan<string> for args | Avoids string[] heap allocations when slicing arguments                                      |
| ValueStringBuilder            | Stack-allocated string building, avoids StringBuilder heap allocation                        |
| Single type resolution        | Type resolved once in ValidateArgs, passed via context to Execute (was resolved 2-3x before) |
2026-01-06 21:21:29 -08:00
Kamron Batman
fffda53263
fix: Adds command help, webpage, and fixes issues with other commands (#1669)
### Summary

- Fixes `[AdvancedSearch` being accessible by players 😱 
- Adds `[GenCommands` to generate the same commands html page on https://muo.gg/commands.
- Fixes `[helpinfo` so all commands properly show up!

> [!WARNING]  
> ### Developer Warning:
> Commands must now be registered in the `Configure` bootup phase.
> If a command is not registered early enough, it may not be available to systems like [helpinfo
> that cache their information.

> [!NOTE]  
> ### Developer Note:
> Various commands related to generating content have been changed to _Developer_ and above access level.

### Screenshots
<img width="673" alt="image" src="https://github.com/modernuo/ModernUO/assets/3953314/b105b5c9-5eb4-4ace-93ff-1bfb31e7132f">

<img width="547" alt="image" src="https://github.com/modernuo/ModernUO/assets/3953314/e97487e8-47a5-4aa7-89cc-9fe3deda584d">
2024-02-10 00:19:19 -08:00
Fabrizio
537526a328
fix: Update LabelTo, SendMessage, etc to Interpolated Strings (#1283) 2022-11-28 19:58:12 -08:00
Kamron Batman
3f6a87483b
feat: Adds auto archiving (#794)
## Adds Auto Archiving
Backups are archived once an hour, day, and month. Archives older than 60 days are pruned automatically.

_Note: Automatic pruning is off by default_

### Archive compression format
The following formats are supported:
* Zstd (The fastest with best compression ratio)
* GZip
* Zip
* None (Tar)

_Note: By default archives use [zstandard](http://facebook.github.io/zstd/) format.
The archive format can be changed in modernuo.json `autoArchive.compressionFormat`_

### Restoring world from archive
Move the archive to the Saves folder (tar.zst file). On startup the server will extract the file and restore the latest save. See pictures below.

### Manually extracting an archives
#### Windows
* Use the latest version of [7-zip w/ ZStandard](https://github.com/mcmilk/7-Zip-zstd/releases/latest)
  1. Extract the `.tar.zst` file.
  2. Extract the `.tar` file. (Yes you have to do it in two steps)
* On Windows 10 you can use the command line. `zstd.exe` is in the Assemblies folder after building ModernUO.
  1. `tar --use-compress-program "Distribution\Assemblies\zstd.exe -d" -xvf "Archives\Hourly\archivefile.tar.zst" -C "path to where you want to extract it"`
#### Mac
* Install [Keka](https://www.keka.io)
  1. Drop the .tar.zst onto the keka interface.
#### Linux
  1. Install zstd from a package manager
  2. Run `tar -I zstd -xvf "Archives\Hourly\archivefile.tar.zst" -C "path to where you want to extract it"`

### Other changes
* Changes Autosave to occur at the same time no matter when the server is booted.
* Adds `[SaveFrequency <delay> [warning]`command to set save frequency and warning frequency in-game.
* Adds support for time zones that are configurable. The system timezone can also be manually configured. Check `TimeZoneHandler.cs` for details.

<img width="257" alt="Screen Shot 2021-09-22 at 11 23 18 PM" src="https://user-images.githubusercontent.com/3953314/134464929-a5bf3cd8-2ef0-4476-a9ad-71d0816a19ac.png">
<img width="241" alt="Screen Shot 2021-09-22 at 11 23 39 PM" src="https://user-images.githubusercontent.com/3953314/134464945-5f6f96dc-3d1e-434d-8256-5c6b3705e786.png">
<img width="836" alt="Screen Shot 2021-09-22 at 11 34 39 PM" src="https://user-images.githubusercontent.com/3953314/134464957-625e57f2-ef47-4ca1-a0a7-cd40c9b6539c.png">
<img width="631" alt="Screen Shot 2021-09-22 at 11 34 50 PM" src="https://user-images.githubusercontent.com/3953314/134464969-b2d65c0d-f8f5-497a-a832-5d805e8e0b22.png">
2021-09-25 17:22:05 -07:00