From 8509b1f8a015a47b40ed31b0f5ece94194930e91 Mon Sep 17 00:00:00 2001 From: Kamron Batman <3953314+kamronbatman@users.noreply.github.com> Date: Sun, 19 Jun 2022 19:55:55 -0700 Subject: [PATCH] chore: Updates whats changed (#1082) --- FAQ.md | 21 ++++---- GOALS.md | 63 ------------------------ README.md | 4 +- RUNUO_MIGRATIONS.md | 73 ---------------------------- RUNUO_TO_MODERNUO.md | 113 +++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 127 insertions(+), 147 deletions(-) delete mode 100644 GOALS.md delete mode 100644 RUNUO_MIGRATIONS.md create mode 100644 RUNUO_TO_MODERNUO.md diff --git a/FAQ.md b/FAQ.md index 4b7d5c3e4..aa3f178c5 100644 --- a/FAQ.md +++ b/FAQ.md @@ -1,21 +1,24 @@ # Frequently Asked Questions -- [Why do I have thousands of compile errors](#why-do-i-have-thousands-of-compile-errors) -- [Where do I go to run ModernUO?](#where-do-i-go-to-run-modernuo) -- [I don't see my scripts compiling, what happened?](#i-dont-see-my-scripts-compiling-what-happened) -- [Why can't I run ModernUO on a 32-bit operating system?](#why-cant-i-run-modernuo-on-a-32-bit-operating-system) + +## What has changed since RunUO 2.7? +We have compiled a non-exhaustive list of the features changes, technical improvements, and new additions. +- See: [RunUO to ModernUO](./RUNUO_TO_MODERNUO.md) ## Why do I have thousands of compile errors? -ModernUO uses a new technique called source generation. Most likely the compile errors are related to that. -To resolve the compile errors make sure you are using Visual Studio 2019 v16.10+, or Rider 2021.2+ for development and the newest .NET available. +ModernUO uses a _very_ new technique called source generation. Most likely the compile errors are related to the source generator crashing. +To resolve the compile errors make sure you are using Visual Studio 2022, or Rider 2022+ for development and the newest .NET available. Specifically for Visual Studio, you must build the entire solution once, restart Visual Studio entirely, and then build again. -This is required every time code changes in the `SerializationGenerator` or `SerializationSchemaGenerator` projects. + +It is possible that while you are developing, many of your files will go red and properties/symbols cannot be resolved. Contact us on discord +and let us know how to reproduce it so we can get it fixed! Under the same vein, we are working on building an analyzer for the IDE so if there +are errors, you are not in the dark. ## Where do I go to run ModernUO? Everything is run from the `Distribution` directory. This folder is portable, so it can be moved to the production server for deployments. ## I don't see my scripts compiling, what happened? -.NET 5 does not support dynamic compiling newer versions of C#. Scripts are now compiled separately and moved to the `Distribution/Assemblies` directory. +Scripts are now compiled separately (UOContent project) and moved to the `Distribution/Assemblies` directory. ## Why can't I run ModernUO on a 32-bit operating system? -ModernUO is optimized for performance and modern hardware. For this reason we no longer support 32-bit operating systems +ModernUO is optimized for modern hardware. We cannot support 32-bit systems, sorry. diff --git a/GOALS.md b/GOALS.md deleted file mode 100644 index 1995b9dc1..000000000 --- a/GOALS.md +++ /dev/null @@ -1,63 +0,0 @@ -# GOALS - -The goal of ModernUO is a complete rewrite of RunUO that focuses on performance. -Some of the many high level goals include: - -### Architecture -- [x] Upgrade Server/Scripts to target .NET Core 3 -- [X] Upgrade the code to use C# 8 standards -- [X] Remove compilation of scripts at startup in favor of loading DLLs. -- [X] Support configuration via a `modernuo.json` file - -### Networking -- [ ] Replace Packet classes with functions -- [X] Improve asynchronous socket handling using Pipes -- [X] Improve socket handling (2-5x) and event loop using libuv - -### Administration -- [ ] Move IP logging and account data to SQL -- [ ] Move account management to web - -### Serialization & World Saves -- [ ] Create a flag to indicate when objects will be serialized. - * Players, all items held by players, houses, etc. -- [ ] Create transactions which handle changes to objects including: - * Transitioning from not serialized to serialized (created) - * Transitioning from serialized to not serialized (deleted) - * Property value changes -- [ ] Create asynchronous request from cold storage - * External system must provide objects and their nested objects (e.g. containers) - -### Optimizations for large shards -- [ ] Eject inactive players and their items after they are serialized to cold storage -- [ ] Create hydration techniques for non-persisted systems (e.g. spawners, decorations, static, champions, etc) -- [ ] Simplify certain items by eliminating all of their individual types and using an enum to reference their individual type - * Items subject to change: reagents, logs, ore, ingots, potions, and containers - * Syntax change: `new MandrakeRoot()` -> `new Reagent(ReagentType.MandrakeRoot)` -- [ ] Create object pools for high availability items such as gold and reagents - * For example, `new MandrakeRoot()` -> `ObjectPool.Get(ReagentType.MandrakeRoot)`. - * Pools should be elastic and adjust according to nominal usage. For example, if thousands of gold objects are created and destroyed in a small period of time, the pool should be expanded and replenished properly so it is never empty, or full. -- [ ] Replace timer system with libuv implementation -- [X] Create `DefaultName` for mobiles - -### Plugins (Separate Repos & Optional) -- [ ] RunUO 2.0 Migration Utility - * Template code for updating serializations - * Convert World file and dump to new serialization technique if there is one -- [ ] Login/Auth Gateway (Like OSI!) - * Use central AuthID via redis or similar - * Support 2FA Auths via at login gump -- [ ] 2FA Auth at Login to prevent password theft - * Support Authy via universal interface - * Support external app/device, magic number email, SMS, and FCM - * Support user identity via web browser to curb flooding -- [ ] Serialization via Database (no world saves) - * Asynchronously offload the transaction to another service for permanent serialization - * Support basic KVP databases such as MongoDB, RocksDB, and Redis, etc. - -### Bugs & Misc -- [X] Rounding errors with new bank system -- [X] Replace specific checks for `Item`/`Mobile` by extended `IEntity` and mainstreaming the `NEWPARENT` code -- [ ] TryDropItem dropping items until it fails when it should be all-or-nothing -- [ ] Consumable items are deleted directly in several places instead of using `Item.Consume()` -- [ ] Stack/Hold checks are broken in certain situations diff --git a/README.md b/README.md index c19bfc451..558a44ef1 100644 --- a/README.md +++ b/README.md @@ -69,7 +69,7 @@ Rider 2021.3+           & ## Thanks - RunUO Team & Community -- ServUO Team & Community +- [Voxpire](https://github.com/Voxpire), the ServUO Team & Community - [Karasho](https://github.com/andreakarasho), [Jaedan](https://github.com/jaedan) and the ClassicUO Community ## Troubleshooting / FAQ @@ -79,6 +79,6 @@ Rider 2021.3+           & Thank you for supporting us! You can find out how by visiting the [sponsors](./SPONSORS.md) page.

-

Development Tools & Plugins provided with ♥ by
JetBrains +

Development Tools & Plugins provided with ♥ by
JetBrains Material Theme

diff --git a/RUNUO_MIGRATIONS.md b/RUNUO_MIGRATIONS.md deleted file mode 100644 index 4c7b890d8..000000000 --- a/RUNUO_MIGRATIONS.md +++ /dev/null @@ -1,73 +0,0 @@ -### Migrating from RunUO 2.0 -There are several considerations before migration from RunUO 2.0 to ModernUO. -Many of these issues can be fixed by writing scripts that do world file fixes. -Others will require careful code review to fix compile/runtime issues. - -We will introduce templates to create migration scripts in a future iteration. - -#### Deserialization Issues (Startup Issues) -* All BODs will fail to deserialize due to the introduction of BaseBOD - * Recommend a migration script to fix this. -* Serial is now a `uint` which could break certain scripts expecting negative numbers or fixed encoding. - -#### Constructor Inheritance (Runtime Errors, Corruption, or Crashes) -* Items that inherit `Food` will pass the wrong arguments to their parent constructor. -* Items that inherit spells or scrolls will pass the wrong arguments to their parent constructors. - -#### Code Changes (Possible Compile Errors) -* The following functions no longer take a Type as their argument and now utilize generics: - * CanBeginAction, BeginAction and EndAction - * FindRegion and IsPartOf - * FindGump, HasGump, and CloseGump -* Timers use lambda functions instead of callback objects with generic `object state` argument. -* Various functions such as `OnAdded` which previously took `object parent` now take `IEntity` -* Functions that utilized non-generic structures (ArrayList, etc) have been changed to use generics. - -### Profiling -* Disabled - -#### Reports -* Removed - -#### RemoteAdmin -* Removed - -#### MyRunUO -* Removed - -#### EventLog -* Removed - -### Future Migration Concerns - -#### Account System -* Accounts from the XML format will need to be migrated the new interface/format. -* IP Logs from the Accounts XML will need to be offloaded to a new database. - -#### Deserialization -* All resources including but not limited to the following will need to be migrated: - * Gold, Silver, Platinum - * Arrow, Bolt, Feature, Shaft - * BaseOre, BaseIngot, BaseScale and all children - * BigFish, Fish, MagicFish - * Granite, Sand - * BaseReagent and all children - * BoltOfCloth, Bone, Cloth, Cotton, Flax, BaseHides, BaseLeather, UncutCloth, Wool, YarnAndThreads and all children - * ML Resources (Check Scripts/Items/Resources/MiscMLResources.cs on RunUO 2.0) - * Log and Board - * Bottle - * Solen Items (e.g. ZoogiFungus) - * Other commodity types I can't remember -* Spawners will need to be migrated since the old Spawner has been replaced and deserialization is not guaranteed. - -#### Timers -* Scripts that rely on Timers, TimerPriority, etc will need to be fixed to work with the format. - -#### Fastwalk -* Other fastwalk/speedhack detection systems will need to be deleted or fixed. - -#### Packets -* Packet, ByteQueue, PacketWriter, and PacketReader have been removed. -* `ns.Send(packet)` has been changed to `Packets.Send(ns, PacketFunc, arg1, arg2...)` -* Packets that are not used in the core have been moved to content distribution -* God client packets have been removed diff --git a/RUNUO_TO_MODERNUO.md b/RUNUO_TO_MODERNUO.md new file mode 100644 index 000000000..b4e3a3d0f --- /dev/null +++ b/RUNUO_TO_MODERNUO.md @@ -0,0 +1,113 @@ +# From RunUO to ModernUO +RunUO was built using C# from .NET 1.1 in 2002. There have been massive changes to technology in the past 20+ years. +We believe it is time for Ultima Online to take advantage of this technology so a server can provide a richer experience with never before seen scale. + +While it is possible (and many have done it) to migrate an _active server_ from RunUO to ModernUO, it is a daunting task. +Please ask for help in our discord! + +## Technology +| | RunUO | ModernUO | +|:-------------|:------------------------------------------------------------------------------------------------------|:------------------------------------------------------------| +| Language | C# 4 | C# 11 (.NET 6/7) | +| Supported OS | 32 & 64bit Windows or Mono | 64bit Windows, MacOS & Linux | +| IDEs | [VS](https://visualstudio.microsoft.com/downloads/), [VSCode](https://code.visualstudio.com/download) | VS 2022+ or [Rider 2022+](https://www.jetbrains.com/rider/) | + +## Code API Changes +* **ModernUO can source generator [serialization/deserialization](https://github.com/modernuo/SerializationGenerator#basic-usage) automatically.** + * This is the biggest change to the API! While intimidating and different, it unlocks the ability for ModernUO to only serialize _data that has changed_. + * We estimate a world save with 10mill objects on a busy server will take less than 1 second. + * This feature is _optional and will remain optional indefinitely_. +* `Serialize(GenericWriter writer)` and equiv deserialize was changed to `Serialize(IGenericWriter writer)`. +* The following now use generics, e.g. `BeginAction(typeof(X))` is now `BeginAction`. + * CanBeginAction, BeginAction and EndAction + * FindRegion and IsPartOf + * FindGump, HasGump, and CloseGump +* Functions such as `OnAdded(object)` for both Items/Mobiles are now `OnAdded(IEntity)`. +* Most `delegate` have been changed to `Action`. + * Example: `EventSink.PlayerDeath += new PlayerDeathEventHandler(EventSink_PlayerDeath);` is now `EventSink.PlayerDeath += EventSink_PlayerDeath;`. +* `ObjectPropertyList` is now `IPropertyList`, + * Example: `GetProperties(ObjectPropertyList list)` is now `GetProperties(IPropertyList list)`. +* `[Constructable]` attribute is now `[Constructible]`. + +### Object Property List API Changes +ObjectPropertyList has been drastically optimized, which means the API has been modernized: + +❌ _Not valid_ +```cs +list.Add(1061837, "{0}\t{1}", m_CurArcaneCharges, m_MaxArcaneCharges); +``` +✅ +```cs +list.Add(1061837, $"{_curArcaneCharges}\t{_maxArcaneCharges}"); +``` + +Clilocs that use arguments: + +❌ _Not valid_ +```cs +list.Add(1060659, "Level\t{1}", m_Level); +``` +✅ - _Note the string as an argument, this is mandatory!_ +```cs +list.Add(1060659, $"{"Level"}\t{Level}"); // ~1_val~: ~2_val~ +``` + +Clilocs that use other clilocs as an argument: + +❌ _Do not prepend #_ +```cs +list.Add(1060830, $"#{dirt.ToString()}"); +``` +✅ _Use the new custom cliloc argument formatter_ +```cs +list.Add(1060830, $"{dirt:#}"); +``` + +## Core Changes +* ModernUO is not inherently thread safe. Overall infrastructure improved with CPU and minimizing memory garbage in mind. +* Timer system improved drastically by using [Timer Wheels](http://www.cs.columbia.edu/~nahum/w6998/papers/sosp87-timing-wheels.pdf). +* Networking is 5-10x faster by using fixed [Circular Buffers](https://en.wikipedia.org/wiki/Circular_buffer). +* Network packets are no longer objects and write directly to the network buffer, improving performance by 10x. +* World saves are 10x faster by saving to memory and flushing to disk in the background. +* Improved RNG accuracy and performance by 5x using [Xoshiro256++](https://prng.di.unimi.it/) +* Converted quite a bit of configuration to JSON with a central settings file. +* Logging changed from Console.WriteLine to Serilog (still work in progress). +* Eliminated calling `DateTime.Now` which had a huge performance penalty. +* Accounts are now saved to a binary file (will eventually change to a databse) to improve world save performance by eliminating XML. +* Strings are now built using the new interpolated string syntax, and highly performant string builders. + +## New Features +* IPv6 support. +* Generic serialization that is automatically done in parallel with the world save. + * The [faction system](https://github.com/modernuo/ModernUO/blob/7adf52ef48df7ae2b034c27e67b0c332b37fb053/Projects/UOContent/Engines/Factions/Core/FactionSystem.cs#L15) is no longer powered by a single item and instead uses generic persistence. +* Timezone support for custom scripts that might need it. +* Hourly/daily/weekly/monthly backups & archiving using [Z-Standard](https://facebook.github.io/zstd). +* Client version detection (including CUO) for easy configuration. +* Localization (Cliloc) support. +* Better encryption for passwords using [Argon2](https://en.wikipedia.org/wiki/Argon2). +* Packet throttling that is configurable per packet and per connection. +* Enable packet logging per connection. +* Owner accounts can be protected from being locked out. The first account created is automatically added to this list. + +## Major Feature Changes +* By default, world saves are now every 5th minute of a real world hour. + * E.g. 5:00, 5:05, 5:10, regardless of when the server is booted. + +## Removed Features +* Reporting +* Remote Admin +* My RunUO +* Event Log + +## New Development Features & Changes +* Added a shared list/queue for temporary processing such as accumulating players to damage/kill. + * [PooledRefList](https://github.com/modernuo/ModernUO/blob/main/Projects/Server/Collections/PooledRefList.cs) + * [PooledRefQueue](https://github.com/modernuo/ModernUO/blob/main/Projects/Server/Collections/PooledRefQueue.cs) +* Adds HashSet that is ordered by insertion time + * [OrderedHashSet](https://github.com/modernuo/ModernUO/blob/main/Projects/Server/Collections/OrderedHashSet.cs) + * [PooledOrderedHashSet](https://github.com/modernuo/ModernUO/blob/main/Projects/Server/Collections/PooledOrderedHashSet.cs) +* Adds [StringBuilder](https://github.com/modernuo/ModernUO/blob/main/Projects/Server/Buffers/ValueStringBuilder.cs) that is fast and does not impose garbage collection. +* Adds [BitArray](https://github.com/modernuo/ModernUO/blob/main/Projects/Server/Collections/BitArray.cs) that can be serialized/deserialized directly to a buffer. +* Adds performant [JSON](https://github.com/modernuo/ModernUO/blob/main/Projects/Server/Json/JsonConfig.cs) support with simple API. +* Adds performant, thread _unsafe_, [ArrayPool](https://github.com/modernuo/ModernUO/blob/main/Projects/Server/Buffers/STArrayPool.cs) +* Adds highly performant and easy to use converters for [HexString](https://github.com/modernuo/ModernUO/blob/main/Projects/Server/Text/HexStringConverter.cs) representation of data