Highly Performant & Scalable Ultima Online Server Emulator https://www.modernuo.com
Find a file
Kamron Batman 0404251638
feat: Adds Latin1 text support (#2317)
## Summary

- Adds proper Latin1 encoding support, replacing CP1252 usage throughout the codebase
- Adds specialized, optimized string decoding methods with safe string filtering for each encoding type
- Filters invalid Unicode characters (C0/C1 control codes, non-characters) by removal rather than replacement since
the UO client renders nothing for these characters
- Fixes UTF-16 null terminator position handling to correctly advance by 2 bytes

## Changes

TextEncoding.cs

- Added SearchValues-based invalid byte/char detection for efficient filtering
- Added encoding-specific GetString methods: GetStringAscii, GetStringLatin1, GetStringUtf8, GetStringBigUni,
GetStringLittleUni
- Each method supports a safeString parameter for filtering invalid characters
- Little-endian UTF-16 uses direct memory cast for zero-copy decoding on LE systems
- Invalid characters are removed (not replaced with U+FFFD) since the client renders nothing for them

SpanReader.cs

- Added ReadLatin1() and ReadLatin1Safe() methods
- Rewrote encoding-specific read methods to use optimized TextEncoding.GetString* methods
- Fixed UTF-16 null terminator handling: position now correctly advances by byteLength (2) instead of 1

SpanWriter.cs

- Added WriteLatin1 and WriteLatin1Null methods

## Packet Updates

- Updated all packet code to use Latin1 encoding instead of CP1252
- Affected: account packets, equipment packets, menu packets, message packets, mobile packets, player packets, secure
trade packets, vendor packets, gump packets, book packets, mahjong packets

## Filtering Behavior

Invalid characters filtered in safe mode:
```
┌───────────────┬────────────────────────┐
│     Range     │      Description       │
├───────────────┼────────────────────────┤
│ 0x00-0x1F     │ C0 control codes       │
├───────────────┼────────────────────────┤
│ 0x7F          │ DEL                    │
├───────────────┼────────────────────────┤
│ 0x80-0x9F     │ C1 control codes       │
├───────────────┼────────────────────────┤
│ 0xFFFE-0xFFFF │ Unicode non-characters │
└───────────────┴────────────────────────┘
```

Note: Surrogate pairs (0xD800-0xDFFF) are not filtered because proper validation requires context checking for paired
vs unpaired surrogates. The UO client renders nothing for these anyway.

## Test Plan

- All 631 Server.Tests pass
- Verified client rendering behavior using TestUnicodeGump command (pages 1-5)
- Confirmed U+FFFD, unpaired surrogates, and non-characters all render as blank in client
- Verified Latin1 characters (0xA0-0xFF) display correctly
- Verified C1 control codes (0x80-0x9F) are filtered and don't display
2026-01-22 15:51:00 -08:00
.config feat: Bumps Serialization Generator for better support (#2292) 2025-12-26 14:27:24 -08:00
.github feat: Adds Network Packet Documentation (#2302) 2026-01-01 17:52:56 -08:00
Distribution/Data feat: Adds Network Packet Documentation (#2302) 2026-01-01 17:52:56 -08:00
docs fix: Fixes Packet Documentation scrolling on mobile. (#2305) 2026-01-02 22:17:01 -08:00
Projects feat: Adds Latin1 text support (#2317) 2026-01-22 15:51:00 -08:00
.editorconfig chore(docs): Adds UnmanagedDataReader & BinaryFileReader documentation (#1968) 2024-10-01 17:07:51 -07:00
.gitattributes fix: Adds migration checks to CICD, updates dependencies, adds Fedora 37, alpine 3.17 support (#1309) 2022-12-22 12:12:47 -08:00
.gitignore feat: Moves logger to a separate assembly for reuse (#2001) 2024-12-30 20:25:34 -08:00
azure-pipelines.yml feat: Adds .NET 10 / C# 14 support. (#2258) 2025-11-11 11:26:34 -08:00
CODE_OF_CONDUCT.md Create CODE_OF_CONDUCT.md 2020-05-05 08:51:39 -07:00
CONTRIBUTING.md chore(license): Removes CLA requirement. All contributors moving forward will retain copyright. (#2038) 2024-12-31 02:52:58 -08:00
Directory.Build.props feat: Adds .NET 10 / C# 14 support. (#2258) 2025-11-11 11:26:34 -08:00
FAQ.md docs: Updates FAQ for time zone issues. (#1368) 2023-03-10 00:18:36 -08:00
global.json feat: Adds .NET 10 / C# 14 support. (#2258) 2025-11-11 11:26:34 -08:00
LICENSE Fixes more line ending issues (#267) 2020-09-27 13:54:56 -07:00
mkdocs.yml chore: Fixes github pages (#2303) 2026-01-01 19:35:35 -08:00
ModernUO.sln fix: Updates Visual Studio version in solution file to Visual Studio 2026 (#2285) 2025-12-01 14:46:36 -08:00
publish.cmd fix: Fixes publishing (#1808) 2024-05-31 12:59:59 -07:00
publish.sh fix: Fixes publishing (#1808) 2024-05-31 12:59:59 -07:00
README.md chore: Update README with logo link and image sources 2025-11-25 20:05:05 -08:00
rider-settings.zip Cleanup/Housekeeping (#242) 2020-09-12 15:31:21 -07:00
Rules.ruleset chore(docs): Adds UnmanagedDataReader & BinaryFileReader documentation (#1968) 2024-10-01 17:07:51 -07:00
SPONSORS.md chore: Updates readme files (#2247) 2025-07-27 11:04:32 -07:00
stylecop.json Updates formatting rules (#199) 2020-08-25 18:00:51 -07:00
THIRD-PARTY-NOTICES chore: Updates 3rd party notices (#1042) 2022-06-01 09:42:11 -07:00
version.json feat: Add CanSpawnMobile overload with props Z-range support. (#2293) 2025-12-27 17:01:15 -08:00

ModernUO - Ultima Online Server Emulator for the modern era!

ModernUO Discord Subreddit subscribers Twitter Follow

Ultima Online Server Emulator for the modern era!

GitHub license GitHub stars GitHub issues
GitHub build Azure Pipelines build

Requirements

Supported Operating Systems

Windows 10/11/2012/2016/2019/2022/2025 MacOS 14+ Debian 12+ Ubuntu 22+ LTS
Alpine 3.22+ Fedora 42+ RedHat 9+ CentOS Stream 9+ openSUSE 15.6+ SUSE Enterprise 15 SP6 Linux Mint 21+ Arch

Required Frameworks

All Operating Systems

.NET

Windows

VC++ Redistributable v14

Development

git .NET

Supported IDEs

Jetbrains Rider 2025.3+ space VSCode space Visual Studio 2026

Getting Started

  • Install prerequisite requirements
  • Clone this repository (or download the latest):
    • git clone https://github.com/modernuo/ModernUO.git
  • Open ModernUO.sln to start developing

Building/Publishing

  • Run ./publish.cmd [release|debug (default: release)] [os] [arch (default: x64)]

Linux Prerequisites

Fedora, CentOS, RHEL, etc

dnf upgrade --refresh -y
# CentOS does not come with EPEL enabled
dnf install -y epel-release epel-next-release
dnf install -y findutils libicu libdeflate-devel zstd libargon2-devel

Ubuntu, Debian, etc

apt-get update -y
apt-get install -y libicu-dev libdeflate-dev zstd libargon2-dev

OSX Requirements

brew install icu4c libdeflate zstd argon2

Running the Server

  • Follow the publish instructions
  • Run ModernUO.exe or dotnet ModernUO.dll from the Distribution directory

Troubleshooting / FAQ

Want to sponsor?

Thank you for supporting us! You can find out how by visiting the sponsors page.

Collaborators

Kamron Batman Mark1145

Thanks

  • RunUO Team & Community
  • Voxpire, the ServUO Team & Community
  • Karasho, Jaedan and the ClassicUO Community



Development Tools & Plugins provided with ♥ by

JetBrains
Material Theme