ModernUO/Projects/UOContent/Commands/Generic/Extensions/Compilers/PropertyExpressions.cs
Kamron Batman 535a098996
refactor: compile where/sort/distinct and Advanced Search through expression trees (#2625)
## Summary

`where`, `sort by`, `distinct` and the Advanced Search property test now compile through expression trees instead of the hand-rolled IL in `Emitter.cs`. The emitter and its three `Reflection.Emit` compilers were RunUO-era code from before expression trees existed; they were the only way to avoid per-object reflection at the time, and they are not any more.

Net: ~1,900 lines of IL bookkeeping deleted, one comparison engine instead of two, faster per object, cheaper to compile, collectible, and `Nullable<T>` properties work.

## Why

- **Bugs hid in the IL.** Equality on a type with value semantics but no `IComparable` (`TextDefinition`) was a raw `ceq`, so `where Message = 1060847` never matched. A chained binding (`Message.Number`) dereferenced every link unguarded, so the first swept object with a null intermediate killed the sweep with an NRE. A constant narrower or unsigned than `int` threw before compiling. A struct with no `CompareTo` reached `ceq` on two unboxed values, which is invalid IL. Every dynamic assembly was `Run`, so each `[global where` grew the process for good.
- **Advanced Search had its own engine.** Per entity and per leaf it re-split the expression, scanned the runtime type's properties by name, read the value by reflection, re-parsed the right-hand side and dispatched on type through ~300 lines of `CompareValues`. Same job, second implementation, second set of bugs.

## What changed

- `ICondition.Compile(MethodEmitter)` becomes `ICondition.Build(ParameterExpression)` returning an `Expression`. `ConditionalCompiler` assembles a `Func<object, bool>`, `SortCompiler` a `Comparison<T>`, `DistinctCompiler` both comparer interfaces over one lambda. The parsed constant is an `Expression.Constant`, so the generated type, its constructor and the per-condition field for non-primitive constants disappear with `PropertyValue`.
- `PropertyExpressions` holds the shared pieces: the chain walk with its null-intermediate guard, `CompareTo` resolution with the old null ordering, the integral/enum operator path, and constant parsing.
- `BaseExtension.Optimize` loses its `ref AssemblyEmitter` parameter. An out-of-tree extension that overrides `Optimize` needs to drop it.
- Advanced Search keeps its grammar (`~` negates, `@` is AND, `|` is OR and binds looser, string `>` is "starts with") and translates each leaf into the same conditions, compiled once per declaring type per search and memoized across the workers. Float and double keep their typed-precision tolerance through a small `EpsilonCondition` in the Advanced Search folder.

## Semantics preserved

- Equality on a non-comparable reference type is `object.Equals`, never reference identity. A non-comparable struct boxes into the same call.
- A null intermediate in a chained binding is no match, and stays no match under negation. Sort and distinct read it as `default(T)`.
- Unsigned relational compares stay unsigned; integral primitives and enums use the operator directly, nothing widens to a signed type. `float`, `double`, `decimal`, `string` and structs still go through the type's own `CompareTo`, so `string` equality stays culture-sensitive exactly as before.
- Only `==` and `!=` are valid for non-comparable types; a relational operator still throws at build time.
- `TypeCondition` is still first and still null-checks the cast target.

## Behavior changes

- **`Nullable<T>` works**, with C# lifted semantics in `where`: two nulls are equal, a null and a value are unequal, a null satisfies no relation. Sort keeps a total order with unset values at one end. A null *reference* keeps the ordering it had.
- **Advanced Search**: a leaf that cannot be parsed or resolved is no match even under `~` (it used to negate the failure and match every entity); `null` is the null value for equality on strings, nullables and reference types, as in `where`; dotted names walk into a property; static properties are no longer searchable.

## Measurements

`where`, from the handoff (Debug test host, single condition, ratios not absolutes):

| Approach | ns/object | Compile | Collectible | LOC |
|---|---:|---:|---|---:|
| `AssemblyBuilder` + IL (before) | 20.5 | 0.311 ms | no (`Run`) | ~1,916 |
| Expression trees (after) | ~12 | 0.187 ms warm | yes | ~600 |

Advanced Search, Release, one `SkillTeleporter`, 2M evaluations per leaf:

| Leaf | Before | After |
|---|---:|---:|
| `Hue=5` | 137 ns | 20 ns |
| `Name~~gate` | 121 ns | 39 ns |
| `Skill=Magery` | 116 ns | 21 ns |
| `Weight>0.5` | 122 ns | 28 ns |

Plus 0.5 to 2 ms to compile each declaring type a search meets (12 ms for the first compile in the process). All of it runs on the search workers; nothing new touches the loop.

## Also fixed: Advanced Search map filters

Found while testing the property test in game. The map boxes are independent checkboxes, but the worker applied each ticked map as "must be on this map", so ticking two or more (all maps and Internal, say) rejected every entity before any other filter ran. Present since #1649; the default of Felucca alone never showed it. An entity now passes when its map is any of the ticked ones, with none ticked meaning no map constraint. Pinned by a worker test.

## Test plan

- [x] `UOContent.Tests`: 862 passed, 0 failed
- [x] `Server.Tests`: 869 passed, 0 failed
- [x] Every commit builds and its tests pass on its own (bisectable)
- [ ] In game: `[global where`, `[area where`, `[condition`, `sort by`, `distinct`, Advanced Search property test
2026-09-10 19:01:59 -07:00

407 lines
15 KiB
C#

using System;
using System.Globalization;
using System.Linq.Expressions;
using System.Reflection;
namespace Server.Commands.Generic;
/// <summary>
/// Expression-tree fragments over a bound <see cref="Property" /> chain, shared by the
/// conditional, sort and distinct compilers. Everything here builds an <see cref="Expression" />;
/// the compilers assemble those into a lambda and hand <c>Compile()</c> the codegen.
/// </summary>
public static class PropertyExpressions
{
private static readonly MethodInfo _objectEquals = typeof(object).GetMethod(
nameof(object.Equals),
BindingFlags.Public | BindingFlags.Static,
[typeof(object), typeof(object)]
)!;
/// <summary>
/// Walks a property binding. A binding of more than one property (<c>Message.Number</c>)
/// dereferences each link in turn, and any link but the last can legitimately be null -- an
/// unset TextDefinition, an unparented item. Each reference-typed intermediate link is stored
/// once and null-checked; a null there yields <paramref name="whenUnreadable" /> in place of
/// whatever <paramref name="onValue" /> would have built from the final link.
/// </summary>
public static Expression Chain(
Expression target,
Property prop,
Func<Expression, Expression> onValue,
Expression whenUnreadable
) => ChainFrom(target, prop.Chain, 0, onValue, whenUnreadable);
private static Expression ChainFrom(
Expression current,
PropertyInfo[] chain,
int index,
Func<Expression, Expression> onValue,
Expression whenUnreadable
)
{
var link = Expression.Property(current, chain[index]);
// The last link is the value being tested, so a null there is the caller's business.
if (index == chain.Length - 1)
{
return onValue(link);
}
if (link.Type.IsValueType)
{
return ChainFrom(link, chain, index + 1, onValue, whenUnreadable);
}
var local = Expression.Variable(link.Type, chain[index].Name);
return Expression.Block(
[local],
Expression.Assign(local, link),
Expression.Condition(
Expression.ReferenceNotEqual(local, Expression.Constant(null, local.Type)),
ChainFrom(local, chain, index + 1, onValue, whenUnreadable),
whenUnreadable
)
);
}
/// <summary>
/// Walks a binding for a caller that has no way to express "no match" -- ordering and
/// grouping, where the value itself is the answer rather than a yes or no. An unreadable
/// link yields <c>default(T)</c>, which is what a null link along the way amounts to; the
/// comparers these feed already handle a null value.
/// </summary>
public static Expression ChainOrDefault(Expression target, Property prop) =>
Chain(target, prop, static value => value, Expression.Default(prop.Type));
/// <summary>
/// Equality for the "not comparable" path, which supports only == and !=. Reference equality
/// would miss a type whose equality is by value -- <see cref="TextDefinition" /> among them --
/// so this is static <c>object.Equals</c>, which honors the override and is null-safe on
/// either side. Value types box; they only reach here when they have no <c>CompareTo</c>.
/// </summary>
public static Expression ValueEquals(Expression a, Expression b) =>
Expression.Call(_objectEquals, Box(a), Box(b));
private static Expression Box(Expression e) =>
e.Type == typeof(object) ? e : Expression.Convert(e, typeof(object));
/// <summary>
/// A boolean test of <paramref name="a" /> against <paramref name="b" />. Integral primitives
/// and enums compare with the operator itself -- on the unsigned types as unsigned; nothing
/// here widens to a signed type. Everything else goes through <see cref="TryCompare" /> and
/// tests the sign of the result. <c>Nullable&lt;T&gt;</c> is lifted either way, as C# lifts it:
/// two nulls are equal, a null and a value are unequal, and a null satisfies no relation.
/// (A null <em>reference</em> keeps the ordering <see cref="TryCompare" /> gives it.) False
/// when the type has no <c>CompareTo</c> at all, in which case only equality is meaningful.
/// </summary>
public static bool TryRelational(Expression a, Expression b, ComparisonOperator op, out Expression test)
{
var type = a.Type;
var underlying = Nullable.GetUnderlyingType(type);
var nonNullable = underlying ?? type;
if (underlying != null && !underlying.IsEnum && !IsIntegral(underlying))
{
return TryLiftedRelational(a, b, op, out test);
}
if (nonNullable.IsEnum)
{
// Equal/NotEqual are defined on enums; the relational operators are not, so those
// read the underlying integer. Convert lifts over Nullable<E> on its own.
if (op is not (ComparisonOperator.Equal or ComparisonOperator.NotEqual))
{
var integer = Enum.GetUnderlyingType(nonNullable);
if (underlying != null)
{
integer = typeof(Nullable<>).MakeGenericType(integer);
}
a = Expression.Convert(a, integer);
b = Expression.Convert(b, integer);
}
test = Relational(a, b, op);
return true;
}
if (IsIntegral(nonNullable))
{
test = Relational(a, b, op);
return true;
}
if (!TryCompare(a, b, 1, out var comparison))
{
test = null;
return false;
}
test = Relational(comparison, Expression.Constant(0), op);
return true;
}
// Nullable<T> over a type that compares through CompareTo: the values compare when both are
// present, and the HasValue flags decide otherwise, the way the language lifts an operator.
private static bool TryLiftedRelational(Expression a, Expression b, ComparisonOperator op, out Expression test)
{
var left = Expression.Variable(a.Type, "left");
var right = Expression.Variable(a.Type, "right");
var couldCompare = TryCompareValues(
Expression.Property(left, "Value"),
Expression.Property(right, "Value"),
1,
out var comparison
);
if (!couldCompare)
{
test = null;
return false;
}
var leftHasValue = Expression.Property(left, "HasValue");
var rightHasValue = Expression.Property(right, "HasValue");
var both = Expression.AndAlso(leftHasValue, rightHasValue);
var relation = Relational(comparison, Expression.Constant(0), op);
Expression lifted = op switch
{
ComparisonOperator.Equal => Expression.Condition(both, relation, Expression.Equal(leftHasValue, rightHasValue)),
ComparisonOperator.NotEqual => Expression.Condition(both, relation, Expression.NotEqual(leftHasValue, rightHasValue)),
_ => Expression.AndAlso(both, relation)
};
test = Expression.Block(
[left, right],
Expression.Assign(left, a),
Expression.Assign(right, b),
lifted
);
return true;
}
private static Expression Relational(Expression a, Expression b, ComparisonOperator op) =>
op switch
{
ComparisonOperator.Equal => Expression.Equal(a, b),
ComparisonOperator.NotEqual => Expression.NotEqual(a, b),
ComparisonOperator.Greater => Expression.GreaterThan(a, b),
ComparisonOperator.GreaterEqual => Expression.GreaterThanOrEqual(a, b),
ComparisonOperator.Lesser => Expression.LessThan(a, b),
ComparisonOperator.LesserEqual => Expression.LessThanOrEqual(a, b),
_ => throw new InvalidOperationException("Invalid comparison operator.")
};
private static bool IsIntegral(Type type) =>
type == typeof(int) || type == typeof(long) || type == typeof(uint) || type == typeof(ulong)
|| type == typeof(short) || type == typeof(ushort) || type == typeof(byte) || type == typeof(sbyte);
/// <summary>
/// An <c>int</c>-valued comparison of <paramref name="a" /> against <paramref name="b" />
/// with <c>CompareTo</c> semantics, multiplied by <paramref name="sign" />. A null on either
/// side of a reference or nullable type is handled here rather than in the callee:
/// <c>null.CompareTo(null) = 0</c>, <c>real.CompareTo(null) = -sign</c>,
/// <c>null.CompareTo(real) = +sign</c>. False when the type has no <c>CompareTo</c>.
/// </summary>
public static bool TryCompare(Expression a, Expression b, int sign, out Expression comparison)
{
var type = a.Type;
// Both sides are read more than once below; pin them so a chained binding is walked once.
var left = Expression.Variable(type, "left");
var right = Expression.Variable(type, "right");
if (!TryCompareValues(left, right, sign, out var body))
{
comparison = null;
return false;
}
comparison = Expression.Block(
[left, right],
Expression.Assign(left, a),
Expression.Assign(right, b),
body
);
return true;
}
private static bool TryCompareValues(Expression a, Expression b, int sign, out Expression comparison)
{
var type = a.Type;
var underlying = Nullable.GetUnderlyingType(type);
if (underlying != null)
{
if (!TryCompareValues(Expression.Property(a, "Value"), Expression.Property(b, "Value"), sign, out var inner))
{
comparison = null;
return false;
}
comparison = NullAware(Expression.Property(a, "HasValue"), Expression.Property(b, "HasValue"), inner, sign);
return true;
}
if (type.IsEnum)
{
var integer = Enum.GetUnderlyingType(type);
return TryCompareValues(Expression.Convert(a, integer), Expression.Convert(b, integer), sign, out comparison);
}
var compareTo = FindCompareTo(type);
if (compareTo == null)
{
comparison = null;
return false;
}
var parameterType = compareTo.GetParameters()[0].ParameterType;
var argument = parameterType == type ? b : Expression.Convert(b, parameterType);
Expression call = Expression.Call(a, compareTo, argument);
if (sign == -1)
{
call = Expression.Negate(call);
}
if (type.IsValueType)
{
comparison = call;
return true;
}
var nil = Expression.Constant(null, type);
comparison = NullAware(Expression.ReferenceNotEqual(a, nil), Expression.ReferenceNotEqual(b, nil), call, sign);
return true;
}
private static Expression NullAware(Expression aHasValue, Expression bHasValue, Expression compare, int sign) =>
Expression.Condition(
aHasValue,
Expression.Condition(bHasValue, compare, Expression.Constant(-sign)),
Expression.Condition(bHasValue, Expression.Constant(sign), Expression.Constant(0))
);
private static MethodInfo FindCompareTo(Type type)
{
var compareTo = type.GetMethod("CompareTo", [type]);
if (compareTo != null)
{
return compareTo;
}
/* There's a scenario where we might be trying to use CompareTo on an interface
* which, while it doesn't explicitly implement CompareTo itself, is said to
* extend IComparable indirectly. The implementation is implicitly passed off
* to implementers, so the interface's own GetMethod("CompareTo") returns null.
*/
var ifaces = type.FindInterfaces(
static (iface, _) => iface.IsGenericType && iface.GetGenericTypeDefinition() == typeof(IComparable<>),
null
);
for (var i = 0; i < ifaces.Length; ++i)
{
if (ifaces[i].GetGenericArguments()[0].IsAssignableFrom(type))
{
return ifaces[i].GetMethod("CompareTo", [type]);
}
}
return typeof(IComparable).IsAssignableFrom(type)
? typeof(IComparable).GetMethod("CompareTo", [typeof(object)])
: null;
}
/// <summary>
/// The right-hand side of a condition as a typed constant. A string is parsed the way the
/// props gump would parse it: <c>null</c> for a reference or nullable type, <c>@"null"</c>
/// for the literal string, names for enums, hex with a <c>0x</c> prefix for the numerics,
/// and the type's own static <c>Parse</c> for everything else.
/// </summary>
public static ConstantExpression Constant(Type type, object value)
{
if (value is string text)
{
value = Parse(type, text);
}
return Expression.Constant(value, type);
}
private static object Parse(Type type, string text)
{
var underlying = Nullable.GetUnderlyingType(type);
if (text == "null" && (underlying != null || !type.IsValueType))
{
return null;
}
var target = underlying ?? type;
if (target == typeof(string))
{
return text == @"@""null""" ? "null" : text;
}
if (target.IsEnum)
{
return Enum.Parse(target, text, true);
}
if (target == typeof(bool))
{
return bool.Parse(text);
}
var parseNumber = target.GetMethod(
"Parse",
BindingFlags.Public | BindingFlags.Static,
null,
Types.ParseStringNumericParamTypes,
null
);
if (parseNumber != null)
{
var style = NumberStyles.Integer;
if (text.InsensitiveStartsWith("0x"))
{
style = NumberStyles.HexNumber;
text = text[2..];
}
return parseNumber.Invoke(null, [text, style]);
}
var parseGeneral = target.GetMethod(
"Parse",
BindingFlags.Public | BindingFlags.Static,
null,
Types.ParseStringParamTypes,
null
);
if (parseGeneral != null)
{
return parseGeneral.Invoke(null, [text, null]);
}
throw new InvalidOperationException($"Unable to convert string \"{text}\" into type '{type}'.");
}
}