## Summary `where`, `sort by`, `distinct` and the Advanced Search property test now compile through expression trees instead of the hand-rolled IL in `Emitter.cs`. The emitter and its three `Reflection.Emit` compilers were RunUO-era code from before expression trees existed; they were the only way to avoid per-object reflection at the time, and they are not any more. Net: ~1,900 lines of IL bookkeeping deleted, one comparison engine instead of two, faster per object, cheaper to compile, collectible, and `Nullable<T>` properties work. ## Why - **Bugs hid in the IL.** Equality on a type with value semantics but no `IComparable` (`TextDefinition`) was a raw `ceq`, so `where Message = 1060847` never matched. A chained binding (`Message.Number`) dereferenced every link unguarded, so the first swept object with a null intermediate killed the sweep with an NRE. A constant narrower or unsigned than `int` threw before compiling. A struct with no `CompareTo` reached `ceq` on two unboxed values, which is invalid IL. Every dynamic assembly was `Run`, so each `[global where` grew the process for good. - **Advanced Search had its own engine.** Per entity and per leaf it re-split the expression, scanned the runtime type's properties by name, read the value by reflection, re-parsed the right-hand side and dispatched on type through ~300 lines of `CompareValues`. Same job, second implementation, second set of bugs. ## What changed - `ICondition.Compile(MethodEmitter)` becomes `ICondition.Build(ParameterExpression)` returning an `Expression`. `ConditionalCompiler` assembles a `Func<object, bool>`, `SortCompiler` a `Comparison<T>`, `DistinctCompiler` both comparer interfaces over one lambda. The parsed constant is an `Expression.Constant`, so the generated type, its constructor and the per-condition field for non-primitive constants disappear with `PropertyValue`. - `PropertyExpressions` holds the shared pieces: the chain walk with its null-intermediate guard, `CompareTo` resolution with the old null ordering, the integral/enum operator path, and constant parsing. - `BaseExtension.Optimize` loses its `ref AssemblyEmitter` parameter. An out-of-tree extension that overrides `Optimize` needs to drop it. - Advanced Search keeps its grammar (`~` negates, `@` is AND, `|` is OR and binds looser, string `>` is "starts with") and translates each leaf into the same conditions, compiled once per declaring type per search and memoized across the workers. Float and double keep their typed-precision tolerance through a small `EpsilonCondition` in the Advanced Search folder. ## Semantics preserved - Equality on a non-comparable reference type is `object.Equals`, never reference identity. A non-comparable struct boxes into the same call. - A null intermediate in a chained binding is no match, and stays no match under negation. Sort and distinct read it as `default(T)`. - Unsigned relational compares stay unsigned; integral primitives and enums use the operator directly, nothing widens to a signed type. `float`, `double`, `decimal`, `string` and structs still go through the type's own `CompareTo`, so `string` equality stays culture-sensitive exactly as before. - Only `==` and `!=` are valid for non-comparable types; a relational operator still throws at build time. - `TypeCondition` is still first and still null-checks the cast target. ## Behavior changes - **`Nullable<T>` works**, with C# lifted semantics in `where`: two nulls are equal, a null and a value are unequal, a null satisfies no relation. Sort keeps a total order with unset values at one end. A null *reference* keeps the ordering it had. - **Advanced Search**: a leaf that cannot be parsed or resolved is no match even under `~` (it used to negate the failure and match every entity); `null` is the null value for equality on strings, nullables and reference types, as in `where`; dotted names walk into a property; static properties are no longer searchable. ## Measurements `where`, from the handoff (Debug test host, single condition, ratios not absolutes): | Approach | ns/object | Compile | Collectible | LOC | |---|---:|---:|---|---:| | `AssemblyBuilder` + IL (before) | 20.5 | 0.311 ms | no (`Run`) | ~1,916 | | Expression trees (after) | ~12 | 0.187 ms warm | yes | ~600 | Advanced Search, Release, one `SkillTeleporter`, 2M evaluations per leaf: | Leaf | Before | After | |---|---:|---:| | `Hue=5` | 137 ns | 20 ns | | `Name~~gate` | 121 ns | 39 ns | | `Skill=Magery` | 116 ns | 21 ns | | `Weight>0.5` | 122 ns | 28 ns | Plus 0.5 to 2 ms to compile each declaring type a search meets (12 ms for the first compile in the process). All of it runs on the search workers; nothing new touches the loop. ## Also fixed: Advanced Search map filters Found while testing the property test in game. The map boxes are independent checkboxes, but the worker applied each ticked map as "must be on this map", so ticking two or more (all maps and Internal, say) rejected every entity before any other filter ran. Present since #1649; the default of Felucca alone never showed it. An entity now passes when its map is any of the ticked ones, with none ticked meaning no map constraint. Pinned by a worker test. ## Test plan - [x] `UOContent.Tests`: 862 passed, 0 failed - [x] `Server.Tests`: 869 passed, 0 failed - [x] Every commit builds and its tests pass on its own (bisectable) - [ ] In game: `[global where`, `[area where`, `[condition`, `sort by`, `distinct`, Advanced Search property test
393 lines
12 KiB
C#
393 lines
12 KiB
C#
using System;
|
|
using System.Collections.Concurrent;
|
|
using System.Threading;
|
|
using Server.Items;
|
|
using Server.Logging;
|
|
using Server.Mobiles;
|
|
using Server.Multis;
|
|
|
|
namespace Server.Engines.AdvancedSearch;
|
|
|
|
/// <summary>
|
|
/// Filters entities on a background thread while the main loop keeps mutating them — an
|
|
/// intentional, bounded race. Reads of live <see cref="Item"/>/<see cref="Mobile"/> state are
|
|
/// unsynchronized, so a torn <see cref="Point3D"/> read may report stale coordinates, and any
|
|
/// getter that throws mid-read is caught per-entity in <see cref="DoEntitySearch"/> and skipped.
|
|
/// Results are best-effort and may omit a concurrently modified entity, but never fault or corrupt
|
|
/// server state. Eliminating the race would require snapshotting each read field onto the main
|
|
/// thread before handing entities off; that is deferred.
|
|
/// </summary>
|
|
public class AdvancedSearchThreadWorker
|
|
{
|
|
private static readonly ILogger _logger = LogFactory.GetLogger(typeof(AdvancedSearchThreadWorker));
|
|
|
|
private readonly Thread _thread;
|
|
private readonly AutoResetEvent _startEvent; // Main thread tells the thread to start working
|
|
private readonly AutoResetEvent _stopEvent; // Main thread waits for the worker finish draining
|
|
private bool _pause;
|
|
private bool _exit;
|
|
private readonly ConcurrentQueue<IEntity> _entities;
|
|
private ConcurrentQueue<AdvancedSearchResult> _results;
|
|
private ConcurrentQueue<IEntity> _ignoreQueue;
|
|
private WorldLocation _worldLocation;
|
|
private AdvancedSearchFilter _filter;
|
|
private AdvancedSearchConditions.Cache _predicates;
|
|
|
|
public AdvancedSearchThreadWorker()
|
|
{
|
|
_startEvent = new AutoResetEvent(false);
|
|
_stopEvent = new AutoResetEvent(false);
|
|
_entities = new ConcurrentQueue<IEntity>();
|
|
_thread = new Thread(Execute)
|
|
{
|
|
IsBackground = true
|
|
};
|
|
_thread.Start(this);
|
|
}
|
|
|
|
/// <param name="predicates">
|
|
/// Compiled property-test memo for this search. Shared across the workers so a type is
|
|
/// compiled once per search rather than once per worker; a lone worker may leave it null.
|
|
/// </param>
|
|
public void Wake(
|
|
WorldLocation worldLocation,
|
|
AdvancedSearchFilter filter,
|
|
ConcurrentQueue<AdvancedSearchResult> results,
|
|
ConcurrentQueue<IEntity> ignoreQueue,
|
|
AdvancedSearchConditions.Cache predicates = null
|
|
)
|
|
{
|
|
_worldLocation = worldLocation;
|
|
_filter = filter;
|
|
_ignoreQueue = ignoreQueue;
|
|
_results = results;
|
|
_predicates = predicates ?? new AdvancedSearchConditions.Cache();
|
|
_startEvent.Set();
|
|
}
|
|
|
|
public void Sleep()
|
|
{
|
|
Volatile.Write(ref _pause, true);
|
|
_stopEvent.WaitOne();
|
|
}
|
|
|
|
public void Exit()
|
|
{
|
|
Volatile.Write(ref _exit, true);
|
|
|
|
Wake(WorldLocation.Zero, null, null, null);
|
|
|
|
// Tolerate a worker that has already terminated (e.g. Core.Closing raced us) so
|
|
// shutdown can't deadlock waiting on a stopEvent that will never be set.
|
|
if (_thread.IsAlive)
|
|
{
|
|
Sleep();
|
|
}
|
|
}
|
|
|
|
public void Push(IEntity entity)
|
|
{
|
|
_entities.Enqueue(entity);
|
|
}
|
|
|
|
private static void Execute(object obj)
|
|
{
|
|
var worker = (AdvancedSearchThreadWorker)obj;
|
|
|
|
var reader = worker._entities;
|
|
|
|
while (worker._startEvent.WaitOne())
|
|
{
|
|
while (true)
|
|
{
|
|
var pauseRequested = Volatile.Read(ref worker._pause);
|
|
if (reader.TryDequeue(out var entity))
|
|
{
|
|
var result = worker.DoEntitySearch(entity);
|
|
if (result != null)
|
|
{
|
|
worker._results?.Enqueue(result);
|
|
}
|
|
}
|
|
else if (pauseRequested) // Break when finished
|
|
{
|
|
worker._results = null;
|
|
worker._filter = null;
|
|
worker._predicates = null; // a compiled constant may pin an entity resolved by serial
|
|
break;
|
|
}
|
|
else
|
|
{
|
|
// Transiently empty but not yet paused: yield rather than busy-spin.
|
|
Thread.Yield();
|
|
}
|
|
}
|
|
|
|
// The owning thread may start another cycle the moment _stopEvent is set (Exit does exactly
|
|
// that). Clear _pause and sample the exit condition before signaling, or the new cycle's
|
|
// pause request is clobbered / its Sleep orphaned. Matches SerializationThreadWorker.
|
|
var exiting = Core.Closing || Volatile.Read(ref worker._exit);
|
|
Volatile.Write(ref worker._pause, false);
|
|
|
|
worker._stopEvent.Set(); // Allow the main thread to continue now that we are finished
|
|
|
|
if (exiting)
|
|
{
|
|
return;
|
|
}
|
|
}
|
|
}
|
|
|
|
private AdvancedSearchResult DoEntitySearch(IEntity entity)
|
|
{
|
|
if (entity == null || entity.Deleted)
|
|
{
|
|
return null;
|
|
}
|
|
|
|
try
|
|
{
|
|
return DoEntitySearchCore(entity);
|
|
}
|
|
catch (Exception ex)
|
|
{
|
|
_logger.Warning(ex, "AdvancedSearch: filter threw for {Entity}; skipping", entity);
|
|
return null;
|
|
}
|
|
}
|
|
|
|
private AdvancedSearchResult DoEntitySearchCore(IEntity entity)
|
|
{
|
|
if (_filter == null)
|
|
{
|
|
// Exit() clears the filter; a straggler entity dequeued after teardown bails here.
|
|
return null;
|
|
}
|
|
|
|
if (!OnASelectedMap(entity.Map))
|
|
{
|
|
return null;
|
|
}
|
|
|
|
var location = (entity as Item)?.GetWorldLocation() ?? entity.Location;
|
|
|
|
if (_filter.FilterRange &&
|
|
(_filter.Range == null ||
|
|
_filter.Range < 0 ||
|
|
entity.Map != _worldLocation.Map ||
|
|
!Utility.InRange(_worldLocation.Location, location, _filter.Range.Value)))
|
|
{
|
|
return null;
|
|
}
|
|
|
|
if (_filter.FilterRegion &&
|
|
(string.IsNullOrWhiteSpace(_filter.RegionName) ||
|
|
!Region.Find(location, entity.Map).IsPartOf(_filter.RegionName)))
|
|
{
|
|
return null;
|
|
}
|
|
|
|
// After the cheap filters, so non-qualifying entities skip the house/keyring enumeration.
|
|
if (_filter.HideValidInternalMap)
|
|
{
|
|
HandleValidInternal(entity);
|
|
}
|
|
|
|
if (entity is Mobile mobile)
|
|
{
|
|
return DoMobileSearch(mobile);
|
|
}
|
|
|
|
if (entity is Item item)
|
|
{
|
|
return DoItemSearch(item);
|
|
}
|
|
|
|
return null;
|
|
}
|
|
|
|
// The map boxes are independent checks, so several can be ticked at once: an entity passes when
|
|
// its map is any of them. With none ticked there is no map constraint.
|
|
private bool OnASelectedMap(Map map)
|
|
{
|
|
var f = _filter;
|
|
|
|
var anySelected = f.FilterFelucca || f.FilterTrammel || f.FilterIlshenar || f.FilterMalas ||
|
|
f.FilterTokuno || f.FilterTerMur || f.FilterInternalMap || f.FilterNullMap;
|
|
|
|
if (!anySelected)
|
|
{
|
|
return true;
|
|
}
|
|
|
|
if (map == null)
|
|
{
|
|
return f.FilterNullMap;
|
|
}
|
|
|
|
if (map == Map.Internal)
|
|
{
|
|
return f.FilterInternalMap;
|
|
}
|
|
|
|
return map == Map.Felucca && f.FilterFelucca ||
|
|
map == Map.Trammel && f.FilterTrammel ||
|
|
map == Map.Ilshenar && f.FilterIlshenar ||
|
|
map == Map.Malas && f.FilterMalas ||
|
|
map == Map.Tokuno && f.FilterTokuno ||
|
|
map == Map.TerMur && f.FilterTerMur;
|
|
}
|
|
|
|
private static bool IsValidInternal(Item item)
|
|
{
|
|
if (item.Parent != null || item.HeldBy != null)
|
|
{
|
|
return true;
|
|
}
|
|
|
|
if (item is Fists
|
|
or MountItem
|
|
or EffectItem
|
|
or MovingCrate
|
|
or BaseDockedBoat
|
|
or BaseBoat
|
|
or Plank
|
|
or TillerMan
|
|
or Hold)
|
|
{
|
|
return true;
|
|
}
|
|
|
|
// DisplayCache container
|
|
if (item.GetType().DeclaringType == typeof(GenericBuyInfo))
|
|
{
|
|
return true;
|
|
}
|
|
|
|
return false;
|
|
}
|
|
|
|
private AdvancedSearchResult DoItemSearch(Item item)
|
|
{
|
|
if (_filter.FilterName && !string.IsNullOrWhiteSpace(_filter.Name) && !(item.Name ?? item.ItemData.Name).InsensitiveContains(_filter.Name))
|
|
{
|
|
return null;
|
|
}
|
|
|
|
if (_filter.HideValidInternalMap && item.Map == Map.Internal && !IsValidInternal(item))
|
|
{
|
|
return null;
|
|
}
|
|
|
|
if (_filter.FilterPropertyTest && !PassesPropertyTest(item))
|
|
{
|
|
return null;
|
|
}
|
|
|
|
return new AdvancedSearchResult(item.Name ?? item.ItemData.Name, item.GetType(), item.Location, item.Map, item.RootParent)
|
|
{
|
|
Entity = item,
|
|
};
|
|
}
|
|
|
|
private AdvancedSearchResult DoMobileSearch(Mobile mobile)
|
|
{
|
|
if (_filter.FilterName && !string.IsNullOrWhiteSpace(_filter.Name) && !mobile.Name.InsensitiveContains(_filter.Name))
|
|
{
|
|
return null;
|
|
}
|
|
|
|
if (_filter.HideValidInternalMap && mobile.Map == Map.Internal && !IsValidInternal(mobile))
|
|
{
|
|
return null;
|
|
}
|
|
|
|
if (_filter.FilterPropertyTest && !PassesPropertyTest(mobile))
|
|
{
|
|
return null;
|
|
}
|
|
|
|
return new AdvancedSearchResult(mobile.Name, mobile.GetType(), mobile.Location, mobile.Map, null)
|
|
{
|
|
Entity = mobile
|
|
};
|
|
}
|
|
|
|
private static bool IsValidInternal(Mobile m)
|
|
{
|
|
// Logged out players
|
|
if (m.Account != null)
|
|
{
|
|
return true;
|
|
}
|
|
|
|
// Stabled pets
|
|
if (m is BaseCreature creature && creature.IsStabled)
|
|
{
|
|
return true;
|
|
}
|
|
|
|
// Internalized vendors
|
|
if (m is PlayerVendor playerVendor && playerVendor.House != null)
|
|
{
|
|
return true;
|
|
}
|
|
|
|
// Currently mounted creatures
|
|
if (m is IMount mount && mount.Rider != null)
|
|
{
|
|
return true;
|
|
}
|
|
|
|
return false;
|
|
}
|
|
|
|
public void HandleValidInternal(IEntity entity)
|
|
{
|
|
if (entity is CommodityDeed deed && deed.Commodity != null && deed.Commodity.Map == Map.Internal)
|
|
{
|
|
_ignoreQueue.Enqueue(entity);
|
|
return;
|
|
}
|
|
|
|
// Keys don't have a backreference, so we just ignore them for now
|
|
if (entity is KeyRing keyring && keyring.Keys?.Count > 0)
|
|
{
|
|
foreach (var k in keyring.Keys)
|
|
{
|
|
_ignoreQueue.Enqueue(k);
|
|
}
|
|
|
|
return;
|
|
}
|
|
|
|
if (entity is BaseHouse house)
|
|
{
|
|
foreach (var relEntity in house.RelocatedEntities)
|
|
{
|
|
if (relEntity.Entity is Item)
|
|
{
|
|
_ignoreQueue.Enqueue(relEntity.Entity);
|
|
}
|
|
}
|
|
|
|
foreach (var inventory in house.VendorInventories)
|
|
{
|
|
foreach (var subItem in inventory.Items)
|
|
{
|
|
_ignoreQueue.Enqueue(subItem);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
// The test is compiled once per runtime type for the search and memoized; after that each
|
|
// entity costs a dictionary lookup and a delegate call.
|
|
private bool PassesPropertyTest(IEntity entity)
|
|
{
|
|
var test = _filter.PropertyTest;
|
|
|
|
return !string.IsNullOrWhiteSpace(test) &&
|
|
AdvancedSearchConditions.GetPredicate(_predicates, entity.GetType(), test)(entity);
|
|
}
|
|
}
|