07 · Queries¶
What it is¶
A query is "every entity that has these components and matches these filters, with access to their
data". It is how systems find the entities they work on: "all entities with Position and
Velocity", "all enemies that are not Dead".
Why we need it¶
Queries are the main way systems read and write the world.
- R2 (Luau): scripts cannot name C++ types, so queries must also work with
ComponentIds (dynamic queries). - R3 (network):
Changed<T>filters let replication find what changed. - R4 (generic): the query is the boundary that hides storage, so sparse sets and tables can coexist.
How others do it¶
| ECS | Query style |
|---|---|
| EnTT | registry.view<A, B>(entt::exclude<C>); groups for faster combinations |
| Bevy | Query<(&mut A, &B), (With<C>, Without<D>, Changed<A>)> as a system parameter |
| flecs | Query builder and a query language with variables, relationships, optional terms |
| Unity DOTS | SystemAPI.Query<RefRW<A>, RefRO<B>>().WithNone<C>() |
Our choice¶
- Typed queries for C++, Bevy-style: non-
constcomponents are written,constones are read. - Dynamic queries for Luau and tools, built from ids.
- One engine underneath: typed queries compile down to a dynamic query, so there is one iteration algorithm to test and optimize.
How it works¶
Typed¶
void movement(rtype::ecs::Query<Position, const Velocity> query, rtype::ecs::Res<Time> time) {
for (auto [entity, position, velocity] : query) {
position->x += velocity.x * time->fixedDelta; // Mut<Position>: writing marks it changed
position->y += velocity.y * time->fixedDelta;
}
}
void cleanup(rtype::ecs::Query<const Transform, rtype::ecs::With<Enemy>, rtype::ecs::Without<Dead>> query,
rtype::ecs::Commands& commands);
| Term | Meaning | Yields |
|---|---|---|
T |
required, written | Mut<T> (see Change detection) |
const T |
required, read | const T& |
Optional<T> / Optional<const T> |
may be absent | Mut<T> that may be empty / const T* that may be null |
With<T> / Without<T> |
presence / absence, no access | nothing |
Added<T> / Changed<T> |
added / written since this system last ran | nothing |
A typed query has two parts. The QueryState<Terms...> is built once (it resolves and checks the terms,
registering their components); a system keeps one per Query parameter. The Query<Terms...> a system
receives is a light view of it, made for each run with the system's lastRun and iterated directly. Outside
a system, iterate the state with an explicit lastRun:
const rtype::ecs::QueryState<Position, const Velocity, rtype::ecs::Without<Dead>> movement{world.getComponents()};
for (auto [entity, position, velocity] : movement.iter(lastRun)) { // Mut<Position>, const Velocity&
position->x += velocity.x;
}
rtype::ecs::Query<Position, const Velocity, rtype::ecs::Without<Dead>> view{movement, lastRun};
for (auto [entity, position, velocity] : view) { /* what a system's body does */ }
- A match is a
std::tupleof the entity and one item per term that has one (filters yield nothing), in term order; bind it withauto [...]. - Building registers each term's component with the default options if it is not registered yet (one already registered keeps its options), so a query can exist before its first entity.
- Invalid combinations do not compile (
ValidQuery<Terms...>, see Rules). - No allocation: the typed pass keeps its buffers in
std::arrays sized by the term count.
Dynamic¶
rtype::ecs::Result<rtype::ecs::DynamicQuery> query = rtype::ecs::QueryBuilder{}
.write(shieldId)
.read(positionId)
.without(deadId)
.build(world.getComponents()); // an Error (kInvalidQuery) instead of an exception
for (rtype::ecs::QueryMatch& match : query->iter(lastRun)) {
rtype::ecs::Entity entity = match.getEntity();
std::span<std::byte> shield = match.getMut(0); // data term 0: game.Shield, marked changed
std::span<const std::byte> position = match.get(1); // data term 1: rtype.Position, read only
}
| Builder call | Term | Data term |
|---|---|---|
read(id) / write(id) |
required | yes |
optionalRead(id) / optionalWrite(id) |
may be absent (empty bytes then) | yes |
with(id) / without(id) |
presence / absence | no |
added(id) / changed(id) |
required, stamped after lastRun |
no |
- Values are numbered by data term, in the order given; filters are not numbered.
get(i)never marks anything;getMut(i)marks the value changed (it only has bytes, so it cannot tell a read from a write) and returns empty for a read term: the bridge cannot write what the script declared read-only.-
The pass allocates its two small buffers once, when
iter()starts; stepping never allocates. -
Components registered at run time work the same: a component described by the Luau bridge (no C++ type behind it) is read, written, filtered and change-tracked exactly like a C++ one.
In a system, dynamic queries come through the DynamicQueries parameter, configured with a QueryList
when the system is added, so their access is declared to the scheduler (see
Systems). The Luau bridge wraps this so a
script iterates for entity, shield, position in ctx.query do.
The join: smallest pool first¶
Example: Query<Position, const Velocity, With<Player>>.
| Pool | Entities | Role |
|---|---|---|
Position |
1200 | checked with contains |
Velocity |
900 | checked with contains |
Player |
2 (e17, e42) |
driver: 2 iterations instead of 1200 |
flowchart LR
driver["Iterate the Player pool<br/>e17, e42"] --> pos{"Position pool<br/>contains it?"}
pos -- "no" --> skip["skip"]
pos -- "yes" --> vel{"Velocity pool<br/>contains it?"}
vel -- "no" --> skip
vel -- "yes" --> yield["yield entity,<br/>Position, Velocity"]
- Among the required components, pick the pool with the fewest entities: the driver.
- Iterate the driver's dense entity array.
- For each entity, check the other required pools (
contains), theWithoutpools (must not contain), and the tick filters. - Yield the entity and pointers to its components.
Without, Optional and filters never drive iteration. A query with no required component (only
Optional) is rejected: it would mean "every entity".
With archetype tables (v3), table components are matched by walking the cached list of matching tables, and sparse components are checked per entity.
Building once¶
Resolving T to ids, checking the terms and (v3) matching tables happen when the query is built,
which for a system is when it is registered. Per frame, only the iteration runs: iter() looks the pools
up again (registering a component may move them), picks the driver, and walks.
A query keeps a pointer to the world's components, so it only iterates the world it was built for, and the world must outlive its queries (the App will own both).
Rules¶
The same rules for both kinds, from one function (findConflict()): DynamicQuery::create() returns an
error naming the component, and Query<Terms...> does not compile.
| Rejected | Example | Why |
|---|---|---|
| No required term | Optional<Position> alone, Without<Dead> alone |
it would mean "every entity", with no pool to walk |
| Two data terms on one component | Position, const Position |
which one gives the value? |
| The same term twice | Position, Position, Changed<A>, Changed<A> |
two writable terms of one type are a compile error |
Without<T> with another term on T |
const Position, Without<Position> |
it could never match |
| A data term on a tag (typed only) | Query<Dead> |
a tag has no value: use With<Dead> |
| An unknown id (dynamic only) | read(42) |
not registered in this world |
Allowed on purpose: a data term with a filter on the same component (Position, Changed<Position>:
"write the positions that changed"), and Added<T> with Changed<T>.
Access and conflicts¶
A query's access set (getAccess(): each component it reads or writes, once, a write winning over a read)
is what the scheduler uses to order systems, and later to run disjoint ones in
parallel. Added / Changed count as reads (they read the ticks); With / Without access nothing.
How the code is split¶
| Piece | Role |
|---|---|
QueryTerm |
a component id and a TermKind; the term classifications and findConflict() |
QueryBuilder / DynamicQuery |
collect and check the terms; data term numbering; access set; iter() |
detail::QueryMatcher |
findDriver() and matchEntity(): the one join algorithm |
detail::QueryCursor<Pools, Positions> |
the walk over the driver pool, shared by both passes (vectors or arrays) |
QueryIteration / QueryMatch |
the dynamic pass and what its loop body sees |
QueryTerms / TermTraits / QueryState |
the C++ terms, their mapping to TermKinds and items, the typed query and its pass |
Query |
the per-run view a system receives (state + lastRun) |
Connections¶
- Uses: World, Pool, Archetype tables, Change detection.
- Used by: Systems, Luau bridge, Replication (dynamic queries over replicated components).
Pitfalls¶
- Structural changes while iterating. Adding or removing components of a pool being iterated moves elements under the iterator. Use commands.
- Assuming an order. Iteration follows the driver pool's dense order, which changes with swap-and-pop. Sort explicitly when order matters (draw layers).
- Writing through a read-only term.
const Tgivesconst T&; for Luau the bridge enforces it by refusing writes to components declared asread. - Marking everything changed. Iterating
Query<Position>without writing must not stamp ticks;Mut<T>only stamps on mutable access. By id, preferget(i)and callgetMut(i)only to write. - A query outliving its world. It keeps a plain pointer to the world's components: destroy queries first (the App will).
- Iterating a pass twice. A pass is single: a second
begin()continues where the first stopped. Calliter()again for a new pass.
Open questions¶
- EnTT-style groups (keeping two pools aligned) for very hot pairs, before v3 tables exist? Only if benchmarks ask for it. (ECS)