Skip to content

07 · Queries

What it is

A query is "every entity that has these components and matches these filters, with access to their data". It is how systems find the entities they work on: "all entities with Position and Velocity", "all enemies that are not Dead".

Why we need it

Queries are the main way systems read and write the world.

  • R2 (Luau): scripts cannot name C++ types, so queries must also work with ComponentIds (dynamic queries).
  • R3 (network): Changed<T> filters let replication find what changed.
  • R4 (generic): the query is the boundary that hides storage, so sparse sets and tables can coexist.

How others do it

ECS Query style
EnTT registry.view<A, B>(entt::exclude<C>); groups for faster combinations
Bevy Query<(&mut A, &B), (With<C>, Without<D>, Changed<A>)> as a system parameter
flecs Query builder and a query language with variables, relationships, optional terms
Unity DOTS SystemAPI.Query<RefRW<A>, RefRO<B>>().WithNone<C>()

Our choice

  • Typed queries for C++, Bevy-style: non-const components are written, const ones are read.
  • Dynamic queries for Luau and tools, built from ids.
  • One engine underneath: typed queries compile down to a dynamic query, so there is one iteration algorithm to test and optimize.

How it works

Typed

void movement(rtype::ecs::Query<Position, const Velocity> query, rtype::ecs::Res<Time> time) {
    for (auto [entity, position, velocity] : query) {
        position->x += velocity.x * time->fixedDelta;  // Mut<Position>: writing marks it changed
        position->y += velocity.y * time->fixedDelta;
    }
}

void cleanup(rtype::ecs::Query<const Transform, rtype::ecs::With<Enemy>, rtype::ecs::Without<Dead>> query,
             rtype::ecs::Commands& commands);
Term Meaning Yields
T required, written Mut<T> (see Change detection)
const T required, read const T&
Optional<T> / Optional<const T> may be absent Mut<T> that may be empty / const T* that may be null
With<T> / Without<T> presence / absence, no access nothing
Added<T> / Changed<T> added / written since this system last ran nothing

A typed query has two parts. The QueryState<Terms...> is built once (it resolves and checks the terms, registering their components); a system keeps one per Query parameter. The Query<Terms...> a system receives is a light view of it, made for each run with the system's lastRun and iterated directly. Outside a system, iterate the state with an explicit lastRun:

const rtype::ecs::QueryState<Position, const Velocity, rtype::ecs::Without<Dead>> movement{world.getComponents()};

for (auto [entity, position, velocity] : movement.iter(lastRun)) {  // Mut<Position>, const Velocity&
    position->x += velocity.x;
}

rtype::ecs::Query<Position, const Velocity, rtype::ecs::Without<Dead>> view{movement, lastRun};
for (auto [entity, position, velocity] : view) { /* what a system's body does */ }
  • A match is a std::tuple of the entity and one item per term that has one (filters yield nothing), in term order; bind it with auto [...].
  • Building registers each term's component with the default options if it is not registered yet (one already registered keeps its options), so a query can exist before its first entity.
  • Invalid combinations do not compile (ValidQuery<Terms...>, see Rules).
  • No allocation: the typed pass keeps its buffers in std::arrays sized by the term count.

Dynamic

rtype::ecs::Result<rtype::ecs::DynamicQuery> query = rtype::ecs::QueryBuilder{}
    .write(shieldId)
    .read(positionId)
    .without(deadId)
    .build(world.getComponents());          // an Error (kInvalidQuery) instead of an exception

for (rtype::ecs::QueryMatch& match : query->iter(lastRun)) {
    rtype::ecs::Entity entity = match.getEntity();
    std::span<std::byte> shield = match.getMut(0);         // data term 0: game.Shield, marked changed
    std::span<const std::byte> position = match.get(1);    // data term 1: rtype.Position, read only
}
Builder call Term Data term
read(id) / write(id) required yes
optionalRead(id) / optionalWrite(id) may be absent (empty bytes then) yes
with(id) / without(id) presence / absence no
added(id) / changed(id) required, stamped after lastRun no
  • Values are numbered by data term, in the order given; filters are not numbered.
  • get(i) never marks anything; getMut(i) marks the value changed (it only has bytes, so it cannot tell a read from a write) and returns empty for a read term: the bridge cannot write what the script declared read-only.
  • The pass allocates its two small buffers once, when iter() starts; stepping never allocates.

  • Components registered at run time work the same: a component described by the Luau bridge (no C++ type behind it) is read, written, filtered and change-tracked exactly like a C++ one.

In a system, dynamic queries come through the DynamicQueries parameter, configured with a QueryList when the system is added, so their access is declared to the scheduler (see Systems). The Luau bridge wraps this so a script iterates for entity, shield, position in ctx.query do.

The join: smallest pool first

Example: Query<Position, const Velocity, With<Player>>.

Pool Entities Role
Position 1200 checked with contains
Velocity 900 checked with contains
Player 2 (e17, e42) driver: 2 iterations instead of 1200
flowchart LR
    driver["Iterate the Player pool<br/>e17, e42"] --> pos{"Position pool<br/>contains it?"}
    pos -- "no" --> skip["skip"]
    pos -- "yes" --> vel{"Velocity pool<br/>contains it?"}
    vel -- "no" --> skip
    vel -- "yes" --> yield["yield entity,<br/>Position, Velocity"]
  1. Among the required components, pick the pool with the fewest entities: the driver.
  2. Iterate the driver's dense entity array.
  3. For each entity, check the other required pools (contains), the Without pools (must not contain), and the tick filters.
  4. Yield the entity and pointers to its components.

Without, Optional and filters never drive iteration. A query with no required component (only Optional) is rejected: it would mean "every entity".

With archetype tables (v3), table components are matched by walking the cached list of matching tables, and sparse components are checked per entity.

Building once

Resolving T to ids, checking the terms and (v3) matching tables happen when the query is built, which for a system is when it is registered. Per frame, only the iteration runs: iter() looks the pools up again (registering a component may move them), picks the driver, and walks.

A query keeps a pointer to the world's components, so it only iterates the world it was built for, and the world must outlive its queries (the App will own both).

Rules

The same rules for both kinds, from one function (findConflict()): DynamicQuery::create() returns an error naming the component, and Query<Terms...> does not compile.

Rejected Example Why
No required term Optional<Position> alone, Without<Dead> alone it would mean "every entity", with no pool to walk
Two data terms on one component Position, const Position which one gives the value?
The same term twice Position, Position, Changed<A>, Changed<A> two writable terms of one type are a compile error
Without<T> with another term on T const Position, Without<Position> it could never match
A data term on a tag (typed only) Query<Dead> a tag has no value: use With<Dead>
An unknown id (dynamic only) read(42) not registered in this world

Allowed on purpose: a data term with a filter on the same component (Position, Changed<Position>: "write the positions that changed"), and Added<T> with Changed<T>.

Access and conflicts

A query's access set (getAccess(): each component it reads or writes, once, a write winning over a read) is what the scheduler uses to order systems, and later to run disjoint ones in parallel. Added / Changed count as reads (they read the ticks); With / Without access nothing.

How the code is split

Piece Role
QueryTerm a component id and a TermKind; the term classifications and findConflict()
QueryBuilder / DynamicQuery collect and check the terms; data term numbering; access set; iter()
detail::QueryMatcher findDriver() and matchEntity(): the one join algorithm
detail::QueryCursor<Pools, Positions> the walk over the driver pool, shared by both passes (vectors or arrays)
QueryIteration / QueryMatch the dynamic pass and what its loop body sees
QueryTerms / TermTraits / QueryState the C++ terms, their mapping to TermKinds and items, the typed query and its pass
Query the per-run view a system receives (state + lastRun)

Connections

Pitfalls

  • Structural changes while iterating. Adding or removing components of a pool being iterated moves elements under the iterator. Use commands.
  • Assuming an order. Iteration follows the driver pool's dense order, which changes with swap-and-pop. Sort explicitly when order matters (draw layers).
  • Writing through a read-only term. const T gives const T&; for Luau the bridge enforces it by refusing writes to components declared as read.
  • Marking everything changed. Iterating Query<Position> without writing must not stamp ticks; Mut<T> only stamps on mutable access. By id, prefer get(i) and call getMut(i) only to write.
  • A query outliving its world. It keeps a plain pointer to the world's components: destroy queries first (the App will).
  • Iterating a pass twice. A pass is single: a second begin() continues where the first stopped. Call iter() again for a new pass.

Open questions

  • EnTT-style groups (keeping two pools aligned) for very hot pairs, before v3 tables exist? Only if benchmarks ask for it. (ECS)