Skip to content

Luau type definitions

How the engine generates .d.luau files, the type definitions that editors (through luau-lsp) use to complete and check scripts. The files are generated from the C++ code, never written by hand, so that they follow the API when it changes.

Status: design, not implemented yet. The steps are listed in Plan.

Goals

  • One source of truth: the C++ code that binds a function or a type to Luau also describes its type.
  • Two files:
    • engine.d.luau: the API written in C++ (engine.ecs.*, Entity, ComponentView, Commands, the ctx of systems...). It is the same for every game, generated by a tool and committed.
    • game.d.luau: the data of one game (its components, resources, events and the ctx of each of its systems). It depends on the plugins the game adds and the scripts it loads, so the game process writes it.
  • Deterministic output: declarations are sorted by name, so a change of API gives a readable diff.

Not a goal for now: a CI job that checks the committed engine.d.luau is up to date or type-checks the scripts.

Approach: record at registration

Every binding already goes through a small set of calls: LibraryBuilder::function(), Usertype::constructor(), getter(), setter(), method(), index(), newIndex(), eq(), tostring(), and globals().registerLibrary(). Each of them also records a description of what it registers in a types::TypeRegistry owned by the Runtime. A types::DefinitionWriter prints what was recorded.

Rejected alternatives:

  • Parsing the C++ (libclang): a heavy dependency, fragile, and blind to the names, which are runtime strings ("component", "Entity").
  • A separate description of the API (a hand-written file or a DSL): a second source of truth.

Recording at runtime is also what makes game.d.luau possible, since script components only exist once the scripts ran.

Phase 1: engine.d.luau

Inferred types

detail::FunctionTraits already splits a handler into its return type and its parameters. A trait, types::TypeOf<T>, gives the Luau type of each:

C++ Luau
bool, double, std::string, Value::Vector, Value::Nil boolean, number, string, vector, nil
std::optional<U> U?
void (return) ()
T, T&, const T& where T was registered with usertype<T>("Name") Name
Table, FunctionRef, Userdata, Value any
ValuePack (return) ...any

A usertype is resolved by std::type_index when the file is written, not when it is recorded, so a type may be used before it is registered. A type that is still unregistered when the file is written is an error of the writer.

TypeOf<T> can be specialized for other types.

Signatures

Two things cannot be inferred:

  • Parameter names: C++ cannot reflect the names of a lambda's parameters.
  • What a dynamic type really is: a spec parameter is a luau::Table, but scripts must pass a SystemSpec.

Every registration call takes an optional, trailing types::Signature:

struct Signature {
    std::vector<std::string> params;  ///< One per parameter, "name" or "name: Type".
    std::string returns;              ///< Return type, overriding the inferred one; empty to infer it.
    std::string doc;                  ///< Documentation comment, written above the declaration.
};
  • A bare name ("name") keeps the inferred type, "name: Type" overrides it.
  • params is empty, or has exactly one entry per parameter of the handler (without self for methods, getters and setters). A mismatch is API misuse: exceptions::InvalidOperation, thrown when the handler is bound. Without names, parameters are written arg1, arg2...
  • For index(), returns is the type of the values it reads: the class gets the indexer [string]: <returns>.
ecsBuilder.function("system", {.params = {"name", "spec: SystemSpec"}, .doc = "Declares a script system."})
    .bind([this](const std::string& name, const luau::Table& spec) { declareSystem(name, spec); });

entityType->getter("bits", bitsOf, {.doc = "Raw bits of the entity."});
viewType->index(readField, {.returns = "any"});

A struct, rather than chained calls (.params(...).doc(...)), because:

  • It works the same way on Usertype, where method() returns the Usertype to chain the next member. A chained .params() would describe "the last member added", which is hidden state.
  • It keeps the type metadata out of Function, which is also used outside libraries (Function::fromRef).
  • It is plain data: the registry stores it as is.

Type aliases

Named shapes are declared as Luau text, next to the C++ code that reads them, so that both change together:

// Next to SystemDeclarationReader
runtime.types().alias("SystemSpec",
                      "{ query: {string}?, resources: {string}?, events: {string}?, run: (ctx: Context) -> () }");

Text rather than a C++ builder of types: it is shorter, and it reads like the generated file.

Changes to the binding API

Two changes make inference cover most of the API:

  1. Function::bind accepts the same handlers as Usertype::method: returning void, a ValuePack, or a value a Value can be built from. Today it only accepts ValuePack, so handlers that return nothing return luau::ValuePack{}, which would be typed ...any.
  2. Handlers may take a registered usertype as T& or const T& (beyond the self of methods). The library checks the userdata's type at call time and raises a Luau error on a mismatch. Today usertypes arrive as an untyped Userdata that each handler unwraps.

Values set on a library table with Table::set (such as engine.isClient) are not recorded. LibraryBuilder gets a call to add a constant with its Signature, so that they are.

Output

-- Generated from the C++ bindings. Do not edit.
type SystemSpec = { query: {string}?, resources: {string}?, events: {string}?, run: (ctx: Context) -> () }

declare class Entity
    bits: number
    function __tostring(self): string
end

declare engine: {
    isClient: boolean,
    isServer: boolean,
    isDebug: boolean,
    ecs: {
        component: (name: string, spec: ComponentSpec) -> (),
        system: (name: string, spec: SystemSpec) -> (),
        plugin: (name: string, spec: PluginSpec) -> (),
        addPlugin: (name: string) -> (),
    },
}
  • A usertype is a declare class: getters and setters are fields, methods take self, eq and tostring are __eq and __tostring, and index() adds an indexer.
  • A usertype with a constructor also gets a global table: declare Name: { new: (...) -> Name }.
  • A library registered as a global is a declare name: { ... } of its members, sub-libraries nested.

Code layout

In luau, a types/ folder (rtype::luau::types):

  • TypeOf.hpp / .tpp: the trait, with the specializations of the table above.
  • Signature.hpp: the struct.
  • Declarations.hpp: a set of declarations (aliases, classes, globals). The writer takes one, so that scripting can build its own for game.d.luau, separate from the runtime's.
  • TypeRegistry.hpp: the declarations of a Runtime, filled by LibraryBuilder and Usertype, reached through Runtime::types().
  • DefinitionWriter.hpp: turns a Declarations into the text of a .d.luau. It returns a Result, failing on an unresolved usertype.

luau knows nothing about the ECS. scripting adds the signatures and aliases of its bindings.

A luau-types tool target builds a ScriptHost without running any script, then writes engine.d.luau. An xmake task runs it. The output is committed.

Phase 2: game.d.luau

What it types

Game data is addressed by name: ctx.resources["rtype.Input"], ctx.events.read("rtype.Collision"), ctx.commands:add(entity, "game.Shield", {...}). game.d.luau maps each name to its type:

  • Components, script and C++ alike, since scripts see both through ComponentView: one type per component, built from its fields.
  • ctx.resources: a table type with quoted keys, one per plain-data resource (the ones ctx hands out).
  • ctx.events.read and ctx.events.send: one overload per event, selected by its name.
  • ctx.commands:add and spawn: the table of initial values each component takes.
  • The ctx of each system: its query yields the entity, then one view per term, in the order of the terms.
-- Generated from the game's ECS registry. Do not edit.
type GameShield = { strength: number, regen: number, owner: Entity }
type RtypeTransform = { position: vector, rotation: number, scale: vector }

type GameResources = { ["rtype.Input"]: RtypeInput, ["rtype.Time"]: RtypeTime }
type GameEvents = {
    read: ((name: "rtype.Collision") -> () -> RtypeCollision?) & ((name: "game.Hello") -> () -> GameHello?),
    send: ((name: "game.Hello", data: GameHello) -> ()) & ...,
}
type GameShieldRegenContext = Context & {
    query: () -> (Entity?, GameShield),
    resources: GameResources,
    events: GameEvents,
}

Sources

  • The ECS registry: the name, kind and FieldDescriptions of every component, resource and event, the same for C++ and script types.
  • The declarations of script systems: terms, resources and events. ScriptHost keeps a copy of them after registerSystem(), which consumes them today.

Field types

The Luau type of each ecs::FieldType is defined in view/FieldAccess.cpp, next to readField() and writeField(), so that the types match what the conversions accept: kBool is boolean, integers and floats are number, kEntity is Entity, kAssetId is AssetId, vectors are what readVector() returns.

Names

A dotted name becomes PascalCase segments joined together: game.Shield is GameShield, and the context of the system game.ShieldRegen is GameShieldRegenContext. Two names that give the same type name (game.Shield and gameShield) are an error, never merged.

Code layout and emission

  • scripting/types/ (rtype::scripting::types) builds a luau::types::Declarations from the ECS registry and the kept system declarations, and writes it with the luau writer.
  • The types of game.d.luau refer to those of engine.d.luau (Entity, Context...) by name; both files are given to luau-lsp.
  • The C++ components come from the plugins the game adds, so only the game process knows all of them. ScriptingPlugin gets an option to write game.d.luau once the App is complete: every plugin added and every script loaded.

Plan

Each step builds on the previous one and can be its own pull request.

# Module Change
1 luau bind accepts void and plain values; the scripting handlers drop return luau::ValuePack{}
2 luau Usertype parameters (T&, const T&), checked through the UsertypeRegistry
3 luau types/: TypeOf, Signature, Declarations, TypeRegistry filled by LibraryBuilder and Usertype, the arity check, library constants
4 luau DefinitionWriter, tested against expected text
5 scripting Signatures and aliases (ComponentSpec, SystemSpec, PluginSpec, Context, Commands...) next to their readers
6 tool luau-types target and xmake task; engine.d.luau generated and committed
7 — Check with luau-lsp, on a hand-written game.d.luau: overloads selected by a string literal, quoted keys in table types, global types of a definition file seen by scripts
8 scripting Luau types of the field types next to the conversions; ScriptHost keeps its system declarations
9 scripting types/: component, resource and event types, GameResources, GameEvents, the context of each system
10 scripting ScriptingPlugin option to write game.d.luau, with tests

Open points

  • How a system's run gets its typed ctx:
    • an annotation in the script, run = function(ctx: GameShieldRegenContext), which always works;
    • or no annotation, with one overload of engine.ecs.system per system name, each with its own run type. This needs the type checker to select overloads by a string literal reliably; step 7 tells whether it does.
  • How a game turns on writing game.d.luau:
    • a command-line flag (rtype_duel client --emit-types), explicit and leaving normal runs untouched (preferred);
    • a setting in project.luau;
    • on every debug run.