Luau type definitions¶
How the engine generates .d.luau files, the type definitions that editors (through luau-lsp) use to complete and
check scripts. The files are generated from the C++ code, never written by hand, so that they follow the API when it
changes.
Status: design, not implemented yet. The steps are listed in Plan.
Goals¶
- One source of truth: the C++ code that binds a function or a type to Luau also describes its type.
- Two files:
engine.d.luau: the API written in C++ (engine.ecs.*,Entity,ComponentView,Commands, thectxof systems...). It is the same for every game, generated by a tool and committed.game.d.luau: the data of one game (its components, resources, events and thectxof each of its systems). It depends on the plugins the game adds and the scripts it loads, so the game process writes it.
- Deterministic output: declarations are sorted by name, so a change of API gives a readable diff.
Not a goal for now: a CI job that checks the committed engine.d.luau is up to date or type-checks the scripts.
Approach: record at registration¶
Every binding already goes through a small set of calls: LibraryBuilder::function(), Usertype::constructor(),
getter(), setter(), method(), index(), newIndex(), eq(), tostring(), and globals().registerLibrary().
Each of them also records a description of what it registers in a types::TypeRegistry owned by the Runtime. A
types::DefinitionWriter prints what was recorded.
Rejected alternatives:
- Parsing the C++ (libclang): a heavy dependency, fragile, and blind to the names, which are runtime strings
(
"component","Entity"). - A separate description of the API (a hand-written file or a DSL): a second source of truth.
Recording at runtime is also what makes game.d.luau possible, since script components only exist once the scripts
ran.
Phase 1: engine.d.luau¶
Inferred types¶
detail::FunctionTraits already splits a handler into its return type and its parameters. A trait,
types::TypeOf<T>, gives the Luau type of each:
| C++ | Luau |
|---|---|
bool, double, std::string, Value::Vector, Value::Nil |
boolean, number, string, vector, nil |
std::optional<U> |
U? |
void (return) |
() |
T, T&, const T& where T was registered with usertype<T>("Name") |
Name |
Table, FunctionRef, Userdata, Value |
any |
ValuePack (return) |
...any |
A usertype is resolved by std::type_index when the file is written, not when it is recorded, so a type may be used
before it is registered. A type that is still unregistered when the file is written is an error of the writer.
TypeOf<T> can be specialized for other types.
Signatures¶
Two things cannot be inferred:
- Parameter names: C++ cannot reflect the names of a lambda's parameters.
- What a dynamic type really is: a
specparameter is aluau::Table, but scripts must pass aSystemSpec.
Every registration call takes an optional, trailing types::Signature:
struct Signature {
std::vector<std::string> params; ///< One per parameter, "name" or "name: Type".
std::string returns; ///< Return type, overriding the inferred one; empty to infer it.
std::string doc; ///< Documentation comment, written above the declaration.
};
- A bare name (
"name") keeps the inferred type,"name: Type"overrides it. paramsis empty, or has exactly one entry per parameter of the handler (withoutselffor methods, getters and setters). A mismatch is API misuse:exceptions::InvalidOperation, thrown when the handler is bound. Without names, parameters are writtenarg1,arg2...- For
index(),returnsis the type of the values it reads: the class gets the indexer[string]: <returns>.
ecsBuilder.function("system", {.params = {"name", "spec: SystemSpec"}, .doc = "Declares a script system."})
.bind([this](const std::string& name, const luau::Table& spec) { declareSystem(name, spec); });
entityType->getter("bits", bitsOf, {.doc = "Raw bits of the entity."});
viewType->index(readField, {.returns = "any"});
A struct, rather than chained calls (.params(...).doc(...)), because:
- It works the same way on
Usertype, wheremethod()returns theUsertypeto chain the next member. A chained.params()would describe "the last member added", which is hidden state. - It keeps the type metadata out of
Function, which is also used outside libraries (Function::fromRef). - It is plain data: the registry stores it as is.
Type aliases¶
Named shapes are declared as Luau text, next to the C++ code that reads them, so that both change together:
// Next to SystemDeclarationReader
runtime.types().alias("SystemSpec",
"{ query: {string}?, resources: {string}?, events: {string}?, run: (ctx: Context) -> () }");
Text rather than a C++ builder of types: it is shorter, and it reads like the generated file.
Changes to the binding API¶
Two changes make inference cover most of the API:
Function::bindaccepts the same handlers asUsertype::method: returningvoid, aValuePack, or a value aValuecan be built from. Today it only acceptsValuePack, so handlers that return nothing returnluau::ValuePack{}, which would be typed...any.- Handlers may take a registered usertype as
T&orconst T&(beyond theselfof methods). The library checks the userdata's type at call time and raises a Luau error on a mismatch. Today usertypes arrive as an untypedUserdatathat each handler unwraps.
Values set on a library table with Table::set (such as engine.isClient) are not recorded. LibraryBuilder gets a
call to add a constant with its Signature, so that they are.
Output¶
-- Generated from the C++ bindings. Do not edit.
type SystemSpec = { query: {string}?, resources: {string}?, events: {string}?, run: (ctx: Context) -> () }
declare class Entity
bits: number
function __tostring(self): string
end
declare engine: {
isClient: boolean,
isServer: boolean,
isDebug: boolean,
ecs: {
component: (name: string, spec: ComponentSpec) -> (),
system: (name: string, spec: SystemSpec) -> (),
plugin: (name: string, spec: PluginSpec) -> (),
addPlugin: (name: string) -> (),
},
}
- A usertype is a
declare class: getters and setters are fields, methods takeself,eqandtostringare__eqand__tostring, andindex()adds an indexer. - A usertype with a constructor also gets a global table:
declare Name: { new: (...) -> Name }. - A library registered as a global is a
declare name: { ... }of its members, sub-libraries nested.
Code layout¶
In luau, a types/ folder (rtype::luau::types):
TypeOf.hpp/.tpp: the trait, with the specializations of the table above.Signature.hpp: the struct.Declarations.hpp: a set of declarations (aliases, classes, globals). The writer takes one, so thatscriptingcan build its own forgame.d.luau, separate from the runtime's.TypeRegistry.hpp: the declarations of aRuntime, filled byLibraryBuilderandUsertype, reached throughRuntime::types().DefinitionWriter.hpp: turns aDeclarationsinto the text of a.d.luau. It returns aResult, failing on an unresolved usertype.
luau knows nothing about the ECS. scripting adds the signatures and aliases of its bindings.
A luau-types tool target builds a ScriptHost without running any script, then writes engine.d.luau. An xmake
task runs it. The output is committed.
Phase 2: game.d.luau¶
What it types¶
Game data is addressed by name: ctx.resources["rtype.Input"], ctx.events.read("rtype.Collision"),
ctx.commands:add(entity, "game.Shield", {...}). game.d.luau maps each name to its type:
- Components, script and C++ alike, since scripts see both through
ComponentView: one type per component, built from its fields. ctx.resources: a table type with quoted keys, one per plain-data resource (the onesctxhands out).ctx.events.readandctx.events.send: one overload per event, selected by its name.ctx.commands:addandspawn: the table of initial values each component takes.- The
ctxof each system: itsqueryyields the entity, then one view per term, in the order of the terms.
-- Generated from the game's ECS registry. Do not edit.
type GameShield = { strength: number, regen: number, owner: Entity }
type RtypeTransform = { position: vector, rotation: number, scale: vector }
type GameResources = { ["rtype.Input"]: RtypeInput, ["rtype.Time"]: RtypeTime }
type GameEvents = {
read: ((name: "rtype.Collision") -> () -> RtypeCollision?) & ((name: "game.Hello") -> () -> GameHello?),
send: ((name: "game.Hello", data: GameHello) -> ()) & ...,
}
type GameShieldRegenContext = Context & {
query: () -> (Entity?, GameShield),
resources: GameResources,
events: GameEvents,
}
Sources¶
- The ECS registry: the name, kind and
FieldDescriptions of every component, resource and event, the same for C++ and script types. - The declarations of script systems: terms, resources and events.
ScriptHostkeeps a copy of them afterregisterSystem(), which consumes them today.
Field types¶
The Luau type of each ecs::FieldType is defined in view/FieldAccess.cpp, next to readField() and writeField(),
so that the types match what the conversions accept: kBool is boolean, integers and floats are number, kEntity
is Entity, kAssetId is AssetId, vectors are what readVector() returns.
Names¶
A dotted name becomes PascalCase segments joined together: game.Shield is GameShield, and the context of the system
game.ShieldRegen is GameShieldRegenContext. Two names that give the same type name (game.Shield and
gameShield) are an error, never merged.
Code layout and emission¶
scripting/types/(rtype::scripting::types) builds aluau::types::Declarationsfrom the ECS registry and the kept system declarations, and writes it with theluauwriter.- The types of
game.d.luaurefer to those ofengine.d.luau(Entity,Context...) by name; both files are given to luau-lsp. - The C++ components come from the plugins the game adds, so only the game process knows all of them.
ScriptingPlugingets an option to writegame.d.luauonce the App is complete: every plugin added and every script loaded.
Plan¶
Each step builds on the previous one and can be its own pull request.
| # | Module | Change |
|---|---|---|
| 1 | luau |
bind accepts void and plain values; the scripting handlers drop return luau::ValuePack{} |
| 2 | luau |
Usertype parameters (T&, const T&), checked through the UsertypeRegistry |
| 3 | luau |
types/: TypeOf, Signature, Declarations, TypeRegistry filled by LibraryBuilder and Usertype, the arity check, library constants |
| 4 | luau |
DefinitionWriter, tested against expected text |
| 5 | scripting |
Signatures and aliases (ComponentSpec, SystemSpec, PluginSpec, Context, Commands...) next to their readers |
| 6 | tool | luau-types target and xmake task; engine.d.luau generated and committed |
| 7 | — | Check with luau-lsp, on a hand-written game.d.luau: overloads selected by a string literal, quoted keys in table types, global types of a definition file seen by scripts |
| 8 | scripting |
Luau types of the field types next to the conversions; ScriptHost keeps its system declarations |
| 9 | scripting |
types/: component, resource and event types, GameResources, GameEvents, the context of each system |
| 10 | scripting |
ScriptingPlugin option to write game.d.luau, with tests |
Open points¶
- How a system's
rungets its typedctx:- an annotation in the script,
run = function(ctx: GameShieldRegenContext), which always works; - or no annotation, with one overload of
engine.ecs.systemper system name, each with its ownruntype. This needs the type checker to select overloads by a string literal reliably; step 7 tells whether it does.
- an annotation in the script,
- How a game turns on writing
game.d.luau:- a command-line flag (
rtype_duel client --emit-types), explicit and leaving normal runs untouched (preferred); - a setting in
project.luau; - on every debug run.
- a command-line flag (