Network Overview¶
The short version of the network design, for a first read. Every section links to the full document,
design.md, which has the reasons behind each choice. The bytes on the wire, precise enough to write
another client, are specified in protocol.md.
In one paragraph¶
The server decides everything; clients only send their input states (buttons and axes) and permitted
intentions, never world state, and display what the server sends back. Everything goes over UDP on a
single port, with a small reliability layer written by us on top of Asio. The server runs one thread for the network and one thread per game room. Each room is a complete
ECS App with its own world and Luau VM. About 60 times per second, each client receives a snapshot: the
entities that appeared, changed or disappeared since the last snapshot it confirmed.
The pieces¶
flowchart BT
net["network (lib)<br/>sockets, connections, reliability<br/>knows nothing about the ECS"]
ecs["ecs (lib)"]
repl["replication (lib)<br/>ECS ↔ network:<br/>snapshots, inputs, manifest"]
engine["engine (lib)"]
client["client (executable)<br/>+ window, Vulkan"]
server["server (executable)<br/>headless"]
net --> repl
ecs --> repl
ecs --> engine
engine --> client
engine --> server
repl --> client
repl --> server
network: moves bytes between machines. It handles connections, packet numbering, acks and resending. It never includes an ECS header.replication: the only code that knows both the ECS and the network. It turns the world into snapshots, applies them on the client, and turns client inputs intoPlayerInputcomponents.clientlinks the window and the renderer.servernever does, so it runs on a machine with no screen or GPU.
Details: Targets and dependencies.
How you play¶
| Mode | What runs |
|---|---|
| Dedicated server | server hosts every room on port 4242. |
| Client | client connects to a server. |
| Host a game | client starts a server thread inside itself and connects to 127.0.0.1; friends connect to the host's IP. |
| Single player | client with no network at all (the ECS StandalonePlugin). |
Details: Topology and run modes.
Threads: one owner for everything¶
Room thread (one per room) Network thread (owns the socket and all connections)
────────────────────────── ────────────────────────────────────────────────────
read inbox ◄──────────────────────────────── receive, check, split into messages
run gameplay
write outbox ────────────────────────────────► pack messages, number them, send
- A room never touches the socket, and the network thread never touches a world. They only exchange messages through two queues (a mutex and a vector each).
- One thread per room: a slow script in one room does not slow down the others.
- The client has no network thread: it polls the socket at the start of each frame.
This is what the subject asks for: the server never waits for clients, and the game keeps running whatever they do.
Details: Threads.
Connecting¶
sequenceDiagram
participant C as Client
participant S as Server
C->>S: connect request (protocol version)
S->>C: challenge (random number)
C->>S: same number back
S->>C: accepted (session token, resume token, client id)
S->>C: manifest (every component, packet, capsule, input action)
C->>S: ready → lobby
- Challenge: someone pretending to have another player's address never receives the number, so they cannot connect in their name.
- Session token: a random number carried by every later packet. A packet with the wrong token is dropped.
- Manifest: the list of every component and packet the server knows, each with a hash of its layout. If the client's scripts differ ("update your game"), the client refuses to connect instead of misreading data later. After that, the wire uses 2-byte ids instead of names.
- Crash detection: a client silent for 2 seconds is considered lost. It then has a grace period (60 s by
default, a setting) to reconnect; if it does not, it is removed and the other players of its room are told
(
kClientLeft).
Details: Connection lifecycle, Manifest.
Reconnecting¶
A lost wifi must not end the game. After 2 s of silence, the server keeps the player's session lingering instead of closing it, and the client keeps retrying with a secret resume token it received when it connected.
- On a valid token the server rebinds the session to the client's new address (a network change is fine), gives it fresh session and resume tokens, and sends the whole world again as spawns. The old tokens are dead.
- After the grace period (
reconnectGrace, 60 s by default,0to disable) the session is closed as for any departure. - The other players of the room are told by default (
kClientLost,kClientReconnected), and every side gets an ECS event (ClientLost,ClientReconnected;ConnectionLost,Reconnectedfor the client's own connection) that the game can use or ignore. - The engine is generic: it keeps the ship in the world with a neutral input. Whether it freezes, is invulnerable or disappears is up to the game.
Details: Reconnecting after a connection loss.
Packets and reliability¶
Every datagram is at most 1200 bytes (bigger ones would be fragmented by the network, and lose everything if one piece is lost: the protocol never fragments, senders split large reliable payloads into messages, and an oversized datagram is dropped). It contains a 13-byte header and a list of messages on one of two channels:
| Channel | Guarantee | Used for |
|---|---|---|
| Unreliable | May be lost | Snapshots (a newer one replaces a lost one) and inputs (each datagram repeats the last 3 per-tick samples, so the next one recovers a lost one) |
| Reliable | Arrives exactly once, in order | Manifest, lobby, "player X left", game events like PlayerKilled |
Each datagram confirms the last 33 datagrams it received from the other side (ack and ackBits). Reliable
messages are put again in later datagrams until a datagram carrying them is confirmed. The receiver drops
duplicates by message id.
Details: Transport.
Snapshots: send states, never differences¶
Why not just send "what changed this tick"? Because if that one packet is lost, the change is lost forever:
| Tick | Server sends | Arrives? | Client shows |
|---|---|---|---|
| 100 | "boss HP = 40" | lost | 50 |
| 101+ | nothing about HP (it did not change again) | ✓ | 50 forever |
Instead, the server keeps sending each change until the client confirms it:
| Tick | Client confirmed the boss up to | Server sends | Arrives? |
|---|---|---|---|
| 100 | 98 | "boss HP = 40" | lost |
| 101 | 98 | "boss HP = 40" again | ✓ (the client confirms 101) |
| 103 | 101 | nothing about HP: already confirmed | ✓ |
This works because a snapshot carries states ("HP = 40"), not differences ("HP −10"): receiving one twice does no harm. The client confirms the last snapshot it applied in every input message.
A snapshot holds four lists:
| List | Meaning |
|---|---|
| Spawns | Entities this client has not confirmed yet, with all their components |
| Updates | Components changed since the client last confirmed that entity |
| Removals | Components removed |
| Despawns | Entities destroyed |
Spawns and despawns are in the snapshot, not on the reliable channel: one lost bullet spawn never delays anything else. If a snapshot would be too big, what does not fit simply goes in the next one. (Until deltas land, every snapshot lists every entity: one too big for a message is split into parts of the same tick.)
Bullets are kSpawnOnly: sent once when they appear, then moved by the same system on both sides. With
positions packed into 16-bit integers, a steady-state snapshot of 300 entities fits in one datagram instead of
ten; a burst of new bullets (about 20 bytes each) spreads over the next snapshots.
On the client, entities are drawn about 100 ms in the past, smoothly interpolated between two snapshots, so motion does not stutter. The player's own ship is the exception: the client moves it immediately from the inputs and the server corrects it if they disagree.
Details: Snapshots, Bandwidth estimate.
Inputs¶
- The client sends its held buttons and stick axes, never positions or actions with coordinates.
- Every input message also contains the two previous inputs: if one packet is lost, the next one still has it, with no delay.
- The server applies one input per tick to the player's ship and repeats the last one if nothing arrived.
Shooting is an input, not a packet. The client says "fire is held"; the server decides where the bullet
appears, and everyone sees it as a spawn in the next snapshot. A client that could send PlayerShoot {
origin = ... } could shoot from anywhere.
Details: Inputs, Shooting is an input.
Custom packets (from Luau)¶
For what is not state, like a kill feed, a "ready" button or a chat line:
-- shared/packets/PlayerKilled.luau, loaded by the server and the client
return net.packet("game.PlayerKilled", {
direction = "toClients",
channel = "reliable",
fields = {
{ name = "victim", type = "Entity" },
{ name = "killer", type = "Entity" },
},
})
-- server: send it
ctx.net:sendToRoom("game.PlayerKilled", { victim = ship, killer = enemy })
-- client: read it like any ECS event, inside a system
for packet in ctx.events["game.PlayerKilled"] do
showKillFeed(packet.victim, packet.killer)
end
- Fields are an ordered list, using the same type names as components (
u8,f32,vec2,Entity,string...). - Sending checks every value (
300into au8is a script error); receiving drops anything invalid. - This one is 11 bytes on the wire.
Implemented subset (client → server commands, validated, from Luau): Custom packets. Full design: Custom packets.
Never trust the network¶
- Every byte is read through a bounds-checked reader; a bad packet is dropped, never a crash, a
throwor anassert. - Floats must be finite, counts and string lengths are capped, enum values are checked.
- Each client has rate limits and an error budget; past it, it is disconnected.
- Windows trap: after a client crashes, Windows reports an error on the server's next receive. The server turns that report off and never stops its receive loop on an error, or one crash would cut off every player.
- Encryption is out of scope: the session token stops impersonation, not eavesdropping.
Details: Robustness and security.
What the subject asks, and where it is answered¶
| The subject says | We do |
|---|---|
| Notify clients of spawns, moves, deaths, shots, kills, other players' actions | Snapshots, plus custom packets like PlayerKilled |
| The server is multithreaded and never waits for clients | Network thread plus one thread per room |
| A client crash does not stop the server, and the others are told | 2 s timeout, a grace period to reconnect, then kClientLeft to the room |
| The server is robust | Checked parsing, limits, the Windows fix, fuzz tests |
| Asio or OS sockets | Asio, with our own reliability layer (no ENet) |
Build order¶
network: byte reader and writer, packet numbering, acks, the reliable channel, tested with a fake network that loses and reorders packets.- Connections (challenge, token, timeouts), and the
clientandserverexecutables. replication: manifest and first snapshots, so the client sees the server world.- Inputs: the ship moves.
- Updates, despawns, the size limit,
kSpawnOnly: many bullets. - Lobby and rooms, custom packets, interpolation.
- Client prediction of the local ship, then benchmarks to tune the defaults.
Still to decide¶
The snapshot rate, players per room and the timeouts are settings, tuned later from benchmarks. The local ship is
predicted by the client and corrected by the server (design 7.6),
so it does not wait for the 100 ms display delay. The protocol is specified in protocol.md; what
remains is a few tuning values: see Decisions and open questions. Points where this design differs from the ECS
docs are listed in Differences with the ECS docs.