# Proto Socket Protocol Binary TCP and WebSocket protocol using Protocol Buffers for message framing and type-based routing. Designed for bidirectional, heterogeneous communication across languages and platforms. Current protocol version: `0.1` --- ## Scope This protocol intentionally defines only the transport-level contract shared across implementations. - In scope: TCP/WebSocket framing, protobuf payload transport, `typeName` routing, `nonce`, `responseNonce`, TLS/WSS transport variants, and heartbeat - Out of scope: authentication, session lifecycle, agent semantics, chat semantics, game rules, room logic, and other application-specific behavior - Higher-level concerns should be implemented as messages and conventions on top of this protocol, not inside the protocol core --- ## Transport별 Wire Format ### TCP / SSL+TCP ``` ┌──────────────────────────────────────────┐ │ Header (4 bytes, big-endian integer) │ ← PacketBase payload length ├──────────────────────────────────────────┤ │ PacketBase (protobuf, N bytes) │ ← typeName + nonce + data └──────────────────────────────────────────┘ ``` #### Header - 4 bytes, big-endian positive integer. Dart reads/writes it as `int32`; Go reads/writes the same 4 bytes as `uint32`. - Value: byte length of the following `PacketBase` protobuf payload - Value of `0`: reserved / no-op, receiver clears buffer - Implementations may apply safety limits to payload size. The current Dart, Go, and Kotlin implementations reject TCP packets larger than 64 MiB. ### WebSocket / WSS ``` ┌──────────────────────────────────────────┐ │ PacketBase (protobuf, N bytes) │ ← typeName + nonce + data └──────────────────────────────────────────┘ ``` - Length 헤더 없음 — WebSocket 프로토콜이 메시지 경계를 보장한다 - Binary frame 사용 (text frame 아님) - 연결: `ws://host:port/path` (plain) / `wss://host:port/path` (TLS) ### PacketBase ```protobuf message PacketBase { string typeName = 1; // protobuf full message name used for routing int32 nonce = 2; // monotonically increasing per-connection counter bytes data = 3; // serialized inner message bytes int32 responseNonce = 4; // > 0: this packet is a response to request with that nonce } ``` | Field | Description | |-------|-------------| | `typeName` | Protobuf full message name used as the language-agnostic routing key | | `nonce` | Starts at 1, increments by 1 per send call per connection | | `data` | Protobuf-serialized bytes of the inner message | | `responseNonce` | `0` for regular messages. Set to the request's `nonce` when sending a response | --- ## Built-in Messages ### HeartBeat ```protobuf message HeartBeat {} ``` Automatically handled by the framework. Applications do not need to register or handle this manually. --- ## Heartbeat Lifecycle ``` Side A Side B │ │ │── (no activity for intervalTime) ──────▶│ │ send(HeartBeat) │ │ │── onHeartBeat() called │ │ send(HeartBeat) [response] │◀─────────────────────────────────────────│ │ onHeartBeat() called │ │ _waitingHeartbeatResponse = false │ │ timer reset │ ``` - After any received message → heartbeat interval timer resets - If no data received within `heartbeatIntervalTime` seconds → send `HeartBeat` - If no response within `heartbeatWaitTime` seconds → `onDisconnected()` → `dispose()` - A side that is already waiting for a heartbeat response consumes the echo and does not send another heartbeat. A side that is not waiting replies with `HeartBeat`. Changing heartbeat timing semantics, response behavior, or disconnect rules is a breaking protocol candidate. Document the compatibility plan in [VERSIONING.md](VERSIONING.md) before changing it. --- ## Optional Capability Announcement Implementations may define an application-level `ProtoSocketHello` message, or an equivalent project-specific hello message, to announce protocol and implementation capabilities after connection open. This message is optional and non-blocking in protocol `0.1`: - It is a normal `PacketBase` payload sent through the existing `typeName` routing path. - It must not be required before normal messages, requests, responses, or heartbeats can flow. - Peers that do not recognize its `typeName` may ignore it according to their normal unknown-message policy. - Implementations must not treat a missing hello message as a protocol error in `0.1`. Recommended payload fields: ```protobuf message ProtoSocketHello { string protocolVersion = 1; // e.g. "0.1" repeated string capabilities = 2; // stable capability tokens map implementation = 3; // optional language/library metadata } ``` Capability tokens should be stable ASCII strings such as `std_error.v1` or `ws.wss`. Application-specific tokens should use a project prefix to avoid collisions. Promoting this message to a required handshake is a future protocol-version decision and is not part of protocol `0.1`. --- ## typeName Convention `typeName` uses the protobuf full message name on the send side. For messages in the common proto package, the canonical values are `proto_socket.PacketBase`, `proto_socket.HeartBeat`, and `proto_socket.TestData`; `PacketBase.typeName` normally carries the inner message name such as `proto_socket.HeartBeat` or `proto_socket.TestData`. On the receive side, use the same canonical value as the registration key. Proto package names are part of the wire identity. Package names must use lowercase snake_case segments joined by dots, for example `proto_socket` or `example_chat.v1`. Language-specific namespace options such as Go `go_package` or Java/Kotlin package names do not change the wire `typeName`. The canonical package for `proto/message_common.proto` is `proto_socket`. | Language | Registration key | |----------|-----------------| | Dart | protobuf descriptor/native full name when available, otherwise static metadata aligned with the canonical full name | | Go | `string(proto.MessageName(m))` via `TypeNameOf(m)` | | C# | protobuf descriptor full name — verify matches the canonical full name | | Kotlin | `descriptorForType.fullName` via `typeNameOf(m)` | | Swift | `Message.protoMessageName` or equivalent descriptor full name — verify matches the canonical full name | | TypeScript | local `MessageType.typeName` value — verify matches the canonical full name | | Python | `MessageClass.DESCRIPTOR.full_name` | Implementations may accept legacy simple-name values such as `TestData` and `HeartBeat` for receive compatibility, but new sends must use the canonical full proto name once the implementation supports this convention. **Important**: Verify typeName consistency across languages before connecting heterogeneous clients. Changing the proto package or `typeName` derivation rule is a breaking protocol change unless all released implementations keep a compatibility path. See [VERSIONING.md](VERSIONING.md). --- ## nonce - Starts at `1` per connection - Increments by `1` on every send call (including requests and responses) - If a candidate nonce matches a currently pending request correlation key (`sendRequest` in-flight `nonce`), implementations must skip the candidate and try the next nonce. - If every positive `int32` nonce value is currently pending, outbound send should fail with an explicit error instead of reusing an active pending nonce. - Maximum nonce is `2,147,483,647` (`int32` max), the lowest common signed integer max across supported protocol fields - After issuing `2,147,483,647`, implementations reset the internal counter to `0`; the next emitted packet nonce is `1` - `0` is reserved and must not be emitted as `PacketBase.nonce`, because `responseNonce == 0` means "not a response" - Used for request-response correlation via `responseNonce` Changing `nonce` or `responseNonce` semantics is a breaking protocol change unless older peers can continue to correlate requests and responses correctly. --- ## Request-Response Pattern Fire-and-forget (`send`) and request-response (`sendRequest`) coexist on the same connection. ``` Requester Responder │ │ │── PacketBase { nonce=5, responseNonce=0, │ │ typeName="GetUser", │ │ data=... } ─────────────▶│ │ │── addRequestListener │ │ handler called → returns UserData │◀─ PacketBase { nonce=12, responseNonce=5,│ │ typeName="UserData", │ │ data=... } ──────────────│ │ sendRequest Future completes │ ``` **Rules:** - `responseNonce == 0`: regular message or outgoing request → routed to `addListener` or `addRequestListener` - `responseNonce > 0`: response → matched to the pending `sendRequest` by `responseNonce` - A response must also have the expected `typeName` for the waiting `sendRequest`; mismatches are protocol errors - Both sides can be requester and responder simultaneously on the same connection - For a given `typeName` on one connection, register it with either `addListener` or `addRequestListener`, not both. Mixed registration is ambiguous and rejected by the Dart and Go implementations. ### Standard Error Response Convention Implementations may define a `ProtoSocketError` message, or an equivalent project-specific error message, for request handlers that fail before producing the expected response. Recommended payload fields: ```protobuf message ProtoSocketError { string code = 1; // stable machine-readable error code string message = 2; // safe human-readable message string detail = 3; // optional non-sensitive detail string debug = 4; // optional local/debug-only detail } ``` When used as a request failure response: - The packet's `responseNonce` should be set to the failing request's `nonce`. - The packet's `typeName` should identify the error message, for example `proto_socket.ProtoSocketError` if this becomes a shared common message in a future schema. - A requester that understands the convention should complete the pending request with a typed error instead of waiting for timeout. - A requester that does not understand the convention may see an expected-response type mismatch or may continue to timeout, depending on its current response handling. Protocol `0.1` does not require every request handler failure to send an error response. Adding a common error message to the canonical proto schema requires parser map updates and cross-language tests before it can become a shared compatibility contract. --- ## Receive Ordering and Backpressure ### Inbound Queue Each connection owns one bounded FIFO inbound queue between the frame parser and stateful dispatch. - The read loop parses complete frames and enqueues packet envelopes in arrival order. - Enqueue order is the authoritative dispatch order; implementations must not reorder enqueued packets before dispatch. - Recommended default capacity: **64 packets**. Implementations may choose a different bound but must document it. - When the inbound queue is full, the implementation applies backpressure by suspending frame reads from the transport until space is available. The underlying TCP flow control propagates this pressure to the sender. - **WebSocket Runtime Limitations**: - Some WebSocket client runtimes (such as Browser WebSocket and Kotlin OkHttp WebSocket) do not support transport-level pause/resume (suspending frame reads). - To prevent unbounded promise/coroutine staging backlog under heavy load, these runtimes must implement a **Bounded Input Gate** with a documented limit (e.g., **64 packets** max staging boundary). - If the input gate capacity is exceeded (i.e. the dispatcher queue is full and the staging buffer overflows the limit), the runtime must apply a strict discard/backpressure policy: **disconnect the connection (forcefully close / destroy)** immediately to protect resources from memory exhaustion. ### Receive Coordinator A single receive coordinator dequeues packets one at a time and dispatches them: 1. If `responseNonce > 0`: match to the pending `sendRequest` and complete the waiting future/callback. 2. If `responseNonce == 0` and the `typeName` is registered with `addRequestListener`: invoke the handler, then enqueue the automatic response through the write queue in the same coordinator loop iteration. 3. If `responseNonce == 0` and the `typeName` is registered with `addListener`: invoke the listener. 4. `HeartBeat` is handled internally before any listener dispatch. **Ordering guarantee**: for a given connection, listeners and request handlers are invoked strictly in packet arrival order. Automatic responses are enqueued into the write queue in the same order their triggering requests were dispatched. ### Response Correlation Path Packets with `responseNonce > 0` complete an already-pending request and do not invoke listeners or request handlers. Implementations may therefore complete those pending requests through a direct correlation path before or instead of the listener/request-handler FIFO queue. This response path does not weaken the listener/request-handler ordering guarantee: packets that dispatch to `addListener` or `addRequestListener` must still be processed in arrival order, and automatic responses produced by request handlers must still be enqueued in request-dispatch order. Implementations that enqueue response packets in the same inbound queue must produce equivalent pending-request completion semantics. ### Close, Drain, and Cancel - When `Close()` is called, the implementation stops accepting new inbound packets and drains the already-enqueued packets in the inbound queue before tearing down the receive coordinator. - In-flight `sendRequest` futures that have not yet received a response are cancelled with an error when the connection closes. - The inbound queue is not drained after `Close()` if the transport signals an error (e.g., read error, heartbeat timeout). In that case the queue is discarded immediately. - Implementations must ensure `Close()` is idempotent (see `connCloseOnce` pattern in `PORTING_GUIDE.md`). ### Worker Gateway and Internal seq - Implementations may introduce an optional parallel worker gateway for decoding or preprocessing large/burst packets. - In such multi-worker designs, the gateway input/output carries an internal `seq` assigned by the read loop to track the arrival order. - The `seq` sequence is purely an internal implementation detail and **NOT** part of the wire format (it is not added to `PacketBase` or the transport framing). - The gateway must perform only stateless operations (e.g., raw bytes decoding or preprocessing) and is prohibited from possessing stateful dispatcher logic. - The receive coordinator remains the sole owner of ordering restoration (sorting by `seq`), pending response matching, listener/request handler dispatching, and automated response enqueuing. --- ## Example Packet (hex) Sending `HeartBeat {}`: ``` 00 00 00 18 ← header: PacketBase length = 24 bytes 0A 16 70 72 6F 74 6F 5F 73 6F 63 6B 65 74 2E 48 65 61 72 74 42 65 61 74 ← PacketBase { typeName: "proto_socket.HeartBeat" } ``` --- ## Multi-language Implementations | Language | Status | Path | |----------|--------|------| | Dart | Available | `dart/` | | C# (Unity) | Planned / On hold | `csharp/` | | Kotlin | Available | `kotlin/` | | Swift | Planned / On hold | `swift/` | | Go | Available | `go/` | | TypeScript | Available | `typescript/` | | Python | Available | `python/` | --- ## Proto Source `proto/message_common.proto` is the canonical packet definition. All language implementations generate bindings from it. The canonical package for this proto is `proto_socket`. New application protos must choose package names using lowercase snake_case segments joined by dots, and those package names define the wire `typeName` prefix. Language-specific generation options such as `go_package` and Java/Kotlin package options are kept in each language's own proto copy under `go/packets/` and `kotlin/src/main/proto/`. These options affect generated code namespaces only; they do not change the protobuf package or wire identity. Keep those copies' message schemas in sync with `proto/message_common.proto` before regenerating language bindings.