Proto Socket

Binary socket protocol library for bidirectional, heterogeneous communication across languages and platforms.

Built on Protocol Buffers with TCP length-prefixed framing, WebSocket binary frames, type-based message routing, request-response correlation, and built-in heartbeat.

---

Current Status

Proto Socket currently treats the Dart, Go, Kotlin, Python, and TypeScript implementations as the available compatibility set for protocol 0.1.

The project is in a stabilization and maintenance pass before adding more language targets or publishing package registry releases. Git-based consumption from this repository is the primary supported workflow for now. C# and Swift remain planned targets, but implementation work is deferred until there is concrete demand.

Use the local validation matrix as the stability gate for protocol or implementation changes:

bash agent-ops/skills/project/run-proto-socket-test-matrix/scripts/run_matrix.sh --all

Protocol or public API changes should be treated as compatibility work: update the relevant docs, regenerate protobuf bindings when needed, and keep the full validation matrix passing.

---

Design Principles

• Keep the core transport layer thin and stable
• Provide only the minimum common foundation for cross-language communication
• Standardize framing, serialization, routing, request-response correlation, and heartbeat
• Treat Protocol Buffers as the core protocol dependency. Other runtime dependencies should be native-first, narrowly scoped, and proportional to the feature they implement
• Do not embed application semantics into the protocol core
• Domain-specific concerns such as auth, session, agent workflow, chat features, and game logic belong in upper-layer implementations

---

Protocol

See PROTOCOL.md for the full wire format specification.

[4-byte big-endian length] [PacketBase protobuf bytes]

For WebSocket/WSS transports, each binary frame contains one PacketBase protobuf payload without the TCP length header.

Protocol compatibility is tracked separately from language package versions. See VERSIONING.md. Even while packages are not published, the checked-in implementations must preserve the documented protocol contract.

---

Implementations

Language	Status	Path	Use case
Dart	Available	dart/	Flutter, Dart server
Kotlin	Available	kotlin/	Android, JVM
Go	Available	go/	Server, tooling, scripting
TypeScript	Available	typescript/	Browser, Node.js
Python	Available	python/	Server, tooling, scripting
C#	Planned / deferred	csharp/	Unity, .NET
Swift	Planned / deferred	swift/	iOS, macOS

New language implementations should start from PORTING_GUIDE.md and the templates in agent-ops/skills/project/add-proto-socket-crosstest-language/templates/. Mark an implementation available only after its same-language tests and cross-language tests pass.

Keep implementation status here as the public snapshot. Current work context is selected through agent-ops/roadmap/current.md, and broader roadmap direction lives in agent-ops/roadmap/ROADMAP.md.

---

Common Commands

Purpose	Command	Notes
Full local validation	bash agent-ops/skills/project/run-proto-socket-test-matrix/scripts/run_matrix.sh --all	Proto sync, same-language tests, and all available cross-language checks
Proto schema sync check	tools/check_proto_sync.sh	Verifies language proto copies match proto/message_common.proto except allowed language options
Regenerate protobuf bindings	tools/generate_proto.sh	Run after changing the canonical proto schema
Dart tests	cd dart && dart pub get && dart test && dart compile js test/browser_ws_import_compile.dart -o /tmp/proto_socket_browser_ws_import_compile.js	Dart VM/IO tests and web import compile check
Go tests	cd go && go test ./...	Go same-language checks
Kotlin tests	cd kotlin && ./gradlew test	Kotlin same-language checks
Python tests	cd python && python3 -m pytest -q	Python same-language checks
TypeScript checks	cd typescript && npm run check && npm test	TypeScript type check and tests

---

Repository Structure

Path	Role
proto/	Canonical protobuf schema source
dart/	Dart / Flutter implementation and tests
go/	Go implementation, tests, and Go-led crosstests
kotlin/	Kotlin / JVM implementation and crosstests
python/	Python implementation and crosstests
typescript/	TypeScript implementation and crosstests
tools/	Proto generation and sync helpers
examples/	Consumer-oriented examples
agent-ops/	Project rules, roadmap, and repeatable agent workflows

---

Quick Start (Dart)

import 'package:proto_socket/proto_socket.dart';

// 1. Define your message in message_common.proto, generate with protoc

// 2. Implement a client
class MyClient extends ProtobufClient {
  MyClient(Socket socket) : super(socket, 30, 10, {
    MyMessage.getDefault().info_.qualifiedMessageName: MyMessage.fromBuffer,
  });
}

// 3. Implement a server
class MyServer extends ProtobufServer {
  MyServer() : super('0.0.0.0', 9090, (socket) => MyClient(socket));

  @override
  void onClientConnected(ProtobufClient client) {
    client.addListener<MyMessage>((msg) => print('Received: ${msg}'));
  }
}

// 4. Start
final server = MyServer();
await server.start();

// 5. Connect and send
final socket = await Socket.connect('localhost', 9090);
final client = MyClient(socket);
await client.send(MyMessage()..text = 'hello');

---

Adding Message Types

Edit the canonical proto at proto/message_common.proto, then regenerate all checked-in bindings:

tools/generate_proto.sh
tools/check_proto_sync.sh

The Go and Kotlin proto copies are allowed to keep only language-specific options such as option go_package or Java package/class options. tools/check_proto_sync.sh fails with a diff when their message schema drifts from proto/message_common.proto.

---

Quick Start (Go)

package main

import (
	"context"
	"net"
	"time"

	"google.golang.org/protobuf/proto"

	protoSocket "git.toki-labs.com/toki/proto-socket/go"
	"git.toki-labs.com/toki/proto-socket/go/packets"
)

func parserMap() protoSocket.ParserMap {
	return protoSocket.ParserMap{
		protoSocket.TypeNameOf(&packets.TestData{}): func(b []byte) (proto.Message, error) {
			m := &packets.TestData{}
			return m, proto.Unmarshal(b, m)
		},
	}
}

func main() {
	ctx := context.Background()

	server := protoSocket.NewTcpServer("127.0.0.1", 9090, func(conn net.Conn) *protoSocket.TcpClient {
		return protoSocket.NewTcpClient(conn, 30, 10, parserMap())
	})
	server.OnClientConnected = func(client *protoSocket.TcpClient) {
		protoSocket.AddRequestListenerTyped[*packets.TestData, *packets.TestData](
			&client.Communicator,
			func(req *packets.TestData) (*packets.TestData, error) {
				return &packets.TestData{Index: req.GetIndex(), Message: "echo: " + req.GetMessage()}, nil
			},
		)
	}
	if err := server.Start(ctx); err != nil {
		panic(err)
	}
	defer server.Stop()

	client, err := protoSocket.DialTcp(ctx, "127.0.0.1", 9090, 30, 10, parserMap())
	if err != nil {
		panic(err)
	}
	defer client.Close()

	res, err := protoSocket.SendRequestTyped[*packets.TestData, *packets.TestData](
		&client.Communicator,
		&packets.TestData{Index: 1, Message: "hello"},
		2*time.Second,
	)
	if err != nil {
		panic(err)
	}
	println(res.GetMessage())
}

Go also provides:

• TCP: NewTcpServer, DialTcp, NewTcpServerTLS, DialTcpTLS
• WebSocket: NewWsServer, DialWs, NewWsServerTLS, DialWss
• Shared helpers: Send, SendRequest, AddListenerTyped, AddRequestListenerTyped, Broadcast

If you want a copyable scaffold for using this module from another repository, see examples/go-module-consumer/.

---

Running Tests

Local commands are documented here for development and troubleshooting. The repeatable repository-local validation entry point is:

bash agent-ops/skills/project/run-proto-socket-test-matrix/scripts/run_matrix.sh --all

It runs proto schema sync, same-language tests, and cross-language checks, then prints PASS/FAIL tables with reproduction commands for failures. Use --proto, --unit, or --cross to run a smaller slice.

External CI runner integration is deferred until the parent ../oto project standardizes its execution path. Until then, the local matrix above is the project stability gate.

Proto schema sync:

tools/check_proto_sync.sh

Same-language checks:

cd dart
dart pub get
dart test
dart compile js test/browser_ws_import_compile.dart -o /tmp/proto_socket_browser_ws_import_compile.js

cd go
go test ./...

cd kotlin
./gradlew test

cd python
python3 -m pytest -q

cd typescript
npm run check
npm test

Cross-language checks:

cd go
go run ./crosstest/go_dart.go
go run ./crosstest/go_kotlin.go
go run ./crosstest/go_python.go
go run ./crosstest/go_typescript.go

cd dart
dart run crosstest/dart_go.dart
dart run crosstest/dart_kotlin.dart
dart run crosstest/dart_python.dart
dart run crosstest/dart_typescript.dart

cd kotlin
./gradlew run -PmainClass=com.tokilabs.proto_socket.crosstest.KotlinDartKt
./gradlew run -PmainClass=com.tokilabs.proto_socket.crosstest.KotlinGoKt
./gradlew run -PmainClass=com.tokilabs.proto_socket.crosstest.KotlinPythonKt
./gradlew run -PmainClass=com.tokilabs.proto_socket.crosstest.KotlinTypescriptKt

cd python
python3 crosstest/python_dart.py
python3 crosstest/python_go.py
python3 crosstest/python_kotlin.py
python3 crosstest/python_typescript.py

cd typescript
./node_modules/.bin/tsx crosstest/typescript_dart.ts
./node_modules/.bin/tsx crosstest/typescript_go.ts
./node_modules/.bin/tsx crosstest/typescript_kotlin.ts
./node_modules/.bin/tsx crosstest/typescript_python.ts

---

Work Context

Before making code or protocol changes, check the project rules in agent-ops/rules/project/rules.md and any matching domain rule listed there. The active roadmap window is in agent-ops/roadmap/current.md.

Stabilization boundaries:

• Keep protocol 0.1 behavior stable unless compatibility work is explicitly requested.
• Maintenance-mode work may include bug fixes, documentation corrections, test hardening, and compatibility-preserving implementation fixes.
• Resume C#/Swift, package registry release, or protocol/API change work only when there is concrete consumer demand, a compatibility plan, and a passing full validation matrix.
• Do not connect external CI/CD runners from this repository until the parent ../oto workflow is decided.