proto-socket/README.md

Toki Socket

Binary socket protocol library for bidirectional, heterogeneous communication across languages and platforms.

Built on Protocol Buffers with TCP length-prefixed framing, WebSocket binary frames, type-based message routing, request-response correlation, and built-in heartbeat.

---

Design Principles

• Keep the core transport layer thin and stable
• Provide only the minimum common foundation for cross-language communication
• Standardize framing, serialization, routing, request-response correlation, and heartbeat
• Do not embed application semantics into the protocol core
• Domain-specific concerns such as auth, session, agent workflow, chat features, and game logic belong in upper-layer implementations

---

Protocol

See PROTOCOL.md for the full wire format specification.

[4-byte big-endian length] [PacketBase protobuf bytes]

For WebSocket/WSS transports, each binary frame contains one PacketBase protobuf payload without the TCP length header.

Protocol compatibility is tracked separately from language package versions. See VERSIONING.md. Even while packages are not published, the checked-in implementations must preserve the documented protocol contract.

---

Implementations

Language	Status	Path	Use case
Dart	Available	dart/	Flutter, Dart server
C#	Planned	csharp/	Unity, .NET
Kotlin	Planned	kotlin/	Android
Swift	Planned	swift/	iOS, macOS
Go	Available	go/	Server, tooling, scripting
Python	Planned	python/	Server, tooling, scripting
Rust	Planned	rust/	High-performance server, embedded

New language implementations should start from PORTING_GUIDE.md and the templates in templates/language/. Mark an implementation available only after its same-language tests and cross-language tests pass.

---

Quick Start (Dart)

import 'package:toki_socket/toki_socket.dart';

// 1. Define your message in message_common.proto, generate with protoc

// 2. Implement a client
class MyClient extends ProtobufClient {
  MyClient(Socket socket) : super(socket, 30, 10, {
    MyMessage.getDefault().info_.qualifiedMessageName: MyMessage.fromBuffer,
  });
}

// 3. Implement a server
class MyServer extends ProtobufServer {
  MyServer() : super('0.0.0.0', 9090, (socket) => MyClient(socket));

  @override
  void onClientConnected(ProtobufClient client) {
    client.addListener<MyMessage>((msg) => print('Received: ${msg}'));
  }
}

// 4. Start
final server = MyServer();
await server.start();

// 5. Connect and send
final socket = await Socket.connect('localhost', 9090);
final client = MyClient(socket);
await client.send(MyMessage()..text = 'hello');

---

Adding Message Types

Edit the canonical proto at dart/lib/src/packets/message_common.proto, then regenerate all checked-in bindings:

tools/generate_proto.sh
tools/check_proto_sync.sh

The Go proto copy is allowed to keep only its Go-specific option go_package difference. tools/check_proto_sync.sh fails with a diff when the message schema drifts.

---

Quick Start (Go)

package main

import (
	"context"
	"net"
	"time"

	"google.golang.org/protobuf/proto"

	toki "toki-labs.com/toki_socket/go"
	"toki-labs.com/toki_socket/go/packets"
)

func parserMap() toki.ParserMap {
	return toki.ParserMap{
		toki.TypeNameOf(&packets.TestData{}): func(b []byte) (proto.Message, error) {
			m := &packets.TestData{}
			return m, proto.Unmarshal(b, m)
		},
	}
}

func main() {
	ctx := context.Background()

	server := toki.NewTcpServer("127.0.0.1", 9090, func(conn net.Conn) *toki.TcpClient {
		return toki.NewTcpClient(conn, 30, 10, parserMap())
	})
	server.OnClientConnected = func(client *toki.TcpClient) {
		toki.AddRequestListenerTyped[*packets.TestData, *packets.TestData](
			&client.Communicator,
			func(req *packets.TestData) (*packets.TestData, error) {
				return &packets.TestData{Index: req.GetIndex(), Message: "echo: " + req.GetMessage()}, nil
			},
		)
	}
	if err := server.Start(ctx); err != nil {
		panic(err)
	}
	defer server.Stop()

	client, err := toki.DialTcp(ctx, "127.0.0.1", 9090, 30, 10, parserMap())
	if err != nil {
		panic(err)
	}
	defer client.Close()

	res, err := toki.SendRequestTyped[*packets.TestData, *packets.TestData](
		&client.Communicator,
		&packets.TestData{Index: 1, Message: "hello"},
		2*time.Second,
	)
	if err != nil {
		panic(err)
	}
	println(res.GetMessage())
}

Go also provides:

• TCP: NewTcpServer, DialTcp, NewTcpServerTLS, DialTcpTLS
• WebSocket: NewWsServer, DialWs, NewWsServerTLS, DialWss
• Shared helpers: Send, SendRequest, AddListenerTyped, AddRequestListenerTyped, Broadcast

---

Running Tests

Local commands are documented here for development and troubleshooting. Continuous verification is expected to run in an external tool, with Jenkins planned to execute the full Dart, Go, and cross-language test suite.

cd dart
dart pub get
dart test

cd go
go test ./...

When proto files change, also run:

tools/check_proto_sync.sh