proto-socket/README.md

# Proto Socket

Binary socket protocol library for bidirectional, heterogeneous communication across languages and platforms.

Built on Protocol Buffers with TCP length-prefixed framing, WebSocket binary frames, type-based message routing, request-response correlation, and built-in heartbeat.

---

## Design Principles

- Keep the core transport layer thin and stable
- Provide only the minimum common foundation for cross-language communication
- Standardize framing, serialization, routing, request-response correlation, and heartbeat
- Treat Protocol Buffers as the core protocol dependency. Other runtime dependencies should be native-first, narrowly scoped, and proportional to the feature they implement
- Do not embed application semantics into the protocol core
- Domain-specific concerns such as auth, session, agent workflow, chat features, and game logic belong in upper-layer implementations

---

## Protocol

See [PROTOCOL.md](PROTOCOL.md) for the full wire format specification.

```
[4-byte big-endian length] [PacketBase protobuf bytes]
```

For WebSocket/WSS transports, each binary frame contains one `PacketBase` protobuf payload without the TCP length header.

Protocol compatibility is tracked separately from language package versions. See [VERSIONING.md](VERSIONING.md). Even while packages are not published, the checked-in implementations must preserve the documented protocol contract.

---

## Implementations

| Language | Status | Path | Use case |
|----------|--------|------|----------|
| Dart | Available | [dart/](dart/) | Flutter, Dart server |
| C# | Planned | `csharp/` | Unity, .NET |
| Kotlin | Available | kotlin/ | Android, JVM |
| Swift | Planned | `swift/` | iOS, macOS |
| Go | Available | [go/](go/) | Server, tooling, scripting |
| TypeScript | Available | [typescript/](typescript/) | Browser, Node.js |
| Python | Available | [python/](python/) | Server, tooling, scripting |

New language implementations should start from [PORTING_GUIDE.md](PORTING_GUIDE.md) and the templates in [agent-ops/skills/project/add-proto-socket-crosstest-language/templates/](agent-ops/skills/project/add-proto-socket-crosstest-language/templates/). Mark an implementation available only after its same-language tests and cross-language tests pass.

Roadmap direction: every implementation should stay as close to native platform behavior as possible outside the protobuf layer. Prefer built-in TCP/TLS/WebSocket, concurrency, timer, and binary APIs over external packages. A focused library that directly matches a protocol need is acceptable when native support is missing or impractical, but broad frameworks or runtime layers must not be introduced just to cover a narrow feature such as WebSocket transport. Runtime dependencies beyond protobuf must be explicitly justified, protocol-relevant, and kept minimal.

---

## Quick Start (Dart)

```dart
import 'package:proto_socket/proto_socket.dart';

// 1. Define your message in message_common.proto, generate with protoc

// 2. Implement a client
class MyClient extends ProtobufClient {
  MyClient(Socket socket) : super(socket, 30, 10, {
    MyMessage.getDefault().info_.qualifiedMessageName: MyMessage.fromBuffer,
  });
}

// 3. Implement a server
class MyServer extends ProtobufServer {
  MyServer() : super('0.0.0.0', 9090, (socket) => MyClient(socket));

  @override
  void onClientConnected(ProtobufClient client) {
    client.addListener<MyMessage>((msg) => print('Received: ${msg}'));
  }
}

// 4. Start
final server = MyServer();
await server.start();

// 5. Connect and send
final socket = await Socket.connect('localhost', 9090);
final client = MyClient(socket);
await client.send(MyMessage()..text = 'hello');
```

---

## Adding Message Types

Edit the canonical proto at `proto/message_common.proto`, then regenerate all checked-in bindings:

```bash
tools/generate_proto.sh
tools/check_proto_sync.sh
```

The Go and Kotlin proto copies are allowed to keep only language-specific options such as `option go_package` or Java package/class options. `tools/check_proto_sync.sh` fails with a diff when their message schema drifts from `proto/message_common.proto`.

---

## Quick Start (Go)

```go
package main

import (
	"context"
	"net"
	"time"

	"google.golang.org/protobuf/proto"

	protoSocket "git.toki-labs.com/toki/proto-socket/go"
	"git.toki-labs.com/toki/proto-socket/go/packets"
)

func parserMap() protoSocket.ParserMap {
	return protoSocket.ParserMap{
		protoSocket.TypeNameOf(&packets.TestData{}): func(b []byte) (proto.Message, error) {
			m := &packets.TestData{}
			return m, proto.Unmarshal(b, m)
		},
	}
}

func main() {
	ctx := context.Background()

	server := protoSocket.NewTcpServer("127.0.0.1", 9090, func(conn net.Conn) *protoSocket.TcpClient {
		return protoSocket.NewTcpClient(conn, 30, 10, parserMap())
	})
	server.OnClientConnected = func(client *protoSocket.TcpClient) {
		protoSocket.AddRequestListenerTyped[*packets.TestData, *packets.TestData](
			&client.Communicator,
			func(req *packets.TestData) (*packets.TestData, error) {
				return &packets.TestData{Index: req.GetIndex(), Message: "echo: " + req.GetMessage()}, nil
			},
		)
	}
	if err := server.Start(ctx); err != nil {
		panic(err)
	}
	defer server.Stop()

	client, err := protoSocket.DialTcp(ctx, "127.0.0.1", 9090, 30, 10, parserMap())
	if err != nil {
		panic(err)
	}
	defer client.Close()

	res, err := protoSocket.SendRequestTyped[*packets.TestData, *packets.TestData](
		&client.Communicator,
		&packets.TestData{Index: 1, Message: "hello"},
		2*time.Second,
	)
	if err != nil {
		panic(err)
	}
	println(res.GetMessage())
}
```

Go also provides:

- TCP: `NewTcpServer`, `DialTcp`, `NewTcpServerTLS`, `DialTcpTLS`
- WebSocket: `NewWsServer`, `DialWs`, `NewWsServerTLS`, `DialWss`
- Shared helpers: `Send`, `SendRequest`, `AddListenerTyped`, `AddRequestListenerTyped`, `Broadcast`

If you want a copyable scaffold for using this module from another repository, see [examples/go-module-consumer/](examples/go-module-consumer/).

---

## Running Tests

Local commands are documented here for development and troubleshooting. Continuous verification is expected to run in an external tool, with Jenkins planned to execute the full Dart, Go, and cross-language test suite.

```bash
cd dart
dart pub get
dart test
```

```bash
cd go
go test ./...
```

```bash
cd kotlin
./gradlew test
```

Cross-language checks:

```bash
cd go
go run ./crosstest/go_dart.go
go run ./crosstest/go_kotlin.go
```

```bash
cd dart
dart run crosstest/dart_go.dart
```

```bash
cd kotlin
./gradlew run -PmainClass=com.tokilabs.proto_socket.crosstest.KotlinGoKt
```

When proto files change, also run:

```bash
tools/check_proto_sync.sh
```