Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

Welcome to the dnet book.

dnet is a unified messaging abstraction library for Rust that makes it easy to send and receive messages over multiple transport protocols with a consistent async interface.

The core idea behind dnet is to delegate message encoding and decoding to serde. This allows library users to focus on the networking logic of their application instead of spending time on low-level details of the specific transport protocol used to send messages.

Security

Read the security section first, before using the dnet crate.

What you will learn

This book is designed to help you understand:

  • What dnet is and when to use it
  • How to work with common transports such as TCP, UDP, QUIC, WebSocket, MessagePort and transport for communication with web workers
  • How to extend dnet with new transports or custom codecs
  • How to implement RPC over dnet transports

Getting started

If you are new to dnet, it’s best to start by learning the core concepts of dnet and then explore the examples.

Security

Security is intentionally left out of scope of the dnet crate.

dnet project provides a transport abstraction and networking implementation helpers, it focuses on flexible transport composition and message delivery semantics, but it does not provide encryption, authentication, or other security primitives by itself.

  • Use dnet over insecure transports only in trusted or local environments.
  • For untrusted networks, rely on a secure underlying transport layer (like TCP + TLS or QUIC - assuming secure configuration).

Always read underlying transport documentation to learn about its security semantics.

Implementing secure protocols over dnet transports

Building a custom secure protocol on top of dnet transports, wrapping insecure lower-level protocols - like UDP or raw TCP without TLS - is technically possible, but not recommended for most users.

If you choose to do this, follow these guidelines:

  1. Prefer well-vetted cryptographic libraries and protocols.
  2. Keep cryptographic responsibilities separate from the dnet transport logic.
  3. Avoid inventing custom encryption schemes unless you are a cryptography expert.

For most applications it’s best to use an existing underlying secure transport layer instead of building security directly into dnet messages.

Core concepts

In dnet, there are three central concepts that structure how messaging works:

Message

A “Message” is a struct defined by the user of the library. It represents the payload that will be sent or received over a dnet transport.

  • Outgoing messages are sent by your application.
  • Incoming messages are received from a peer.
  • The types used for outgoing and incoming messages may differ.

dnet itself does not impose a fixed message schema; you define the message shapes that make sense for your application.

use serde::{Deserialize, Serialize};

#[derive(Debug, Clone, Serialize, Deserialize)]
struct OutgoingMessage {
    content: String,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
struct IncomingMessage {
    content: String,
    timestamp: u64,
}

Codec

A “Codec” is a struct(/enum) implementing dnet’s Codec that defines how messages are encoded and decoded.

The codec is passed as a parameter to a Transport and is responsible for converting message values to and from a wire format.

“Codec” implementations work with message types that implement Serde’s Serialize and Deserialize traits.

let codec = dnet::codecs::BincodeCodec::default();

Transport

A Transport is a struct(/enum) defined by the dnet library (or by a third-party implementation that adheres to the dnet transport contract).

A transport wraps a lower-level networking protocol (like TCP or UDP), channel, or other communication medium and provides an asynchronous interface for sending and receiving messages.

See Transport contract for details on the dnet transport interface.

use futures::SinkExt;
use tokio::net::TcpStream;

use dnet::Receive as _;

let stream = TcpStream::connect("127.0.0.1:1234").await?;
let mut transport = TcpTransport::buffered(stream, codec);

transport.send(OutgoingMessage { content: "Hello from dnet".to_string() }).await?;
let incoming: IncomingMessage = transport.receive().await?;

Transport contract

A dnet transport implements both futures::Sink and futures::Stream to provide a unified asynchronous transport interface.

A dnet transport must satisfy these requirements:

  • Implement Sink<Outgoing, Error = dnet::Error<Other>> for outgoing messages.
  • Implement Stream<Item = Result<Incoming, Other>> for incoming messages.
  • The Sink error type is always dnet::Error<Other>, where Other is the transport-specific error type.
  • Receiving from the Stream yields Result<Incoming, Other>; the stream does not use dnet::Error for the item error.
  • The generic Other type of the result yielded by the Stream matches the generic Other error of the dnet::Error<Other> used by the Sink.
  • The Stream returns None to signal that the transport is closed, regardless of the underlying cause.
  • When the logging feature is enabled, transports also implement logging::Logging.
/// Transport error.
#[derive(Debug, PartialEq, Eq)]
pub enum Error<Other> {
    /// Occurs when transport is closed.
    Closed,

    /// Other non-predefined transport-specific error.
    Other(Other),
}

/// Trait for transports implementing `dnet` interface.
pub trait Transport<Incoming, Outgoing, Error>:
    Sink<Outgoing, Error = crate::Error<Error>>
    + Stream<Item = Result<Incoming, Error>>
    + conditional::Logging
{
}

impl<T, Incoming, Outgoing, Error> Transport<Incoming, Outgoing, Error> for T where
    T: Sink<Outgoing, Error = crate::Error<Error>>
        + Stream<Item = Result<Incoming, Error>>
        + conditional::Logging
{
}

Note

conditional::Logging extends logging::Logging when “logging” feature is enabled; it’s an empty trait otherwise.

How to use dnet

dnet is a easy-to-use async messaging abstraction. You define the message shapes your application needs, choose a transport and a codec, then send and receive messages over that transport.

This page uses TCP examples, but the same concepts apply to other transports such as WebSocket, UDP, channel transport, etc.

Dependencies

To use dnet, add it to your Cargo.toml, along with serde for message serialization and futures for async stream and sink extensions:

[dependencies]
dnet = "*"
serde = { version = "*", features = ["derive"] }
futures = "*"

Messages

Define the messages your application will send and receive. Use serde to derive serialization and deserialization for your message types.

use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize)]
pub struct ServerMessage {
    some_string: String,
    some_int: u32,
}

Incoming and outgoing messages can be of different types:

#[derive(Serialize, Deserialize)]
pub struct ClientMessage {
    some_float: f64,
}

Transport

A transport wraps an underlying I/O channel and provides dnet message send/receive operations.

Note

Different transports require different setup of the underlying protocol/channel/etc. before they can wrap them. See the transport documentation for details.

use dnet::{codecs::BincodeCodec, tcp::TcpTransport};
use tokio::net::TcpStream;

let stream = TcpStream::connect("127.0.0.1:8080").await?;
let mut transport = TcpTransport::<_, _, ClientMessage, ServerMessage>::buffered(
    stream,
    BincodeCodec::default(),
);

Sending

Send values with future’s SinkExt send(...):

use futures::SinkExt;

transport.send(ServerMessage { 
    some_string: "Hello, world!".into(), 
    some_int: 42 
}).await?;

Receiving

Receive values with future’s StreamExt next():

use futures::StreamExt;

match transport.next().await {
    Some(Ok(message)) => {
        println!("received message from the server: {}", message.some_string);
        transport.send(ClientMessage { some_float: 3.14 }).await?;
    }
    Some(Err(error)) => {
        // handle receive error
    }
    None => {
        // transport closed
    }
}

or by using the receive() method:

use dnet::Receive as _;

let incoming: ClientMessage = transport.receive().await?;
println!("received message from the client: {}", message.some_float);

Logging

dnet transports support logging via the Logging trait (enabled with the “logging” feature). To see transport logs, initialize a tracing subscriber from tracing-subscriber, then enable logging on the transport.

[dependencies]
dnet = { version = "*", features = ["logging"] }
tracing-subscriber = "*"
use dnet::{codecs::BincodeCodec, tcp::TcpTransport};
use tracing_subscriber;

tracing_subscriber::fmt().init();

let stream = TcpStream::connect("127.0.0.1:8080").await?;
let mut transport = TcpTransport::<_, _, ServerMessage, ClientMessage>::buffered(
    stream,
    BincodeCodec::default(),
);
// optionally: transport.set_logging_name("tcp-client");
transport.enable_logging();

Examples

See the repository examples for complete working code.

Codecs

A dnet codec is responsible for converting messages between Rust values and wire format.

dnet separates encoding and decoding into two traits, then combines them into a single Codec trait.

Encode

/// Trait for encoders.
pub trait Encode {
    /// Error type.
    type Error: std::error::Error;

    /// Encode message into writer.
    fn encode<W, T>(&mut self, writer: W, message: &T) -> Result<(), Self::Error>
    where
        W: Write,
        T: Serialize;
}

Encode takes a writable sink and a serializable message. Implementations may write messages in any wire format, such as JSON, binary, or length-delimited frames.

Decode

/// Trait for decoders.
pub trait Decode {
    /// Error type.
    type Error: std::error::Error;

    /// Decode message from reader.
    fn decode<R, T>(&mut self, data: R) -> Result<T, Self::Error>
    where
        R: Read,
        for<'de> T: Deserialize<'de>;
}

Decode reads bytes from a readable source and produces a deserialized message value. The generic error type allows decoding failures to be reported precisely.

Codec

/// Trait for `dnet` codecs.
pub trait Codec: Encode + Decode {}

Codec is a simple alias trait combining Encode and Decode. A dnet transport uses a Codec to transform outgoing messages into bytes and incoming bytes back into message values.

Provided codecs

dnet ships with the following codecs in dnet’s codecs module (behind the “codecs” feature, enabled by default):

  • BincodeCodec
    • Binary codec using bincode.
    • Best for compact, fast wire format when size is important.
  • JsonCodec
    • Text codec using serde_json.
    • Best for readable, interoperable payloads and debugging.

In addition, dnet includes length-delimiting framing codec for use with the FramedTransport:

bincode codec

The BincodeCodec serializes and deserializes messages using bincode.

When to use it

Use BincodeCodec when you want a fast, compact wire format and when message size and speed matter.

JSON codec

The JsonCodec serializes and deserializes messages using serde_json. It is a text-based codec that is easy to inspect and useful for debugging and interoperability thanks to its human-readability.

When to use it

Use JsonCodec when readability and interoperability are more important than binary compactness.

It is a good fit for HTTP-friendly transports, browser interoperability, debugging, or when message structure must be inspected easily.

Framing codecs

Framing codecs are used by FramedTransport.

What framing means

FramedTransport wraps a lower-level transport implementing AsyncRead and AsyncWrite. That means the transport sends a continuous stream of bytes, which must first be split into frames if we want to decode them into separate messages on the receiving side.

Framing codecs are responsible for this split.

How framing codecs work

Framing codecs differ from normal codecs in that their error type implements NotEnoughData - this tells FramedTransport that the codec needs more bytes before it can decode the incoming message.

/// Codecs implementing this trait can be used in [FramedTransport].
pub trait Framing: DecodeWithMessageLength {}

impl<T> Framing for T
where
    T: Decode + DecodeWithMessageLength,
    <T as Decode>::Error: NotEnoughData,
{
}

Warning

The codec must preserve partial input (resulting in NotEnoughData error) in some internal buffer until a full frame can be decoded - FramedTransport won’t send earlier bytes to the codec again.

Note

DecodeWithMessageLength can be used to obtain the byte length of a message in addition to the decoded message itself. This functionality is currently used only for logging.

Framing schemes

Different frame splitting methods are possible. The dnet library currently includes one:

Length-delimited (framing) codec

A dnet::io::length_delimited::Codec prefixes each frame with its length in bytes.

The length prefix is used to divide the continuous byte stream into discrete frames: the codec reads the 4-byte length in big-endian order, then reads that many bytes as a single message frame.

Note

A u32 (4 bytes) is used to encode the message length in big-endian byte order. The 4 length-prefix bytes themselves are not counted in that length; the length value refers only to the subsequent message bytes.

Transports

dnet provides a variety of transport implementations for different networking scenarios.

Transports are implemented for both native and WebAssembly targets (when appropriate).

Provided transports

Available for licensing

Logging

The “logging” feature is not enabled by default. If you want transport-level diagnostics, enable it in your Cargo.toml:

[dependencies]
dnet = { version = "0.1.4", features = ["logging"] }

When the “logging” feature is enabled, supported transports implement the Logging trait. You can then enable logging at runtime and assign a logical transport name for easier filtering and debugging:

use dnet::Logging;

let mut transport = /* create transport */;
transport.set_logging_name("peer-1");
transport.enable_logging();

Named transports are especially useful when you use multiple transports of the same kind in your application.

TCP transport

TCP transport wrappers in dnet are intended for Tokio TCP streams and can be used for plain TCP or TLS-over-TCP.

Buffered constructors

The buffered variants are recommended when the transport sees many small messages or when you want to reduce syscall overhead. They wrap the underlying stream in buffered I/O and can improve throughput without changing framing behavior.

Examples

Plain TCP

Warning

We are creating an insecure (unencrypted) connection here.

Server

A server accepts a plain TCP connection, receives a String, then sends an i32 back.

use futures::SinkExt;
use tokio::net::TcpListener;
use dnet::{codecs::BincodeCodec, tcp::TcpTransport, Receive};

let listener = TcpListener::bind("127.0.0.1:8080").await?;
let (tcp_stream, _peer_addr) = listener.accept().await?;
let mut transport = TcpTransport::buffered(tcp_stream, BincodeCodec::default());

let name: String = transport.receive().await?;
transport.send(name.len() as i32).await?;

Client

A client connects over plain TCP, sends a String, and receives an i32 reply.

use futures::SinkExt;
use tokio::net::TcpStream;
use dnet::{codecs::BincodeCodec, tcp::TcpTransport, Receive};

let tcp_stream = TcpStream::connect("127.0.0.1:8080").await?;
let mut transport = TcpTransport::buffered(tcp_stream, BincodeCodec::default());

transport.send("hello".to_string()).await?;
let length: i32 = transport.receive().await?;

TLS

TLS (Transport Layer Security) encrypts and authenticates the TCP connection, protecting messages from eavesdropping and tampering. See Transport Layer Security for more details.

Warning

Cryptography is hard - make sure you know what you’re doing.

Server

A TLS server accepts the socket first, then upgrades it before wrapping it in TcpTransport.

use futures::SinkExt;
use native_tls::{Identity, TlsAcceptor};
use tokio_native_tls::TlsAcceptor as TokioTlsAcceptor;
use tokio::net::TcpListener;
use dnet::{codecs::BincodeCodec, tcp::TcpTransport, Receive};

let listener = TcpListener::bind("0.0.0.0:8443").await?;
let (tcp_stream, _peer_addr) = listener.accept().await?;

let identity = Identity::from_pkcs12(&pkcs12_bytes, "password")?;
let tls_acceptor = TlsAcceptor::new(identity)?;
let tokio_acceptor = TokioTlsAcceptor::from(tls_acceptor);
let tls_stream = tokio_acceptor.accept(tcp_stream).await?;

let mut transport = TcpTransport::buffered(tls_stream, BincodeCodec::default());
let message: String = transport.receive().await?;
transport.send(message.len() as i32).await?;

Client

A TLS client connects to the server and then negotiates TLS before wrapping the stream.

use futures::SinkExt;
use native_tls::TlsConnector;
use tokio_native_tls::TlsConnector as TokioTlsConnector;
use tokio::net::TcpStream;
use dnet::{codecs::BincodeCodec, tcp::TcpTransport, Receive};

let tcp_stream = TcpStream::connect("example.com:8443").await?;
let connector = TlsConnector::new()?;
let tokio_connector = TokioTlsConnector::from(connector);
let tls_stream = tokio_connector.connect("example.com", tcp_stream).await?;

let mut transport = TcpTransport::buffered(tls_stream, BincodeCodec::default());
transport.send("hello".to_string()).await?;
let length: i32 = transport.receive().await?;

UDP transport

UDP transport wrapper for Tokio’s UdpSocket.

This transport inherits UDP semantics:

  • unreliable: messages may be dropped,
  • unordered: messages may arrive out of order or be duplicated,
  • size-limited: messages are bounded by the datagram size for the underlying socket.

The transport type is UdpTransport<U, Codec, Incoming, Outgoing> where U borrows a UdpSocket. It supports both connected sockets via Sink<Outgoing> and unconnected send via send_to.

Warning

This is an unreliable transport.
Messages may be dropped, arrive out of order, or be delivered multiple times.

Example

Connected UDP

use futures::SinkExt;
use tokio::net::UdpSocket;
use dnet::{codecs::BincodeCodec, udp::UdpTransport, Receive};

let udp_socket = UdpSocket::bind("127.0.0.1:1234").await?;
udp_socket.connect("127.0.0.1:1235").await?;
let mut transport = UdpTransport::new(udp_socket, BincodeCodec::default());

transport.send("hello".to_string()).await?;
let reply: String = transport.receive().await?;

Unconnected send/receive

use tokio::net::UdpSocket;
use dnet::{codecs::BincodeCodec, udp::UdpTransport};

let udp_socket = UdpSocket::bind("127.0.0.1:1236").await?;
let mut transport = UdpTransport::new(udp_socket, BincodeCodec::default());

transport.send_to("hello".to_string(), "127.0.0.1:1237").await?;
let (message, peer_addr) = transport.receive_from().await?;
println!("received {:?} from {}", message, peer_addr);

Notes

  • Use send_to when you need to send to different endpoints.
  • Use the sink interface after UdpSocket::connect(...) for a simple connected UDP workflow.
  • Use receive_from() if you need the sender address for each incoming message.

QUIC transports

dnet provides wrappers around Quinn primitives. The QUIC implementation includes both reliable stream-based transports over SendStream/RecvStream and an unreliable datagram transport over Connection.

Available transports

Reliable QUIC streams

QuicTransport is the most convenient option for typical message passing over QUIC streams. It composes QuicFramedTransport with dnet::io::length_delimited::Codec.

QuicTransport supports:

  • bidirectional streams (wrapping SendStream and RecvStream).
  • unidirectional send-only and receive-only streams (wrapping SendStream or RecvStream).
  • buffered versions for better throughput on many small messages.
  • configurable maximum message length.

Note

This is a reliable transport.
Reliable delivery and in-order delivery are handled by QUIC stream semantics.

Unreliable QUIC datagrams

QuicUnreliableTransport wraps a quinn::Connection and uses QUIC datagrams for message transport.

This transport inherits QUIC datagram semantics:

  • unreliable: messages may be dropped.
  • unordered: messages can arrive out of order.
  • duplicate delivery is possible.
  • messages are bounded by the underlying datagram size.

QuicUnreliableTransport can be configured with Configuration to:

Warning

This is an unreliable transport.
Messages may be dropped, arrive out of order, or be delivered multiple times.

When to use which transport

  • Use QuicTransport when you need a reliable, stream-based message channel.
  • Use QuicFramedTransport when you want a custom framing codec on top of QUIC streams.
  • Use QuicUnreliableTransport when you want datagram-style messaging with lower latency and no reliability guarantees.

Example

See QUIC example.

Framed transport

FramedTransport is a generic transport wrapper for AsyncRead + AsyncWrite streams that uses a framing codec to divide a continuous byte stream into discrete messages.

This transport requires a codec implementing dnet::io::framed::Framing.

What it does

  • Wraps a lower-level transport that implements AsyncRead and AsyncWrite.
  • Encodes outgoing messages into frames using the provided codec.
  • Reads incoming bytes into an internal buffer and decodes complete frames as they arrive.

Constructors

When to use it

Notes

  • A framing codec can implement custom delimiting strategies beyond length-prefix framing.
  • The codec must preserve partial input until a full frame can be decoded.
  • FramedTransport::buffered is often a good default for network streams.

Length-delimited transport

The LengthDelimitedTransport is a convenience wrapper around the FramedTransport transport that uses a length-delimited framing codec.

It prefixes each message with a 4‑byte big-endian length (in bytes) and then writes the encoded message payload. This is the common framing scheme when sending discrete messages over a byte stream (for example TCP, IPC, or QUIC streams).

Under the hood the module composes a FramedTransport with dnet::io::length_delimited::Codec, so you get the same framed transport semantics but with the length-prefixed codec already set up.

When to use

Use LengthDelimitedTransport when you want a simple length-prefixed framing for sending serialized messages over a byte stream.

If you need a different framing scheme, use FramedTransport directly with a custom codec.

Example

See IPC example.

WebSocket transport

WebSocketTransport wraps a WebSocket connection and exposes it as a dnet transport. It sends and receives binary messages using a configured Codec to (de)serialize them.

On wasm32, this transport wraps a browser web_sys::WebSocket. On native targets it wraps a tokio_tungstenite’s WebSocketStream. For axum servers, use dnet::websocket::axum::WebSocketTransport with axum::extract::ws::WebSocket.

Note

This transports WebSocket binary frames only. Text, ping, pong and other non-binary frames are ignored by the receiver.

Browser example

use futures::SinkExt;
use dnet::{codecs::BincodeCodec, websocket::WebSocketTransport, Receive};
use web_sys::WebSocket;

let web_socket = WebSocket::new("ws://localhost:8080/ws")?;
let mut transport = WebSocketTransport::new(web_socket, BincodeCodec::default()).await?;

transport.send("hello".to_string()).await?;
let reply: String = transport.receive().await?;

Native example

use futures::SinkExt;
use dnet::{codecs::BincodeCodec, websocket::WebSocketTransport, Receive};
use tokio_tungstenite::connect_async;

let (stream, _) = connect_async("ws://localhost:8080/ws").await?;
let mut transport = WebSocketTransport::new(stream, BincodeCodec::default());

transport.send("hello".to_string()).await?;
let reply: String = transport.receive().await?;

If you want the transport to create the connection for you, native clients can use WebSocketTransport::new_with_address(url, codec).await.

Axum server example

use futures::SinkExt;
use dnet::{codecs::BincodeCodec, websocket::axum::WebSocketTransport, Receive};
use axum::extract::ws::WebSocket;

async fn handle_socket(socket: WebSocket) {
    let (mut sender, mut receiver) =
        WebSocketTransport::new(socket, BincodeCodec::default()).split();

    sender.send("welcome".to_string()).await.unwrap();
    let message: String = receiver.receive().await.unwrap();
    println!("received {message}");
}

Web Worker transport

WebWorkerTransport is used for communication between a host page (or host worker) and a Web Worker. It uses the worker’s postMessage method and onmessage events (and their equivalents in worker context - DedicatedWorkerGlobalScope.postMessage and DedicatedWorkerGlobalScope.onmessage) to exchange messages.

Note

new_in_worker and new_with_name_in_worker must be called inside a worker context, Error::NotInWorker is returned otherwise.

Named transports are useful when the host or worker needs multiple channels. Multiple named transports can be used simultaneously alongside an unnamed transport in the same page or worker.

Workers can themselves host nested workers and use WebWorkerTransport::new(...) from within a worker context to create transports to child workers.

This transport is appropriate when you want a direct worker-host channel. If you need to use a dedicated MessagePort or send transferable objects, see the MessagePort and Transferable transport sections.

Host example

use futures::SinkExt;
use dnet::{codecs::BincodeCodec, webworker::WebWorkerTransport, Receive};
use web_sys::{Worker, WorkerOptions, WorkerType};

let options = WorkerOptions::new();
options.set_type(WorkerType::Module);
let worker = Worker::new_with_options("./worker.js", &options).unwrap();

let mut transport = WebWorkerTransport::new(worker, BincodeCodec::default()).await?;

transport.send("hello from host".to_string()).await?;
let reply: String = transport.receive().await?;

Worker example

use futures::SinkExt;
use dnet::{codecs::BincodeCodec, webworker::WebWorkerTransport, Receive};

let mut transport = WebWorkerTransport::new_in_worker(BincodeCodec::default()).await?;

let request: String = transport.receive().await?;
transport.send(format!("hello from worker: {}", request)).await?;

MessagePort transport

MessagePortTransport wraps a browser MessagePort and exposes it as a dnet transport. It is useful when you need a dedicated channel between two JavaScript contexts, such as:

  • two sides of a MessageChannel,
  • a page and an iframe,
  • a page and a worker when the port is passed with postMessage(),
  • or any other API that uses MessagePort.

Constructors:

Note

The underlying MessagePort is started automatically and the transport waits for the peer side to open before returning.

Multiple named transports can coexist on the same MessagePort, and they can be used at the same time as an unnamed transport on that port. This makes it easy to multiplex several logical channels over the same underlying port pair.

If you only need a direct worker-host channel, prefer WebWorkerTransport.

If you need to send transferable objects across the port, see the Transferable transport section.

Example

Create a MessageChannel, then wrap each port with MessagePortTransport.

use futures::SinkExt;
use web_sys::MessageChannel;
use dnet::{codecs::BincodeCodec, message_port::MessagePortTransport, Receive};

let channel = MessageChannel::new().unwrap();
let port1 = channel.port1();
let port2 = channel.port2();

let mut left = MessagePortTransport::new(port1, BincodeCodec::default()).await?;
left.send("hello from left".to_string()).await?;

// pass the other port to another context, or just use it in the same context:

let mut right = MessagePortTransport::new(port2, BincodeCodec::default()).await?;

let reply: String = right.receive().await?;
right.send(format!("hello from right, got: {}", reply)).await?;

Transferable transport

TransferableTransport is a dnet transport for browser postMessage targets that need to transfer both data and transferable objects.

It wraps a target implementing PostMessage such as:

TransferableTransport works with messages that implement the IntoTransferable / FromTransferable conversion traits:

  • IntoTransferable<Context, Error> first converts outgoing messages into an (arbitrary) type implementing Transferable trait.
  • Transferable trait implementation is then responsible for preparing message for transfer through the transport by converting it into WithTransferable type, which contains both the serialized message data (packed into JS object) and an JS array of transferable objects.
  • For incoming messages (on the receiving end), Transferable trait implementation is responsible for reconstructing the original message from the received JS object (handling of transferable objects is done automatically by the browser).
  • FromTransferable<Context, Error> reconstructs incoming messages from the (arbitrary) Transferable type.

Note

Transferable trait uses arbitrary Context (passed by the user to TransferableTransport during construction).
It may be used to store codec, works buffers, etc., required for message serialization and deserialization.

When to use

Use TransferableTransport when you need to send browser transferable objects, like:

If you only need serializable messages - without transferable objects - prefer dnet::js::Transport or its more specialized variants, such as MessagePortTransport or WebWorkerTransport.

Note

When directly using this or dnet::js::Transport over a MessagePort, remember to call port.start() on both sides before sending messages.

Implementing IntoTransferable and FromTransferable

Implementing IntoTransferable, FromTransferable and Transferable manually is straightforward, but can be tedious.

Use #[derive(IntoTransferable)] to implement those traits automatically:

use js_sys::ArrayBuffer;
use dnet::js::{IntoTransferable, Transferable};

#[derive(IntoTransferable)]
struct Message {
    some_field: String,

    #[transferable]
    buffer: ArrayBuffer,
}

Transferable fields must be marked with #[transferable].

Note

Generated IntoTransferable implementation expects you to use dnet::js::wrapper::Context with your transport.
If you need to use a different context type, implement IntoTransferable and FromTransferable manually.

Nested IntoTransferables

Messages deriving IntoTransferable can contain nested fields that also implement IntoTransferable.

Mark such fields with #[into_transferable]:

use js_sys::ArrayBuffer;
use dnet::js::IntoTransferable;

#[derive(IntoTransferable)]
struct Nested {
    #[transferable]
    buffer: ArrayBuffer,
}

#[derive(IntoTransferable)]
struct Message {
    some_field: String,

    #[into_transferable]
    nested: Nested,
}

Note

For convenience IntoTransferable is implemented for typed JS typed array types - see IntoTransferable’s foreign impls.

Example:

use js_sys::Uint16Array;
use dnet::js::IntoTransferable;

#[derive(Debug, IntoTransferable)]
struct Message {
     #[into_transferable]
     array: Uint16Array,
}

Warning

#[derive(IntoTransferable)] macro will generate extra structures named:

  • <YourTypeName>Wrapper,
  • <YourTypeName>Stripped,
  • <YourTypeName>IntoTransferables (only if nested #[into_transferable] fields are present),

where <YourTypeName> is the name of your message.

If name collision(s) occurs, consider putting your message type into a separate module or - if not possible - implementing IntoTransferable and FromTransferable manually.

Example

The following example demonstrates how a host and a worker can exchange messages with transferable objects using TransferableTransport:

use futures::SinkExt;
use js_sys::{ArrayBuffer, Uint8Array};
use std::rc::Rc;
use web_sys::{MessageChannel, Worker, WorkerOptions, WorkerType};

use dnet::{
    codecs::BincodeCodec,
    js::{wrapper::Context, IntoTransferable, TransferableTransport},
    Receive,
};

#[derive(IntoTransferable)]
struct Message {
    #[transferable]
    buffer: ArrayBuffer,
}

async fn host_example() -> Result<(), Box<dyn std::error::Error>> {
    let options = WorkerOptions::new();
    options.set_type(WorkerType::Module);
    let worker = Worker::new_with_options("./worker.js", &options)?;

    let context = Context::new(BincodeCodec::default());
    let mut transport = TransferableTransport::<_, _, Message, Message, _>::new(
        &worker,
        None,
        context,
        true,
    )
    .await?;

    let buffer = ArrayBuffer::new_with_length(8);
    Uint8Array::new(&buffer).copy_from(&[1, 2, 3, 4, 5, 6, 7, 8]);

    transport.send(Message { buffer }).await?;
    Ok(())
}

async fn worker_example(
    global: Rc<web_sys::DedicatedWorkerGlobalScope>,
) -> Result<(), Box<dyn std::error::Error>> {
    let context = Context::new(BincodeCodec::default());
    let mut transport = TransferableTransport::<_, _, Message, Message, _>::new(
        &global,
        None,
        context,
        false,
    )
    .await?;

    let received = transport.receive().await?;
    let Message { buffer } = received;
    assert_eq!(Uint8Array::new(&buffer).to_vec(), vec![1, 2, 3, 4, 5, 6, 7, 8]);

    Ok(())
}

WebRTC data channel

The WebRTC data channel transport lets peers exchange arbitrary messages using WebRTC data channels. It provides a peer-to-peer messaging option that works both in browser environments and natively, enabling low-latency, direct connections between endpoints.

Beyond the core transport, the WebRTC support in this project includes higher-level features that simplify real-world use:

  • built-in handling of signaling so peers can be connected with minimal setup,
  • server- and client-side connection management to coordinate multiple peer connections through a shared signaling channel,
  • an option for manual (custom) signaling when you need tighter control; and automatic and robust ICE candidate negotiation.

The implementation supports both browser and native runtimes.

Note

The WebRTC data channel transport is part of the dnet-webrtc offering and is available under a commercial license.
The commercial package includes full source access, production-focused support, and example applications illustrating common deployment patterns.

See dnet README for more information.

dnet utilities

dnet comes with the following set of utilities (under “utils” feature, enabled by default):

  • Filtering - wraps a transport and drops incoming or outgoing messages based on a predicate.
  • Mapping - adapts a transport to a different message type by mapping outgoing messages and unmapping incoming ones.
  • Split2, Split3, Split4, and Split5 - split a transport carrying a tagged message enum into multiple transports with different message types.
  • Unwrapping - converts incoming messages that implement Unwrap into their inner payloads.
  • Latest - wraps a numbered transport and only exposes the newest messages, discarding older ones.
  • ChannelTransport - transport over futures channels, useful for testing and debugging. Can be conveniently created with transports() function.
  • MergedTransport - merges a sink and stream into a single transport.
  • Numbered - wraps a transport and adds sequence numbers to messages.
  • Wall - a transport that rejects every operation, returning an error on send, receive, flush, or close.
  • Void - a transport that accepts outgoing messages (but “sends” them into a ‘void’) and never produces incoming ones.

And:

  • Pipe - connects two transports in both directions so messages can flow back and forth. It is not a transport itself; it is a helper for linking two transports together.

Remote Procedure Call

dnet’s rpc module (enabled with the “rpc” feature, which is enabled by default) provides a type-safe implementation of Remote Procedure Calls (RPC) that works over any transport supported by dnet.

What is RPC

Remote Procedure Call (RPC) allows code running in one execution context to invoke functions that execute in another context as if they were ordinary local function calls.

The two sides may be running:

  • in different processes,
  • on different computers connected over a network,
  • in different browser tabs,
  • inside Web Workers,
  • or on any other pair of peers connected through a dnet transport.

With dnet, you define your API once as a Rust trait, implement it on one side, and obtain a strongly typed client on the other. Serialization, message routing, request/response matching, and asynchronous communication are handled automatically.

Steps

Implementing an RPC service consists of a few simple steps:

  1. Define your service interface (see API section).
  2. Implement a producer for your service (see Producing section).
  3. Establish a connection between two peers using any supported dnet transport.
  4. On the server side, register the service by calling produce(...).
  5. On the client side, create a typed client by calling consume(...).

Once the connection is established, you simply call methods on the generated consumer (see Consuming section). The RPC layer transparently serializes arguments, sends requests over the selected transport, executes the corresponding server-side method, and returns the result.

Example

For a complete example, see the RPC example in the dnet repository.

API

The first step when using dnet RPC is defining your service API.

An API is a Rust trait that describes the remote methods exposed by a producer (server) and available to a consumer (client). The trait acts as the single source of truth for your RPC interface - dnet generates all request/response types and client/server glue code from it.

Defining the API trait

Use #[api] marker on a trait that describes your remote service interface:

// ...

use dnet::rpc::{api, no_ack};

#[api]
pub trait Api {
    /// Print "Hello World!" message on the server.
    #[no_ack]
    async fn hello_world(&self);

    /// Add two numbers together.
    async fn add_numbers(&self, a: i32, b: i32) -> i32;

    /// Concatenate two strings together.
    async fn concatenate_strings(&self, a: String, b: String) -> String;

    /// Repeatedly send server time at a given interval.
    async fn stream_time(&self, interval: Duration) -> impl Stream<Item = NaiveTime>;
}

The macro generates the following items:

  • Request enum - serializable request payloads for each method
  • Response enum - serializable response payloads and stream items
  • Consumer struct - client-side API for making RPC calls
  • impl_produce macro - producer-side helper for wiring your implementation to the RPC runtime

Warning

The generated types (Request, Response, Consumer, and impl_produce) have fixed names. Because of this, defining multiple #[api] traits in the same Rust module will result in name collisions.

Define each API trait in its own module.

Method types

The generated API understands several kinds of methods:

  • ordinary asynchronous request/response methods
  • methods marked with #[no_ack] are fire-and-forget and do not wait for a response
  • async methods that return impl Stream<Item = T> create a stream from producer to consumer
  • methods marked with #[abortable] use an AbortionToken argument on the producer side that can be used to react to abort event triggered on the client-side

Fire-and-forget methods

Annotate a method with #[no_ack] marker when the consumer should not wait for a response. The producer receives the request, but it does not send a response back.

Streaming methods

If a method returns impl Stream<Item = T>, the generated RPC consumer will receive a stream handle when the request is made. Values produced by the server are sent to the consumer until the stream closes or is aborted.

Aborting requests

The #[abortable] attribute adds an AbortionToken to the producer method signature. The consumer can abort a request or stream, and the token allows the producer to react to that event.

Serialization and no_serde

By default, generated Request and Response enums derive Serialize and Deserialize using serde. If you need to disable serialization for one or both enums, use the #[no_serde] attribute.

The #[no_serde] macro can be used as:

  • #[no_serde] - disable serialization for both request and response
  • #[no_serde(request)] - disable (de)serialization for request only
  • #[no_serde(response)] - disable (de)serialization for response only
  • #[no_serde(request_serialize)] - disable request serialization only
  • #[no_serde(request_deserialize)] - disable request deserialization only
  • #[no_serde(response_serialize)] - disable response serialization only
  • #[no_serde(response_deserialize)] - disable response deserialization only

This is useful when used with arguments/return types that are not serializable and RPC is meant to be used over dnet transports that don’t require serialization before sending messages or message encoding/decoding is meant to be implemented manually, using non-serde scheme.

Transferable objects

Use the #[transferable] and #[into_transferable] attributes when your API needs to use browser transferable objects (your RPC service is meant to be used over a TransferableTransport).

Attributes can be applied to individual function arguments.

Because Rust does not allow placing attributes directly on a return type, mark the whole method with #[transferable] or #[into_transferable] when you intend the return value (or stream items) to be transferable.

Note

#[transferable] is not supported for stream responses - use #[into_transferable] for streaming methods.

When any #[transferable] / #[into_transferable] attribute is used in an #[api] trait, the API generation automatically switches to the transferable serialization mode. The generated Request and Response enums derive IntoTransferable and the appropriate fields are annotated (for example #[transferable] on transferable fields and #[into_transferable] on nested/into-converted fields).

Because transferable mode uses IntoTransferable conversions instead of serde, use of these attributes implies #[no_serde] for the API.

This feature is intended for use with the transferable transport - see its documentation for more details.

Example API using transferable attributes:

use dnet_rpc::api;
use js_sys::{ArrayBuffer, Float32Array, Uint32Array, Uint8Array};
use futures::Stream;

#[api]
pub trait Api {
    #[no_ack]
    async fn test_no_ack(&self, #[transferable] some_array_buffer: ArrayBuffer);

    async fn add(&self, #[into_transferable] floats: Float32Array) -> f32;

    #[transferable]
    async fn get_some_transferable(&self) -> ArrayBuffer;

    #[into_transferable]
    async fn stream(&self) -> impl Stream<Item = Uint32Array>;
}

Generated request/response enums (transferable mode) derive IntoTransferable and annotate fields appropriately. For example:

#[derive(Debug, Clone, ::dnet_js::IntoTransferable)]
pub enum Request {
    SendTransferable { #[transferable] data: OffscreenCanvas },
    GetData {},
}

#[derive(Debug, Clone, ::dnet_js::IntoTransferable)]
pub enum Response {
    SendTransferable(()),
    GetData(#[into_transferable] Data),
}

Where to go next

Read the Producing page for the producer implementation details, and the Consuming page for the client-side consumption of the API.

Producing

A producer is the server-side implementation of your RPC API. It receives requests from the consumer, executes the requested methods, and sends responses or stream items back.

Implementing a producer

Suppose you have the following API:

#[api]
pub trait Api {
    async fn add_numbers(&self, a: i32, b: i32) -> i32;

    #[no_ack]
    async fn hello_world(&self);

    async fn stream_time(&self, interval: Duration) -> impl Stream<Item = NaiveTime>;
}

You can implement it on any type:

#[derive(Produce)]
pub struct Producer;

impl Producer {
    async fn add_numbers(&self, a: i32, b: i32) -> i32 {
        a + b
    }

    async fn hello_world(&self) {
        println!("Hello World!");
    }

    fn stream_time(
        &self,
        interval: Duration,
    ) -> impl Stream<Item = NaiveTime> {
        // ...
    }
}

The Produce derive macro uses the generated impl_produce! macro from #[api] to generate the RPC dispatch implementation.

Note

impl_produce!, Request, Response must be in scope for the Produce derive macro to work.
It is generated by the #[api] macro alongside the Request, Response, and Consumer types.

Why not implement API’s async trait?

dnet deliberately does not expect producers to implement the API trait to avoid current limitations of the Rust’s async traits.

Instead Produce derive macro uses impl_produce! internally to glue itself to methods implemented directly on your type.

Abortable methods

An abortable API method:

#[abortable]
async fn long_task(&self);

expects the producer implementation to have an additional AbortionToken argument:

async fn long_task(
    &self,
    token: AbortionToken,
) {
    // ...
}

The token becomes cancelled when the consumer aborts the request or stream, allowing the producer to stop any ongoing work.

Producing

Once your producer implementation is ready, you can call produce(...) over a dnet transport:

use dnet::rpc::producer::Produce;
use dnet::{codecs::BincodeCodec, tcp::TcpTransport};

let listener = tokio::net::TcpListener::bind(address).await?;
loop {
    let (stream, peer) = listener.accept().await?;
    let transport = TcpTransport::buffered(stream, BincodeCodec::default());
    let producer = Producer::new();

    tokio::spawn(async move {
        let shutdown = producer
            .produce(transport, Default::default(), Default::default())
            .await;
        tracing::info!("Connection {} finished: {:?}", peer, shutdown);
    });
}

Where to go next

Read the Consuming page to learn how to consume the API on the client side.

Consuming

Consumers are used to consume the service API on the client side.

Creating a consumer

The #[api] macro generates a Consumer type for your API trait.

Create it with Consumer::consume function over a dnet transport:

use dnet::rpc::Consume;
use dnet::{codecs::BincodeCodec, tcp::TcpTransport};

let stream = tokio::net::TcpStream::connect(address).await?;
let transport = TcpTransport::buffered(stream, BincodeCodec::default());
let consumer = Consumer::consume(transport, Default::default(), Default::default());

Calling remote methods

Once the consumer is created, you call remote methods as ordinary async Rust methods:

consumer.hello_world().await?;
let sum = consumer.add_numbers(2, 3).await?;

Consumer methods return (after awaiting) a Result which error is Error inside rpc module.

Stream methods

If the API declares a streaming method, the consumer call returns a stream handle:

let mut time_stream = consumer.stream_time(std::time::Duration::from_secs(1)).await?;

Note

Notice that you have to await your streaming request first - only then you will receive an actual stream instance.

You can then read values from the stream until it closes:

use futures::StreamExt;
while let Some(time) = time_stream.next().await {
    println!("Time: {}", time);
}

Streaming consumer methods return type Stream inside rpc’s consumer module.

Aborting requests and streams

Generated request futures and stream handles support aborting.
For example:

use tokio::time::{sleep, Duration};

let mut long_task_future = consumer.long_task();
let aborter = long_task_future.aborter();

tokio::select! {
    result = &mut long_task_future => {
        println!("Task completed: {result:?}");
    }

    _ = sleep(Duration::from_secs(10)) => {
        aborter.abort();
        println!("Task aborted due to taking too long.");
    }
}

Notes

  • A consumer can make many requests over the same transport.
  • Each request is matched to a response using an internal request ID.
  • Streaming values are delivered in order until the producer closes the stream.

Writing a custom dnet transport

A dnet transport must implement the transport contract described in the transport contract.

What your transport must do

  • Implement futures::Sink<Outgoing, Error = dnet::Error<Other>>.
  • Implement futures::Stream<Item = Result<Incoming, Other>>.
  • Preferably implement futures::stream::FusedStream as well.
  • Conditionally implement dnet_base::Logging behind a “logging” feature in your crate.

The Other type is your transport-specific error type.

Sink

The outgoing side of a transport always uses dnet::Error<Other> as the Sink error type.

Example:

impl<Codec, Incoming, Outgoing> Sink<Outgoing> for MyTransport<Codec, Incoming, Outgoing> {
    type Error = dnet::Error<MyError>;
    // ... poll_ready, start_send, poll_flush, poll_close ...
}

Stream

The incoming side is a Stream of Result<Incoming, Other>, where the same Other error type is used directly in the stream item.

Note

Do not wrap stream item errors in dnet::Error.

Example:

impl<Codec, Incoming, Outgoing> Stream for MyTransport<Codec, Incoming, Outgoing> {
    type Item = Result<Incoming, MyError>;
    // ... poll_next ...
}

The stream must return None to signal that the transport is closed (regardless of cause).

FusedStream

Implementing FusedStream is not strictly required but recommended.

impl<Codec, Incoming, Outgoing> FusedStream for MyTransport<Codec, Incoming, Outgoing> {
    fn is_terminated(&self) -> bool {
        // todo
    }
}

Logging support

Conditionally implement dnet_base::Logging behind a “logging” feature in your crate.

See dnet transport implementations for reference of what to log, how and where.

Example:

pub struct MyTransport<Codec, Incoming, Outgoing> {
    // ...

    #[cfg(feature = "logging")]
    logger: dnet::Logger,
}

impl<Codec, Incoming, Outgoing> MyTransport<Codec, Incoming, Outgoing> {
    pub fn new(/* ... */) -> Self {
        WebSocketTransport {
            // ...

            #[cfg(feature = "logging")]
            logger: dnet::Logger::new::<Self>(),
        }
    }
}

// ... todo: remember to log in your poll methods 

#[cfg(feature = "logging")]
impl<Codec, Incoming, Outgoing> dnet_base::Logging for MyTransport<Codec, Incoming, Outgoing> {
    const KIND: &'static str = "MyTransport";

    fn with_logger<F, R>(&self, f: F) -> R
    where
        F: FnOnce(&dnet_base::Logger) -> R,
    {
        f(&self.logger)
    }

    fn with_logger_mut<F, R>(&mut self, f: F) -> R
    where
        F: FnOnce(&mut dnet_base::Logger) -> R,
    {
        f(&mut self.logger)
    }
}

Testing transports

Transports may be tested using the dnet-tests crate. It provides helpers such as dnet_tests::test_transport, dnet_tests::test_unit_message, and dnet_tests::test_stream for validating that your transport correctly sends, receives, and closes.

You can use dnet_tests::dtest_configure!() and #[dtest] attribute macro to mark your tests in your test modules when your transport supports both: native and WASM targets.

Common mistakes

  • Transport not closing on drop. Some underlying transports require an explicit Drop implementation to flush or close resources correctly.
  • Returning Result<Incoming, dnet::Error<TransportError>> from the stream. The stream item should be Result<Incoming, TransportError> and the closing of the transport is notified by returning None.
  • Returning Poll::Pending too early. If the underlying transport returned Poll::Ready, but you can’t construct (and return) a message yet, keep polling in a loop until you can construct a concrete return value - return Poll::Pending only when underlying transport returns Poll::Pending (or you track waking with some other mechanism).
  • Forgetting to implement Logging. When the logging feature is enabled, the transport should implement dnet_base::Logging or it may not work correctly with other dnet features.
  • Implementing Logging trait but forgetting to actually log anything.