moonnet

    Networking for MoonBit: connections, listeners and dialers over TCP, TLS and QUIC.

    networking
    network
    socket
    udp
    tcp
    tls
    https
    http
    http1
    http2
    http3
    h2
    h3
    quic
    websocket
    ws
    alpn
    connection
    connection-pool
    keep-alive
    listener
    dialer
    client
    server
    proxy
    async
    rfc9110
    rfc9112
    rfc9113
    rfc9114
    rfc6455
    rfc7301
    rfc9000
    moonbit
    Download zip
    Version
    0.1.11
    License
    Apache-2.0
    Last updated
    11 hours ago
    Downloads
    2K

    #moonnet

    The connection layer: one contract for a request and a reply, whatever carries it, and the sockets underneath.

    The wire formats are pure state machines — bytes in, events out — and touch no socket. This is what puts them on one: TCP and UDP, TLS over TCP, QUIC over UDP, and an adapter for each protocol that turns it into the same open / send / response / read.

    // One request, whichever of the four carries it.
    let conn = @h2.connect("example.com", port=443, tls=@tls.Client::new(@roots.mozilla(), alpn=["h2"]))
    let (response, body) = @spec.call(conn, @spec.Request::new("GET", "/"))
    println("\{response.status} \{body.length()}")

    #Install

    moon add moonbitstack/moonnet

    #The contract

    Everything above this library is written against moonnet/spec, and nothing in it names a protocol:

    DeclarationShape
    Field{ name, value }, one field of a head
    BodyEmpty, Bytes(Bytes), Streaming
    Request{ verb, target, fields, body }
    Response{ status : Int32, fields } — the head; the body is read through the connection
    Streamone exchange, numbered by the connection rather than by the protocol under it
    RefusedMalformed, Unsupported, Closed, Broken

    pub(open) trait Conn {
    async fn open(self : Self, request : Request) -> Stream
    async fn send(self : Self, stream : Stream, bytes : Bytes, end~ : Bool) -> Unit
    async fn response(self : Self, stream : Stream) -> Response
    async fn read(self : Self, stream : Stream) -> Bytes?
    fn trailers(self : Self, stream : Stream) -> Array[Field]
    fn is_ready(self : Self) -> Bool
    fn kind(self : Self) -> String
    async fn close(self : Self) -> Unit
    }

    The trait is open: a new protocol is an implementation of it, and nothing above changes. call and read_to_end are the two helpers over it, for the common shape of "send this, read all of that".

    async fn get(conn : &@spec.Conn, path : String) -> String {
    let stream = conn.open(@spec.Request::new("GET", path))
    let response = conn.response(stream)
    let body = @spec.read_to_end(conn, stream)
    "\{response.status} \{body.length()}"
    }

    The same get runs on every adapter below. is_ready is what a connection that carries one exchange at a time says about itself: HTTP/1.1 is False while one is in flight, HTTP/2, HTTP/3 and a WebSocket session are True.

    #Adapters

    PackageProtocolPublic entry points
    adapt/h1HTTP/1.1, RFC 9112client(tcp_conn, authority), connect(host, port~, tls?)
    adapt/h2HTTP/2, RFC 9113client(tcp_conn, scheme~, authority~), connect(host, port~, tls?)
    adapt/h3HTTP/3, RFC 9114connect(host, port~, tls~)
    adapt/wsWebSocket, RFC 6455client(tcp_conn, host~), connect(url, tls?)

    HTTP/1.1 frames a body three ways: a declared length, chunks with trailers, and a body that runs to the close. A reply shorter than its length is a truncation and raises rather than returning a short body; an informational head is passed over for the final one; the Host field is the authority the connection was opened to, not whatever the caller passed.

    HTTP/2 sends its preface and settings when the connection opens, puts every exchange on its own stream, and honours flow control both ways: a body goes out as the window allows and waits to be widened, and the octets read are handed back so that a reply larger than one window arrives instead of stalling at 64 KiB.

    HTTP/3 states its settings on a control stream — with the QPACK dynamic table shut, so no reply can refer to an entry this side does not have — and puts each exchange on a bidirectional stream. A peer that says nothing for thirty seconds is given up on rather than waited for.

    WebSocket makes the upgrade its exchange: open sends the handshake and response is the 101 with its accept checked against the key. After that read yields whole messages and a ping is answered on the way without reaching the caller. send writes a binary message, since the contract's octets do not say whether they are text; send_text is the one that writes text, and close_code / close_reason report how a session ended.

    // A session, and the close the peer gave.
    let ws = @ws.connect("wss://example.com/chat", tls=client)
    let stream = ws.open(@spec.Request::new("GET", "/chat"))
    let _ = ws.response(stream)
    ws.send_text("hello")
    match ws.next() {
    Some(@frame.Message::Text(reply)) => println(reply)
    Some(@frame.Message::Close(status~, ..)) => println("closed \{status}")
    _ => ()
    }

    #The pool

    Connections are kept by what makes two of them interchangeable: the host, the port, and the protocol ALPN settled on. Every knob is a field of a record with a published preset, so a caller changes one without guessing at the rest.

    let pool = @pool.Pool::new(
    key => @tcp.dial(key.host, port=key.port, tls=tls_for(key)),
    config={ ..@pool.defaults, idle: 60000, max_total: 128 },
    )
    let key = @pool.Key::new("example.com", 443, alpn="h2")
    let conn = pool.get(key)
    // ... one connection's worth of work ...
    pool.put(key, conn)

    FieldPresetWhat it is
    idle30000 mshow long a connection may sit idle before it is closed instead of reused
    max_idle8how many idle connections one key keeps
    max_total64how many the pool holds open at once, in use or idle
    livenessPeekwhat a reused connection is asked first: nothing, or a read that does not wait
    wait5000 mshow long a caller waits for a slot when the pool is full

    Peek is a read that does not wait: a peer that has closed while the connection sat idle is dropped, and one that said something is kept along with what it said, so the next exchange starts from those octets rather than after them. A pool that is full raises Busy rather than opening past max_total.

    #UDP and QUIC

    moonnet/udp is the datagram socket and the client end of a QUIC connection on it.

    let socket = @udp.bind(port=12000)
    socket.send(to, datagram)
    let next = socket.recv(timeout=100)

    let quic = @udp.dial("example.com", port=443, tls=client)
    let stream = quic.open_bi()
    quic.write(stream, b"hello", fin=true)

    Quic::step is one turn of the connection — send what is queued, take one datagram in — and take, ended, open_bi and open_uni read what came of it. Reading is a poll rather than a blocking call because the same connection also carries control and QPACK streams.

    #TCP

    moonnet/tcp is the connection underneath the adapters: TCP, TLS 1.3 over it, the listener and accept loop, the dialler, and one port that serves TLS and the clear.

    Conn::plain(sock)a socket in the clear
    Conn::server(sock, config)the server side of TLS, returned after the handshake
    Conn::client(sock, config, host)the client side, returned after the server's chain is judged
    starts_with(prefix)whether the stream opens with prefix, reading only as far as the first byte that differs
    opens_tls(), secure(config)serve TLS and the clear on one port
    is_alive(wait~)whether the peer is still there, asked without waiting
    shutdown(), close()a close_notify, then the socket

    @tcp.serve((conn, peer) => {
    while conn.read_until("\n") is Some(line) {
    conn.write(line + "\n")
    }
    }, port=12000)

    serve listens on 127.0.0.1:12000 unless told otherwise. One task may read while another writes, and writes are serialised, so concurrent writers put whole messages on the wire and TLS records stay in sequence.

    The sockets are moonbitlang/async's, which exist on the native and wasm backends; on js and wasm-gc the packages that drive them are empty.

    #Examples

    DirectoryWhat it does
    src/examples/toura TLS server and a client talking over it in one run
    src/examples/01-interopone client that speaks all four protocols through the contract, for driving against another server
    src/examples/02-pluga connection implemented outside this library, run through the same helpers
    src/examples/03-front-doorthe front door on its own: a listener and a client with one import
    src/examples/04-ws-conformancethe client end of a WebSocket conformance run, for a fuzzing server to drive
    src/examples/tls-peera TLS peer for interop runs against OpenSSL and tlsfuzzer

    moon run src/examples/tour --target native

    #Conformance

    The adapters are clients, so the evidence is what they carry against implementations they did not write, over real sockets:

    WhatAgainstResult
    HTTP/1.1 — a head, a declared-length body, a chunked body with trailers, a body to the closetwo independent serversall exchanges answered, bodies byte for byte
    HTTP/2 — preface, settings, one exchange per stream, flow control both waysthe same twoall exchanges answered; a reply over one window arrives whole
    HTTP/3 — QUIC handshake, settings with the QPACK table shut, one stream per exchangeboth, where the server offers h3all exchanges answered
    WebSocket — the upgrade, framing, ping and pong, close, invalid UTF-8, reserved bitsAutobahn's fuzzing server244 graded cases pass, 4 of them non-strict; no failures
    TLS 1.3 over TCP — handshake, certificate chain, alertsOpenSSL, and tlsfuzzerOpenSSL interop passes; tlsfuzzer runs in CI

    scripts/http_interop.sh is the run behind the first four rows: it starts a server of this family and one in a container, and asks each of them for HTTP/1.1, HTTP/2 and HTTP/3, plus a WebSocket message. scripts/ws_conformance.sh is the fifth: it starts Autobahn's fuzzing server and plays its cases with this library's client, and the verdict is the server's report. Both are jobs in CI, on real sockets, on every push.

    The WebSocket run leaves out two sections of the suite: 9.* measures throughput with messages from 64 KiB to 16 MiB, and 12.* and 13.* exercise permessage-deflate, which this adapter does not negotiate. The four non-strict cases are 6.4.1–6.4.4, where a message that is not valid UTF-8 is rejected when the message ends rather than at the first bad octet.

    h2spec and h3spec are conformance suites for servers; nothing here serves HTTP/2 or HTTP/3, so they do not apply to these adapters. The codecs underneath have their own conformance runs.

    #Licence

    Apache-2.0. See LICENSE.

    Body

    Where a message's body comes from.

    Conn

    The names this library is used by, in one import: the connection, the datagram socket and the contract. A caller that wants an adapter names that package, so a program that only opens TCP links nothing else.

    Field

    One field of a message head: the name, and the value, as they go on the wire.

    Listener

    The names this library is used by, in one import: the connection, the datagram socket and the contract. A caller that wants an adapter names that package, so a program that only opens TCP links nothing else.

    Refused

    What a connection will not do.

    Request

    A request: what to ask for, and what to carry along.

    Response

    A reply head: the status and the fields. The body is read through the connection, because it may be larger than anything worth holding at once.

    Socket

    A datagram socket. One task may receive while others send; two tasks receiving at once would each take half the datagrams, so a connection that needs one reader keeps one.

    Stream

    One exchange on a connection, named by the connection rather than by the protocol underneath: on a connection that carries one exchange at a time it is 1, and on one that multiplexes it is whatever that connection gave out.

    Opened

    pub(all) struct Opened {
    conn :
    Conn

    alpn : String
    target : String
    }

    What connect settled on.

    bind

    async fn bind(host? : String, port? : Int) ->
    Socket

    Bind host:port. Port 0 asks the system for a free one, which addr reports.

    connect

    async fn connect(url : StringView, alpn? : Array[String], roots? :
    Roots
    , now? : Int64) -> Opened

    Connect to url in one line: the scheme decides TLS, the ALPN list is offered, and the chain is judged against the roots the caller passes — Mozilla's when none are given.

    The connection is returned before any protocol is chosen for it: alpn says what the server picked, and the matching adapter is the one to hand it to.

    let opened = @moonnet.connect("https://example.com/", alpn=["h2", "http/1.1"])
    if opened.alpn == "h2" {
    let conn = @h2.client(opened.conn, scheme="https", authority="example.com")
    let (response, body) = @spec.call(conn, @spec.Request::new("GET", opened.target))
    println("\{response.status} \{body.length()}")
    }

    dial

    async fn dial(host : String, port~ : Int, tls? :
    Client
    , now? : Int64) ->
    Conn

    Connect to host:port, under TLS when tls is given, and return once the handshake is done. The host is resolved, then names the server for SNI and for checking its certificate.

    listen

    async fn listen(host? : String, port? : Int, tls? :
    Server
    , reuse_addr? : Bool, handshake? : Int?, cleartext? : Bool) ->
    Listener

    Bind host:port and listen, for TLS connections when tls is given. Port 0 asks the system for a free one; addr says which. reuse_addr lets a restarted server bind while the old socket lingers in TIME_WAIT, as uvicorn does. handshake bounds each TLS handshake; None waits for ever.

    cleartext lets a TLS listener take connections in the clear too, on the same port: one that opens with a TLS handshake record is served TLS, any other is left in the clear. Off, a TLS listener takes TLS only.

    serve

    async fn serve(handler : async (
    Conn
    ,
    Addr
    ) -> Unit, host? : String, port? : Int, tls? :
    Server
    , limit? : Int, reuse_addr? : Bool, handshake? : Int?) -> Unit

    listen, then serve: accept connections on host:port and run handler on each until cancelled.

    Source Files