🥮 mooncakes.io

    moon-mysql

    Pure-MoonBit MySQL wire-protocol driver implementing @moondb.Driver — no C, like PyMySQL.

    mysql
    driver
    moondb
    wire
    database
    protocol
    Download zip
    Author
    Version
    0.3.1
    License
    Apache-2.0
    Repository
    Last updated
    5 days ago
    Downloads
    27

    Dependencies

    #moon-mysql

    A pure-MoonBit MySQL / MariaDB wire-protocol driver — no C, no bindings, like PyMySQL. It speaks the MySQL client/server protocol (version 10 handshake, mysql_native_password auth, the text COM_QUERY protocol) directly over a raw TCP socket, and implements the @moondb.Driver contract so a moondb-based query layer (moonorm) can talk to MySQL through the same seam as the SQLite and Postgres drivers.

    MariaDB speaks the same wire protocol and is a first-class target: the handshake parser recognises MariaDB's 5.5.5- version sentinel (recovering the real version and reporting server_kind = MariaDB), reads past its extended capability bits without disturbing the MySQL-8 offsets, and authenticates over the shared mysql_native_password path. CI runs the same integration suite against MySQL 8, MariaDB 11, and MariaDB 10.11.

    $ moon add Lfan-ke/moon-mysql

    • API reference: https://lfan-ke.github.io/moon-mysql/
    • License: Apache-2.0

    #Quickstart

    The synchronous @moondb.Driver adapter is the one-call-per-statement surface:

    let db = @client.MysqlDriver::new(
      user="root", password="root", database="test",
      host="127.0.0.1", port=3306,
    )
    db.execute("CREATE TABLE t (id INT PRIMARY KEY, name TEXT)", [])
    db.execute("INSERT INTO t VALUES (?, ?)", [@moondb.Int(1), @moondb.Text("a")])
    let rows = db.query("SELECT id, name FROM t", [])
    rows[0].int_by("id")   // => 1
    rows[0].text_by("name") // => "a"

    For long-lived connections and real transactions, use the asynchronous MysqlConn directly (see The async wall below):

    async fn run() -> Unit raise {
      let conn = @client.MysqlConn::connect("127.0.0.1", 3306, "root", "root", "test")
      conn.begin()
      conn.execute("INSERT INTO t VALUES (?, ?)", [@moondb.Int(2), @moondb.Text("b")])
      conn.commit()
      let rows = conn.query("SELECT * FROM t", [])
      conn.close()
    }

    #Architecture

    The library is split so the entire wire format is testable off any backend, and only the socket transport is native-bound.

    flowchart TD
  subgraph root["Lfan-ke/moon-mysql · pure codec · target: all backends"]
    sha1["sha1 — FIPS 180-4"]
    packet["packet — lenenc / fixed-int / string cursor"]
    handshake["handshake — parse + scramble + response"]
    response["response — OK/ERR/EOF + text-row decode"]
    binding["binding — ? placeholders → escaped literals"]
  end
  subgraph client["Lfan-ke/moon-mysql/client · native only"]
    conn["MysqlConn — async transport over @socket.Tcp"]
    driver["MysqlDriver — sync @moondb.Driver adapter"]
  end
  handshake --> conn
  response --> conn
  binding --> conn
  packet --> handshake
  packet --> response
  sha1 --> handshake
  conn --> driver
  driver -. implements .-> moondb["@moondb.Driver"]

    %% root (pure, all backends)          client (native)
%%   sha1 ─┐                            MysqlConn ── async socket transport
%%   packet┼─ handshake ─┐                 │
%%         └─ response ──┼──────────────►  │
%%            binding ───┘              MysqlDriver ── impl @moondb.Driver

    Every packet field type — the 3-byte length + sequence-id framing, length-encoded and fixed-width little-endian integers, NUL- and length-encoded strings — is parsed through an in-memory cursor in the pure half, so the codec is unit-tested on wasm, wasm-gc, js, and native. The client package adds the async socket transport and the driver, and is native-only because moonbitlang/async/socket has no JS/wasm backend.

    #The async wall

    moonbitlang/async sockets are asynchronous, but @moondb.Driver's methods are synchronous, and MoonBit forbids calling an async function from a non-async one. The synchronous adapter bridges each call through @async.run_async_main, which runs an async body to completion on a fresh event loop.

    A socket does not survive across two such event loops (each tears its file descriptors down — verified empirically), so the synchronous MysqlDriver opens a fresh connection, authenticates, runs one statement, and closes, on every execute/query. This is correct and efficient enough for autocommit work (which is exactly what a moondb roundtrip is), but it means a transaction cannot be held open across separate Driver calls. begin/commit/rollback on the synchronous adapter therefore raise a clear DbError directing you to the asynchronous MysqlConn, which owns its socket for its whole lifetime and brackets real server-side transactions. This is a language constraint, not a shortcut.

    #Injection safety

    Values never reach the server as SQL text unescaped. The text protocol has no out-of-band parameter binding (that arrives with prepared statements — see the roadmap), so this round renders each bound @moondb.Value as an escaped SQL literal: strings are single-quoted with every metacharacter backslash-escaped, blobs use the x'…' hex form, and the placeholder scanner tracks single-, double-, and backtick-quoted spans so a literal ? inside a string is never mistaken for a placeholder.

    #Type mapping (text protocol)

    MySQL column type@moondb.Value
    TINYINT / SMALLINT / INT / MEDIUMINT / YEARInt
    BIGINTInt64
    FLOAT / DOUBLEDouble
    binary-collation columns (BLOB, VARBINARY, …)Blob
    VARCHAR / TEXT / DECIMAL / JSON / temporal (ISO text)Text
    SQL NULLNull

    #Tested against a real MySQL and MariaDB

    CI (.github/workflows/ci.yml) starts a real MySQL 8 (pinned to mysql_native_password) and, in separate jobs, MariaDB 11 and MariaDB 10.11, then runs moon check/build/test on native against each. The one integration test — gated by MYSQL_TEST so a local moon test without a database skips it — connects, authenticates, creates a table, inserts parameter-bound rows (integers, UTF-8 text, an embedded quote), and asserts the decoded result set cell by cell; the same suite is green on all three servers. The pure codec additionally runs on every backend, and parse_server_version has a unit test covering both MariaDB's 5.5.5- sentinel and a plain MySQL version string.

    #MariaDB auth negotiation

    mysql_native_password is the tested path on both servers. If a server's default plugin differs but it still offers pluggable auth, the client advertises mysql_native_password and answers the server's AuthSwitchRequest for it — the down-negotiation MariaDB may require. A switch to any other plugin, or a caching_sha2_password full-auth exchange, raises a clear UnsupportedError pointing here rather than silently mis-authenticating.

    #Roadmap

    This round covers connect + mysql_native_password auth + the text protocol + value decode + the @moondb.Driver adapter + a real roundtrip. Planned next:

    • Prepared statements — COM_STMT_PREPARE/COM_STMT_EXECUTE, the binary protocol, and true out-of-band parameter binding.
    • caching_sha2_password — MySQL 8's default plugin, including the RSA full-auth exchange over a plain connection.
    • client_ed25519 — MariaDB's native Ed25519 auth plugin, for MariaDB accounts not using mysql_native_password.
    • Full type coverage — binary-protocol temporal/decimal/bit decoding, and a dedicated temporal Value case once moondb grows one.
    • Connection pooling and a streaming Rows cursor.
    • TLS once moonbitlang/async exposes a client-side entry point.

    #Parameter binding

    The text protocol has no out-of-band binding, so bind_params renders each value as an escaped literal. That makes the escaping load-bearing rather than cosmetic: a connection asks the server for @@sql_mode and @@character_set_connection at connect time and escapes for what it finds — a doubled quote under NO_BACKSLASH_ESCAPES, a backslash otherwise. A charset whose characters can end in 0x5C (gbk, big5, sjis, cp932, gb18030) cannot be escaped safely with backslashes at all, so such a connection is refused rather than served.

    Prepared statements (COM_STMT_PREPARE + the binary protocol) are what remove the question entirely, and are the next thing this driver needs.

    MysqlError

    source
    pub(all) suberror MysqlError {
      ProtocolError(String)
      ServerError(Int, String, String)
      UnsupportedError(String)
    } derive(Eq)

    A failure raised while speaking the MySQL wire protocol. It is deliberately distinct from @moondb.DbError: the pure codec and the socket transport raise MysqlError (which carries wire-level specifics — a server error code, an unsupported auth plugin, a malformed packet), and the @moondb.Driver adapter maps it onto the coarse DbError cases at the public boundary.

    • ProtocolError — a packet did not parse: short read, bad prefix byte, an out-of-place packet in the result-set state machine.
    • ServerError — the server sent an ERR packet. Carries the numeric error code, the 5-char SQLSTATE, and the human message verbatim.
    • UnsupportedError — a code path this round does not implement (e.g. the caching_sha2_password full-auth exchange, a LOCAL INFILE request).
    impl Show for MysqlError

    MysqlError::to_string

    source
    fn MysqlError::to_string(self : MysqlError) -> String

    A one-line rendering, e.g. ServerError(1146, 42S02): Table 'test.t' doesn't exist.

    ColumnDef

    source
    pub(all) struct ColumnDef {
      name : String
      column_type : Int
      charset : Int
      flags : Int
    } derive(Eq)

    One result-set column's definition, reduced to what text decoding needs: the projected name, the MySQL type code, the collation (to tell text from binary), and the column flags.

    Handshake

    source
    pub struct Handshake {
      server_version : String
      server_kind : ServerKind
      connection_id : Int
      salt : Bytes
      capability : Int
      mariadb_capability : Int
      charset : Int
      status : Int
      auth_plugin : String
    }

    The server's initial Handshake packet (protocol version 10), decoded down to the fields the client needs: the 20-byte auth salt, the negotiated capabilities, and the auth-plugin name that selects the scramble.

    OkPacket

    source
    pub(all) struct OkPacket {
      affected_rows : Int64
      last_insert_id : Int64
      status : Int
      warnings : Int
    } derive(Eq)

    An OK packet: a statement that returned no result set (INSERT/UPDATE/DELETE/DDL) or the terminator of a successful command.

    PacketReader

    source
    pub struct PacketReader {
      data : Bytes
      pos : Int
    }

    A cursor over one decoded MySQL packet payload. Every field type the protocol uses — fixed-width little-endian integers, length-encoded integers and strings, NUL-terminated strings, and raw byte runs — is read through this, advancing a position and raising [MysqlError::ProtocolError] on any short read. It holds no socket: the transport reads a full payload into Bytes first, then parses it here, which is what lets the whole codec be tested off any backend.

    PacketReader::at_end

    source
    fn PacketReader::at_end(self : PacketReader) -> Bool

    Whether the cursor has consumed the whole payload.

    PacketReader::bytes

    source
    fn PacketReader::bytes(self : PacketReader, n : Int) -> Bytes raise MysqlError

    Read n raw bytes.

    PacketReader::lenenc_bytes

    source
    fn PacketReader::lenenc_bytes(self : PacketReader) -> Bytes? raise MysqlError

    Read a length-encoded string (the string<lenenc> type), returning None for the 0xFB NULL sentinel that appears in text-protocol rows.

    PacketReader::lenenc_uint

    source
    fn PacketReader::lenenc_uint(self : PacketReader) -> Int64 raise MysqlError

    Read a length-encoded unsigned integer (the int<lenenc> type). The 0xFB NULL sentinel and 0xFF are rejected here — NULL only has meaning inside a row, which [lenenc_bytes] handles.

    PacketReader::new

    source
    fn PacketReader::new(data : Bytes) -> PacketReader

    Wrap a decoded packet payload for reading from the start.

    PacketReader::peek

    source
    fn PacketReader::peek(self : PacketReader) -> Int

    Peek the next byte without advancing; -1 at end of payload.

    PacketReader::remaining

    source
    fn PacketReader::remaining(self : PacketReader) -> Int

    Bytes not yet consumed.

    PacketReader::rest

    source
    fn PacketReader::rest(self : PacketReader) -> Bytes

    Read everything left in the payload (an EOF-terminated string field).

    PacketReader::skip

    source
    fn PacketReader::skip(self : PacketReader, n : Int) -> Unit raise MysqlError

    Skip n bytes.

    PacketReader::string_nul

    source
    fn PacketReader::string_nul(self : PacketReader) -> Bytes raise MysqlError

    Read a NUL-terminated string, consuming the terminator.

    PacketReader::u8

    source
    fn PacketReader::u8(self : PacketReader) -> Int raise MysqlError

    Read one byte as an Int in 0..=255.

    PacketReader::uint_le

    source
    fn PacketReader::uint_le(self : PacketReader, n : Int) -> Int64 raise MysqlError

    Read an n-byte little-endian unsigned integer.

    QuoteMode

    source
    pub(all) enum QuoteMode {
      Backslash
      DoubleQuote
    } derive(Eq)

    How the server reads a string literal, which decides how a value has to be escaped. NO_BACKSLASH_ESCAPES is part of ANSI and of several stock sql_mode combinations: there, a backslash is an ordinary character and the only escape is a doubled quote — so a backslash-escaping writer leaves the value's own quote live and the statement is injectable.

    ServerKind

    source
    pub(all) enum ServerKind {
      MySQL
      MariaDB
    } derive(Eq, 
    Debug    )

    Which server dialect answered the handshake. Both speak the MySQL wire protocol; they diverge in the version string and the extended capabilities.
    impl Show for ServerKind

    bind_params

    source
    fn bind_params(sql : String, params : Array[
    Value    ], mode? : QuoteMode) -> Bytes raise MysqlError

    Substitute the ordered params for the ? placeholders in sql, producing the UTF-8 query bytes a COM_QUERY carries.

    The text protocol has no out-of-band parameter binding — that arrives with prepared statements (COM_STMT_PREPARE, binary protocol) in a later round — so values are rendered as escaped literals here. The scanner tracks single-, double-, and backtick-quoted spans (honouring backslash escapes) so a literal ? inside a string or identifier is never mistaken for a placeholder, and it raises when the placeholder and parameter counts disagree.

    build_handshake_response

    source
    fn build_handshake_response(handshake : Handshake, user : String, password : Bytes, database : String) -> Bytes raise MysqlError

    Build the client's HandshakeResponse41 payload for user/password/database.

    mysql_native_password and caching_sha2_password (MySQL 8's default, fast-path scramble here and full auth driven in [MysqlConn::connect]) are both handled directly. For any other plugin the client advertises mysql_native_password and sends a native token, so a native-capable account authenticates and otherwise the server drives an AuthSwitchRequest the connection layer answers (native or caching_sha2) or rejects. Only a server that requires a non-native plugin and does not offer pluggable auth is rejected here. client_ed25519 remains on the README roadmap.

    build_text_rows

    source
    fn build_text_rows(columns : Array[ColumnDef], row_payloads : Array[Bytes]) -> Array[
    Row    ] raise MysqlError

    Build the materialised rows of a text-protocol result set from its column definitions and the raw row packets. This is the pure heart of query: the socket transport collects columns and row_payloads, and every cell decode happens here, off any backend.

    bytes_to_string

    source
    fn bytes_to_string(b : Bytes) -> String

    Bytes → String, lossily (server version, error text, text-protocol cells).

    caching_sha2_full_auth_token

    source
    fn caching_sha2_full_auth_token(password : Bytes, nonce : Bytes, pem : String, seed : Bytes) -> Bytes raise MysqlError

    The caching_sha2_password full-auth token: the NUL-terminated password XORed with the nonce (cycled), then RSA-OAEP encrypted under the server's public key.

    caching_sha2_scramble

    source
    fn caching_sha2_scramble(password : Bytes, nonce : Bytes) -> Bytes

    The caching_sha2_password fast-auth scramble: SHA256(pw) XOR SHA256( SHA256(SHA256(pw)) ‖ nonce ), 32 bytes. An empty password sends an empty token.

    client_connect_with_db

    source
    let client_connect_with_db : Int

    client_long_flag

    source
    let client_long_flag : Int

    client_long_password

    source
    let client_long_password : Int

    client_plugin_auth

    source
    let client_plugin_auth : Int

    client_protocol_41

    source
    let client_protocol_41 : Int

    client_secure_connection

    source
    let client_secure_connection : Int

    client_transactions

    source
    let client_transactions : Int

    concat_bytes

    source
    fn concat_bytes(a : Bytes, b : Bytes) -> Bytes

    Concatenate two byte strings.

    decode_text_value

    source
    fn decode_text_value(raw : Bytes?, col : ColumnDef) -> 
    Value     raise MysqlError

    Map one text-protocol cell (None = the 0xFB NULL sentinel) onto a [@moondb.Value] using the column's declared type. Integers narrow to Int except BIGINT, which keeps 64 bits; FLOAT/DOUBLE become Double; binary-collation strings become Blob; everything else (VARCHAR, TEXT, DECIMAL, temporal types as ISO text, JSON) becomes Text.

    ed25519_sign

    source
    fn ed25519_sign(secret : Bytes, msg : Bytes) -> Bytes

    Ed25519 signing (RFC 8032 §5.1.6), deterministic, keyed by secret (expanded via [ed_expand]). Returns the 64-byte R || S: r = SHA-512(prefix || M) modl, R = [r]B, k = SHA-512(R || A || M) mod l, S = (r + k·s) mod l.

    is_auth_switch_request

    source
    fn is_auth_switch_request(payload : Bytes) -> Bool

    Whether payload is an AuthSwitchRequest (0xFE lead byte with a body — the length is what tells it apart from a 5-byte EOF, exactly as [is_eof_packet] keys off < 9).

    is_eof_packet

    source
    fn is_eof_packet(payload : Bytes) -> Bool

    Whether payload is an EOF packet (first byte 0xFE, fewer than 9 bytes — which is what tells it apart from a row whose first cell is an 8-byte length-encoded value).

    is_err_packet

    source
    fn is_err_packet(payload : Bytes) -> Bool

    Whether payload is an ERR packet (first byte 0xFF).

    is_ok_packet

    source
    fn is_ok_packet(payload : Bytes) -> Bool

    Whether payload is an OK packet (first byte 0x00, at least 7 bytes).

    mariadb_client_cache_metadata

    source
    let mariadb_client_cache_metadata : Int

    mariadb_client_com_multi

    source
    let mariadb_client_com_multi : Int

    mariadb_client_extended_metadata

    source
    let mariadb_client_extended_metadata : Int

    mariadb_client_progress

    source
    let mariadb_client_progress : Int

    mariadb_client_stmt_bulk_operations

    source
    let mariadb_client_stmt_bulk_operations : Int

    mariadb_ed25519_response

    source
    fn mariadb_ed25519_response(password : Bytes, scramble : Bytes) -> Bytes

    The MariaDB client_ed25519 response: the 64-byte Ed25519 signature of the server's 32-byte challenge, keyed by the password. Unlike mysql_native_password, the ed25519 plugin signs unconditionally — an empty password expands to SHA-512("") and still yields a valid signature.

    native_password_scramble

    source
    fn native_password_scramble(password : Bytes, salt : Bytes) -> Bytes

    The mysql_native_password challenge response: SHA1(password) XOR SHA1(salt ++ SHA1(SHA1(password))), 20 bytes. An empty password produces an empty response (the server accepts a zero-length token).

    parse_auth_switch_request

    source
    fn parse_auth_switch_request(payload : Bytes) -> (String, Bytes) raise MysqlError

    Decode an AuthSwitchRequest into the plugin name the server wants and the fresh 20-byte auth salt for it. The trailing NUL that follows the scramble is dropped so the salt feeds [native_password_scramble] directly.

    parse_column_def

    source
    fn parse_column_def(payload : Bytes) -> ColumnDef raise MysqlError

    Decode a column-definition packet (protocol 41). Only the fields that steer text decoding are kept; catalog/schema/table names and the length/decimals fields are read past.

    parse_ed25519_challenge

    source
    fn parse_ed25519_challenge(payload : Bytes) -> Bytes raise MysqlError

    The 32-byte nonce from a client_ed25519 AuthSwitchRequest. MariaDB sends exactly NONCE_BYTES (= 32) with no NUL terminator and the client signs all of them, so — unlike the native scramble — the full 32 bytes are read raw rather than through the NUL-stripping [parse_auth_switch_request].

    parse_err

    source
    fn parse_err(payload : Bytes) -> MysqlError raise MysqlError

    Decode a full ERR packet (including its 0xFF marker) into the [MysqlError::ServerError] it represents.

    parse_handshake

    source
    fn parse_handshake(payload : Bytes) -> Handshake raise MysqlError

    Parse a protocol-10 initial Handshake payload. An ERR packet in its place (0xFF, e.g. "Host is blocked" / "Too many connections") is surfaced as a [MysqlError::ServerError].

    parse_ok

    source
    fn parse_ok(payload : Bytes) -> OkPacket raise MysqlError

    Decode an OK packet (including its 0x00 marker).

    parse_rsa_public_key

    source

    Parse a PEM SubjectPublicKeyInfo (-----BEGIN PUBLIC KEY-----) into the RSA modulus and public exponent. Walks SEQUENCE { AlgorithmIdentifier, BIT STRING{ RSAPublicKey { INTEGER n, INTEGER e } } }.

    parse_server_version

    source
    fn parse_server_version(raw : String) -> (ServerKind, String)

    Classify a raw handshake version string and recover the real version.

    MariaDB 10+ prefixes its version with the fake 5.5.5- sentinel (MySQL 5.5.5 was never released), so pre-10 clients that gate on the leading 5. still connect. Strip it to get the true version — e.g. 5.5.5-11.4.2-MariaDB-ubu2404 → 11.4.2-MariaDB-ubu2404 as [ServerKind::MariaDB]; a plain 8.0.35 is returned unchanged as [ServerKind::MySQL].

    put_lenenc_bytes

    source
    fn put_lenenc_bytes(buf : 
    Buffer    , s : Bytes) -> Unit

    Append a length-encoded string (its lenenc length prefix, then the bytes).

    put_lenenc_uint

    source
    fn put_lenenc_uint(buf : 
    Buffer    , v : Int64) -> Unit

    Append a length-encoded unsigned integer.

    put_string_nul

    source
    fn put_string_nul(buf : 
    Buffer    , s : Bytes) -> Unit

    Append raw bytes followed by a NUL terminator.

    put_uint_le

    source
    fn put_uint_le(buf : 
    Buffer    , v : Int64, n : Int) -> Unit

    Append an n-byte little-endian unsigned integer.

    rsa_oaep_sha1_encrypt

    source
    fn rsa_oaep_sha1_encrypt(msg : Bytes, n : 
    BigInt    , e : 
    BigInt    , seed : Bytes) -> Bytes raise MysqlError

    EME-OAEP encode msg (SHA-1, empty label) to k octets with the given seed, then RSA-encrypt: EM^e mod n. seed must be 20 random bytes (RFC 8017 §7.1.1).

    sha1

    source
    fn sha1(msg : Bytes) -> Bytes

    SHA-1 (FIPS 180-4) over msg, returning the 20-byte digest.

    MoonBit's core ships no SHA-1, but mysql_native_password's challenge-response is built entirely on it (SHA1(password) XOR SHA1(salt + SHA1(SHA1(password)))), so the driver carries its own. Pure and allocation-simple; verified against the FIPS/NIST test vectors in the suite.

    sha256

    source
    fn sha256(msg : Bytes) -> Bytes

    SHA-256 (FIPS 180-4).

    sha512

    source
    fn sha512(msg : Bytes) -> Bytes

    SHA-512 (FIPS 180-4), 64-bit words over 80 rounds. Messages here are far under 2^64 bits, so the 128-bit length field's high half is always zero.

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io