moon-postgres

Pure-MoonBit PostgreSQL wire-protocol driver implementing @moondb.Driver — no C, like asyncpg.

postgres
driver
moondb
wire
database
moon add Lfan-ke/moon-postgres@0.2.0
Download zip
Author
Version
0.2.0
License
Apache-2.0
Last updated
17 days ago
Downloads
65

Dependencies

README

#moon-postgres

A pure-MoonBit PostgreSQL driver — the v3 frontend/backend wire protocol spoken directly over async TCP, with zero C and no libpq. Like asyncpg or pg8000, it talks to a real PostgreSQL server itself. It implements the @moondb.Driver contract, so a moondb-based stack (e.g. moonorm) can sit on top of it.

moon add Lfan-ke/moon-postgres

Native-only: the connection layer rides moonbitlang/async sockets, which have no JS/wasm backend.

#Quickstart

The real driver is the async PgConn (asyncpg-shaped), used inside an event loop (async fn main / async test):

async fn run() -> Unit raise {
let conn = @moon_postgres.PgConn::connect(
"127.0.0.1", 5432, "postgres", "postgres", "test",
)
conn.execute("CREATE TABLE hero (id int, name text)", []) |> ignore

// '?' placeholders are translated to '$1','$2' and bound out-of-band
// (never spliced into the SQL) — injection-safe to the wire.
conn.execute("INSERT INTO hero VALUES (?, ?)", [
@moondb.Int(1), @moondb.Text("Boromir"),
]) |> ignore

let rows = conn.query("SELECT id, name FROM hero WHERE id = ?", [@moondb.Int(1)])
println(rows[0].text_by("name")) // Boromir

conn.begin()
conn.execute("UPDATE hero SET name = ? WHERE id = ?", [
@moondb.Text("Faramir"), @moondb.Int(1),
]) |> ignore
conn.commit()

conn.close()
}

#What it speaks

sequenceDiagram participant C as PgConn (client) participant S as PostgreSQL C->>S: StartupMessage (user, database, client_encoding) S-->>C: Authentication (Ok / MD5 / cleartext) C->>S: PasswordMessage (md5… token) %% if requested S-->>C: AuthenticationOk S-->>C: ParameterStatus*, BackendKeyData, ReadyForQuery Note over C,S: simple query (no params) C->>S: Query 'Q' S-->>C: RowDescription 'T', DataRow 'D'*, CommandComplete 'C', ReadyForQuery 'Z' Note over C,S: extended query (bound params, ?→$n) C->>S: Parse 'P', Bind 'B', Describe 'D', Execute 'E', Sync 'S' S-->>C: ParseComplete, BindComplete, RowDescription, DataRow*, CommandComplete, ReadyForQuery

  • Framing — every message is a 1-byte type tag, a big-endian Int32 self-inclusive length, and a payload (the untagged StartupMessage aside).
  • AuthAuthenticationOk (trust), cleartext, and MD5 (md5(md5(password+user)+salt), with a self-contained pure-MoonBit MD5). SCRAM-SHA-256 is on the roadmap.
  • Simple query (Q) when there are no parameters; extended query (Parse/Bind/Describe/Execute/Sync) when there are, with ?$n translation and out-of-band text-format binding.
  • DecodingRowDescription + DataRow decode into @moondb.Row; column type OIDs map int2/int4Int, int8Int64, float4/float8Double, boolBool, byteaBlob, everything else→Text.

#The async wall

@moondb.Driver's methods are synchronous (fn execute(...) raise DbError), which fits an FFI-backed driver like moon-sqlite whose C calls block. PostgreSQL is reached over TCP, and MoonBit's only socket stack (moonbitlang/async) is async-only: an async fn cannot be called from a synchronous one, and the runtime exposes no public "run this async thunk to completion" bridge (with_event_loop lives in an import-blocked internal package). A synchronous method therefore cannot perform a PostgreSQL round trip.

So the faithful, working driver is the async PgConn, and it is what the CI integration suite drives against a live server. PgDriver still implements every @moondb.Driver method to demonstrate the seam and give moondb-based code a stable target; its row-touching methods raise a precise ConnectError directing callers to PgConn, and PgDriver::connect returns the real async connection. When moondb grows an async Driver variant — or MoonBit ships a blocking socket / a public event-loop entry — PgDriver becomes a thin adapter over PgConn with no change for callers.

This is the language-idiomatic equivalent, not a shortcut: asyncpg is async for exactly the same reason.

#Testing

  • Unit (moon test) — MD5 vectors, ?$n translation (literals/identifiers/comments/dollar-quotes/??), value encode/decode by OID, and message framing + RowDescription/DataRow decode against synthetic backend messages.
  • Integration (moon test, gated on MOON_PG_TEST) — connect / DDL / parameterised INSERT / SELECT round trip and transaction commit+rollback against a real PostgreSQL. Skipped when MOON_PG_TEST is unset (local checkout without a database); CI stands up a postgres:16 service and runs it. Connection parameters come from the standard PGHOST/PGPORT/PGUSER/PGPASSWORD/PGDATABASE variables.

The DataRow decoder is mutation-checked: breaking it makes the decode test fail with a type error.

#Roadmap

This round covers connect + auth (MD5/trust) + simple query + text decode + ?$n + @moondb.Driver conformance + a real round trip. Later rounds:

  • SCRAM-SHA-256 authentication (the default for PostgreSQL 14+ with a password set).
  • Binary result/parameter format and a type-OID registry (numeric, dates/times, arrays, JSON, UUID).
  • Prepared statements (named, cached) and a streaming Rows cursor.
  • RETURNING-based last_insert_id, COPY, LISTEN/NOTIFY, TLS, and a connection pool.
  • An async Driver trait in moondb so PgDriver becomes a first-class synchronous-signature adapter over PgConn.

#License

Apache-2.0 © Leo Cheng.

#
DbError

using @Lfan-ke/moondb { type DbError }

The single database error type every fallible operation raises, aliased from moondb so driver signatures read raise DbError.

#
ExecResult

The outcome of a non-row statement, aliased from moondb.

#
Row

using @Lfan-ke/moondb { type Row }

One decoded result row (column names + values), aliased from moondb.

#
Value

using @Lfan-ke/moondb { type Value }

The dialect-neutral value type this driver binds and decodes. Aliased so the wire code reads Value rather than @moondb.Value throughout; the enum constructors (Int, Text, …) are still spelled @moondb.Int where a value is built.

#
PgConn

pub struct PgConn {
tcp :
Tcp

closed : Bool
}

A live connection to a PostgreSQL backend, already past the startup handshake and sitting at ReadyForQuery. Runs the v3 frontend/backend protocol over a single @socket.Tcp. Not safe for concurrent use by multiple tasks: one statement must complete (drain to ReadyForQuery) before the next begins, as the wire protocol is a strict request/response pipeline per connection.

#
PgConn::begin

async fn PgConn::begin(self : PgConn) -> Unit raise
DbError

Begin an explicit transaction (BEGIN).

#
PgConn::close

async fn PgConn::close(self : PgConn) -> Unit

Close the connection: best-effort Terminate, then close the socket. Idempotent — a second call is a no-op.

#
PgConn::commit

async fn PgConn::commit(self : PgConn) -> Unit raise
DbError

Commit the current transaction (COMMIT).

#
PgConn::connect

async fn PgConn::connect(host : String, port : Int, user : String, password : String, database : String) -> PgConn raise
DbError

Open a connection and complete the startup handshake: send StartupMessage, satisfy the authentication request (trust/AuthenticationOk, cleartext, or MD5), and drain server parameters up to the first ReadyForQuery. host may be a hostname or a literal IP; port is typically 5432. Raises ConnectError on any transport or handshake failure, including an unsupported auth method.

#
PgConn::execute

Run a non-row statement (INSERT/UPDATE/DELETE/DDL) and report rows changed. last_insert_id is 0: the simple/extended protocols do not surface a generated key without a RETURNING clause (roadmap).

#
PgConn::query

Run a row-returning statement and materialise every row. Uses the simple Query protocol when params is empty, and the extended Parse/Bind/Execute protocol (with ?$n translation and out-of-band text-format binding) when parameters are supplied.

#
PgConn::rollback

async fn PgConn::rollback(self : PgConn) -> Unit raise
DbError

Roll back the current transaction (ROLLBACK).

#
PgDriver

pub struct PgDriver {
host : String
port : Int
user : String
password : String
database : String
}

A connection descriptor that carries the parameters needed to reach a PostgreSQL backend and conforms to the synchronous [@moondb.Driver] seam.

The async wall (why this is a façade)

@moondb.Driver's methods are synchronous (fn execute(...) raiseDbError), matching an FFI-backed driver like moon-sqlite whose C calls block. PostgreSQL, however, is spoken over TCP, and MoonBit's only socket stack (moonbitlang/async) is async-only: an async fn cannot be called from a synchronous one, and the runtime exposes no public "run-this-async-thunk-to-completion" bridge (with_event_loop lives in an import-blocked internal package). A synchronous method therefore cannot perform a PostgreSQL round trip.

The faithful, working driver is the async [PgConn] (see conn.mbt), which mirrors asyncpg: PgConn::connect then .execute / .query / .begin / .commit / .rollback / .close, all async, used inside an event loop (async test / async fn main). The CI integration suite drives a real PostgreSQL through exactly that API.

PgDriver exists to demonstrate the seam and to give moondb-based code a stable target: it implements every @moondb.Driver method. The row-touching methods raise a precise ConnectError explaining that the round trip must run through PgConn under an event loop; close is a no-op. When moondb grows an async Driver variant (or MoonBit ships a blocking socket / a public event-loop entry), this façade becomes a thin adapter over PgConn with no behavioural change to callers.
impl Driver for PgDriver

#
PgDriver::connect

async fn PgDriver::connect(self : PgDriver) -> PgConn raise
DbError

Open the async connection this descriptor points at. This is the intended entry point: call it inside an event loop and use the returned [PgConn].

#
PgDriver::new

fn PgDriver::new(host~ : String, port? : Int, user~ : String, password? : String, database~ : String) -> PgDriver

Build a driver descriptor. Does not connect — the async wall means the actual connection is opened by [PgConn::connect] inside an event loop.

#
OID_BOOL

let OID_BOOL : Int

#
OID_BYTEA

let OID_BYTEA : Int

#
OID_FLOAT4

let OID_FLOAT4 : Int

#
OID_FLOAT8

let OID_FLOAT8 : Int

#
OID_INT2

let OID_INT2 : Int

#
OID_INT4

let OID_INT4 : Int

#
OID_INT8

let OID_INT8 : Int

#
build_bind

fn build_bind(params : Array[
Value
]) -> Bytes

Bind ('B'): bind params (all text format) to the unnamed statement, producing the unnamed portal, and request all result columns in text format.

#
build_describe_portal

fn build_describe_portal() -> Bytes

Describe ('D') the unnamed portal, so the server sends a RowDescription before the DataRows (giving column names + type OIDs for decoding).

#
build_execute

fn build_execute() -> Bytes

Execute ('E') the unnamed portal with no row limit (0 = all rows).

#
build_parse

fn build_parse(sql : String) -> Bytes

Parse ('P'): prepare the unnamed statement from sql (with $n placeholders). Zero declared parameter types — the server infers them.

#
build_password

fn build_password(token : String) -> Bytes

PasswordMessage ('p'): the auth response token (cleartext, or the md5… digest), as a C string.

#
build_query

fn build_query(sql : String) -> Bytes

Simple Query ('Q'): one SQL string, no bound parameters.

#
build_startup

fn build_startup(user : String, database : String) -> Bytes

StartupMessage: protocol version 3.0 (196608) plus the user / database parameters and a client_encoding=UTF8 request, terminated by an empty key.

#
build_sync

fn build_sync() -> Bytes

Sync ('S'): close the extended-query batch; the server replies ReadyForQuery.

#
build_terminate

fn build_terminate() -> Bytes

Terminate ('X'): ask the backend to close the connection.

#
concat_bytes

fn concat_bytes(a : Bytes, b : Bytes) -> Bytes

Concatenate two byte strings. A small helper the MD5 auth path and message framing lean on; MoonBit's Bytes is immutable so this allocates once.

#
count_placeholders

fn count_placeholders(sql : String) -> Int

The number of ? placeholders [translate_placeholders] would consume — the count of parameters a statement expects. Shares the same scanner discipline so ?? escapes and quoted/comment spans do not count.

#
decode_bytea

fn decode_bytea(s : String) -> Bytes

Decode PostgreSQL bytea hex text (\xDEADBEEF) back to raw bytes. A value not in hex form (legacy escape format) rides back as its UTF-8 bytes.

#
decode_value

fn decode_value(oid : Int, is_null : Bool, raw : String) ->
Value

Decode a text-format result cell (raw, already UTF-8) tagged with its column oid into a [Value]. is_null marks a wire NULL (length -1), which decodes to Null regardless of type. Numeric and boolean OIDs decode to their typed cases; everything else — including numeric, dates, and unknown OIDs — rides back as Text, exactly the dialect-neutral contract moondb documents (temporal/numeric typing is a roadmap item).

#
encode_param

fn encode_param(v :
Value
) -> Bytes?

Encode a bound parameter to its text-format bytes, or None for SQL NULL (which the Bind message sends as a length of -1). Binding is out-of-band — the value never touches the SQL string — so this is the injection-safe path.

  • integers / doubles / bools render to their canonical PostgreSQL text literals (t/f for booleans);
  • Text passes through UTF-8;
  • Blob uses the bytea hex format (\x + lowercase hex), which the server accepts for a text-format bytea parameter.

#
hex_lower

fn hex_lower(data : Bytes) -> String

Lowercase hex of data, e.g. the 16-byte MD5 digest rendered as its 32-char hex string — the form PostgreSQL's MD5 auth concatenates and re-hashes.

#
md5

fn md5(msg : Bytes) -> Bytes

The MD5 digest of msg as 16 raw bytes (RFC 1321). Used only for PostgreSQL's AuthenticationMD5Password handshake; not a general-purpose hashing API. Runs the standard little-endian padding + four-round compression over 64-byte blocks.

#
parse_double

fn parse_double(s : String) -> Double?

Parse a floating-point literal from PostgreSQL text (123.45, -1e10, Infinity, NaN), or None if malformed. Handles sign, fraction, and a base-10 exponent; the special IEEE tokens PostgreSQL emits are recognised.

#
parse_int

fn parse_int(s : String) -> Int?

Parse a base-10 Int, or None if s is not a well-formed integer. Used to decode int2/int4 result text; a parse failure falls back to Text so a surprising server rendering never silently becomes a wrong number.

#
parse_int64

fn parse_int64(s : String) -> Int64?

Parse a base-10 Int64, or None on any non-digit (after an optional sign).

#
pg_md5_password

fn pg_md5_password(user : String, password : String, salt : Bytes) -> String

The AuthenticationMD5Password response token: "md5" ++hex(md5(hex(md5(password ++ user)) ++ salt)), exactly per the PostgreSQL frontend/backend protocol. salt is the four bytes the server sent.

#
translate_placeholders

fn translate_placeholders(sql : String) -> String

Rewrite moondb's dialect-neutral ? positional placeholders into PostgreSQL's numbered $1, $2, … form. moondb fixes the calling convention (an ordered params array) but leaves the placeholder spelling to the driver; PostgreSQL's extended-query protocol requires $n, so a query layer that emits ? for portability is translated here before Parse.

A ? is only a placeholder in SQL text — never inside a single-quoted string literal, a dollar-quoted string, a "-quoted identifier, or a -- / /* */ comment. Those spans are scanned through verbatim so a literal ? in data or a comment is left untouched and does not shift the parameter numbering. A literal ? an application genuinely needs in output can be written ??, which collapses to a single ? (mirroring JDBC-style escaping).

#
utf8_decode

fn utf8_decode(data : Bytes) -> String

Decode UTF-8 bytes to a String. PostgreSQL text values, column names, and error fields all arrive UTF-8 (the startup message negotiates client_encoding). Decodes the ASCII fast path directly and multi-byte sequences by code point; malformed input yields U+FFFD rather than raising, matching a lenient text codec.