moondb

The standard database-access interface for MoonBit — the driver↔query-layer contract, transliterated from Go's database/sql/driver and Python's DB-API 2.0. ORMs (moonorm) build on it; drivers (moon-sqlite / moon-postgres / moon-mysql) implement it.

database
sql
driver
db-api
moonbit
interface
moon add Lfan-ke/moondb@0.1.5
Download zip
Author
Version
0.1.5
License
Apache-2.0
Last updated
17 days ago
Downloads
570
README

#moondb

The standard database-access interface for MoonBit.

The small, pure contract that sits between database drivers and query layers — MoonBit's answer to Go's database/sql/driver and Python's DB-API 2.0 (PEP 249).

Check and Test tests API docs mooncakes license

moondb defines one thing: the boundary every SQL database backend implements and every ORM / query builder is written against. It has zero dependencies, is pure (compiles on wasm, wasm-gc, js, and native alike), and ships a dependency-free reference driver so the whole stack above it can be tested without a database.

It deliberately does not connect to a database, speak a wire protocol, or build SQL. Those belong on either side of the seam:

flowchart TD subgraph layers["query layers — build on moondb"] ORM["moonorm<br/><small>SQLAlchemy-style ORM / query builder</small>"] APP["your app / repository code"] end IFACE(["<b>@moondb</b><br/><small>Value · Row · ExecResult · DbError · Driver</small>"]) subgraph drivers["drivers — implement moondb"] SQLITE["moon-sqlite<br/><small>C-FFI, native</small>"] PG["moon-postgres<br/><small>pure-MoonBit wire</small>"] MYSQL["moon-mysql<br/><small>pure-MoonBit wire</small>"] MOCK["MockDriver<br/><small>in-memory, ships here</small>"] end APP --> ORM --> IFACE IFACE --> SQLITE IFACE --> PG IFACE --> MYSQL IFACE --> MOCK

The value of a contract package is that the two sides are written by different people at different times and still fit: a driver author implements Driver once, and every query layer works against their backend; an ORM author targets Driver once, and every driver works under their ORM.

#Install

moon add Lfan-ke/moondb

#The interface at a glance

TypeRoleTransliterated from
Valueone dialect-neutral cell — the unit crossing the boundary both waysGo driver.Value, DB-API type objects
Rowone result row: aligned column names + Values, with typed accessorssql.Rows / DB-API row tuple
ExecResultoutcome of a non-query: rows affected + last insert idGo sql.Result, DB-API rowcount/lastrowid
DbErrorthe one error every operation raisesDB-API exception hierarchy (flattened)
Driverthe trait a backend implements / a query layer targets (with a default ping health probe)Go driver.Conn+Execer+Queryer+Pinger
Poola fixed-ceiling connection pool over any DriverGo sql.DB pool, SQLAlchemy QueuePool
MockDriverdependency-free in-memory reference driver, for tests

#The binding contract

execute and query both take (sql, params). The SQL string carries positional placeholders and params supplies one Value per placeholder, in order. Values are bound out-of-band by the driver and are never spliced into the SQL text — that is what makes a moondb-based stack injection-safe all the way to the wire.

The placeholder token is the driver's dialect (? for SQLite/MySQL, $1, $2, … for PostgreSQL); moondb fixes the calling convention — an ordered Array[Value] — not the spelling.

#Quickstart

test "round-trip through the interface" {
// A query layer is written against the `Driver` trait, not a concrete backend.
let db = @moondb.MockDriver::new()

// Bind values as parameters — never string-interpolated into the SQL.
db.execute("INSERT INTO hero (id, name) VALUES (?, ?)", [Int(1), Text("Nova")])
|> ignore

db.begin()
db.execute("INSERT INTO hero (id, name) VALUES (?, ?)", [Int(2), Text("Iris")])
|> ignore
db.rollback() // Iris is undone; Nova remains.

let rows = db.query("SELECT id, name FROM hero", [])
assert_eq(rows.length(), 1)
assert_eq(rows[0].int_by("c0"), 1)
assert_eq(rows[0].text_by("c1"), "Nova")
}

Swap MockDriver for moon-sqlite / moon-postgres / moon-mysql and the same code runs against a real database — that substitutability is the point of the package.

#Implementing a driver

A backend implements the six-method Driver trait. DbError is pub(all), so a driver in its own package can construct and raise every case:

pub impl @moondb.Driver for MyConn with execute(self, sql, params) {
// ... bind `params` positionally, run `sql`, then:
{ rows_affected: n, last_insert_id: id }
}
// query / begin / commit / rollback / close likewise.

execute/query/begin/commit/rollback raise DbError on failure; close is best-effort and idempotent.

#Design notes

  • Why raise, not Result. Typed accessors and driver calls raise DbError rather than returning Result[_, DbError], so a decode bug or a dropped connection surfaces at the call site instead of being silently swallowed. A caller opts into recovery with try/catch.
  • Typed accessors are strict. row.int(i) raises TypeError if the cell is not an integer — including when it is NULL. Guard nullable columns with is_null first. Integer→integer and integer→double conversions are allowed and lossless; int narrows an Int64 and says so.
  • DbError is pub(all). A plain pub suberror can be caught from another package but not constructed — which would stop out-of-tree drivers from raising it. pub(all) opens the constructors.
  • The pool is synchronous. Pool[D] reuses idle connections under a size ceiling, evicts a connection that fails its pre_ping probe or outlives max_lifetime (age measured by an injected clock, the way database/sql swaps nowFunc in tests), and offers a non-blocking try_acquire. Because moondb's base contract is sync and the pure backends have no threads, an exhausted acquire fails immediately rather than blocking — the acquire_timeout is the budget an async driver layers real waiting on top of. ping is a default Driver method (SELECT 1), so every driver gets a health probe for free.

#Roadmap

v0.1 fixes the smallest contract that a relational backend and a query layer both need, so it can stabilise. Planned, additive extensions (none of which change the v0.1 surface):

  • Prepared statements — a Stmt handle for repeated execution with rebinding.
  • Streaming rows — a Rows cursor so large result sets need not fully materialise.
  • A temporal Value case — dates/times currently ride as ISO-8601 Text; a dedicated case (with a decided epoch/precision) will follow.
  • Named parameters and nested transactions (savepoints) as a layer over the flat begin/commit/rollback.

#Tests

The suite covers the value model, every typed accessor and its error path, the error type, the reference driver — including real transaction rollback/commit semantics — and the connection pool. Key behaviours are mutation-verified: breaking rollback turns the transaction tests red, and disabling the pool's pre_ping eviction or its max_lifetime retirement turns the corresponding pool tests red. They run on all four backends:

moon test --target all

#License

Apache-2.0 © Leo Cheng

#
Cursor

pub(open) trait Cursor {
fn next(Self) -> Row? raise DbError
fn close(Self) -> Unit
}

A forward-only cursor over a query's rows — the streaming counterpart to [Driver::query]. Where query materialises the whole result, a cursor yields one [Row] at a time, so a large result set is consumed in bounded memory (SQLAlchemy's yield_per / server-side cursors, Go's sql.Rows, Python DB-API's fetchone).

next advances and returns the next row, or None once the result is exhausted; close releases the cursor early (a driver holding a server-side cursor or prepared statement frees it here). A driver with real incremental fetch — moon-sqlite steps its prepared statement, an async driver reads rows off the wire on demand — returns a live cursor; the default [Driver::query_stream] falls back to an [ArrayCursor] over a materialised result, which honours the same interface without the memory bound.

pub(open) so out-of-tree drivers implement it.

#
Driver

pub(open) trait Driver {
fn execute(Self, String, Array[Value]) -> ExecResult raise DbError
fn query(Self, String, Array[Value]) -> Array[Row] raise DbError
fn query_stream(Self, String, Array[Value]) -> &Cursor raise DbError = _
fn begin(Self) -> Unit raise DbError
fn commit(Self) -> Unit raise DbError
fn rollback(Self) -> Unit raise DbError
fn close(Self) -> Unit
fn ping(Self) -> Bool = _
}

The contract every database backend implements and every query layer is written against — the seam moondb exists to define. It is the MoonBit transliteration of Go's database/sql/driver.Conn/Execer/Queryer and Python's DB-API 2.0 Connection/Cursor, reduced to the smallest set of operations a relational backend must offer.

The binding contract

execute and query both take (sql, params). sql carries positional placeholders and params supplies one [Value] per placeholder, in order. Values are bound out-of-band by the driver and are never interpolated into the SQL text — that is what makes a moondb-based stack injection-safe all the way to the wire.

The placeholder token itself is dialect-specific and chosen by the driver (SQLite/MySQL use ?, PostgreSQL uses $1, $2, …); a query layer that emits SQL for a given backend uses that backend's token. moondb fixes the calling convention (ordered params array), not the spelling.

  • execute runs a statement that returns no rows (INSERT/UPDATE/DELETE/DDL) and reports an [ExecResult].
  • query runs a statement that returns rows and materialises every [Row].
  • begin / commit / rollback bracket an explicit transaction. Nesting (savepoints), isolation levels, and read-only hints are driver concerns layered on top; the base contract is the three flat operations.
  • close releases the connection. It does not raise: closing is best-effort and idempotent, matching Go's io.Closer discipline for connections.

This trait is pub(open) so out-of-tree drivers (moon-sqlite, moon-postgres, moon-mysql, …) can implement it. A future prepared-Stmt handle and a streaming Rows cursor are noted in the README roadmap; v0.1 fixes exactly the operations below so the contract everything pins to stays small and stable.

#
DbError

pub(all) suberror DbError {
ConnectError(String)
QueryError(String)
TypeError(String)
Closed
}

A database-layer failure. Every fallible operation on the interface raises this one error type, so a query layer catches a single kind. The cases mirror the coarse failure classes both Go's database/sql and Python's DB-API 2.0 draw a line around, kept deliberately few so drivers can classify without a taxonomy:

  • ConnectError — opening, attaching, or handshaking a connection failed, or the connection dropped mid-flight.
  • QueryError — the backend rejected a statement: a prepare/step/bind error, a constraint violation, or a protocol error. Carries the backend's message.
  • TypeError — a [Row] typed accessor was asked to read a column as a type it does not hold (e.g. int on a Text), or a column index/name that does not exist in the row.
  • Closed — the connection or statement was used after close.

Declared pub(all) so out-of-tree drivers (moon-postgres, moon-mysql, …) that live in their own packages can construct and raise these cases — a plain pub suberror would let them catch a DbError but not build one.
impl Show for DbError

#
DbError::to_string

fn DbError::to_string(self : DbError) -> String

A one-line rendering of the error, e.g. QueryError: no such table: hero.

#
ArrayCursor

pub struct ArrayCursor {
rows : Array[Row]
pos : Int
}

A [Cursor] over an already-materialised Array[Row]. The fallback the default [Driver::query_stream] hands back for any driver that has not overridden it: it preserves the streaming interface (callers pull rows one at a time) even though the rows were fetched up front. The mock driver and the reference query layer test against it.

#
ArrayCursor::new

fn ArrayCursor::new(rows : Array[Row]) -> ArrayCursor

A cursor positioned before the first of rows.

#
ExecResult

pub(all) struct ExecResult {
rows_affected : Int64
last_insert_id : Int64
} derive(Eq)

The outcome of a non-query statement (INSERT/UPDATE/DELETE/DDL): how many rows it changed and the id of the last inserted row.

rows_affected is 64-bit to hold bulk DML counts; last_insert_id is the backend's auto-increment / rowid value and is backend-defined (0 when the statement produced none). This mirrors Go's sql.Result (RowsAffected / LastInsertId) and DB-API's cursor.rowcount / lastrowid.

#
MockDriver

pub struct MockDriver {
rows : Array[Row]
next_id : Int64
savepoints : Array[Int]
closed : Bool
}

A dependency-free, in-memory reference [Driver]. It exists for two reasons: to prove the interface is implementable end to end, and to give query layers built on moondb (moonorm and friends) a real test double they can run against with no database and no native backend — it compiles on every target.

It is a deliberately naive echo store, not a SQL engine: it does not parse sql. Each execute appends one [Row] built from the bound params (columns named c0, c1, …) and hands back an [ExecResult] with a monotonically increasing last_insert_id; each query returns every stored row. What it does model faithfully is the transaction bracket: begin snapshots the store, rollback restores it, commit keeps the changes — so a test can assert real rollback semantics against it.

#
MockDriver::in_transaction

fn MockDriver::in_transaction(self : MockDriver) -> Bool

Whether a transaction is currently open (at least one un-committed begin).

#
MockDriver::is_closed

fn MockDriver::is_closed(self : MockDriver) -> Bool

Whether the connection has been closed.

#
MockDriver::new

fn MockDriver::new() -> MockDriver

A fresh, empty mock connection.

#
MockDriver::row_count

fn MockDriver::row_count(self : MockDriver) -> Int

How many rows the store currently holds. A test-facing helper, not part of the [Driver] contract.

#
Pool

pub struct Pool[D] {
make : () -> D raise DbError
idle : Array[(D, Int64)]
checked_out : Array[(D, Int64)]
open_count : Int
max_size : Int
max_lifetime_ms : Int64
pre_ping : Bool
acquire_timeout_ms : Int64
clock : () -> Int64
closed : Bool
}

A fixed-ceiling connection pool over any [Driver]. Opening a database connection is expensive — a TCP handshake and auth round trip for a networked backend — so a server hands out a small set of connections and reuses them instead of opening one per request. This is the moondb counterpart of Go's sql.DB connection pool and SQLAlchemy's QueuePool.

The pool is generic in the concrete driver D rather than holding &Driver trait objects, so acquire gives back the real driver type and a caller keeps access to backend-specific methods (SQLite's exec_script, a driver's prepared-statement handle). new takes a make factory that opens one fresh connection; the pool calls it only when no idle connection is available and the open count is still below max_size.

Beyond plain reuse, the pool keeps a connection healthy over its lifetime, matching the knobs both database/sql and SQLAlchemy's QueuePool expose:

  • pre-ping (pre_ping) — before an idle connection is handed back it is probed with [Driver::ping]; a connection that fails the probe is closed and skipped, so a caller never receives a connection the backend has already dropped. This is SQLAlchemy's pool_pre_ping.
  • max lifetime (max_lifetime_ms) — a connection older than this is retired on acquire and replaced with a fresh one, so long-lived pools recycle connections a load balancer or the server may have aged out. This is Go's SetConnMaxLifetime. Age is measured with an injected clock, exactly as database/sql swaps nowFunc in tests; the default clock disables the check.
  • acquire timeout (acquire_timeout_ms) — the wait budget an exhausted acquire is allowed. See the note on synchrony below.

It is synchronous and single-threaded by design — moondb's base contract is sync, and MoonBit's pure backends have no shared-memory threads — so acquire never truly blocks: nothing else can return a connection while one call waits. A request that finds the pool exhausted therefore fails immediately rather than sleeping; acquire_timeout_ms is carried into that failure's message and is the budget a driver running on an async runtime honours when it layers real waiting on top. Use [try_acquire] for the non-raising "give me one only if free" path. The reuse, ceiling, health, and lifecycle bookkeeping all live here.

#
Pool::acquire

fn[D : Driver] Pool::acquire(self : Pool[D]) -> D raise DbError

Take a connection: reuse an idle one if the pool has a healthy, unexpired one (the common path, avoiding a fresh handshake), otherwise open a new one through make as long as the open count is below max_size.

An idle connection past max_lifetime_ms, or one that fails the [Driver::ping] probe when pre_ping is on, is closed and dropped rather than handed back; the pool then tries the next idle connection or opens a fresh one. Raises QueryError if the pool is exhausted (all max_size connections are checked out) or [Closed] if it has been closed. The caller must return the connection with [release] — or use [with_conn], which does so even on error.

#
Pool::acquire_timeout_ms

fn[D] Pool::acquire_timeout_ms(self : Pool[D]) -> Int64

The configured acquire wait budget in milliseconds (0 = fail immediately).

#
Pool::close_all

fn[D : Driver] Pool::close_all(self : Pool[D]) -> Unit

Close every idle connection and mark the pool closed. Connections still checked out are not touched — each is closed as it is [release]d back. After this, acquire raises Closed. Idempotent.

#
Pool::idle_count

fn[D] Pool::idle_count(self : Pool[D]) -> Int

How many connections are currently idle (checked in and reusable).

#
Pool::is_closed

fn[D] Pool::is_closed(self : Pool[D]) -> Bool

Whether [close_all] has been called.

#
Pool::max_lifetime_ms

fn[D] Pool::max_lifetime_ms(self : Pool[D]) -> Int64

The configured maximum connection lifetime in milliseconds (0 = unlimited).

#
Pool::max_size

fn[D] Pool::max_size(self : Pool[D]) -> Int

The pool's ceiling on simultaneously open connections.

#
Pool::new

fn[D] Pool::new(make : () -> D raise DbError, max_size? : Int, max_lifetime_ms? : Int64, pre_ping? : Bool, acquire_timeout_ms? : Int64, clock? : () -> Int64) -> Pool[D] raise DbError

Build a pool whose connections come from make.

  • max_size caps how many connections may be open at once (in use plus idle); it defaults to 10, the ceiling Go's database/sql uses out of the box, and must be positive.
  • max_lifetime_ms retires a connection older than this many milliseconds on acquire (0, the default, means no age limit). Enforcing it requires a real clock; with the default clock every connection reads as age 0.
  • pre_ping turns on the [Driver::ping] health probe before an idle connection is reused (off by default).
  • acquire_timeout_ms records the wait budget for an exhausted acquire; 0 (the default) means "fail immediately". In this synchronous pool the wait is always degenerate (see the type doc), so the value serves the error message and an async layer above.
  • clock returns a monotonically non-decreasing millisecond reading and exists so max_lifetime is testable and portable; it defaults to a constant 0, which disables age-based retirement.

#
Pool::open_count

fn[D] Pool::open_count(self : Pool[D]) -> Int

How many connections the pool has open in total: those checked out plus those sitting idle. Never exceeds max_size.

#
Pool::pre_ping

fn[D] Pool::pre_ping(self : Pool[D]) -> Bool

Whether the pre-ping health probe is enabled.

#
Pool::release

fn[D : Driver] Pool::release(self : Pool[D], conn : D) -> Unit

Return a connection to the idle set so a later [acquire] can reuse it, preserving the birth time it was opened with so max_lifetime keeps counting from creation rather than resetting on every checkout. A connection released back into a closed pool is closed immediately rather than pooled, so no handle outlives close_all.

#
Pool::try_acquire

fn[D : Driver] Pool::try_acquire(self : Pool[D]) -> D? raise DbError

Take a connection only if one is immediately available — an idle one or fresh headroom under max_size — returning None instead of raising when the pool is exhausted. This is the non-blocking counterpart of [acquire]: it never reports the exhaustion case as an error, so a caller can choose to shed load or retry. Health and lifetime eviction apply exactly as in acquire; a genuine failure while opening a fresh connection (the make factory raising) still propagates.

#
Pool::with_conn

fn[D : Driver, R] Pool::with_conn(self : Pool[D], f : (D) -> R raise DbError) -> R raise DbError

Run f with a pooled connection, returning it afterwards whether f returns or raises. This is the leak-proof way to use the pool: the release happens on every path, so a raising query never strands a connection checked out.

#
Row

pub(all) struct Row {
columns : Array[String]
values : Array[Value]
}

One row of a result set: the column names in projection order alongside the decoded [Value] for each. columns and values are the same length and are index-aligned — columns[i] names values[i].

This is the shape drivers hand back from [Driver::query] and the shape a query layer decodes. Read cells either positionally ([get]/[int]/…) or by column name ([by_name]/[int_by]/…). The typed accessors raise on a type or lookup mismatch rather than returning a sentinel, so a decode bug surfaces as a DbError at the call site instead of silently reading a zero value.

#
Row::blob

fn Row::blob(self : Row, idx : Int) -> Bytes raise DbError

Read column idx as an opaque blob. Raises TypeError unless the cell is Blob.

#
Row::blob_by

fn Row::blob_by(self : Row, name : String) -> Bytes raise DbError

Read the column named name as an opaque blob.

#
Row::bool

fn Row::bool(self : Row, idx : Int) -> Bool raise DbError

Read column idx as a Bool. Raises TypeError unless the cell is Bool.

#
Row::bool_by

fn Row::bool_by(self : Row, name : String) -> Bool raise DbError

Read the column named name as a Bool.

#
Row::by_name

fn Row::by_name(self : Row, name : String) -> Value?

The [Value] of the column named name, or None if the row has no such column. Names match exactly as the driver reported them (case-sensitive).

#
Row::double

fn Row::double(self : Row, idx : Int) -> Double raise DbError

Read column idx as a Double, widening an integer. Raises TypeError unless the cell is Double, Int, or Int64.

#
Row::double_by

fn Row::double_by(self : Row, name : String) -> Double raise DbError

Read the column named name as a Double.

#
Row::get

fn Row::get(self : Row, idx : Int) -> Value raise DbError

The raw [Value] at column idx (0-based, projection order). Raises QueryError if idx is out of range.

#
Row::index_of

fn Row::index_of(self : Row, name : String) -> Int raise DbError

Resolve a column name to its index, raising QueryError if absent. The positional accessors are the primitives; the _by-name accessors resolve through this.

#
Row::int

fn Row::int(self : Row, idx : Int) -> Int raise DbError

Read column idx as an Int, narrowing an Int64 to its low 32 bits. Raises TypeError unless the cell is Int or Int64.

#
Row::int64

fn Row::int64(self : Row, idx : Int) -> Int64 raise DbError

Read column idx as an Int64, widening an Int. Raises TypeError unless the cell is Int or Int64.

#
Row::int64_by

fn Row::int64_by(self : Row, name : String) -> Int64 raise DbError

Read the column named name as an Int64.

#
Row::int_by

fn Row::int_by(self : Row, name : String) -> Int raise DbError

Read the column named name as an Int.

#
Row::is_null

fn Row::is_null(self : Row, idx : Int) -> Bool raise DbError

Whether column idx is SQL NULL. Raises QueryError if idx is out of range. Call this before a typed accessor when a column is nullable, since the typed accessors treat NULL as a type mismatch.

#
Row::is_null_by

fn Row::is_null_by(self : Row, name : String) -> Bool raise DbError

Whether the column named name is SQL NULL. Raises QueryError if the column is absent.

#
Row::text

fn Row::text(self : Row, idx : Int) -> String raise DbError

Read column idx as text. Raises TypeError unless the cell is Text.

#
Row::text_by

fn Row::text_by(self : Row, name : String) -> String raise DbError

Read the column named name as text.

#
Row::width

fn Row::width(self : Row) -> Int

The number of columns in the row.

#
Value

pub(all) enum Value {
Null
Bool(Bool)
Int(Int)
Int64(Int64)
Double(Double)
Text(String)
Blob(Bytes)
} derive(Eq)

A single dialect-neutral database value — the unit of everything that crosses the driver boundary in either direction. Bound parameters travel in as Values (never spliced into the SQL string, so binding is injection-safe by construction) and result columns come back as Values.

This is the moondb analogue of Go's driver.Value and Python DB-API's input type objects: a small, closed sum every backend can map onto its own wire types. Drivers are expected to normalise their column types down to these seven cases; a query layer built on moondb reads them back through [Row]'s typed accessors.

  • Null — SQL NULL / a missing value.
  • Bool — a boolean; backends without a native boolean map 0/1.
  • Int — a 32-bit signed integer.
  • Int64 — a 64-bit signed integer (rowids, BIGINT, counters).
  • Double — an IEEE-754 double (REAL / FLOAT8).
  • Text — a UTF-8 string (TEXT / VARCHAR); dates and times ride here as ISO-8601 text until a dedicated temporal case lands (see the README roadmap).
  • Blob — an opaque byte string (BLOB / BYTEA).
impl Show for Value

#
Value::is_null

fn Value::is_null(self : Value) -> Bool

Whether this value is SQL NULL.

#
Value::kind

fn Value::kind(self : Value) -> String

The name of a value's variant ("Null", "Int", "Blob", …). Used to build legible [DbError::TypeError] messages when a typed accessor is asked for the wrong shape.

#
drain

fn drain(cursor : &Cursor) -> Array[Row] raise DbError

Drain a cursor into an array — the inverse of streaming, for callers that do want every row (and for asserting a cursor yields exactly what query would).