superbigcup325/moonlake/catalog

superbigcup325/moonlake/catalog does not have a README file

BatchStream

pub(open) trait BatchStream {
  fn next(self : Self) -> @superbigcup325/moonlake/types.DataBatch
DataBatch? raise @superbigcup325/moonlake/types.SqlError
SqlError
}

A cursor over the batches of one scan. next returns None exactly once at the end; batches are independent and may be read lazily. Column i of every batch is schema column request.projection[i].

ScanSource

source

pub(open) trait ScanSource {
  fn open(self : Self, request : ScanRequest) -> &BatchStream raise @superbigcup325/moonlake/types.SqlError
SqlError
  fn plan(self : Self, request : ScanRequest) -> ScanPlan raise @superbigcup325/moonlake/types.SqlError
SqlError
}

A lazily scanned table: the schema is known up front, the data is not.

Catalog

source

pub struct Catalog {
  tables : Map[String, TableEntry]
}

The registry execute reads from: map table name to its entry. Plain values everywhere — build one per execution (or share one across threads that never mutate it).

Catalog::lookup

source

fn Catalog::lookup(self : Catalog, name : String) -> TableEntry?

Catalog::new

source

fn Catalog::new() -> Catalog

An empty catalog; register scanned tables into it.

Catalog::register

source

fn Catalog::register(self : Catalog, entry : TableEntry) -> Unit

Column

source

pub(all) struct Column {
  name : String
  dtype : @superbigcup325/moonlake/types.DataType
DataType
} derive(Eq, @moonbitlang/core/debug.Debug
Debug)

One column of a table schema: its name and columnar type.

Column::equal

source

fn Column::equal(Column, Column) -> Bool

Column::not_equal

source

fn Column::not_equal(x : Column, y : Column) -> Bool

Column::to_repr

source

fn Column::to_repr(Column) -> @moonbitlang/core/debug.Repr
Repr

ScanPlan

source

pub(all) struct ScanPlan {
  files : Int
  row_groups : Int
  row_groups_skipped : Int
  rows : Int64?
}

One part of a scan's work that is known before reading any row: how many files and row groups exist, and how many row groups statistics eliminated.

ScanPlan::none

source

fn ScanPlan::none() -> ScanPlan

ScanPredicate

source

pub(all) struct ScanPredicate {
  column : Int
  op : @superbigcup325/moonlake/types.CmpOp
CmpOp
  value : @superbigcup325/moonlake/types.Scalar
Scalar
}

One conjunctive predicate hint, in the table's own column numbering (not the projection's): column op constant.

Hints are advisory. The engine applies the predicate itself, so a source may ignore every hint; a source that uses one must only skip rows that provably cannot satisfy it (that is what Parquet row-group statistics are good for). Column pruning is not a hint: it is the projection.

ScanRequest

source

pub(all) struct ScanRequest {
  projection : Array[Int]
  predicates : Array[ScanPredicate]
}

What one scan of a table should produce: the columns the query reads and the predicates it may use to skip data.

ScanRequest::all

source

fn ScanRequest::all(width : Int) -> ScanRequest

Every column of a width-column table, no predicates: what a whole-file read (a writer, SELECT *) asks for.

ScanRequest::new

source

fn ScanRequest::new(projection : Array[Int], predicates : Array[ScanPredicate]) -> ScanRequest

TableEntry

source

pub(all) struct TableEntry {
  schema : TableSchema
  batches : Array[@superbigcup325/moonlake/types.DataBatch
DataBatch]
  source : &ScanSource?
}

A registered table: its schema, and either materialized batches or a lazy source. Exactly one of batches / source carries data — a lazy entry keeps batches empty and answers scan from source.

TableEntry::from_source

source

fn TableEntry::from_source(schema : TableSchema, source : &ScanSource) -> TableEntry

A table that scans its data on demand. Registering it reads no rows.

TableEntry::is_lazy

source

fn TableEntry::is_lazy(self : TableEntry) -> Bool

Whether this entry streams from a source instead of holding its batches.

TableEntry::new

source

fn TableEntry::new(schema : TableSchema, batches : Array[@superbigcup325/moonlake/types.DataBatch
DataBatch]) -> TableEntry

A table already in memory (tests, derived scratch tables, the spill bench).

TableEntry::plan

source

fn TableEntry::plan(self : TableEntry, request : ScanRequest) -> ScanPlan raise @superbigcup325/moonlake/types.SqlError
SqlError

Planning information for EXPLAIN: row groups statistics eliminated. Materialized entries have no plan to report.

TableEntry::scan

source

fn TableEntry::scan(self : TableEntry, request : ScanRequest) -> Array[@superbigcup325/moonlake/types.DataBatch
DataBatch] raise @superbigcup325/moonlake/types.SqlError
SqlError

Produce the batches one query needs: only request.projection columns, in that order, from the materialized batches or from a fresh stream. Predicates are advisory (see ScanPredicate); the caller still filters.

TableSchema

source

pub(all) struct TableSchema {
  table : String
  columns : Array[Column]
} derive(Eq, @moonbitlang/core/debug.Debug
Debug)

A table's schema: its name and ordered columns. Field names are the user-facing SQL identifiers.

TableSchema::equal

source

fn TableSchema::equal(TableSchema, TableSchema) -> Bool

TableSchema::index_of

source

fn TableSchema::index_of(self : TableSchema, name : String) -> Int?

TableSchema::not_equal

source

fn TableSchema::not_equal(x : TableSchema, y : TableSchema) -> Bool

TableSchema::to_repr

source

fn TableSchema::to_repr(TableSchema) -> @moonbitlang/core/debug.Repr
Repr

SCAN_BATCH_ROWS

source

let SCAN_BATCH_ROWS : Int

Rows per batch a scan source should aim for. Sources may emit a final smaller batch; a zero-row table emits one empty batch so the engine still sees the column layout.

drain_stream

source

fn drain_stream(stream : &BatchStream) -> Array[@superbigcup325/moonlake/types.DataBatch
DataBatch] raise @superbigcup325/moonlake/types.SqlError
SqlError

Drain a stream to a batch array. The engine stops here because joins, aggregation and sorting are still eager; a source still decides how much it decodes per batch, and a predicate passed to drain_stream keeps the materialized row count below the table's own.

project_batches

source

fn project_batches(batches : Array[@superbigcup325/moonlake/types.DataBatch
DataBatch], columns : Array[Int]) -> Array[@superbigcup325/moonlake/types.DataBatch
DataBatch]

Drop all but columns (ascending) from each batch, sharing the vectors.

Source Files

Site source Report issue Packages Build queue Skills Statistics

#BatchStream

#ScanSource

#Catalog

#Catalog::lookup

#Catalog::new

#Catalog::register

#Column

#Column::equal

#Column::not_equal

#Column::to_repr

#ScanPlan

#ScanPlan::none

#ScanPredicate

#ScanRequest

#ScanRequest::all

#ScanRequest::new

#TableEntry

#TableEntry::from_source

#TableEntry::is_lazy

#TableEntry::new

#TableEntry::plan

#TableEntry::scan

#TableSchema

#TableSchema::equal

#TableSchema::index_of

#TableSchema::not_equal

#TableSchema::to_repr

#SCAN_BATCH_ROWS

#drain_stream

#project_batches

Source Files

BatchStream

ScanSource

Catalog

Catalog::lookup

Catalog::new

Catalog::register

Column

Column::equal

Column::not_equal

Column::to_repr

ScanPlan

ScanPlan::none

ScanPredicate

ScanRequest

ScanRequest::all

ScanRequest::new

TableEntry

TableEntry::from_source

TableEntry::is_lazy

TableEntry::new

TableEntry::plan

TableEntry::scan

TableSchema

TableSchema::equal

TableSchema::index_of

TableSchema::not_equal

TableSchema::to_repr

SCAN_BATCH_ROWS

drain_stream

project_batches