🥮 mooncakes.io

    ihb2032/MoonFrame/io does not have a README file

    CsvReadOptions

    source
    pub struct CsvReadOptions {
      has_header : Bool
      delimiter : Char
      infer_schema_rows : Int
      strict_column_count : Bool
      on_parse_error : OnParseError
      allow_nonfinite_floats : Bool
      strict_quotes : Bool
      // private fields
    }

    Options that control how a CSV input is parsed into a DataFrame.

    • has_header — when true, the first row supplies column names. When false, every row is treated as data and column names are synthesised as "column1", "column2", … in declaration order; the first data row fixes the column count, so a later wider row's trailing cells are dropped under the lenient default (set strict_column_count = true to reject a ragged row instead).
    • delimiter — field separator passed through to NyaCSV.
    • infer_schema_rows — number of leading rows scanned when guessing each column's dtype. 0 (or any value <= 0) lifts the cap and scans every row, trading a slower inference pass for never mis-guessing a column from a prefix. Cells past a finite window are still parsed under the chosen dtype; one that doesn't fit is handled per on_parse_error.
    • null_values — raw strings that should be treated as null cells (both for type inference, which skips them, and for the final typed column, where they become None). The default [""] matches the empty string only, which is how an absent CSV cell is typically represented. A cell a ragged short row never held is null regardless of this list — the mapping decides what a read value means, and no value was read. Read it back through the null_values() accessor: the field itself is private, and both it and the constructor copy, so the token list a reader uses cannot be changed after the options are built.
    • strict_column_count — when true, every data row must have exactly as many cells as the header declares; a ragged row (too few or too many cells) raise ParseError instead of being silently null-padded (short row) or truncated (long row). Defaults to false, preserving the lenient behaviour that tolerates ragged rows (a permissive CSV idiom, and the reader's long-standing default).
    • on_parse_error — what to do when a non-null cell past the inference window fails to parse under its column's locked-in dtype. Raise (default) fails with ParseError(Cell(...)); Null downgrades the offending cell to a null cell and keeps the column's inferred dtype.
    • dtypes — per-column dtype overrides that pin a column's type instead of inferring it, as a name → DataType map. Only the four dtypes a reader can parse (Int / Float / Bool / String) can be pinned — anything else is refused with InvalidOperation at read time, as is a name the input does not carry (a pin is an assertion about the input; a misspelt one fails rather than silently does nothing). Columns not named keep their inferred dtype. A pinned column skips inference entirely, so it locks in even when its cells would have inferred differently — and a cell that does not parse under the pin is handled per on_parse_error, exactly as a post-window mismatch is. The map is copied, so mutating it afterwards cannot alter the options; read it back through the dtypes() accessor.
    • allow_nonfinite_floats — when true (default), the float probe accepts every non-finite result: the nan / inf / infinity literals, and a finite literal that overflows Double to a signed Infinity (1e999), so a column of those tokens infers as Float. When false, the probe rejects them, so such a column falls back to String instead of being silently retyped to Float (a non-finite token past the window is then a parse failure handled per on_parse_error).
    • strict_quotes — when true, the input is pre-scanned and an unterminated quoted field, text after a closing quote, or a bare quote inside an unquoted field raise ParseError instead of being repaired by the lenient tokeniser. Defaults to false.

    CsvReadOptions::CsvReadOptions

    source
    fn CsvReadOptions::CsvReadOptions(has_header? : Bool, delimiter? : Char, infer_schema_rows? : Int, null_values? : Array[String], strict_column_count? : Bool, on_parse_error? : OnParseError, allow_nonfinite_floats? : Bool, strict_quotes? : Bool, dtypes? : Map[String, 
    DataType    ]) -> CsvReadOptions

    Build read options. Every field has a default: header on, comma delimiter, scan the first 100 rows for inference, treat the empty string as null, tolerate ragged rows, fail the read on a cell that doesn't fit its inferred dtype, accept non-finite float literals, and tokenise leniently. CsvReadOptions::CsvReadOptions() is the all-defaults reader; name only what differs, as in CsvReadOptions::CsvReadOptions(delimiter=';', strict_quotes=true).

    null_values and dtypes are copied, so mutating them afterwards cannot alter the options.

    CsvReadOptions::dtypes

    source

    The per-column dtype overrides. A fresh Map is returned, so a caller cannot reach into the options and change what the reader pins — the same reason the constructor copies its input.

    CsvReadOptions::null_values

    source
    fn CsvReadOptions::null_values(self : CsvReadOptions) -> Array[String]

    The raw strings treated as null cells. A fresh array is returned, so a caller cannot reach into the options and change what the reader nulls out — the same reason the constructor copies its input.

    CsvWriteOptions

    source
    pub struct CsvWriteOptions {
      header : Bool
      delimiter : Char
      null_value : String
      sanitize_formulas : Bool
      line_terminator : String
    }

    Options that control how a DataFrame is rendered as CSV text.

    • header — when true, the first emitted row is the column names; when false, only data rows are written.
    • delimiter — field separator written between cells.
    • null_value — the literal string written in place of a null cell. The default empty string round-trips with the reader's default null_values.
    • sanitize_formulas — when true, a String cell beginning with =, +, -, @, a tab, or a carriage return is prefixed with a single quote so a spreadsheet reads it as text. Deliberately lossy: such a cell reads back with the leading quote. Defaults to false.
    • line_terminator — the string written after every row. The default "\n" (LF) matches every mainstream writer; RFC 4180's letter calls for "\r\n" (CRLF), and the reader accepts LF, CRLF, and CR alike, so either choice round-trips. Must not be empty — an empty terminator would fuse the rows into one.

    CsvWriteOptions::CsvWriteOptions

    source
    fn CsvWriteOptions::CsvWriteOptions(header? : Bool, delimiter? : Char, null_value? : String, sanitize_formulas? : Bool, line_terminator? : String) -> CsvWriteOptions raise 
    DataError

    Build write options. Every field has a default: header on, comma delimiter, null cells emitted as the empty string, no formula sanitisation, and LF line endings. CsvWriteOptions::CsvWriteOptions() is the all-defaults writer; name only what differs, as in CsvWriteOptions::CsvWriteOptions(delimiter=';'). The line_terminator must be composed of \n and \r alone (an empty one, or one holding any other character, raises InvalidOperation): the reader breaks lines on those two characters only, so any other terminator would write a file whose rows fuse or whose last cell glues the terminator on — every CR/LF combination ("\n", "\r\n", "\r", "\n\n", …) round-trips cleanly.

    JsonReadOptions

    source
    pub struct JsonReadOptions {
      infer_schema_rows : Int
      on_parse_error : OnParseError
      reject_duplicate_keys : Bool
      allow_nonfinite_floats : Bool
      // private fields
    }

    Options that control how a JSON records payload is parsed into a DataFrame.

    • infer_schema_rows — number of leading records inspected when guessing each column's dtype. 0 (or any value <= 0) lifts the cap and scans every record. Records past a finite window are still parsed under the chosen dtype; one that doesn't fit is handled per on_parse_error.
    • on_parse_error — what to do when a non-null cell past the inference window fails to parse under its column's locked-in dtype: Raise (default) fails with ParseError(Cell(...)); Null downgrades the offending cell to a null and continues.
    • reject_duplicate_keys — whether an object repeating a key is an error. The default false keeps the common JSON reading: a repeated key survives only as its last value, the earlier ones silently dropped. true rejects the read instead — ParseError(Message(...)) naming the key and the line / column it repeated at — so a payload that would silently lose cells fails loudly. The check rides the parser itself (which reports whichever trouble it meets first: a repeat ahead of a syntax error surfaces as the repeat, and vice versa).
    • allow_nonfinite_floats — when true (default), a number whose magnitude overflows Double (e.g. 1e999) infers the column as Float holding ±Infinity. When false, such a number is not Float evidence — the column stays String, the CSV reader's allow_nonfinite_floats rule, so a silent retype to Float(∞) cannot happen either way a file is read.
    • dtypes — per-column dtype overrides that pin a column's type instead of inferring it, as a name → DataType map. Only the four dtypes a reader can parse (Int / Float / Bool / String) can be pinned — anything else is refused with InvalidOperation at read time, as is a name the input does not carry (a pin is an assertion about the input; a misspelt one fails rather than silently does nothing). Columns not named keep their inferred dtype. A pinned column skips inference entirely, and a cell that does not parse under the pin — a JSON String in a column pinned Int, say, since JSON has strong types and no "1" → 1 coercion — is handled per on_parse_error. The map is copied, so mutating it afterwards cannot alter the options; read it back through the dtypes() accessor.

    The NDJSON reader shares this type: the two formats differ in framing, not in what there is to configure.

    JsonReadOptions::JsonReadOptions

    source
    fn JsonReadOptions::JsonReadOptions(infer_schema_rows? : Int, on_parse_error? : OnParseError, reject_duplicate_keys? : Bool, allow_nonfinite_floats? : Bool, dtypes? : Map[String, 
    DataType    ]) -> JsonReadOptions

    Build read options. Every field has a default: scan the first 100 records for inference, fail the read on a cell that doesn't fit its inferred dtype, let a repeated object key keep its last value (the common JSON reading), and accept overflow-to-Infinity numbers as Float cells. JsonReadOptions::JsonReadOptions() is the all-defaults reader; name only what differs, as in JsonReadOptions::JsonReadOptions(on_parse_error=Null). dtypes is copied, so mutating the map afterwards cannot alter the options.

    JsonReadOptions::dtypes

    source

    The per-column dtype overrides. A fresh Map is returned, so a caller cannot reach into the options and change what the reader pins — the same reason the constructor copies its input.

    OnParseError

    source
    pub(all) enum OnParseError {
      Raise
      Null
    } derive(Eq, 
    Debug    )

    What a reader does when a non-null cell fails to parse under the dtype that inference locked in for its column — in practice a value past the inference window that doesn't fit (e.g. a String in a column inferred Int from its first rows).

    • Raise (every reader's default) — surface ParseError(Cell(...)), failing the whole read. The strict, lossless behaviour: a malformed cell is never silently dropped.
    • Null — downgrade the offending cell to a null cell and keep going, the resilient behaviour. The column keeps its inferred dtype; only the unparseable cells become null.

    Shared by CsvReadOptions and JsonReadOptions and consumed in build_typed_column's parse-failure branch, so the three readers agree on the policy.

    OnParseError::equal

    source

    OnParseError::not_equal

    source
    fn OnParseError::not_equal(x : OnParseError, y : OnParseError) -> Bool

    OnParseError::to_repr

    source

    format_csv

    source

    Render a DataFrame as a CSV string.

    • Header (when options.header == true): column names joined by options.delimiter, each quoted only when it would otherwise be ambiguous.
    • Each row: cells joined by options.delimiter, terminated by options.line_terminator (LF by default). Null cells are written as options.null_value; other cells render via Scalar::to_string.
    • Quoting: a cell containing the delimiter, a double quote, \r, or \n is wrapped in double quotes; interior double quotes are doubled (" → ""). Cells that need no quoting are written verbatim so the output matches the simple "no quotes" CSV idiom whenever possible. Two cells are quoted anyway: in a single-column frame an empty cell (or an empty column name) is written as "" rather than a bare blank, so the row is not a blank line the reader would skip; and a first field opening with a byte-order mark is quoted so the mark cannot be read as part of the file's own preamble.
    • Round-trip caveat: cells render via Scalar::to_string, so a Float whose value is a whole number is written without a fractional part (2.0 → 2) and is re-inferred as Int on read — the same dtype narrowing the JSON writer documents. The I64 riders narrow the same way: a written Date / Timestamp column (ISO text) reads back as String, and a Decimal column (fixed-point text) as Float — the readers never infer the rider dtypes. Negative zero is one such whole value: -0.0 writes as 0, so its sign (observable under division) does not survive the round-trip. A String cell with significant leading / trailing whitespace is written bare (only the delimiter, a quote, \r, or \n force quoting), so this reader round-trips it (trim_spaces off), but an RFC-4180 consumer that trims surrounding whitespace may drop it. A String cell whose text equals a reader null token — the default null_value is the empty string "", so this includes empty strings, plus any custom null_values entry — is read back as null and cannot be distinguished from a true null. A String cell whose text parses as another dtype is likewise re-inferred on read: inference is content-based with no header type hints, so a column of numeric-looking strings ("01", "2") returns as Int / Float — dropping any leading zeros — and "true" / "false" as Bool, the dtype-from-content narrowing inherent to typeless CSV. Quote / escape handling is otherwise round-trip safe.
    • When sanitize_formulas is true, a String cell beginning with =, +, -, @, tab, or carriage return is prefixed with an apostrophe before CSV quoting. This opt-in, intentionally lossy transformation prevents spreadsheet applications from interpreting such cells as formulas. It does not affect headers, nulls, numbers, booleans, or other strings; the default false preserves exact output.
    • Raises InvalidOperation if options.delimiter is a double quote or a line terminator (\n / \r): those collide with the hard-coded quote character and the row terminator, so no escaping could make the output frame fields unambiguously.
    • Raises InvalidOperation for an N×0 frame — rows but no columns — with N > 0. CSV frames a row by its fields, so a row with no fields is an empty line, and an empty line is exactly what the reader skips: such a frame would write as N blank lines and read back as the empty 0×0 frame, losing every row without an error. The format has no spelling for "a row with zero fields", so the write is refused rather than silently made lossy — the delimiter refusal's refuse-don't-corrupt stance, applied to the shape. The column-less 0×0 frame has no rows to lose and still writes: the empty string, or the lone \n of its (field-less) header row.

    format_json

    source
    fn format_json(df : 
    DataFrame    ) -> String

    Render a DataFrame as a JSON records string [ {...}, ... ].

    Per cell:
    • Null → JSON null
    • Int → JSON number (no fractional part)
    • Float→ JSON number for finite values. NaN / ±Infinity have no JSON literal, so they are emitted as null to keep the output valid JSON — a round-trip reads a non-finite cell back as a null.
    • Bool → JSON true / false
    • String → JSON string (escaping ", \, control chars via the builtin stringifier — we deliberately do not roll our own escape so any future Unicode spec tweaks land transparently)
    • Date / Timestamp / Decimal → JSON string holding the cell's value-form rendering (ISO calendar text / ISO instant / fixed-point decimal); the readers never infer these dtypes, so a round-trip reads the column back as String / Float respectively.

    Keys appear in the column declaration order, preserved by the builtin Map (linked-hash-map) used for each record object.

    format_ndjson

    source
    fn format_ndjson(df : 
    DataFrame    ) -> String

    Render a DataFrame as an NDJSON string: one flat JSON object per row, each terminated by \n (including the last, matching the CSV writer's per-row LF). A 0-row frame renders the empty string.

    Per-cell conventions are shared with format_json via scalar_to_json: Null → null; Int → JSON number (no fractional part; exact across the full Int64 range via the preserved repr); finite Float → JSON number, non-finite (NaN / ±Infinity) → null so each line stays valid JSON; Bool → true / false; String → escaped JSON string; Date / Timestamp / Decimal → JSON string holding the value-form rendering. Keys appear in df.columns() order, preserved by the linked-hash-map Map backing each record object.

    parse_csv_str

    source
    fn parse_csv_str(content : String, options? : CsvReadOptions, projection? : Array[String]) -> 
    DataFrame     raise 
    DataError

    Parse a CSV-encoded string into a DataFrame. The pipeline is:
    1. NyaCSV tokenises the text into headers + rows. NyaCSV always treats the first row as a header, so the has_header = false case re-folds that row into the data section and synthesises column1, column2, … names.
    2. Per-column type inference walks the first infer_schema_rows rows in order Int → Float → Bool → String (a null cell within them is skipped but still counts toward the window). The first option that accepts every probed cell wins; a column with no non-null probes lands on String (no information to disambiguate). A column named in options.dtypes skips this pass and locks in its pinned dtype instead.
    3. Null mapping replaces any raw cell whose verbatim string sits in null_values with None. The mapping runs after tokenisation, so quoted empty cells ("") and bare empty cells are both honoured.
    4. Each column is assembled into a typed Series and the result is wrapped through DataFrame::from_parts (an empty input short-circuits to DataFrame::DataFrame([]) before this point).

    Raises:
    • DuplicateColumn(name) — two headers share a name.
    • InvalidOperation — options.dtypes names a column the input does not carry, or pins a dtype that has no read parser (anything but Int / Float / Bool / String).
    • ParseError(Message(...)) — when strict_quotes is true, the input contains an unterminated quoted field, text after a closing quote, or a quote inside an unquoted field; or — when options.strict_column_count is true — a data row's cell count differs from the header's (reported as row N: expected C columns, got G, 1-based over the data rows).
    • ParseError(Cell(...)) — a non-null cell does not parse under the inferred dtype for its column (typically because a sample row outside the inference window is malformed for the inferred type), unless options.on_parse_error is Null, which downgrades that cell to a null instead of failing the read.
    • Whatever DataFrame::from_parts propagates — DuplicateColumn / LengthMismatch / a negative-height InvalidOperation, none of which can actually fire here (uniqueness is checked above; every column is built off the same non-negative rows.length()).

    By default, tokenisation keeps NyaCSV's lenient quote handling. An unterminated quoted field at end-of-input is read as a complete field (1,"unclosed → last cell unclosed); text after a closing quote is folded into the same field ("ab"cd → abcd); and a lone quote inside an unquoted field is kept literally (ab"cd → ab"cd). A strict RFC-4180 reader rejects all three; CsvReadOptions::CsvReadOptions(strict_quotes=true) runs a linear validation pass before tokenisation and rejects them as ParseError.

    projection narrows the read to the named columns — the same knob the lazy scan_csv source's projection pushdown rides. None (the default) builds every column; Some(names) builds only the columns whose header name appears in names, in the file's own header order, leaving the rest uninferred, unparsed, and unmaterialised. A requested name that matches no header is silently ignored (lenient projection — the lazy scan driver relies on it when it hands down its pruned column list), and a projection matching no header builds no column at all and yields the N×0 frame — the file's rows are still counted, so a downstream head / slice bounds-checks against the real height. Whole-input checks (strict quote validation, the empty-input short-circuit, duplicate-header rejection, the ragged-row guard) run before the projection filter and so are unaffected by it; the one behavioural difference is the point of the knob — a parse error confined to a dropped column is not raised, that column is never parsed.

    parse_json_str

    source

    Parse a JSON records string ([ {...}, ... ]) into a DataFrame.

    @json.parse produces the Json AST; a top-level value that is not an array surfaces as ParseError. The array's elements are then handed to frame_from_json_records, which validates each is an object, collects headers in first-seen order across all records, infers one dtype per column, and materialises the frame. An empty array — like empty or whitespace-only input — yields a 0×0 frame.

    Raises:
    • ParseError(Message(...)) — malformed JSON, a top-level value that is not an array, or a record that is not an object.
    • InvalidOperation — options.dtypes names a column the input does not carry, or pins a dtype that has no read parser (anything but Int / Float / Bool / String).
    • ParseError(Cell(...)) — a non-null cell does not fit the column's inferred dtype, unless options.on_parse_error is Null, which downgrades that cell to a null instead.
    • Whatever DataFrame::from_parts propagates (duplicate-header is pre-empted upstream; length-mismatch cannot fire because every column is built off records.length()).

    parse_ndjson_str

    source
    fn parse_ndjson_str(content : String, options? : JsonReadOptions, projection? : Array[String]) -> 
    DataFrame     raise 
    DataError

    Parse an NDJSON string (one JSON object per line) into a DataFrame.

    Pipeline:
    1. Split content on \n. Each non-blank line is parsed with @json.parse; a malformed line surfaces as ParseError(Message("line N: ...")) (1-based). Blank / whitespace-only lines are skipped, so the writer's trailing newline — and any incidental blank lines — never produce phantom records.
    2. The parsed values are handed to frame_from_json_records, which validates each is an object, collects headers in first-seen order across all records (sparse records → null cells), infers one dtype per column, and materialises the frame under per-column rules shared with parse_json_str.

    Empty input — or input that is entirely blank lines — yields a 0×0 frame, matching parse_json_str's empty-array behaviour.

    Raises:
    • ParseError(Message(...)) — a malformed line or a line whose value is not a JSON object.
    • ParseError(Cell(...)) — a non-null cell does not fit its inferred dtype, unless options.on_parse_error is Null, which downgrades that cell to a null instead.
    • Whatever DataFrame::from_parts propagates.

    projection narrows the read to the named columns — the same knob the lazy scan_ndjson source's projection pushdown rides. None (the default) builds every column; Some(names) builds only the columns whose header name appears in names, in first-seen header order. Every line is still parsed and record validation runs before the column filter, so a projected read fails on a malformed or non-object line exactly as a full read does; the one behavioural difference is the point of the knob — a parse error confined to a dropped column is not raised, that column is never built.

    read_csv

    source

    Read a CSV file. options defaults to CsvReadOptions::CsvReadOptions(), so read_csv(path) is the default-options read. IOError from the file system surfaces as raise IoError(message). CsvReadOptions::CsvReadOptions(strict_quotes=true) rejects malformed quoting before NyaCSV tokenisation, exactly as parse_csv_str does.

    read_json

    source

    Read a JSON records file. options defaults to JsonReadOptions::JsonReadOptions(), so read_json(path) is the default-options read. Filesystem errors surface as raise IoError(message).

    read_ndjson

    source

    Read an NDJSON file. options defaults to JsonReadOptions::JsonReadOptions(), so read_ndjson(path) is the default-options read. Filesystem errors surface as raise IoError(message).

    write_csv

    source

    Write a DataFrame to a CSV file. options defaults to CsvWriteOptions::CsvWriteOptions(), so write_csv(path, df) is the default-options write. IOError from the file system surfaces as raise IoError(message). A string cell (or column name) holding an unpaired UTF-16 surrogate is refused with raise InvalidOperation — the UTF-8 file encoding would swallow the following delimiter and silently shift every later cell boundary. CsvWriteOptions::CsvWriteOptions(sanitize_formulas=true) applies the same spreadsheet-safety transformation as format_csv before writing. Rendering runs through format_csv first, so its refusals come first too — an unrepresentable delimiter and the N×0 shape (rows but no columns) both raise InvalidOperation before the file is opened, leaving nothing written.

    write_json

    source

    Write a DataFrame to a JSON records file. Filesystem errors surface as raise IoError(message). A string cell (or column name) holding an unpaired UTF-16 surrogate is refused with raiseInvalidOperation — the UTF-8 file encoding would swallow the following code unit and corrupt the JSON framing.

    write_ndjson

    source

    Write a DataFrame to an NDJSON file. Filesystem errors surface as raise IoError(message). A string cell (or column name) holding an unpaired UTF-16 surrogate is refused with raise InvalidOperation — the UTF-8 file encoding would swallow the following code unit and corrupt the line framing.

    Source Files

    Powered by MoonBit

    Site sourceReport issueModulesSkillsCLIBuild queueStatistics

    © 2026 mooncakes.io