🥮 mooncakes.io
    Sign in

    ihb2032/MoonFrame/io does not have a README file

    CsvReadOptions

    source
    pub struct CsvReadOptions {
      has_header : Bool
      delimiter : Char
      infer_schema_rows : Int
      strict_column_count : Bool
      on_parse_error : OnParseError
      allow_nonfinite_floats : Bool
      strict_quotes : Bool
      // private fields
    }

    Options that control how a CSV input is parsed into a DataFrame.

    • has_header — when true, the first row supplies column names. When false, every row is treated as data and column names are synthesised as "column1", "column2", … in declaration order; the first data row fixes the column count, so a later wider row's trailing cells are dropped under the lenient default (set strict_column_count = true to reject a ragged row instead).
    • delimiter — field separator passed through to NyaCSV.
    • infer_schema_rows — number of leading rows scanned when guessing each column's dtype. 0 (or any value <= 0) lifts the cap and scans every row (Polars' infer_schema_length=None), trading a slower inference pass for never mis-guessing a column from a prefix. Cells past a finite window are still parsed under the chosen dtype; one that doesn't fit is handled per on_parse_error.
    • null_values — raw strings that should be treated as null cells (both for type inference, which skips them, and for the final typed column, where they become None). The default [""] matches the empty string only, which is how an absent CSV cell is typically represented. Read it back through the null_values() accessor: the field itself is private, and both it and the constructor copy, so the token list a reader uses cannot be changed after the options are built.
    • strict_column_count — when true, every data row must have exactly as many cells as the header declares; a ragged row (too few or too many cells) raise ParseError instead of being silently null-padded (short row) or truncated (long row). Defaults to false, preserving the lenient behaviour that tolerates ragged rows (a permissive CSV idiom, and the reader's long-standing default).
    • on_parse_error — what to do when a non-null cell past the inference window fails to parse under its column's locked-in dtype. Raise (default) fails with ParseError(Cell(...)); Null downgrades the offending cell to a null cell and keeps the column's inferred dtype (Polars' ignore_errors=True).
    • allow_nonfinite_floats — when true (default), the float probe accepts every non-finite result: the nan / inf / infinity literals, and a finite literal that overflows Double to a signed Infinity (1e999), so a column of those tokens infers as Float. When false, the probe rejects them, so such a column falls back to String instead of being silently retyped to Float (a non-finite token past the window is then a parse failure handled per on_parse_error).
    • strict_quotes — when true, the input is pre-scanned and an unterminated quoted field, text after a closing quote, or a bare quote inside an unquoted field raise ParseError instead of being repaired by the lenient tokeniser. Defaults to false.

    CsvReadOptions::CsvReadOptions

    source
    fn CsvReadOptions::CsvReadOptions(has_header? : Bool, delimiter? : Char, infer_schema_rows? : Int, null_values? : Array[String], strict_column_count? : Bool, on_parse_error? : OnParseError, allow_nonfinite_floats? : Bool, strict_quotes? : Bool) -> CsvReadOptions

    Build read options. Every field has a default: header on, comma delimiter, scan the first 100 rows for inference, treat the empty string as null, tolerate ragged rows, fail the read on a cell that doesn't fit its inferred dtype, accept non-finite float literals, and tokenise leniently. CsvReadOptions::CsvReadOptions() is the all-defaults reader; name only what differs, as in CsvReadOptions::CsvReadOptions(delimiter=';', strict_quotes=true).

    null_values is copied, so mutating the array afterwards cannot alter the options.

    CsvReadOptions::null_values

    source
    fn CsvReadOptions::null_values(self : CsvReadOptions) -> Array[String]

    The raw strings treated as null cells. A fresh array is returned, so a caller cannot reach into the options and change what the reader nulls out — the same reason the constructor copies its input.

    CsvWriteOptions

    source
    pub struct CsvWriteOptions {
      header : Bool
      delimiter : Char
      null_value : String
      sanitize_formulas : Bool
      line_terminator : String
    }

    Options that control how a DataFrame is rendered as CSV text.

    • header — when true, the first emitted row is the column names; when false, only data rows are written.
    • delimiter — field separator written between cells.
    • null_value — the literal string written in place of a null cell. The default empty string round-trips with the reader's default null_values.
    • sanitize_formulas — when true, a String cell beginning with =, +, -, @, a tab, or a carriage return is prefixed with a single quote so a spreadsheet reads it as text. Deliberately lossy: such a cell reads back with the leading quote. Defaults to false.
    • line_terminator — the string written after every row, Polars' knob of the same name. The default "\n" (LF) matches every mainstream writer; RFC 4180's letter calls for "\r\n" (CRLF), and the reader accepts LF, CRLF, and CR alike, so either choice round-trips. Must not be empty — an empty terminator would fuse the rows into one.

    CsvWriteOptions::CsvWriteOptions

    source
    fn CsvWriteOptions::CsvWriteOptions(header? : Bool, delimiter? : Char, null_value? : String, sanitize_formulas? : Bool, line_terminator? : String) -> CsvWriteOptions raise 
    DataError

    Build write options. Every field has a default: header on, comma delimiter, null cells emitted as the empty string, no formula sanitisation, and LF line endings. CsvWriteOptions::CsvWriteOptions() is the all-defaults writer; name only what differs, as in CsvWriteOptions::CsvWriteOptions(delimiter=';'). The line_terminator must be composed of \n and \r alone (an empty one, or one holding any other character, raises InvalidOperation): the reader breaks lines on those two characters only, so any other terminator would write a file whose rows fuse or whose last cell glues the terminator on — every CR/LF combination ("\n", "\r\n", "\r", "\n\n", …) round-trips cleanly.

    JsonReadOptions

    source
    pub struct JsonReadOptions {
      infer_schema_rows : Int
      on_parse_error : OnParseError
      reject_duplicate_keys : Bool
    }

    Options that control how a JSON records payload is parsed into a DataFrame.

    • infer_schema_rows — number of leading records inspected when guessing each column's dtype. 0 (or any value <= 0) lifts the cap and scans every record (Polars' infer_schema_length=None). Records past a finite window are still parsed under the chosen dtype; one that doesn't fit is handled per on_parse_error.
    • on_parse_error — what to do when a non-null cell past the inference window fails to parse under its column's locked-in dtype: Raise (default) fails with ParseError(Cell(...)); Null downgrades the offending cell to a null and continues (Polars' ignore_errors=True).
    • reject_duplicate_keys — whether an object repeating a key is an error. The default false keeps the serde / pandas / Polars behaviour: a repeated key survives only as its last value, the earlier ones silently dropped. true rejects the read instead — ParseError(Message(...)) naming the key and the 1-based object it repeated in — so a payload that would silently lose cells fails loudly. Nested objects are scanned too, and key spellings are compared decoded ("a" and "\u0061" are the same key). The scan runs before the parse and only when the option is on, so the default read pays nothing.

    The NDJSON reader shares this type: the two formats differ in framing, not in what there is to configure.

    JsonReadOptions::JsonReadOptions

    source
    fn JsonReadOptions::JsonReadOptions(infer_schema_rows? : Int, on_parse_error? : OnParseError, reject_duplicate_keys? : Bool) -> JsonReadOptions

    Build read options. Every field has a default: scan the first 100 records for inference, fail the read on a cell that doesn't fit its inferred dtype, and let a repeated object key keep its last value (the serde / pandas / Polars reading). JsonReadOptions::JsonReadOptions() is the all-defaults reader; name only what differs, as in JsonReadOptions::JsonReadOptions(on_parse_error=Null).

    OnParseError

    source
    pub(all) enum OnParseError {
      Raise
      Null
    } derive(Eq, 
    Debug    )

    What a reader does when a non-null cell fails to parse under the dtype that inference locked in for its column — in practice a value past the inference window that doesn't fit (e.g. a String in a column inferred Int from its first rows).

    • Raise (every reader's default) — surface ParseError(Cell(...)), failing the whole read. The strict, lossless behaviour: a malformed cell is never silently dropped.
    • Null — downgrade the offending cell to a null cell and keep going, the resilient behaviour Polars exposes as ignore_errors=True (pandas tolerates the same). The column keeps its inferred dtype; only the unparseable cells become null.

    Shared by CsvReadOptions and JsonReadOptions and consumed in build_typed_column's parse-failure branch, so the three readers agree on the policy.

    OnParseError::equal

    source

    OnParseError::not_equal

    source
    fn OnParseError::not_equal(x : OnParseError, y : OnParseError) -> Bool

    OnParseError::to_repr

    source

    format_csv

    source

    Render a DataFrame as a CSV string.

    • Header (when options.header == true): column names joined by options.delimiter, each quoted only when it would otherwise be ambiguous.
    • Each row: cells joined by options.delimiter, terminated by options.line_terminator (LF by default). Null cells are written as options.null_value; other cells render via Scalar::to_string.
    • Quoting: a cell containing the delimiter, a double quote, \r, or \n is wrapped in double quotes; interior double quotes are doubled (" → ""). Cells that need no quoting are written verbatim so the output matches the simple "no quotes" CSV idiom whenever possible. Two cells are quoted anyway: in a single-column frame an empty cell (or an empty column name) is written as "" rather than a bare blank, so the row is not a blank line the reader would skip; and a first field opening with a byte-order mark is quoted so the mark cannot be read as part of the file's own preamble.
    • Round-trip caveat: cells render via Scalar::to_string, so a Float whose value is a whole number is written without a fractional part (2.0 → 2) and is re-inferred as Int on read — the same dtype narrowing the JSON writer documents. Negative zero is one such whole value: -0.0 writes as 0, so its sign (observable under division) does not survive the round-trip. A String cell with significant leading / trailing whitespace is written bare (only the delimiter, a quote, \r, or \n force quoting), so this reader round-trips it (trim_spaces off), but an RFC-4180 consumer that trims surrounding whitespace may drop it. A String cell whose text equals a reader null token — the default null_value is the empty string "", so this includes empty strings, plus any custom null_values entry — is read back as null and cannot be distinguished from a true null. A String cell whose text parses as another dtype is likewise re-inferred on read: inference is content-based with no header type hints, so a column of numeric-looking strings ("01", "2") returns as Int / Float — dropping any leading zeros — and "true" / "false" as Bool, the dtype-from-content narrowing inherent to typeless CSV (matching Polars' read_csv). Quote / escape handling is otherwise round-trip safe.
    • When sanitize_formulas is true, a String cell beginning with =, +, -, @, tab, or carriage return is prefixed with an apostrophe before CSV quoting. This opt-in, intentionally lossy transformation prevents spreadsheet applications from interpreting such cells as formulas. It does not affect headers, nulls, numbers, booleans, or other strings; the default false preserves exact output.
    • Raises InvalidOperation if options.delimiter is a double quote or a line terminator (\n / \r): those collide with the hard-coded quote character and the row terminator, so no escaping could make the output frame fields unambiguously.
    • Raises InvalidOperation for an N×0 frame — rows but no columns — with N > 0. CSV frames a row by its fields, so a row with no fields is an empty line, and an empty line is exactly what the reader skips: such a frame would write as N blank lines and read back as the empty 0×0 frame, losing every row without an error. The format has no spelling for "a row with zero fields", so the write is refused rather than silently made lossy — the delimiter refusal's refuse-don't-corrupt stance, applied to the shape. The column-less 0×0 frame has no rows to lose and still writes: the empty string, or the lone \n of its (field-less) header row.

    format_json

    source
    fn format_json(df : 
    DataFrame    ) -> String

    Render a DataFrame as a JSON records string [ {...}, ... ].

    Per cell:
    • Null → JSON null
    • Int → JSON number (no fractional part)
    • Float→ JSON number for finite values. NaN / ±Infinity have no JSON literal, so they are emitted as null (matching pandas' to_json) to keep the output valid JSON — a round-trip reads a non-finite cell back as a null.
    • Bool → JSON true / false
    • String → JSON string (escaping ", \, control chars via the builtin stringifier — we deliberately do not roll our own escape so any future Unicode spec tweaks land transparently)

    Keys appear in the column declaration order, preserved by the builtin Map (linked-hash-map) used for each record object.

    format_ndjson

    source
    fn format_ndjson(df : 
    DataFrame    ) -> String

    Render a DataFrame as an NDJSON string: one flat JSON object per row, each terminated by \n (including the last, matching the CSV writer's per-row LF and Polars' write_ndjson). A 0-row frame renders the empty string.

    Per-cell conventions are shared with format_json via scalar_to_json: Null → null; Int → JSON number (no fractional part; exact across the full Int64 range via the preserved repr); finite Float → JSON number, non-finite (NaN / ±Infinity) → null so each line stays valid JSON; Bool → true / false; String → escaped JSON string. Keys appear in df.columns() order, preserved by the linked-hash-map Map backing each record object.

    parse_csv_str

    source
    fn parse_csv_str(content : String, options? : CsvReadOptions, projection? : Array[String]) -> 
    DataFrame     raise 
    DataError

    Parse a CSV-encoded string into a DataFrame. The pipeline is:
    1. NyaCSV tokenises the text into headers + rows. NyaCSV always treats the first row as a header, so the has_header = false case re-folds that row into the data section and synthesises column1, column2, … names.
    2. Per-column type inference walks the first infer_schema_rows rows in order Int → Float → Bool → String (a null cell within them is skipped but still counts toward the window). The first option that accepts every probed cell wins; a column with no non-null probes lands on String (no information to disambiguate).
    3. Null mapping replaces any raw cell whose verbatim string sits in null_values with None. The mapping runs after tokenisation, so quoted empty cells ("") and bare empty cells are both honoured.
    4. Each column is assembled into a typed Series and the result is wrapped through DataFrame::from_parts (an empty input short-circuits to DataFrame::DataFrame([]) before this point).

    Raises:
    • DuplicateColumn(name) — two headers share a name.
    • ParseError(Message(...)) — when strict_quotes is true, the input contains an unterminated quoted field, text after a closing quote, or a quote inside an unquoted field; or — when options.strict_column_count is true — a data row's cell count differs from the header's (reported as row N: expected C columns, got G, 1-based over the data rows).
    • ParseError(Cell(...)) — a non-null cell does not parse under the inferred dtype for its column (typically because a sample row outside the inference window is malformed for the inferred type), unless options.on_parse_error is Null, which downgrades that cell to a null instead of failing the read.
    • Whatever DataFrame::from_parts propagates — DuplicateColumn / LengthMismatch / a negative-height InvalidOperation, none of which can actually fire here (uniqueness is checked above; every column is built off the same non-negative rows.length()).

    By default, tokenisation keeps NyaCSV's lenient quote handling. An unterminated quoted field at end-of-input is read as a complete field (1,"unclosed → last cell unclosed); text after a closing quote is folded into the same field ("ab"cd → abcd); and a lone quote inside an unquoted field is kept literally (ab"cd → ab"cd). A strict RFC-4180 reader rejects all three; CsvReadOptions::CsvReadOptions(strict_quotes=true) runs a linear validation pass before tokenisation and rejects them as ParseError.

    projection narrows the read to the named columns — the same knob the lazy scan_csv source's projection pushdown rides. None (the default) builds every column; Some(names) builds only the columns whose header name appears in names, in the file's own header order, leaving the rest uninferred, unparsed, and unmaterialised. A requested name that matches no header is silently ignored (lenient projection, as in Polars), and a projection matching no header builds no column at all and yields the N×0 frame — the file's rows are still counted, so a downstream head / slice bounds-checks against the real height. Whole-input checks (strict quote validation, the empty-input short-circuit, duplicate-header rejection, the ragged-row guard) run before the projection filter and so are unaffected by it; the one behavioural difference is the point of the knob — a parse error confined to a dropped column is not raised, that column is never parsed.

    parse_json_str

    source

    Parse a JSON records string ([ {...}, ... ]) into a DataFrame.

    @json.parse produces the Json AST; a top-level value that is not an array surfaces as ParseError. The array's elements are then handed to frame_from_json_records, which validates each is an object, collects headers in first-seen order across all records, infers one dtype per column, and materialises the frame. An empty array — like empty or whitespace-only input — yields a 0×0 frame.

    Raises:
    • ParseError(Message(...)) — malformed JSON, a top-level value that is not an array, or a record that is not an object.
    • ParseError(Cell(...)) — a non-null cell does not fit the column's inferred dtype, unless options.on_parse_error is Null, which downgrades that cell to a null instead.
    • Whatever DataFrame::from_parts propagates (duplicate-header is pre-empted upstream; length-mismatch cannot fire because every column is built off records.length()).

    parse_ndjson_str

    source
    fn parse_ndjson_str(content : String, options? : JsonReadOptions, projection? : Array[String]) -> 
    DataFrame     raise 
    DataError

    Parse an NDJSON string (one JSON object per line) into a DataFrame.

    Pipeline:
    1. Split content on \n. Each non-blank line is parsed with @json.parse; a malformed line surfaces as ParseError(Message("line N: ...")) (1-based). Blank / whitespace-only lines are skipped, so the writer's trailing newline — and any incidental blank lines — never produce phantom records.
    2. The parsed values are handed to frame_from_json_records, which validates each is an object, collects headers in first-seen order across all records (sparse records → null cells), infers one dtype per column, and materialises the frame under per-column rules shared with parse_json_str.

    Empty input — or input that is entirely blank lines — yields a 0×0 frame, matching parse_json_str's empty-array behaviour.

    Raises:
    • ParseError(Message(...)) — a malformed line or a line whose value is not a JSON object.
    • ParseError(Cell(...)) — a non-null cell does not fit its inferred dtype, unless options.on_parse_error is Null, which downgrades that cell to a null instead.
    • Whatever DataFrame::from_parts propagates.

    projection narrows the read to the named columns — the same knob the lazy scan_ndjson source's projection pushdown rides. None (the default) builds every column; Some(names) builds only the columns whose header name appears in names, in first-seen header order. Every line is still parsed and record validation runs before the column filter, so a projected read fails on a malformed or non-object line exactly as a full read does; the one behavioural difference is the point of the knob — a parse error confined to a dropped column is not raised, that column is never built.

    read_csv

    source

    Read a CSV file. options defaults to CsvReadOptions::CsvReadOptions(), so read_csv(path) is the default-options read. IOError from the file system surfaces as raise IoError(message). CsvReadOptions::CsvReadOptions(strict_quotes=true) rejects malformed quoting before NyaCSV tokenisation, exactly as parse_csv_str does.

    read_json

    source

    Read a JSON records file. options defaults to JsonReadOptions::JsonReadOptions(), so read_json(path) is the default-options read. Filesystem errors surface as raise IoError(message).

    read_ndjson

    source

    Read an NDJSON file. options defaults to JsonReadOptions::JsonReadOptions(), so read_ndjson(path) is the default-options read. Filesystem errors surface as raise IoError(message).

    write_csv

    source

    Write a DataFrame to a CSV file. options defaults to CsvWriteOptions::CsvWriteOptions(), so write_csv(path, df) is the default-options write. IOError from the file system surfaces as raise IoError(message). A string cell (or column name) holding an unpaired UTF-16 surrogate is refused with raise InvalidOperation — the UTF-8 file encoding would swallow the following delimiter and silently shift every later cell boundary. CsvWriteOptions::CsvWriteOptions(sanitize_formulas=true) applies the same spreadsheet-safety transformation as format_csv before writing. Rendering runs through format_csv first, so its refusals come first too — an unrepresentable delimiter and the N×0 shape (rows but no columns) both raise InvalidOperation before the file is opened, leaving nothing written.

    write_json

    source

    Write a DataFrame to a JSON records file. Filesystem errors surface as raise IoError(message). A string cell (or column name) holding an unpaired UTF-16 surrogate is refused with raiseInvalidOperation — the UTF-8 file encoding would swallow the following code unit and corrupt the JSON framing.

    write_ndjson

    source

    Write a DataFrame to an NDJSON file. Filesystem errors surface as raise IoError(message). A string cell (or column name) holding an unpaired UTF-16 surrogate is refused with raise InvalidOperation — the UTF-8 file encoding would swallow the following code unit and corrupt the line framing.

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io