Sign in

    #Ion Schema (moonrockz/ion/schema)

    Loads Ion Schema 1.0 and 2.0 schemas and validates Ion values and documents against their types. It implements every constraint of both versions, and schema imports. The loaded schema holds ISL 2.0 constraints: the loader translates each ISL 1.0 construct into ISL 2.0 constraints with the same meaning (see ISL 1.0).

    Constraints: type, all_of, any_of, one_of, not, valid_values, element (with distinct::), ordered_elements, contains, fields (with occurs and closed::), field_names (with distinct::), annotations (standard, and simplified with required:: and closed::), container_length, codepoint_length, byte_length, utf8_byte_length, precision, exponent, regex, timestamp_precision, timestamp_offset, and ieee754_float.

    A length is a non-negative integer or a range::[lower, upper] of integers, and so are precision (at least 1) and exponent (any integer). timestamp_precision takes the precision names, from year to nanosecond. valid_values takes a list of values and ranges, or one range. A range there is of numbers or of timestamps. Numbers compare by their exact value, whatever their Ion type, so 1, 1.0, and 1e0 are all in range::[1, 1]. Timestamps compare by the instant they name. In any range, a bound can be exclusive::, and min or max leaves that end open.

    regex takes the ISL 2.0 subset of ECMA 262 regular expressions, and a string or symbol is valid when the expression matches any part of it. \d, \s, and \w are the ASCII classes [0-9], [ \f\n\r\t], and [A-Za-z0-9_]. The flag i:: ignores case for ASCII letters only, and m:: makes ^ and $ match at line breaks. A construct outside the subset, such as a backreference or a lazy quantifier, is an error. Matching runs a Pike VM, which takes time linear in the input, so no expression makes it backtrack without end. Regex is public, so the same expressions can be used on their own.

    In ISL 2.0, a core type such as int or struct matches only the non-null values of its Ion type, and $int or $struct also matches its typed null. text, lob, number, and any cover several Ion types, and $text, $lob, $number, and $any add their nulls. $null matches null.null, nothing matches no value, and $null_or::T matches null.null or a value of T. type takes a type name, an inline type, or an inline import. occurs is allowed only in fields and ordered_elements. element constrains the elements of a list or S-expression, or the field values of a struct. valid_values compares a value without its annotations. ordered_elements matches the elements in order, each type as many times as its occurs allows; an ambiguous match is found without backtracking. annotations validates the value's annotations as a list of symbols.

    A document is a stream of top-level values. validate_document and is_valid_document check one: element, ordered_elements, container_length, and contains apply to its values, and the built-in type document matches it. No other constraint applies to a document.

    A schema that its version of ISL does not allow raises IonError::DataModel when it loads, as the ion-schema-tests suite's invalid schemas and types require.

    Named types may recurse through container children. If validation revisits a named type on the same value without consuming a child, it returns a cyclic schema type reference violation instead of overflowing the stack. This also applies to documents, imported aliases, and annotation values. A finite successful any_of branch still proves validity; a cycle alone cannot become valid through not or count as a match for one_of. These checks run during validation, so loading a cyclic schema remains supported.

    Loading and validation keep a bounded machine stack. Their optional fuel defaults to Unlimited; Limited(@ion.Budget(100)) bounds open schema operations. Loading counts inline type entries, nullable base-type resolution and imported documents. Validation counts nested constraint evaluations, including scalar constraints and logical branches. Regex parsing and compilation also inherit the caller's remaining fuel. Siblings reuse fuel. Cyclic reference detection remains independent.

    validate, is_valid and their document variants can raise IonError::FuelExhausted. Exhaustion is an incomplete validation, and is never returned as a validation violation or inverted into success by not.

    #Schema documents

    An ISL 2.0 schema document has the version marker $ion_schema_2_0, an optional schema_header::{...}, the type::{...} definitions, and an optional schema_footer::{...}, after which nothing counts. Other top-level values are open content, which the loader ignores, unless they are annotated with a reserved symbol: $ion_schema, a symbol that starts with $ion_schema_, or a symbol in lower snake case. The header, the footer, and each type may have fields whose names are not reserved. The header's user_reserved_fields declares reserved symbols, other than ISL keywords, as more such fields. A document with the version marker $ion_schema_1_0, or with no version marker, is ISL 1.0.

    A type reference must name a type that the schema defines or imports. Imports name other schemas by an id, which the resolver given to Schema::load turns into that schema's values, for example by reading a file under a base directory. A header import brings in all the types of a schema, or one type, optionally under another name with as; an inline import { id: ..., type: ... } refers to one type without adding it to the scope. Types that an imported schema imports in turn are not in scope. Schemas may import each other in a cycle, but not themselves. Without a resolver, an import raises Unsupported. types() lists an imported type as <id>#<name>. An ISL 1.0 schema and an ISL 2.0 schema may import each other, and each type keeps the meaning of its own version.

    #ISL 1.0

    ISL 1.0 gives some of the same names other meanings, so the version marker decides how a document loads. The differences from ISL 2.0 are:

    • A type with no type constraint has type: any, so it matches no null. any also matches a document. In ISL 2.0, such a type matches any value.
    • nullable::T matches null.null, the typed nulls of the Ion types of T's base type, and the values of T. The base type is the type that T's type constraint names, or any. For example, nullable::int matches null, null.int, and 5, but not null.string. There is no $null_or::.
    • content: closed allows only the fields in the type's fields, which have no closed::.
    • scale is the number of digits after the decimal point: scale: 2 is exponent: -2. There is no exponent, field_names, or ieee754_float.
    • annotations takes only a list of symbols, annotated with any of required::, ordered::, and closed::. Each symbol can be required:: or optional::. With ordered, the required symbols must appear in order; with closed too, the value's annotations must be the symbols in order, the optional ones at most once.
    • A valid occurs is allowed in any type, and has no effect outside fields and ordered_elements. element takes no distinct::.
    • An inline type may be annotated type::, and an inline import may have an as, which has no effect.
    • A bound of a timestamp range in valid_values must have a known offset.
    • There are no reserved symbols: a top-level value that is not a header, a type, or a footer, and a field that is not an ISL 1.0 keyword, are open content. The header's imports is its only field with a meaning. A schema has a header only if it has a footer.

    ///|
    test "load an ISL 1.0 schema" {
    let schema = @schema.Schema::load_from_text(
    (
    #|$ion_schema_1_0
    #|type::{
    #| name: price,
    #| content: closed,
    #| fields: {
    #| amount: { type: decimal, scale: 2, occurs: required },
    #| note: nullable::string,
    #| },
    #|}
    ),
    )
    assert_true(
    schema.is_valid("price", @text.read_ion("{ amount: 9.99, note: null }")),
    )
    // 9.9 has a scale of 1, `tax` is not a field of the type, and the default
    // type `any` has no null.
    assert_true(!schema.is_valid("price", @text.read_ion("{ amount: 9.9 }")))
    assert_true(
    !schema.is_valid("price", @text.read_ion("{ amount: 9.99, tax: 1.00 }")),
    )
    assert_true(!schema.is_valid("price", @text.read_ion("null.struct")))
    }

    #Validating values

    Schema::load_from_text reads the type::{...} definitions of a schema document. is_valid answers yes or no, and validate lists each violation with the path to the value that broke it.

    ///|
    test "validate records against a type" {
    let definition =
    #|$ion_schema_2_0
    #|type::{
    #| name: person,
    #| type: struct,
    #| fields: {
    #| name: { type: string, occurs: required, codepoint_length: range::[1, 40] },
    #| role: { valid_values: [admin, member, guest] },
    #| tags: { type: list, element: symbol },
    #| },
    #|}
    let schema = @schema.Schema::load_from_text(definition)
    assert_true(
    schema.is_valid("person", @text.read_ion("{ name: \"Ada\", role: admin }")),
    )
    let violations = schema.validate(
    "person",
    @text.read_ion("{ role: owner, tags: [a, \"b\"] }"),
    )
    let report = violations
    .map(violation => violation.path() + ": " + violation.message())
    .join("\n")
    inspect(
    report,
    content=(
    #|$.name: field occurs 0 time(s)
    #|$.role: value is not one of the allowed values
    #|$.tags[1]: expected one of [symbol] but found string
    ),
    )
    }

    #Imports

    ///|
    test "import a type from another schema" {
    let schemas : Map[String, String] = {
    "common.isl": "$ion_schema_2_0 type::{ name: positive_int, type: int, valid_values: range::[1, max] }",
    }
    let resolver = (id : String) => {
    match schemas.get(id) {
    Some(text) => @text.read_ion_datagram(text)
    None => raise @ion.IonError::DataModel("no schema named " + id)
    }
    }
    let schema = @schema.Schema::load_from_text(
    (
    #|$ion_schema_2_0
    #|schema_header::{ imports: [{ id: "common.isl", type: positive_int, as: count }] }
    #|type::{ name: tally, fields: { votes: count } }
    ),
    resolver~,
    )
    assert_true(schema.is_valid("tally", @text.read_ion("{ votes: 3 }")))
    assert_true(!schema.is_valid("tally", @text.read_ion("{ votes: 0 }")))
    // Without a resolver, the import cannot load.
    let message = try
    @schema.Schema::load_from_text(
    "$ion_schema_2_0 schema_header::{ imports: [{ id: \"common.isl\" }] }",
    )
    catch {
    error => error.message()
    } noraise {
    _ => "loaded"
    }
    inspect(
    message,
    content="the schema imports 'common.isl', but no resolver was given to load it",
    )
    }

    Bound

    pub(all) enum Bound {
    Unbounded
    Inclusive(
    Ion
    )
    Exclusive(
    Ion
    )
    } derive(
    Debug
    )

    A bound of a range.

    Bound::to_repr

    Constraint

    pub(all) enum Constraint {
    IsType(Array[
    IonType
    ])
    NullOr(Constraint)
    TypeRef(String)
    Element(Constraint)
    Fields(Array[FieldConstraint])
    ValidValues(Array[
    Ion
    ])
    InRange(ValueRange)
    ContainerLength(Int, Int?)
    CodepointLength(Int, Int?)
    AllOf(Array[Constraint])
    AnyOf(Array[Constraint])
    OneOf(Array[Constraint])
    Not(Constraint)
    Nothing
    IsDocument
    Matches(Regex)
    ByteLength(Int, Int?)
    Utf8ByteLength(Int, Int?)
    Precision(Int, Int?)
    Exponent(Int, Int?)
    TimestampPrecision(Int, Int?)
    TimestampOffsets(Array[
    TimestampOffset
    ])
    Ieee754Float(FloatFormat)
    FieldNames(Constraint)
    DistinctFieldNames
    DistinctElements
    ClosedFields(Array[String])
    Annotations(Constraint)
    OrderedElements(Array[(Constraint, Occurs)])
    Contains(Array[
    Ion
    ])
    } derive(
    Debug
    )

    A constraint over a value.

    Constraint::any

    fn Constraint::any() -> Constraint

    A constraint that always holds, used for $any.

    FieldConstraint

    pub struct FieldConstraint {
    name : String
    ty : Constraint
    occurs : Occurs
    } derive(
    Debug
    )

    A named field within a fields constraint.

    FloatFormat

    pub(all) enum FloatFormat {
    Binary16
    Binary32
    Binary64
    } derive(Eq,
    Debug
    )

    An IEEE 754 binary interchange format.

    FloatFormat::equal

    fn FloatFormat::equal(FloatFormat, FloatFormat) -> Bool

    FloatFormat::not_equal

    fn FloatFormat::not_equal(x : FloatFormat, y : FloatFormat) -> Bool

    Occurs

    pub struct Occurs {
    lower : Int
    upper : Int?
    } derive(
    Debug
    )

    An occurrence range: how many times a matching element or field must (or may) appear.

    Occurs::allows

    fn Occurs::allows(self : Occurs, count : Int) -> Bool

    Whether count is within the range.

    Occurs::new

    fn Occurs::new(lower : Int, upper : Int?) -> Occurs

    An occurrence range with an optional upper bound (None is unbounded).

    Occurs::optional

    fn Occurs::optional() -> Occurs

    Zero or one time.

    Occurs::required

    fn Occurs::required() -> Occurs

    Exactly once.

    Occurs::to_repr

    Regex

    pub struct Regex {
    source : String
    ignore_case : Bool
    multiline : Bool
    // private fields
    } derive(
    Debug
    )

    A regular expression of the ISL 2.0 regex constraint.

    Regex::ignore_case

    fn Regex::ignore_case(self : Regex) -> Bool

    Whether the flag i is set.

    Regex::matches

    fn Regex::matches(self : Regex, text : String) -> Bool

    Whether the expression matches any part of text.

    Regex::multiline

    fn Regex::multiline(self : Regex) -> Bool

    Whether the flag m is set.

    Regex::new

    fn Regex::new(source : String, ignore_case? : Bool, multiline? : Bool, fuel? :
    Fuel
    ) -> Regex raise
    IonError

    Parse source as an ISL 2.0 regular expression. A construct that ISL 2.0 does not allow raises IonError::DataModel.

    Regex::source

    fn Regex::source(self : Regex) -> String

    The expression's source text.

    Regex::to_repr

    Schema

    pub struct Schema {
    types : Array[TypeDefinition]
    } derive(
    Debug
    )

    A loaded Ion Schema.

    Schema::empty

    fn Schema::empty() -> Schema

    An empty schema.

    Schema::find

    fn Schema::find(self : Schema, name : String) -> TypeDefinition?

    The named type definition, when present.

    Schema::from_types

    fn Schema::from_types(types : Array[TypeDefinition]) -> Schema

    A schema built from explicit type definitions.

    Schema::is_valid

    fn Schema::is_valid(self : Schema, type_name : String, value :
    Ion
    , fuel? :
    Fuel
    ) -> Bool raise
    IonError

    Whether value satisfies the named schema type.

    Schema::is_valid_document

    fn Schema::is_valid_document(self : Schema, type_name : String, values : Array[
    Ion
    ], fuel? :
    Fuel
    ) -> Bool raise
    IonError

    Whether a document satisfies the named schema type.

    Schema::load

    Load a schema from a parsed schema document, in ISL 1.0 or ISL 2.0 as its version marker says. A construct that the version does not allow raises IonError::DataModel, and one that this implementation does not implement raises IonError::Unsupported.

    The schema holds ISL 2.0 constraints: the loader translates each ISL 1.0 construct into the ISL 2.0 constraints with the same meaning. For example, the ISL 1.0 nullable::int loads as null.null, null.int, or int.

    resolver returns the parsed values of the schema document that an import names by its id, such as a file under a base directory. Without it, a schema with imports raises IonError::Unsupported. An imported type is in types() as <id>#<name>.

    Schema::load_from_text

    fn Schema::load_from_text(text : StringView, resolver? : (String) -> Array[
    Ion
    ] raise
    IonError
    , fuel? :
    Fuel
    ) -> Schema raise
    IonError

    Load a schema from Ion Schema text.

    Schema::to_repr

    Schema::types

    fn Schema::types(self : Schema) -> Array[TypeDefinition]

    The schema's type definitions.

    Schema::validate

    Validate value against the named schema type, returning every violation found. An empty result means the value is valid. If no finite validation proof exists because type references cycle, return those cycle violations.

    Schema::validate_document

    fn Schema::validate_document(self : Schema, type_name : String, values : Array[
    Ion
    ], fuel? :
    Fuel
    ) -> Array[Violation] raise
    IonError

    Validate a document, a stream of top-level Ion values, against the named schema type, returning every violation found. ISL treats a document like an S-expression without annotations for element, ordered_elements, container_length, and contains, and the built-in type document matches it. No other constraint applies to a document, and no value is a document.

    TypeDefinition

    pub struct TypeDefinition {
    name : String
    constraints : Array[Constraint]
    } derive(
    Debug
    )

    A named Ion Schema type: a name and the constraints its values must satisfy.

    TypeDefinition::constraints

    fn TypeDefinition::constraints(self : TypeDefinition) -> Array[Constraint]

    The type definition's constraints.

    TypeDefinition::name

    fn TypeDefinition::name(self : TypeDefinition) -> String

    The type definition's name.

    ValueRange

    pub struct ValueRange {
    lower : Bound
    upper : Bound
    } derive(
    Debug
    )

    A range of numbers or of timestamps.

    ValueRange::contains

    fn ValueRange::contains(self : ValueRange, value :
    Ion
    ) -> Bool

    Whether value is in the range. A null, a nan, and a value that is not the same kind as the bounds (a number or a timestamp) are not in any range. Annotations do not count.

    ValueRange::lower

    fn ValueRange::lower(self : ValueRange) -> Bound

    The lower bound.

    ValueRange::new

    fn ValueRange::new(lower : Bound, upper : Bound) -> ValueRange

    A range from lower to upper.

    ValueRange::upper

    fn ValueRange::upper(self : ValueRange) -> Bound

    The upper bound.

    Violation

    pub struct Violation {
    path : String
    message : String
    } derive(
    Debug
    )

    A single way in which a value fails to satisfy a schema type.

    Violation::message

    fn Violation::message(self : Violation) -> String

    The human-readable reason for the violation.

    Violation::path

    fn Violation::path(self : Violation) -> String

    The location within the value where the violation occurred ($, $.field, $[0], ...).

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io