moon-hdf5

    A pure MoonBit HDF5 reader and new-file writer

    hdf5
    scientific-data
    file-format
    reader
    writer
    Download zip
    Author
    Version
    0.2.0
    License
    Apache-2.0
    Last updated
    2 days ago
    Downloads
    49

    #moon-hdf5

    Pure MoonBit HDF5 reader and writer for the HDF5 1.14.6 common-file profile. It has no native HDF5 dependency and runs on wasm and native targets.

    This project implements a documented subset of HDF5. The compatibility matrix is the delivery boundary, design notes cover implementation details, and audit notes record verification. Public APIs are listed in src/pkg.generated.mbti.

    #Status

    • Hdf5File reads existing files without modification. Hdf5Writer creates complete new files in memory. Hdf5Resolver follows registered external links and resolver-backed raw-data sources.
    • Reading covers superblocks v0–v3, compact/contiguous/chunked storage, built-in and registered custom filters, dense and shared metadata, structured and variable-length values, references, virtual datasets, and bounded chunk/selection iterators for directly addressable mappings.
    • Writing covers groups, datasets, attributes, links, committed datatypes, filters, fills, dense/SOHM metadata, nested variable-length and reference values, compact/contiguous/chunked/external storage, and resizable datasets.
    • Existing-file editing covers value and attribute patching, attribute and link add/rename/delete, object creation/deletion, and append-only metadata rebuilds. Every editing API returns a new byte image.

    See Compatibility for the exact supported and rejected cases.

    #Installation

    Add the released package to a MoonBit module:

    moon add Zz04321/moon-hdf5@0.2.0 moon update

    The package targets wasm by default.

    #Quick Start

    Install the MoonBit toolchain; version 0.10.19 or newer is required. The native target also needs a C compiler.

    git clone https://github.com/zzSmall6/moon-hdf5.git cd moon-hdf5 moon update moon run src/cmd/main --target native -- testdata/basic_v0.h5

    #Reading

    Imports for the examples are shown in src/cmd/main/moon.pkg.

    ///|
    let bytes = @fs.read_file_to_bytes("data.h5")
    let file = @lib.Hdf5File::parse(bytes)

    let metadata = file.read_dataset_metadata("/group/values")
    let values = file.read_dataset("/group/values")
    let slice = file.read_dataset_selection(
    "/matrix",
    @lib.Hyperslab([0UL, 0UL], [2UL, 2UL], [1UL, 2UL]),
    )
    let attributes = file.attributes("/group/values")
    let nodes = file.walk()

    read_dataset_metadata does not load values. Use chunk_iterator for bounded-memory chunk processing and selection_iterator for large hyperslab or point selections. Simple values use Integers, UnsignedIntegers, Floats, Strings or Enums; other datatypes use recursive Structured values.

    References are returned as encoded bytes. Hdf5File resolves same-file targets; Hdf5Resolver can also follow version-4 references and external links into registered sources.

    ///|
    let resolver = @lib.Hdf5Resolver::new(bytes)
    resolver.add_source("other-file.h5", @fs.read_file_to_bytes("other-file.h5"))
    let remote = resolver.read_dataset("/experiment/remote_counts")

    let object = resolver.resolve_object_reference(reference_bytes)
    let region = resolver.read_region_reference(region_bytes)
    let attribute = resolver.read_attribute_reference(attribute_bytes)

    Third-party chunk filters use Hdf5CustomFilter with Hdf5File::with_custom_filter or Hdf5Resolver::add_custom_filter. Built-in filter IDs cannot be overridden, duplicate IDs are rejected, and decoders must return exactly the expected byte count.

    #Writing

    ///|
    let writer = @lib.Hdf5Writer::new()

    let record = @lib.Datatype::write_compound([
    ("id", @lib.Datatype::write_integer(true, 4)),
    ("value", @lib.Datatype::write_float(8)),
    ])
    writer.add_committed_datatype("/types/record", record)
    writer.add_dataset(
    "/experiment/records",
    [2UL],
    @lib.StructuredValues(record, [
    @lib.Compound([
    { name: "id", value: @lib.SignedInteger(1L) },
    { name: "value", value: @lib.FloatingPoint(1.25) },
    ]),
    @lib.Compound([
    { name: "id", value: @lib.SignedInteger(2L) },
    { name: "value", value: @lib.FloatingPoint(-3.5) },
    ]),
    ]),
    )
    writer.add_chunked_dataset(
    "/experiment/codes",
    [6UL],
    [3U],
    @lib.Int64Values([10L, 11L, 12L, 20L, 21L, 22L]),
    fill_value=@lib.Int64Values([-1L]),
    filters=[@lib.Shuffle, @lib.Deflate(6), @lib.Fletcher32],
    )
    writer.add_attribute(
    "/experiment/records",
    "unit",
    [],
    @lib.FixedStringValues(["items"], 8),
    )
    writer.add_soft_link("/experiment", "latest", "/experiment/records")

    let output = writer.encode()

    Dataset creation options:

    • add_dataset: contiguous storage.
    • add_compact_dataset: inline object-header storage, up to 65,535 bytes.
    • add_chunked_dataset: finite fixed-array chunks or resizable chunks via maximum_dimensions.
    • add_extensible_dataset, resize_dataset, append_dataset and write_dataset_slab: in-memory resizing, append and rectangular writes.
    • add_external_dataset: unfiltered external raw-data slots; the caller creates the external files.

    Fixed-size values support Shuffle, deflate, Fletcher32 and scale-offset. Variable-length values support Shuffle/deflate and must use fixed-size sequence bases. Nested variable-length and reference members are encoded recursively. Register custom encoders with Hdf5Writer::add_custom_filter before adding a dataset that uses Custom(id, parameters). Existing-file add_chunked_dataset accepts the same filters through custom_filters=[filter].

    Version-4 references are opt-in per datatype; legacy references remain the writer default. Use the Datatype::write_version4_*_reference constructors for version-4 object, region or attribute references. Same-file targets use ObjectReference, RegionReference or AttributeReference. External targets use ExternalObjectReference, ExternalRegionReference or ExternalAttributeReference with a file name registered through Hdf5Writer::add_external_source. The writer validates the target object, region dataspace and attribute, supports 4- and 8-byte target files, and stores the file name in the reference. The caller owns and must keep the target file available under that name.

    #Editing Existing Files

    ///|
    let file = @lib.Hdf5File::parse(@fs.read_file_to_bytes("data.h5"))
    let patched = file.patch_dataset_values(
    "/group/values",
    @lib.Int64Values([10L, 20L, 30L, 40L]),
    )
    let with_attribute = @lib.Hdf5File::parse(patched).add_attribute(
    "/group/values",
    "units",
    [],
    @lib.FixedStringValues(["items"], 8),
    )
    let edited = @lib.Hdf5File::parse(with_attribute).add_soft_link(
    "/group", "latest", "/group/values",
    )
    @fs.write_bytes_to_file("edited.h5", edited)

    • patch_dataset_values handles compact, contiguous and chunked datasets, including variable-length values, same-file references and external version-4 references registered through Hdf5File::with_external_source. The replacement must keep the dataset's shape and a compatible datatype.
    • patch_contiguous_dataset is the strict allocated-contiguous-only form.
    • patch_attribute_values replaces existing attribute values without changing names, creation order or storage form when the record can be updated in place.
    • add_attribute, rename_attribute, delete_attribute, add_*_link, rename_link and delete_link rebuild compact/dense collections as needed.
    • add_group, dataset creation APIs and delete_object append objects and attach or detach links through the same rebuild path.

    Editing is append-only. Allocations are limited to existing writable layout or chunk-index storage, and superseded metadata or data remains unreferenced.

    #Command Line

    moon run src/cmd/main -- inspect testdata/basic_v0.h5 moon run src/cmd/main -- metadata testdata/basic_v0.h5 /matrix moon run src/cmd/main -- stats testdata/basic_v0.h5 /matrix moon run src/cmd/main -- read testdata/basic_v0.h5 /matrix 0,0 2,2 1,2 moon run src/cmd/main -- read testdata/basic_v0.h5 /matrix --points "1,2;0,1"

    Coordinates are zero-based. Hyperslabs use comma-separated start, count and stride; point lists use semicolons. Exit status is 0 for success, 1 for data errors and 2 for invalid arguments.

    #Limitations

    moon-hdf5 is not a drop-in replacement for libhdf5.

    • Not yet implemented: streaming output, free-space reclamation, SWMR and parallel HDF5.
    • Virtual datasets are read-only. Same-file and external sources are resolved through Hdf5File::with_external_source or Hdf5Resolver; source files must be registered, printf-style source names and datatype conversion are not supported, and each source mapping must use the virtual dataset's exact datatype. Page and tile reads of all or non-overlapping hyperslab mappings fetch only the requested elements; point and irregular block mappings are expanded in memory per mapping.
    • Fixed boundaries: readers and writers are memory-resident at the API surface; editing is append-only; references and patches target existing objects, and external reference files must be registered; external raw data is resolver-only, contiguous and unfiltered; dense collections are limited to 9–65,535 records; four-byte structured values with variable-length custom fills are rejected for HDF5 1.14.6 interoperability.
    • Detection of unsupported input is not exhaustive. Unsupported structures raise UnsupportedFeature; malformed input raises InvalidField or UnexpectedEof.

    The compatibility matrix is the authoritative list of supported variants and boundaries.

    #Development

    moon info moon fmt moon check moon test --target wasm moon test --target native H5PY_PYTHON=/path/to/python ./scripts/release_check.sh

    scripts/release_check.sh runs formatting, documentation and package checks, both test targets, writer demos and independent h5py verification. The baseline is h5py 3.13.0 with HDF5 1.14.6. Set SKIP_H5PY=1 to omit Python checks. Fixture regeneration is documented in testdata/README.md.

    #License

    Apache-2.0. Zlib decompression uses MoonZip, also Apache-2.0.

    Hdf5CustomFilter

    pub(open) trait Hdf5CustomFilter {
    fn id(Self) -> Int
    fn name(Self) -> String = _
    fn decode(Self, data : Bytes, parameters : Array[UInt], element_size : Int, expected_size : Int) -> Bytes raise Hdf5Error
    fn encode(Self, data : Bytes, parameters : Array[UInt], element_size : Int, expected_size : Int) -> Bytes raise Hdf5Error = _
    }

    Encodes and decodes one third-party HDF5 filter for dataset chunks and fractal-heap metadata.

    HDF5 applies filters in reverse pipeline order. data is the payload at this stage, element_size is the dataset element width (zero for metadata heap blocks), and expected_size is the decoded byte count. A decoder must return exactly expected_size bytes. Implementations that only read files may omit encode; trying to write through such a filter raises UnsupportedFeature.

    Hdf5Source

    pub(open) trait Hdf5Source :
    Debug
    {
    fn length(Self) -> UInt64
    fn read_at(Self, offset : UInt64, length : Int) -> Bytes raise Hdf5Error
    }

    A random-access byte source for an HDF5 file.

    Implementations must return exactly length bytes or raise an error.
    impl Hdf5Source for Bytes

    Hdf5Error

    pub(all) suberror Hdf5Error {
    UnexpectedEof(Int64)
    InvalidSignature
    UnsupportedSuperblockVersion(Int)
    UnsupportedAddressSize(Int)
    UnsupportedFeature(String)
    InvalidArgument(String)
    InvalidField(String)
    } derive(Eq,
    Debug
    )

    Errors raised by HDF5 reading and writing operations.

    Hdf5Error::equal

    fn Hdf5Error::equal(Hdf5Error, Hdf5Error) -> Bool

    Hdf5Error::message

    fn Hdf5Error::message(self : Hdf5Error) -> String

    Hdf5Error::not_equal

    fn Hdf5Error::not_equal(x : Hdf5Error, y : Hdf5Error) -> Bool

    Attribute

    pub(all) struct Attribute {
    name : String
    dataspace : Dataspace
    datatype : Datatype
    values : DatasetValues?
    } derive(
    Debug
    )

    An attribute attached to an HDF5 object.

    CompoundFieldValue

    pub(all) struct CompoundFieldValue {
    name : String
    value : DatasetValue
    } derive(Eq,
    Debug
    )

    A named value decoded from a compound datatype member.

    CompoundFieldValue::equal

    CompoundFieldValue::not_equal

    CompoundMember

    pub(all) struct CompoundMember {
    name : String
    offset : Int
    datatype : Datatype
    } derive(
    Debug
    )

    A named field within a compound datatype.

    DataLayout

    pub struct DataLayout {
    // private fields
    } derive(
    Debug
    )

    DataLayout::chunk_shape

    fn DataLayout::chunk_shape(self : DataLayout) -> Array[UInt]

    Returns the chunk shape, or an empty array for non-chunked datasets.

    DataLayout::kind

    Returns the dataset's storage strategy.

    Dataset

    pub(all) struct Dataset {
    dataspace : Dataspace
    datatype : Datatype
    layout : DataLayout
    values : DatasetValues
    } derive(
    Debug
    )

    A dataset together with its decoded values.

    Dataset::to_repr

    DatasetChunk

    pub(all) struct DatasetChunk {
    offset : Array[UInt64]
    dimensions : Array[UInt64]
    values : DatasetValues
    } derive(
    Debug
    )

    One decoded tile from a dataset. offset is the tile origin in dataset coordinates and dimensions is the edge-clipped tile shape.

    DatasetChunkIterator

    pub struct DatasetChunkIterator {
    // private fields
    }

    Iterates over a dataset in caller-selected tiles without materializing the complete dataset. Storage chunks that intersect a tile are decoded at most once per next call.

    DatasetChunkIterator::next

    Returns the next tile, or None after the dataset has been consumed. Callers can stop early by no longer calling next.

    DatasetLayoutKind

    pub(all) enum DatasetLayoutKind {
    Compact
    Contiguous
    Chunked
    Virtual
    Unknown
    } derive(Eq,
    Debug
    )

    The storage strategy used for a dataset payload.

    DatasetLayoutKind::equal

    DatasetLayoutKind::not_equal

    fn DatasetLayoutKind::not_equal(x : DatasetLayoutKind, y : DatasetLayoutKind) -> Bool

    DatasetMetadata

    pub(all) struct DatasetMetadata {
    dataspace : Dataspace
    datatype : Datatype
    layout : DataLayout
    } derive(
    Debug
    )

    Dataset metadata that can be inspected without reading its stored values.

    DatasetSelection

    pub(all) enum DatasetSelection {
    Hyperslab(Array[UInt64], Array[UInt64], Array[UInt64])
    Points(Array[Array[UInt64]])
    } derive(
    Debug
    )

    A selection of elements from a dataset.

    DatasetSelectionIterator

    pub struct DatasetSelectionIterator {
    // private fields
    }

    Iterates over a dataset selection in bounded-memory pages. Hyperslab ordinals are generated lazily, so creating an iterator for a very large selection does not allocate an index array for the complete result.

    DatasetSelectionIterator::next

    Returns the next selection page, or None after the selection has been consumed. Callers can stop early without reading the remaining pages.

    DatasetSelectionPage

    pub(all) struct DatasetSelectionPage {
    offset : UInt64
    datatype : Datatype
    values : DatasetValues
    } derive(
    Debug
    )

    One page of a dataset selection. offset is the zero-based ordinal of the first value inside the original selection; values are returned in selection order as a flat page.

    DatasetSelectionResult

    pub(all) struct DatasetSelectionResult {
    dimensions : Array[UInt64]
    datatype : Datatype
    values : DatasetValues
    } derive(
    Debug
    )

    Values and shape produced by a dataset selection. Hyperslabs use their count as the shape; point selections use a one-dimensional point count.

    DatasetValue

    pub(all) enum DatasetValue {
    SignedInteger(Int64)
    UnsignedInteger(UInt64)
    BigInteger(
    BigInt
    )
    FloatingPoint(Double)
    Text(String)
    Enumeration(String)
    Bitfield(UInt64)
    Opaque(Bytes)
    Compound(Array[CompoundFieldValue])
    Elements(Array[DatasetValue])
    ObjectReference(String?)
    RegionReference(String?, Hdf5RegionSelection)
    AttributeReference(String?, String)
    ExternalObjectReference(String, String)
    ExternalRegionReference(String, String, Hdf5RegionSelection)
    ExternalAttributeReference(String, String, String)
    Reference(Bytes)
    Sequence(Array[DatasetValue])
    Time(UInt64)
    BigTime(
    BigInt
    )
    BigBitfield(
    BigInt
    )
    Complex(Double, Double)
    } derive(Eq,
    Debug
    )

    One recursively decoded value for structured HDF5 datatypes.

    DatasetValue::equal

    DatasetValue::not_equal

    fn DatasetValue::not_equal(x : DatasetValue, y : DatasetValue) -> Bool

    DatasetValues

    pub(all) enum DatasetValues {
    Integers(Array[Int64])
    UnsignedIntegers(Array[UInt64])
    BigIntegers(Array[
    BigInt
    ])
    Floats(Array[Double])
    Strings(Array[String])
    Enums(Array[String])
    Structured(Array[DatasetValue])
    } derive(
    Debug
    )

    Values decoded from an HDF5 dataset.

    Dataspace

    pub(all) struct Dataspace {
    kind : DataspaceKind
    dimensions : Array[UInt64]
    maximum_dimensions : Array[UInt64]
    } derive(
    Debug
    )

    Dataspace::element_count

    fn Dataspace::element_count(self : Dataspace) -> Int64 raise Hdf5Error

    DataspaceKind

    pub(all) enum DataspaceKind {
    Scalar
    Simple
    Null
    } derive(Eq,
    Debug
    )

    DataspaceKind::equal

    DataspaceKind::not_equal

    fn DataspaceKind::not_equal(x : DataspaceKind, y : DataspaceKind) -> Bool

    Datatype

    pub(all) struct Datatype {
    class : Int
    version : Int
    size : Int
    signed : Bool
    little_endian : Bool
    vlen_is_string : Bool
    enum_members : Array[EnumMember]
    compound_members : Array[CompoundMember]
    base_types : Array[Datatype]
    array_dimensions : Array[UInt64]
    bit_offset : Int
    bit_precision : Int
    float_sign_position : Int
    float_exponent_position : Int
    float_exponent_size : Int
    float_mantissa_position : Int
    float_mantissa_size : Int
    float_exponent_bias : Int
    float_normalization : Int
    string_padding : Int
    string_charset : Int
    opaque_tag : String
    reference_kind : Int
    } derive(
    Debug
    )

    Datatype::to_repr

    Datatype::write_array

    fn Datatype::write_array(base : Datatype, dimensions : Array[UInt64]) -> Datatype raise Hdf5Error

    Creates a fixed-size array datatype.

    Datatype::write_bitfield

    fn Datatype::write_bitfield(size : Int, precision : Int, offset? : Int, little_endian? : Bool) -> Datatype raise Hdf5Error

    Creates a bitfield datatype with a declared offset and precision.

    Datatype::write_compound

    fn Datatype::write_compound(members : Array[(String, Datatype)]) -> Datatype raise Hdf5Error

    Creates a packed compound datatype in member order.

    Datatype::write_enum

    fn Datatype::write_enum(base : Datatype, members : Array[EnumMember]) -> Datatype raise Hdf5Error

    Creates an enumeration datatype with an integer base type.

    Datatype::write_fixed_string

    fn Datatype::write_fixed_string(width : Int) -> Datatype raise Hdf5Error

    Creates a fixed-width UTF-8 string datatype.

    Datatype::write_float

    fn Datatype::write_float(size : Int) -> Datatype raise Hdf5Error

    Creates a standard little-endian IEEE floating-point datatype.

    Datatype::write_float_with_order

    fn Datatype::write_float_with_order(size : Int, little_endian : Bool) -> Datatype raise Hdf5Error

    Creates an IEEE floating-point datatype with an explicit byte order for structured writes.

    Datatype::write_integer

    fn Datatype::write_integer(signed : Bool, size : Int) -> Datatype raise Hdf5Error

    Creates a standard little-endian integer datatype for structured writes.

    Datatype::write_integer_with_order

    fn Datatype::write_integer_with_order(signed : Bool, size : Int, little_endian : Bool) -> Datatype raise Hdf5Error

    Creates an integer datatype with an explicit byte order for structured writes.

    Datatype::write_object_reference

    fn Datatype::write_object_reference() -> Datatype

    Creates a legacy object-reference datatype for structured writes.

    Datatype::write_object_reference_with_address_size

    fn Datatype::write_object_reference_with_address_size(address_size : Int) -> Datatype raise Hdf5Error

    Creates a legacy object-reference datatype for a file with the given on-disk address width.

    Datatype::write_opaque

    fn Datatype::write_opaque(size : Int, tag : String) -> Datatype raise Hdf5Error

    Creates an opaque datatype with an optional ASCII tag.

    Datatype::write_region_reference

    fn Datatype::write_region_reference() -> Datatype

    Creates a legacy dataset-region reference datatype.

    Datatype::write_region_reference_with_address_size

    fn Datatype::write_region_reference_with_address_size(address_size : Int) -> Datatype raise Hdf5Error

    Creates a legacy dataset-region reference datatype for a file with the given on-disk address width.

    Datatype::write_time

    fn Datatype::write_time(size : Int, little_endian? : Bool) -> Datatype raise Hdf5Error

    Creates an HDF5 time datatype backed by an unsigned integer.

    Datatype::write_version4_attribute_reference

    fn Datatype::write_version4_attribute_reference() -> Datatype

    Creates a version-4 attribute-reference datatype.

    Datatype::write_version4_attribute_reference_with_address_size

    fn Datatype::write_version4_attribute_reference_with_address_size(address_size : Int) -> Datatype raise Hdf5Error

    Creates a version-4 attribute-reference datatype for a file with the given on-disk address width.

    Datatype::write_version4_object_reference

    fn Datatype::write_version4_object_reference() -> Datatype

    Creates a version-4 object-reference datatype for structured writes.

    Datatype::write_version4_object_reference_with_address_size

    fn Datatype::write_version4_object_reference_with_address_size(address_size : Int) -> Datatype raise Hdf5Error

    Creates a version-4 object-reference datatype for a file with the given on-disk address width.

    Datatype::write_version4_region_reference

    fn Datatype::write_version4_region_reference() -> Datatype

    Creates a version-4 dataset-region reference datatype.

    Datatype::write_version4_region_reference_with_address_size

    fn Datatype::write_version4_region_reference_with_address_size(address_size : Int) -> Datatype raise Hdf5Error

    Creates a version-4 dataset-region reference datatype for a file with the given on-disk address width.

    Datatype::write_vlen_sequence

    fn Datatype::write_vlen_sequence(base : Datatype) -> Datatype raise Hdf5Error

    Creates a variable-length sequence datatype with a fixed-size base type.

    Datatype::write_vlen_sequence_with_address_size

    fn Datatype::write_vlen_sequence_with_address_size(base : Datatype, address_size : Int) -> Datatype raise Hdf5Error

    Creates a variable-length sequence datatype for a file with the given on-disk heap address width.

    Datatype::write_vlen_string

    fn Datatype::write_vlen_string() -> Datatype

    Creates a variable-length UTF-8 string datatype for structured writes.

    Datatype::write_vlen_string_with_address_size

    fn Datatype::write_vlen_string_with_address_size(address_size : Int) -> Datatype raise Hdf5Error

    Creates a variable-length UTF-8 string datatype for a file with the given on-disk heap address width.

    EnumMember

    pub(all) struct EnumMember {
    name : String
    value : EnumValue
    } derive(
    Debug
    )

    A named member of an enumeration datatype.

    EnumValue

    pub(all) enum EnumValue {
    Signed(Int64)
    Unsigned(UInt64)
    Big(
    BigInt
    )
    } derive(Eq,
    Debug
    )

    One value in an enumeration datatype, preserving the signedness of its integer base type.

    EnumValue::equal

    fn EnumValue::equal(EnumValue, EnumValue) -> Bool

    EnumValue::not_equal

    fn EnumValue::not_equal(x : EnumValue, y : EnumValue) -> Bool

    Hdf5AttributeReferenceTarget

    pub(all) struct Hdf5AttributeReferenceTarget {
    address : UInt64
    kind : Hdf5ObjectKind
    path : String?
    attribute_name : String
    } derive(Eq,
    Debug
    )

    The object and attribute reached by a version-4 attribute reference.

    Hdf5AttributeReferenceTarget::not_equal

    Hdf5ExternalSlot

    pub(all) struct Hdf5ExternalSlot {
    file_name : String
    file_offset : UInt64
    data_size : UInt64
    }

    One external raw-data slot stored outside the HDF5 file.

    Hdf5File

    pub struct Hdf5File {
    // private fields
    }

    A parsed HDF5 file backed by bytes or a random-access source.

    Hdf5File::add_attribute

    fn Hdf5File::add_attribute(self : Hdf5File, path : String, name : String, dimensions : Array[UInt64], values : Hdf5WriteValues) -> Bytes raise Hdf5Error

    Adds a new attribute to an existing group, dataset or committed datatype. Fixed-size values and variable-length UTF-8 strings or sequences are supported. Storage automatically switches between compact and dense forms at the same threshold used by the writer.

    Hdf5File::add_chunked_dataset

    fn Hdf5File::add_chunked_dataset(self : Hdf5File, parent_path : String, name : String, dimensions : Array[UInt64], chunk_dimensions : Array[UInt], values : Hdf5WriteValues, fill_value? : Hdf5WriteValues, maximum_dimensions? : Array[UInt64?], filters? : Array[Hdf5WriteFilter], custom_filters? : Array[&Hdf5CustomFilter]) -> Bytes raise Hdf5Error

    Adds a finite chunked dataset below parent_path and returns the updated file.

    The fixed-array chunk index and chunk payloads are appended at aligned EOF offsets. Built-in Shuffle, deflate, Fletcher32 and scale-offset filters are supported. Caller-defined filters are supplied through custom_filters and referenced by Custom(id, parameters) pipeline entries; IDs must be unique and cannot replace built-in filters. When maximum_dimensions contains None, the dataset uses an extensible-array index for one unlimited axis or a v2 B-tree for multiple unlimited axes.

    Hdf5File::add_dataset

    fn Hdf5File::add_dataset(self : Hdf5File, parent_path : String, name : String, dimensions : Array[UInt64], values : Hdf5WriteValues, fill_value? : Hdf5WriteValues) -> Bytes raise Hdf5Error

    Adds a fixed-size contiguous dataset below parent_path and returns the updated file.

    Numeric, fixed-string, complex, fixed-size structured, variable-length string and object-reference values are supported. The payload is appended first and the new dataset object header links it through a contiguous layout message using the file's address and length widths. Variable-length string descriptors use the file address width, and object references use the file address width rather than a host-sized reference slot. Nested variable-length and reference members in compound, fixed-array and sequence values are encoded recursively. An optional fill value is written through the normal fill-value message.

    Hdf5File::add_external_dataset

    fn Hdf5File::add_external_dataset(self : Hdf5File, parent_path : String, name : String, dimensions : Array[UInt64], values : Hdf5WriteValues, slots : Array[Hdf5ExternalSlot], fill_value? : Hdf5WriteValues) -> Bytes raise Hdf5Error

    Adds an unfiltered dataset whose raw bytes live in external files below parent_path and returns the updated file.

    Slots are concatenated in caller order to form the row-major element stream and must add up exactly to the dataset byte size. The external-data heap is appended to the HDF5 file with the file's address and length widths.
    fn Hdf5File::add_external_link(self : Hdf5File, parent_path : String, name : String, target_file : String, target_path : String) -> Bytes raise Hdf5Error

    Adds an external link to another HDF5 file.

    Hdf5File::add_group

    fn Hdf5File::add_group(self : Hdf5File, parent_path : String, name : String) -> Bytes raise Hdf5Error

    Adds a new empty group below parent_path and returns the updated file.

    The new object header is appended first, then the parent group's complete link collection is rebuilt with a hard link to it. Existing symbol-table parents are converted to link-message storage by the normal link rebuild path. Old metadata remains unreferenced because free-space reclamation is not implemented.
    fn Hdf5File::add_hard_link(self : Hdf5File, parent_path : String, name : String, target_path : String) -> Bytes raise Hdf5Error

    Adds a hard link from a group to an existing object in the same file.
    fn Hdf5File::add_soft_link(self : Hdf5File, parent_path : String, name : String, target : String) -> Bytes raise Hdf5Error

    Adds a soft link whose target is interpreted relative to the parent group.

    Hdf5File::address_size

    fn Hdf5File::address_size(self : Hdf5File) -> Int

    Returns the on-disk address width in bytes.

    Hdf5File::attributes

    fn Hdf5File::attributes(self : Hdf5File, path : String) -> Array[Attribute] raise Hdf5Error

    Reads the attributes attached to an object path.

    Hdf5File::chunk_iterator

    fn Hdf5File::chunk_iterator(self : Hdf5File, path : String, tile_dimensions : Array[UInt64]) -> DatasetChunkIterator raise Hdf5Error

    Creates a bounded-memory iterator over a dataset.

    Hdf5File::delete_attribute

    fn Hdf5File::delete_attribute(self : Hdf5File, path : String, name : String) -> Bytes raise Hdf5Error

    Deletes an attribute and rebuilds the object's compact or dense storage.
    fn Hdf5File::delete_link(self : Hdf5File, parent_path : String, name : String) -> Bytes raise Hdf5Error

    Deletes a link and rebuilds the group's compact or dense link storage.

    Hdf5File::delete_object

    fn Hdf5File::delete_object(self : Hdf5File, path : String) -> Bytes raise Hdf5Error

    Deletes an object by removing every hard link that names it.

    HDF5 has no separate object-deletion operation: an object becomes unreachable when its final hard link is removed. This function also removes hard links in other groups so aliases cannot keep the object reachable. Soft and external links are left untouched and may subsequently dangle. The root group cannot be deleted. Metadata and raw data from the deleted object remain in the append-only file until free-space reclamation is available.

    Hdf5File::parse

    fn Hdf5File::parse(bytes : Bytes) -> Hdf5File raise Hdf5Error

    Parses an HDF5 file from in-memory bytes.

    Hdf5File::parse_source

    fn Hdf5File::parse_source(source : &Hdf5Source) -> Hdf5File raise Hdf5Error

    Parses an HDF5 file from a random-access source.

    Hdf5File::patch_attribute_values

    fn Hdf5File::patch_attribute_values(self : Hdf5File, path : String, name : String, values : Hdf5WriteValues) -> Bytes raise Hdf5Error

    Replaces the values of an existing fixed-size attribute and returns a new byte image. Variable-length UTF-8 strings and fixed-base variable-length sequences are also supported; their new values are placed in an appended global heap. The attribute datatype, dataspace, name and creation order are preserved. Inline messages are updated in their object-header chunk; dense attributes are updated through their fractal-heap record. Legacy same-file object references are rewritten in their existing address-width slots; region references append deduplicated target-address plus selection payloads to a global heap and rewrite their existing heap-address/index slots. Same-file version-4 object references rewrite direct address tokens; version-4 region and attribute references append indirect global-heap payloads and rewrite their kind, length, heap-address and index fields. References and variable-length strings or sequences nested in compound, array and sequence members are encoded recursively. External version-4 references resolve through sources registered with Hdf5File::with_external_source and use flag one plus indirect payloads. Legacy object-reference slots must match the file address size; legacy region-reference slots must be four bytes wider and target existing datasets with non-null dataspaces. Version-4 region and attribute targets follow the same dataset and existing-attribute rules. Shared attribute messages, filtered fractal-heap records, and tiny dense records are rebuilt as a local attribute collection; this may change compact/dense storage and drops metadata-heap filters from the rebuilt collection. Adding, removing or renaming attributes is not supported.

    Hdf5File::patch_contiguous_dataset

    fn Hdf5File::patch_contiguous_dataset(self : Hdf5File, path : String, values : Hdf5WriteValues) -> Bytes raise Hdf5Error

    Replaces the values of an allocated contiguous dataset and returns a new byte image. The shape and datatype must match exactly. Fixed-size values and variable-length UTF-8 strings or fixed-base sequences are supported, as are legacy object and region references in their matching address-width slots.

    Attributes and links are preserved. Variable-length values append an aligned global heap and update the superblock EOF/checksum. Legacy same-file object references are rewritten in their existing address-width slots. Legacy region references append target-address plus selection payloads to an aligned global heap and rewrite their existing heap-address/index slots. Same-file version-4 object references rewrite direct address tokens; version-4 region and attribute references append indirect global-heap payloads and rewrite their kind, length, heap-address and index fields. Chunked, compact, external and filtered storage, reference-bearing compound/array/sequence datatypes and nested variable-length storage are rejected. Use patch_dataset_values to patch compact, unallocated or external-reference storage as well.

    Hdf5File::patch_dataset_values

    fn Hdf5File::patch_dataset_values(self : Hdf5File, path : String, values : Hdf5WriteValues) -> Bytes raise Hdf5Error

    Replaces the values of a compact, contiguous or chunked dataset and returns a new byte image. The shape and datatype must match exactly.

    Compact payloads stay inline; only the containing object-header chunk checksum is recomputed. Allocated contiguous payloads stay at their existing address; unallocated payloads are appended at an aligned file offset and linked by updating the layout address and size. Chunk bytes are updated in place and edge padding is preserved. Filtered chunks are decoded and re-encoded with their existing mask and pipeline. Encodings that fit overwrite the original payload; larger encodings are appended and linked by updating the index address and size fields, the superblock EOF address and affected checksums.

    Missing chunks stay unallocated for fill values. A non-fill chunk can be allocated into writable storage already present in the chunk index, and a missing v1 or v2 B-tree chunk record is created by rebuilding the complete chunk tree from final addresses, stored sizes, masks and coordinates. V1 rebuilds use the superblock's indexed-storage K, sibling links and datatype-sized termination keys; v2 rebuilds preserve the node size and split/merge thresholds. Missing fixed- or extensible-array pages inside an existing data block are initialized and marked allocated; other data blocks, super blocks and index blocks are not created. Fixed-size values, variable-length UTF-8 strings, fixed-base sequences and legacy same-file object or region references are supported, as are same-file 18-byte version-4 object, region and attribute references. Variable-length values use one appended global heap; legacy region references append deduplicated target-address plus selection payloads to a global heap. Version-4 object references write direct address tokens, while version-4 region and attribute references append indirect heap payloads. Variable-length strings and sequences nested in compound, array and sequence members are encoded recursively. Chunked values accept unfiltered storage, Shuffle or deflate, and cannot create a missing fixed-array data block or extensible-array metadata. External storage is read-only; external version-4 references resolve through sources registered with Hdf5File::with_external_source. Legacy object references require a slot width equal to the file address size; legacy region references require four additional bytes and existing dataset targets with non-null dataspaces. Version-4 region and attribute targets are dataset or existing-attribute targets in the local file or a registered external source. The dataset shape and chunk grid are never changed.

    Hdf5File::read_attribute_reference

    fn Hdf5File::read_attribute_reference(self : Hdf5File, reference : Bytes) -> Attribute? raise Hdf5Error

    Reads the attribute selected by a version-4 attribute reference.

    A null reference returns None. The target object must have a named hard link so its attributes can be enumerated, and the named attribute must still exist.

    Hdf5File::read_dataset

    fn Hdf5File::read_dataset(self : Hdf5File, path : String) -> Dataset raise Hdf5Error

    Reads a dataset and decodes all of its values.

    Hdf5File::read_dataset_metadata

    fn Hdf5File::read_dataset_metadata(self : Hdf5File, path : String) -> DatasetMetadata raise Hdf5Error

    Reads a dataset's shape, datatype and storage layout without loading its values.

    Hdf5File::read_dataset_selection

    fn Hdf5File::read_dataset_selection(self : Hdf5File, path : String, selection : DatasetSelection) -> DatasetSelectionResult raise Hdf5Error

    Reads a hyperslab or point selection from a dataset.

    Hdf5File::read_region_reference

    fn Hdf5File::read_region_reference(self : Hdf5File, reference : Bytes) -> DatasetSelectionResult? raise Hdf5Error

    Reads the values selected by a legacy HDF5 dataset-region reference.

    A null reference returns None. The result uses the selection shape for a regular hyperslab and a flat element count for points or irregular blocks.

    Hdf5File::rename_attribute

    fn Hdf5File::rename_attribute(self : Hdf5File, path : String, old_name : String, new_name : String) -> Bytes raise Hdf5Error

    Renames an attribute while preserving its datatype, dataspace, value and creation order. Compact/dense storage is rebuilt consistently.
    fn Hdf5File::rename_link(self : Hdf5File, parent_path : String, old_name : String, new_name : String) -> Bytes raise Hdf5Error

    Renames an existing hard, soft or external link while preserving its target and creation order.

    Hdf5File::resolve

    fn Hdf5File::resolve(self : Hdf5File, path : String) -> Hdf5Node raise Hdf5Error

    Resolves an absolute or root-relative object path.

    Hdf5File::resolve_attribute_reference

    fn Hdf5File::resolve_attribute_reference(self : Hdf5File, reference : Bytes) -> Hdf5AttributeReferenceTarget? raise Hdf5Error

    Resolves a version-4 attribute reference to its object and attribute name.

    Null references return None. External-file and non-native tokens are outside the supported profile. Use read_attribute_reference to load the referenced attribute value.

    Hdf5File::resolve_object_reference

    fn Hdf5File::resolve_object_reference(self : Hdf5File, reference : Bytes) -> Hdf5ReferenceTarget? raise Hdf5Error

    Resolves a legacy HDF5 object reference to its target object.

    The all-zero null reference returns None. Region references must be resolved with resolve_region_reference; malformed references and dangling addresses raise an error. References are local to this file; use Hdf5Resolver to obtain a file that contains a remotely linked dataset before resolving its stored references.

    Hdf5File::resolve_region_reference

    fn Hdf5File::resolve_region_reference(self : Hdf5File, reference : Bytes) -> Hdf5RegionReferenceTarget? raise Hdf5Error

    Resolves a legacy HDF5 dataset-region reference.

    The all-zero null reference returns None. The target must be a dataset in the same file. Legacy and version-4 encoded references are supported, but version-4 references to external files require Hdf5Resolver.

    Hdf5File::selection_iterator

    fn Hdf5File::selection_iterator(self : Hdf5File, path : String, selection : DatasetSelection, page_size : Int) -> DatasetSelectionIterator raise Hdf5Error

    Creates a bounded-memory iterator over an arbitrary dataset selection. Use next to retrieve pages of at most page_size values.

    Hdf5File::superblock_version

    fn Hdf5File::superblock_version(self : Hdf5File) -> Int

    Returns the parsed HDF5 superblock version.

    Hdf5File::walk

    fn Hdf5File::walk(self : Hdf5File) -> Array[Hdf5Node] raise Hdf5Error

    Lists the full object hierarchy below the root group.

    Hdf5File::with_custom_filter

    fn Hdf5File::with_custom_filter(self : Hdf5File, filter : &Hdf5CustomFilter) -> Hdf5File raise Hdf5Error

    Returns a file that can decode the given third-party dataset chunk or fractal-heap metadata filter.

    Registrations are checked for duplicate and built-in filter IDs. The original value is unchanged, so calls can be chained or kept separate.

    Hdf5File::with_external_source

    fn Hdf5File::with_external_source(self : Hdf5File, name : String, source : &Hdf5Source) -> Hdf5File raise Hdf5Error

    Registers a source that version-4 references in this file may target.

    The name is matched against the external reference file name after path normalization. Patching requires a bytes-backed Hdf5File.

    Hdf5Node

    pub(all) struct Hdf5Node {
    path : String
    address : UInt64
    kind : Hdf5ObjectKind
    } derive(Eq,
    Debug
    )

    One named object in the file hierarchy.

    Hdf5Node::equal

    fn Hdf5Node::equal(Hdf5Node, Hdf5Node) -> Bool

    Hdf5Node::not_equal

    fn Hdf5Node::not_equal(x : Hdf5Node, y : Hdf5Node) -> Bool

    Hdf5Node::to_repr

    Hdf5ObjectKind

    pub(all) enum Hdf5ObjectKind {
    Group
    Dataset
    Datatype
    Unknown
    } derive(Eq,
    Debug
    )

    The kind of object stored under an HDF5 link.

    Hdf5ObjectKind::equal

    Hdf5ObjectKind::not_equal

    fn Hdf5ObjectKind::not_equal(x : Hdf5ObjectKind, y : Hdf5ObjectKind) -> Bool

    Hdf5ReferenceTarget

    pub(all) struct Hdf5ReferenceTarget {
    address : UInt64
    kind : Hdf5ObjectKind
    path : String?
    } derive(Eq,
    Debug
    )

    The object reached by a non-null HDF5 object reference.

    path is the first hard-link path found by a bounded hierarchy walk. It is None when the target is valid but has no named hard link.

    Hdf5ReferenceTarget::equal

    Hdf5ReferenceTarget::not_equal

    Hdf5RegionBlock

    pub(all) struct Hdf5RegionBlock {
    start : Array[UInt64]
    end : Array[UInt64]
    } derive(Eq,
    Debug
    )

    One inclusive block in a serialized HDF5 hyperslab selection.

    Hdf5RegionBlock::equal

    Hdf5RegionBlock::not_equal

    fn Hdf5RegionBlock::not_equal(x : Hdf5RegionBlock, y : Hdf5RegionBlock) -> Bool

    Hdf5RegionReferenceTarget

    pub(all) struct Hdf5RegionReferenceTarget {
    address : UInt64
    kind : Hdf5ObjectKind
    path : String?
    dataspace : Dataspace
    selection : Hdf5RegionSelection
    } derive(
    Debug
    )

    The dataset and dataspace selection reached by a region reference.

    Hdf5RegionSelection

    pub(all) enum Hdf5RegionSelection {
    RegionEmpty
    RegionAll
    RegionPoints(Array[Array[UInt64]])
    RegionHyperslab(Array[UInt64], Array[UInt64], Array[UInt64], Array[UInt64])
    RegionBlocks(Array[Hdf5RegionBlock])
    } derive(Eq,
    Debug
    )

    The dataspace selection stored in an HDF5 1.14 region reference.

    Hdf5RegionSelection::equal

    Hdf5RegionSelection::not_equal

    Hdf5ResolvedAttributeReference

    pub(all) struct Hdf5ResolvedAttributeReference {
    file : String
    target : Hdf5AttributeReferenceTarget
    } derive(
    Debug
    )

    An attribute reference resolved across registered HDF5 sources.

    Hdf5ResolvedNode

    pub(all) struct Hdf5ResolvedNode {
    file : String
    path : String
    address : UInt64
    kind : Hdf5ObjectKind
    } derive(Eq,
    Debug
    )

    A concrete object reached through the resolver.

    file is the registered source name, or <root> for the source passed to Hdf5Resolver::new. path is the canonical path inside that source after following soft and external links.

    Hdf5ResolvedNode::equal

    Hdf5ResolvedNode::not_equal

    fn Hdf5ResolvedNode::not_equal(x : Hdf5ResolvedNode, y : Hdf5ResolvedNode) -> Bool

    Hdf5ResolvedObjectReference

    pub(all) struct Hdf5ResolvedObjectReference {
    file : String
    target : Hdf5ReferenceTarget
    } derive(
    Debug
    )

    An object reference resolved across the root file and registered external HDF5 sources.

    Hdf5ResolvedRegionReference

    pub(all) struct Hdf5ResolvedRegionReference {
    file : String
    target : Hdf5RegionReferenceTarget
    } derive(
    Debug
    )

    A dataset-region reference resolved across registered HDF5 sources.

    Hdf5Resolver

    pub struct Hdf5Resolver {
    // private fields
    }

    Resolves paths across a root HDF5 source and registered external-link or external-raw-data targets. Sources are kept in memory and parsed on demand.

    Hdf5Resolver::add_custom_filter

    fn Hdf5Resolver::add_custom_filter(self : Hdf5Resolver, filter : &Hdf5CustomFilter) -> Unit raise Hdf5Error

    Registers a third-party dataset chunk or fractal-heap metadata filter for every source opened by this resolver. Registrations apply to future reads and cannot replace built-in filters or duplicate an existing ID.

    Hdf5Resolver::add_source

    fn Hdf5Resolver::add_source(self : Hdf5Resolver, name : String, source : &Hdf5Source) -> Unit raise Hdf5Error

    Registers an external-link target under a stable source name.

    Hdf5Resolver::attributes

    fn Hdf5Resolver::attributes(self : Hdf5Resolver, path : String) -> Array[Attribute] raise Hdf5Error

    Reads attributes through the resolver.

    Hdf5Resolver::chunk_iterator

    fn Hdf5Resolver::chunk_iterator(self : Hdf5Resolver, path : String, tile_dimensions : Array[UInt64]) -> DatasetChunkIterator raise Hdf5Error

    Creates a bounded-memory dataset iterator through the resolver, including tiles backed by contiguous external raw-data storage.

    Hdf5Resolver::new

    fn Hdf5Resolver::new(source : &Hdf5Source) -> Hdf5Resolver

    Creates a resolver for a root source. External targets must be registered with add_source before they can be reached.

    Hdf5Resolver::open

    fn Hdf5Resolver::open(self : Hdf5Resolver, name : String) -> Hdf5File raise Hdf5Error

    Opens the root source or a registered external-link target.

    Hdf5Resolver::read_attribute_reference

    fn Hdf5Resolver::read_attribute_reference(self : Hdf5Resolver, reference : Bytes) -> Attribute? raise Hdf5Error

    Reads the attribute selected by a reference stored in the root source, including version-4 references to registered external HDF5 files.

    Hdf5Resolver::read_dataset

    fn Hdf5Resolver::read_dataset(self : Hdf5Resolver, path : String) -> Dataset raise Hdf5Error

    Reads a dataset through the resolver, including external-link targets and contiguous external raw-data storage.

    Hdf5Resolver::read_dataset_metadata

    fn Hdf5Resolver::read_dataset_metadata(self : Hdf5Resolver, path : String) -> DatasetMetadata raise Hdf5Error

    Reads dataset metadata through the resolver.

    Hdf5Resolver::read_dataset_selection

    fn Hdf5Resolver::read_dataset_selection(self : Hdf5Resolver, path : String, selection : DatasetSelection) -> DatasetSelectionResult raise Hdf5Error

    Reads a dataset selection through the resolver, including external raw data.

    Hdf5Resolver::read_region_reference

    fn Hdf5Resolver::read_region_reference(self : Hdf5Resolver, reference : Bytes) -> DatasetSelectionResult? raise Hdf5Error

    Reads the values selected by a region reference stored in the root source, including version-4 references to registered external HDF5 files.

    Hdf5Resolver::resolve

    fn Hdf5Resolver::resolve(self : Hdf5Resolver, path : String) -> Hdf5ResolvedNode raise Hdf5Error

    Resolves a path in the root source, following soft and external links.

    Hdf5Resolver::resolve_attribute_reference

    fn Hdf5Resolver::resolve_attribute_reference(self : Hdf5Resolver, reference : Bytes) -> Hdf5ResolvedAttributeReference? raise Hdf5Error

    Resolves an attribute reference stored in the root source, including version-4 references to registered external HDF5 files.

    Hdf5Resolver::resolve_object_reference

    fn Hdf5Resolver::resolve_object_reference(self : Hdf5Resolver, reference : Bytes) -> Hdf5ResolvedObjectReference? raise Hdf5Error

    Resolves an object reference stored in the root source, including version-4 references to registered external HDF5 files.

    Hdf5Resolver::resolve_region_reference

    fn Hdf5Resolver::resolve_region_reference(self : Hdf5Resolver, reference : Bytes) -> Hdf5ResolvedRegionReference? raise Hdf5Error

    Resolves a region reference stored in the root source, including version-4 references to registered external HDF5 files.

    Hdf5Resolver::selection_iterator

    fn Hdf5Resolver::selection_iterator(self : Hdf5Resolver, path : String, selection : DatasetSelection, page_size : Int) -> DatasetSelectionIterator raise Hdf5Error

    Creates a bounded-memory selection iterator through the resolver, including contiguous external raw-data storage.

    Hdf5WriteFilter

    pub(all) enum Hdf5WriteFilter {
    Shuffle
    Deflate(Int)
    Fletcher32
    ScaleOffset(Int)
    Custom(Int, Array[UInt])
    } derive(Eq,
    Debug
    )

    Filters available for chunked writer datasets.

    Hdf5WriteFilter::equal

    Hdf5WriteFilter::not_equal

    fn Hdf5WriteFilter::not_equal(x : Hdf5WriteFilter, y : Hdf5WriteFilter) -> Bool

    Hdf5WriteValues

    pub(all) enum Hdf5WriteValues {
    Int8Values(Array[Int])
    Int16Values(Array[Int])
    Int32Values(Array[Int])
    Int64Values(Array[Int64])
    UInt8Values(Array[UInt])
    UInt16Values(Array[UInt])
    UInt32Values(Array[UInt])
    UInt64Values(Array[UInt64])
    BigIntegerValues(Array[
    BigInt
    ], Int, Bool)
    Float16Values(Array[Float])
    Float32Values(Array[Float])
    Float64Values(Array[Double])
    FixedStringValues(Array[String], Int)
    VariableStringValues(Array[String])
    ObjectReferenceValues(Array[String?])
    Complex64Values(Array[(Float, Float)])
    Complex128Values(Array[(Double, Double)])
    StructuredValues(Datatype, Array[DatasetValue])
    } derive(
    Debug
    )

    Values accepted by the HDF5 new-file writer.

    Hdf5Writer

    pub struct Hdf5Writer {
    // private fields
    }

    In-memory builder for a new HDF5 file.

    Hdf5Writer::add_attribute

    fn Hdf5Writer::add_attribute(self : Hdf5Writer, object_path : String, name : String, dimensions : Array[UInt64], values : Hdf5WriteValues) -> Unit raise Hdf5Error

    Adds an inline attribute to an existing group or dataset.

    Hdf5Writer::add_chunked_dataset

    fn Hdf5Writer::add_chunked_dataset(self : Hdf5Writer, path : String, dimensions : Array[UInt64], chunk_dimensions : Array[UInt], values : Hdf5WriteValues, fill_value? : Hdf5WriteValues, filters? : Array[Hdf5WriteFilter]) -> Unit raise Hdf5Error

    Adds a finite chunked dataset using a fixed-array chunk index.

    Chunks containing only the fill value are left unallocated. Filters are applied in the supplied HDF5 pipeline order.

    Hdf5Writer::add_committed_datatype

    fn Hdf5Writer::add_committed_datatype(self : Hdf5Writer, path : String, datatype : Datatype) -> Unit raise Hdf5Error

    Adds a named committed datatype at path.

    Datasets and unshared attributes whose encoded datatype matches this type reference it through a committed-datatype shared message.

    Hdf5Writer::add_compact_dataset

    fn Hdf5Writer::add_compact_dataset(self : Hdf5Writer, path : String, dimensions : Array[UInt64], values : Hdf5WriteValues) -> Unit raise Hdf5Error

    Adds a compact dataset whose values are stored inline in the object header. Empty dimensions denotes a scalar dataspace.

    Hdf5Writer::add_custom_filter

    fn Hdf5Writer::add_custom_filter(self : Hdf5Writer, filter : &Hdf5CustomFilter) -> Unit raise Hdf5Error

    Registers a caller-defined filter for subsequent chunked datasets. The filter's encode method is used by Custom pipeline entries, and its decode method can read the resulting file through Hdf5File.

    Hdf5Writer::add_dataset

    fn Hdf5Writer::add_dataset(self : Hdf5Writer, path : String, dimensions : Array[UInt64], values : Hdf5WriteValues) -> Unit raise Hdf5Error

    Adds a contiguous dataset. Empty dimensions denotes a scalar dataspace.

    Hdf5Writer::add_extensible_dataset

    fn Hdf5Writer::add_extensible_dataset(self : Hdf5Writer, path : String, dimensions : Array[UInt64], maximum_dimensions : Array[UInt64?], chunk_dimensions : Array[UInt], values : Hdf5WriteValues, fill_value? : Hdf5WriteValues, filters? : Array[Hdf5WriteFilter]) -> Unit raise Hdf5Error

    Adds a chunked dataset with one or more unlimited dimensions.

    A None maximum marks an unlimited axis. One unlimited axis uses an extensible-array index; multiple unlimited axes use a v2 B-tree.

    Hdf5Writer::add_external_dataset

    fn Hdf5Writer::add_external_dataset(self : Hdf5Writer, path : String, dimensions : Array[UInt64], values : Hdf5WriteValues, slots : Array[Hdf5ExternalSlot], fill_value? : Hdf5WriteValues) -> Unit raise Hdf5Error

    Adds an unfiltered dataset whose raw bytes live in external files.

    Slots are concatenated in caller order to form the row-major element stream. Their sizes must add up exactly to the dataset byte size.
    fn Hdf5Writer::add_external_link(self : Hdf5Writer, parent_path : String, name : String, target_file : String, target_path : String) -> Unit raise Hdf5Error

    Adds an external link at parent_path/name.

    target_file is resolved by the HDF5 reader relative to the file containing the link. target_path names an object inside that file.

    Hdf5Writer::add_external_source

    fn Hdf5Writer::add_external_source(self : Hdf5Writer, name : String, source : &Hdf5Source) -> Unit raise Hdf5Error

    Registers a source that external version-4 reference values may target.

    The reference value stores the caller-provided file name; registration is matched after path normalization. Duplicate names are rejected.

    Hdf5Writer::add_group

    fn Hdf5Writer::add_group(self : Hdf5Writer, path : String) -> Unit raise Hdf5Error

    Adds a group and any missing parent groups.
    fn Hdf5Writer::add_soft_link(self : Hdf5Writer, parent_path : String, name : String, target : String) -> Unit raise Hdf5Error

    Adds a soft link at parent_path/name pointing to target.

    Absolute targets start at the root group. Relative targets are resolved from the group containing the link.

    Hdf5Writer::append_dataset

    fn Hdf5Writer::append_dataset(self : Hdf5Writer, path : String, dimensions : Array[UInt64], values : Hdf5WriteValues) -> Unit raise Hdf5Error

    Appends a block at the trailing corner of the dataset's unlimited axes.

    Fixed axes must match the current dataset shape. Each unlimited axis grows by the corresponding block extent. A one-unlimited-axis dataset therefore appends along that axis, while a multi-unlimited dataset grows every unlimited axis and places the block at the old shape's trailing corner.

    Hdf5Writer::encode

    fn Hdf5Writer::encode(self : Hdf5Writer) -> Bytes raise Hdf5Error

    Encodes the builder as a complete HDF5 file.

    Hdf5Writer::new

    fn Hdf5Writer::new() -> Hdf5Writer

    Creates an empty writer containing the root group.

    Hdf5Writer::resize_dataset

    fn Hdf5Writer::resize_dataset(self : Hdf5Writer, path : String, dimensions : Array[UInt64]) -> Unit raise Hdf5Error

    Changes an extensible dataset shape while preserving its overlapping data.

    Hdf5Writer::set_attribute_sharing

    fn Hdf5Writer::set_attribute_sharing(self : Hdf5Writer, enabled : Bool) -> Unit

    Enables or disables writing whole attributes through the shared-object header message (SOHM) table. Attributes whose serialized message is at most 65,535 bytes are shared.

    Hdf5Writer::set_chunk_filter_mask

    fn Hdf5Writer::set_chunk_filter_mask(self : Hdf5Writer, path : String, chunk_index : Int, filter_mask : UInt) -> Unit raise Hdf5Error

    Sets the HDF5 filter mask for one chunk of an existing chunked dataset.

    Bit n set means filter n in the dataset pipeline is skipped for that chunk. The mask is stored in the fixed-array or extensible-array chunk index.

    Hdf5Writer::set_fill_sharing

    fn Hdf5Writer::set_fill_sharing(self : Hdf5Writer, enabled : Bool) -> Unit

    Enables or disables writing complete fill-value messages through the SOHM table. Fill messages at most 65,535 bytes are shared; larger messages remain in the object header.

    Hdf5Writer::set_filter_sharing

    fn Hdf5Writer::set_filter_sharing(self : Hdf5Writer, enabled : Bool) -> Unit

    Enables or disables writing complete filter-pipeline messages through the SOHM table. Pipelines with equal encoded messages reuse one heap ID.

    Hdf5Writer::write_dataset_slab

    fn Hdf5Writer::write_dataset_slab(self : Hdf5Writer, path : String, offsets : Array[UInt64], dimensions : Array[UInt64], values : Hdf5WriteValues) -> Unit raise Hdf5Error

    Writes a rectangular slab at arbitrary dataset coordinates.

    Finite contiguous and chunked datasets require the slab to fit the current shape. Extensible datasets grow only their unlimited axes to the slab's trailing edge, filling newly exposed elements with the dataset fill value. External datasets and dimension-changing writes to finite datasets are rejected.