cf_ocean

    MoonBit CF Ocean Data Exchange and Slicing Tools for NetCDF Classic/64-bit Offset and Oceanographic Data Abstractions

    oceanography
    netcdf
    cf-conventions
    dataset
    slicing
    Download zip
    Author
    Version
    0.1.4
    License
    Apache-2.0
    Last updated
    2 months ago
    Downloads
    52

    No dependencies

    #MoonBit CF Ocean

    CI License MoonBit

    MoonBit CF Ocean is a pure-MoonBit toolkit for inspecting, validating, slicing, quality-controlling, and exchanging oceanographic datasets. It combines NetCDF Classic/CDF-2 binary access with CF-oriented metadata, seawater calculations, multidimensional indexing, deterministic QC reports, and format-specific export helpers.

    #Capabilities and support boundaries

    AreaSupportedExplicit boundary
    NetCDFClassic (CDF-1) and 64-bit Offset (CDF-2) headers and variable payloadsNetCDF-4/HDF5 containers are not implemented
    CF metadataCF-oriented dimensions, attributes, units, standard-name validationThe library does not attempt to validate every CF convention
    SlicingStrided 3D/4D index ranges, masked arrays, lazy scalingData arrays are represented as flat MoonBit arrays
    Quality controlMissing, range, rate, spike, and flatline flags with summariesRules are deterministic thresholds; they are not a replacement for domain review
    Profile screeningDepth-ordered temperature/salinity QC with gradients, density inversions, and stable reportsScreening flags do not sort or repair observations and do not replace mission-specific QC
    Scientific methodsBounded EOS-80/UNESCO-style and TEOS-10-oriented approximationsNot a complete TEOS-10/GSW thermodynamic implementation; see scientific methods
    ZarrZarr v2 metadata and raw little-endian f8 chunksChunks are uncompressed (compressor: null); zlib/Blosc codecs are not included
    ParquetCompact JSON schema representation for ocean profilesThis helper does not emit a complete Parquet binary file or Arrow buffers
    Other exportsCSV, JSON, GeoTIFF metadata, and lineage recordsEach helper documents its representation in its package tests

    #Quick start

    The repository uses the MoonBit stable toolchain installed by the project CI.

    moon fmt --check moon check --target all --deny-warn moon test --target all --deny-warn

    Run the deterministic native workload suite:

    moon run cmd/benchmark --target native

    The command prints workload checksums. Process-level wall-time samples and reproduction instructions are recorded in BENCHMARKS.md.

    To consume the published module from another MoonBit project:

    moon add hrwqe/cf_ocean

    The reusable analytics package is imported as hrwqe/cf_ocean/src/analytics; the repository benchmark remains the smallest complete runnable example.

    #API examples

    #Validate an ocean variable

    let dataset = @dataset.OceanDataset::new("cast")
    dataset.add_axis(@dataset.CoordinateAxis::new(
    "depth",
    @dataset.DepthAxis,
    "m",
    [0.0, 50.0, 100.0],
    ))
    let temperature = @dataset.DatasetVariable::new(
    "temperature",
    "sea_water_temperature",
    "degree_Celsius",
    ["depth"],
    [3],
    [21.0, 18.5, 16.0],
    )
    dataset.add_variable(temperature)
    let issues = @dataset.validate_dataset(dataset)

    #Apply deterministic quality control

    let config : @quality.QCConfig = {
    fill_value: Some(-999.0),
    min_value: Some(-2.0),
    max_value: Some(40.0),
    max_delta: Some(5.0),
    flatline_window: 4,
    }
    let report = @quality.inspect_series(
    "temperature",
    [20.1, 20.2, -999.0, 20.0],
    config,
    )
    let masked = @quality.qc_mask([20.1, 20.2, -999.0, 20.0], report, -999.0)

    #Screen a vertical ocean profile

    let samples = [
    @analytics.ProfileQcSample::new(0.0, 20.0, 35.0),
    @analytics.ProfileQcSample::new(50.0, 18.5, 35.2),
    ]
    let report = @analytics.scan_ocean_profile_default(samples)
    let ready_for_export = report.is_clean()

    The profile scanner preserves input order and reports non-finite values, documented range violations, duplicate or descending depths, large adjacent gradients, and increasing-depth density inversions. It does not silently sort or repair data.

    #Read a typed NetCDF variable

    fn inspect_netcdf(bytes : Array[Byte]) -> Unit {
    match @binary.parse_nc_header(@binary.ByteBuffer::new(bytes)) {
    Some(header) =>
    match @binary.read_nc_variable(bytes, header, "temperature") {
    Some(values, diagnostics) =>
    println("values=" + values.length().to_string() +
    ", diagnostics=" + diagnostics.length().to_string())
    None => println("variable payload is unavailable")
    }
    None => println("invalid NetCDF header")
    }
    }

    #Package map

    src/ ├── binary/ NetCDF Classic/CDF-2 buffers, layouts, diagnostics ├── cf/ CF metadata, units, and compliance diagnostics ├── dataset/ axes, grids, profiles, time series, validation, resampling ├── slice/ ranges, 3D/4D indexing, masked and lazy arrays ├── quality/ deterministic observation QC rules and reports ├── analytics/ seawater physics, profile screening, and spatial-temporal calculations ├── benchmark/ reusable deterministic benchmark workloads ├── export/ NetCDF, Zarr v2, CSV, JSON, Parquet-schema helpers ├── lineage/ transformation and provenance records └── cli/ summaries, validation reports, and benchmark formatting cmd/benchmark/ executable benchmark entrypoint

    #Correctness and known limitations

    • The header parser rejects malformed magic/version/truncated headers; binary layout validation reports invalid dimensions, offsets, and truncated variable payloads instead of silently reading past the available byte array.
    • NetCDF exporter offsets are computed from the encoded header and payloads are padded to the declared data positions.
    • Dataset validation checks axis monotonicity, duplicate axes, dimensions, shapes, and data lengths. Profile resampling supports ascending and descending depth axes with reject or clamp extrapolation.
    • Ocean-physics routines expose a conservative input validator and deterministic regression anchors. They are explicitly bounded approximations; use a dedicated TEOS-10/GSW implementation for publication-grade thermodynamics.
    • Zarr chunk output is intentionally honest raw f8 data. Metadata never claims compression that the writer has not applied.
    • Parquet output is intentionally a schema JSON bridge; use a dedicated Parquet writer when a binary Parquet file is required.

    #Tests and CI

    The local commands mirror the repository workflow:

    moon fmt --check moon check --target all --deny-warn moon build --target all moon test --target all --deny-warn --enable-coverage moon coverage report -f summary moon info

    The workflow also checks that formatting and generated public interfaces are clean. See .github/workflows/ci.yml.

    #License and source attribution

    The project is distributed under the Apache License 2.0. Standards and reference specifications used by the implementation are listed in SOURCE_ATTRIBUTION.md. Scientific method scope and reference links are collected in docs/scientific-methods.md.