cf_ocean

MoonBit CF Ocean Data Exchange and Slicing Tools for NetCDF Classic/64-bit Offset and Oceanographic Data Abstractions

oceanography
netcdf
cf-conventions
dataset
slicing
moon add hrwqe/cf_ocean@0.1.3
Download zip
Author
Version
0.1.3
License
Apache-2.0
Last updated
3 hours ago
Downloads
8
README

#MoonBit CF Ocean

CI License MoonBit

MoonBit CF Ocean is a pure-MoonBit toolkit for inspecting, validating, slicing, quality-controlling, and exchanging oceanographic datasets. It combines NetCDF Classic/CDF-2 binary access with CF-oriented metadata, seawater calculations, multidimensional indexing, deterministic QC reports, and format-specific export helpers.

#Capabilities and support boundaries

AreaSupportedExplicit boundary
NetCDFClassic (CDF-1) and 64-bit Offset (CDF-2) headers and variable payloadsNetCDF-4/HDF5 containers are not implemented
CF metadataCF-oriented dimensions, attributes, units, standard-name validationThe library does not attempt to validate every CF convention
SlicingStrided 3D/4D index ranges, masked arrays, lazy scalingData arrays are represented as flat MoonBit arrays
Quality controlMissing, range, rate, spike, and flatline flags with summariesRules are deterministic thresholds; they are not a replacement for domain review
Profile screeningDepth-ordered temperature/salinity QC with gradients, density inversions, and stable reportsScreening flags do not sort or repair observations and do not replace mission-specific QC
Scientific methodsBounded EOS-80/UNESCO-style and TEOS-10-oriented approximationsNot a complete TEOS-10/GSW thermodynamic implementation; see scientific methods
ZarrZarr v2 metadata and raw little-endian f8 chunksChunks are uncompressed (compressor: null); zlib/Blosc codecs are not included
ParquetCompact JSON schema representation for ocean profilesThis helper does not emit a complete Parquet binary file or Arrow buffers
Other exportsCSV, JSON, GeoTIFF metadata, and lineage recordsEach helper documents its representation in its package tests

#Quick start

The repository uses the MoonBit stable toolchain installed by the project CI.

moon fmt --check moon check --target all --deny-warn moon test --target all --deny-warn

Run the deterministic native workload suite:

moon run cmd/benchmark --target native

The command prints workload checksums. Process-level wall-time samples and reproduction instructions are recorded in BENCHMARKS.md.

To consume the published module from another MoonBit project:

moon add hrwqe/cf_ocean

The reusable analytics package is imported as hrwqe/cf_ocean/src/analytics; the repository benchmark remains the smallest complete runnable example.

#API examples

#Validate an ocean variable

let dataset = @dataset.OceanDataset::new("cast")
dataset.add_axis(@dataset.CoordinateAxis::new(
"depth",
@dataset.DepthAxis,
"m",
[0.0, 50.0, 100.0],
))
let temperature = @dataset.DatasetVariable::new(
"temperature",
"sea_water_temperature",
"degree_Celsius",
["depth"],
[3],
[21.0, 18.5, 16.0],
)
dataset.add_variable(temperature)
let issues = @dataset.validate_dataset(dataset)

#Apply deterministic quality control

let config : @quality.QCConfig = {
fill_value: Some(-999.0),
min_value: Some(-2.0),
max_value: Some(40.0),
max_delta: Some(5.0),
flatline_window: 4,
}
let report = @quality.inspect_series(
"temperature",
[20.1, 20.2, -999.0, 20.0],
config,
)
let masked = @quality.qc_mask([20.1, 20.2, -999.0, 20.0], report, -999.0)

#Screen a vertical ocean profile

let samples = [
@analytics.ProfileQcSample::new(0.0, 20.0, 35.0),
@analytics.ProfileQcSample::new(50.0, 18.5, 35.2),
]
let report = @analytics.scan_ocean_profile_default(samples)
let ready_for_export = report.is_clean()

The profile scanner preserves input order and reports non-finite values, documented range violations, duplicate or descending depths, large adjacent gradients, and increasing-depth density inversions. It does not silently sort or repair data.

#Read a typed NetCDF variable

fn inspect_netcdf(bytes : Array[Byte]) -> Unit {
match @binary.parse_nc_header(@binary.ByteBuffer::new(bytes)) {
Some(header) =>
match @binary.read_nc_variable(bytes, header, "temperature") {
Some(values, diagnostics) =>
println("values=" + values.length().to_string() +
", diagnostics=" + diagnostics.length().to_string())
None => println("variable payload is unavailable")
}
None => println("invalid NetCDF header")
}
}

#Package map

src/ ├── binary/ NetCDF Classic/CDF-2 buffers, layouts, diagnostics ├── cf/ CF metadata, units, and compliance diagnostics ├── dataset/ axes, grids, profiles, time series, validation, resampling ├── slice/ ranges, 3D/4D indexing, masked and lazy arrays ├── quality/ deterministic observation QC rules and reports ├── analytics/ seawater physics, profile screening, and spatial-temporal calculations ├── benchmark/ reusable deterministic benchmark workloads ├── export/ NetCDF, Zarr v2, CSV, JSON, Parquet-schema helpers ├── lineage/ transformation and provenance records └── cli/ summaries, validation reports, and benchmark formatting cmd/benchmark/ executable benchmark entrypoint

#Correctness and known limitations

  • The header parser rejects malformed magic/version/truncated headers; binary layout validation reports invalid dimensions, offsets, and truncated variable payloads instead of silently reading past the available byte array.
  • NetCDF exporter offsets are computed from the encoded header and payloads are padded to the declared data positions.
  • Dataset validation checks axis monotonicity, duplicate axes, dimensions, shapes, and data lengths. Profile resampling supports ascending and descending depth axes with reject or clamp extrapolation.
  • Ocean-physics routines expose a conservative input validator and deterministic regression anchors. They are explicitly bounded approximations; use a dedicated TEOS-10/GSW implementation for publication-grade thermodynamics.
  • Zarr chunk output is intentionally honest raw f8 data. Metadata never claims compression that the writer has not applied.
  • Parquet output is intentionally a schema JSON bridge; use a dedicated Parquet writer when a binary Parquet file is required.

#Tests and CI

The local commands mirror the repository workflow:

moon fmt --check moon check --target all --deny-warn moon build --target all moon test --target all --deny-warn --enable-coverage moon coverage report -f summary moon info

The workflow also checks that formatting and generated public interfaces are clean. See .github/workflows/ci.yml.

#License and source attribution

The project is distributed under the Apache License 2.0. Standards and reference specifications used by the implementation are listed in SOURCE_ATTRIBUTION.md. Scientific method scope and reference links are collected in docs/scientific-methods.md.