🥮 mooncakes.io

moonbit-deid

A MoonBit toolkit for detecting, replacing, and auditing PHI in Chinese and English clinical text.

de-identification
phi
medical
privacy
nlp
moon add ljlljl678/moonbit-deid@0.1.1
Download zip
Author
Version
0.1.1
License
Apache-2.0
Repository
Last updated
7 hours ago
Downloads
5

Dependencies

README

#moonbit-deid

moonbit-deid is a deterministic, library-first toolkit for detecting and redacting personally identifiable information (PII/PHI) in Chinese and English clinical text. A redaction returns the transformed text together with findings, replacement offsets, and an audit report, so downstream systems can review what changed instead of treating masking as a black box.

#Core capabilities

  • Chinese and English rules for names, identifiers, phones, email addresses, dates, medical records, insurance records, addresses, organizations, and context-labelled fields.
  • Regex-backed detection with confidence thresholds, disabled-rule handling, contextual rules, and deterministic overlap resolution.
  • Token, preserve-length, partial-mask, and stable-hash replacement modes.
  • Reusable policies with allowlisted ranges, denied rules, locale hints, and review-only operation.
  • Offset mappings between original and redacted text, audit exports, risk summaries, quality metrics, and integrity contracts.
  • Batch and streaming pipelines, structured field helpers, Unicode-safe boundaries, and an adapter for external NER entities.
  • Time-shift privacy, pseudonym vaults, HL7 v2 and FHIR bundle processing, data minimization, review queues, access decisions, retention planning, lineage graphs, and pipeline observability.
  • Synthetic evaluation fixtures and a deterministic CLI benchmark.

#Quick start

///|
test {
  let result = redact("电话13800138000")
  inspect(result.text, content="电话[PHONE]")
  inspect(result.offsets.length() > 0, content="true")
}

Run the CLI with the latest local MoonBit toolchain:

moon run --target wasm-gc cmd/main -- "姓名：张三 电话13800138000" --audit
moon run --target wasm-gc cmd/main -- --mode=preserve "Phone 415-555-1234"
moon run --target wasm-gc cmd/main -- --mode=partial "Email: zhang@example.com"
moon run --target wasm-gc cmd/main -- --mode=hash "电话13800138000"

#CLI

moon run cmd/main -- [options] [text]

Supported options are --audit, --mode=preserve, --mode=partial, --mode=hash, --benchmark[=N], and --help. The benchmark uses fixed clinical fixtures and the same library pipeline as normal redaction.

#Architecture

  1. rules.mbt, clinical_rules.mbt, and scanner.mbt define detectors and produce normalized findings.
  2. redaction_policy.mbt and pipeline_orchestration.mbt apply policy, confidence, locale, allowlist, and overlap decisions.
  3. deid.mbt, offset_mapping.mbt, and audit_exports.mbt apply replacements and preserve reviewable provenance.
  4. batch_pipeline.mbt, streaming_pipeline.mbt, structured_fields.mbt, ner_adapters.mbt, and evaluation.mbt provide integration surfaces for applications and test fixtures.

The public convenience API is in public_api.mbt; lower-level functions stay available for applications that need explicit rules or policies.

#Benchmarks

Reproduce the local measurement with:

moon run cmd/main -- --benchmark=100

On 2026-08-19, three local runs processed 3 cases, 500 documents, 63,500 input characters, and 3,600 findings each. Elapsed times were 1,224 ms, 926 ms, and 752 ms. Full command output and methodology are recorded in BENCHMARKS.md; elapsed time is host-dependent.

The repository currently contains 21,032 lines of production .mbt source, counted locally while excluding test files and generated build output. CI keeps a transparent lower-bound check of 20,000 production lines.

#Tests and development

The test suite covers ordinary cases and boundary behavior for malformed identifiers, checksums, empty input, Unicode code-unit boundaries, overlapping spans, protected ranges, structured fields, streaming chunks, external entities, mappings, batches, and evaluation denominators.

moon fmt --check
moon info
moon check --target all --deny-warn
moon test --target all --deny-warn
moon run --target wasm-gc cmd/main -- --benchmark=10

Generated interfaces are kept in pkg.generated.mbti and cmd/main/pkg.generated.mbti. Do not edit them manually; regenerate them with moon info.

#CI

.github/workflows/test.yml checks the current stable MoonBit toolchain on Ubuntu, macOS, and Windows. It verifies formatting, generated interfaces, strict type checking, tests, source scale, and the CLI benchmark smoke path. Publishing is deliberately a separate manual workflow so a normal pull request cannot publish a package accidentally.

#Scope and license

The package provides deterministic engineering primitives, not a claim of regulatory compliance. Production deployments should review rule coverage, thresholds, audit retention, and institution-specific policy.

This project is licensed under the Apache License 2.0.

⚠️ Package is too large to be displayed

This package contains over 1372 items.

Source Files

Powered by MoonBit

Site sourceReport issuePackagesBuild queueSkillsStatistics

© 2026 mooncakes.io