🥮 mooncakes.io
    Sign in

    markitdown

    A MoonBit-native document-to-Markdown converter with multi-format parsing, metadata, assets, batch conversion, and validation tooling

    markdown
    pdf
    docx
    xlsx
    pptx
    html
    csv
    json
    xml
    yaml
    epub
    zip
    txt
    Download zip
    Author
    Version
    0.3.3
    License
    Apache-2.0
    Repository
    Last updated
    5 months ago
    Downloads
    88

    #markitdown-mb

    CI MoonBit CLI Platforms Formats Status Validation License

    markitdown-mb is a MoonBit-native document-to-Markdown converter for local document structure extraction, RAG ingestion, and knowledge-base import. The project is built around a native CLI, a unified IR, deterministic metadata and asset sidecars, and checked-in validation and benchmark evidence.

    It is inspired by Microsoft MarkItDown, but it is an independent MoonBit-native implementation and repository design. It is not a Python package, not a Microsoft project, and not affiliated with the AutoGen team.

    Current pipeline:

    multi-format input -> unified IR -> Markdown / assets / metadata sidecar

    Sealed H2++ / H3++ scope: XLSX, HTML, ZIP, EPUB, DOCX, PPTX, and PDF for native text-PDF scope.

    The current 0.3.3 release line packages the repository's unified validation surface around ./samples/check.sh and ./samples/bench.sh, and keeps the checked-in samples/real_world corpus focused on longer complex-scenario documents.

    #Supported Platforms

    markitdown-mb targets MoonBit native builds on:

    • Windows
    • Linux
    • macOS

    The core converter is designed as a MoonBit-native CLI across those platforms. The repository's validation and benchmark scripts are shell-based and are primarily exercised in Unix-like environments; Windows users can use the native build path and run the sample/benchmark script layer through WSL or an equivalent POSIX shell until a dedicated Windows CI/script layer is added. GitHub Actions validation is currently checked in for Ubuntu and macOS.

    #Current Status

    FormatCurrent statusScope
    XLSXH2++ complete / H3++ evidence-backednative overlap corpus; lightweight formula evaluator v1, merged-cell boundary, typed cells, sheet state
    HTML / HTMH2++ complete / H3++ evidence-backedlightweight safe parser; no browser-grade parsing, JS, CSS layout, or remote fetch
    ZIPH2++ complete / H3++ evidence-backedsafe container conversion; nested dispatch and asset remap; no nested archive recursion
    EPUBH2++ complete / H3++ evidence-backedZIP + OPF + spine + nav/NCX + XHTML chapters; no DRM/CSS/JS/remote fetch
    DOCXH2++ complete / H3++ evidence-backedWord document structure recovery; not a Word layout engine
    PPTXH2++ complete / H3++ evidence-backedpresentation information structure recovery; not a PowerPoint layout engine
    PDFH2++ complete for native text-PDF scope / H3++ evidence-backednative text-PDF only; no default OCR/scanned-PDF claim; no full PDF layout engine
    CSV / TSV / JSON / YAML / XML / Markdown / TXTstable structured/text pathsconservative boundaries documented in support docs; not all families are second-round sealed

    Benchmark and quality conclusions are limited to the checked-in corpora and runner contracts named in the repository docs. They are not blanket claims about all documents of a format family.

    #Core Capabilities

    • unified IR across document families
    • shared profile-driven Text Normalization v2 substrate with staged PDF extracted-text and comparison cleanup
    • Markdown main output
    • assets/ export for materialized local images
    • metadata sidecar via --with-metadata
    • batch conversion with isolated per-document roots
    • unified multi-format debug inspect CLI
    • benchmark governance and checked-in benchmark corpora
    • prebuilt-native runner preference for validation and benchmark work

    #Performance Snapshot

    The H3++ performance evidence is based on the prebuilt-native CLI path, not moon run.

    The checked-in overlap comparison uses Microsoft MarkItDown 0.1.5 on named local samples from samples/benchmark/compare_corpus.tsv. Representative single-run examples currently sit in roughly the 20x to 50x range:

    Format / casemarkitdown-mbMicrosoft MarkItDown 0.1.5Ratio
    XLSX formula cached values10 ms480 ms~48x
    DOCX nested lists mixed31 ms821 ms~26x
    PPTX title bullets18 ms710 ms~39x
    PDF URI link basic11 ms516 ms~47x

    These measurements are corpus-scoped local benchmark facts, not universal performance claims. PDF comparison rows apply only to the native text-PDF overlap corpus. Full raw results, representative tables, and caveats live in docs/validation-and-benchmark-summary.md and docs/benchmark-governance.md.

    #CLI

    Build the native product-path binary:

    moon build --target native

    Recommended invocation:

    ./_build/native/debug/build/cli/cli.exe normal <input> [output]

    Other product-path entrypoints:

    ./_build/native/debug/build/cli/cli.exe normal --with-metadata <input> <output.md>
./_build/native/debug/build/cli/cli.exe batch <input_dir> <output_dir>
./_build/native/debug/build/cli/cli.exe debug --json <input>
./_build/native/debug/build/cli/cli.exe ocr <input> [output]

    moon run remains a development fallback. It is not the preferred runner for H3++ performance conclusions.

    #Validation

    Recommended repository verification:

    moon build --target native
moon check
moon test
./samples/check.sh
./samples/bench.sh --suite smoke --kind smoke

    Checked-in GitHub Actions CI now runs moon build --target native, moon check, moon test, and ./samples/check.sh on ubuntu-latest and macos-latest for push and pull_request. ./samples/bench.sh --suite smoke--kind smoke remains available locally and as a manual workflow_dispatch job; it is not part of the default PR gate. Windows core native support remains documented, but the shell validation suite still targets WSL or another POSIX shell rather than native Windows CI. moon publish remains a manual release step. Lower-layer parser/core and unsafe-boundary fixtures now live under samples/fixtures; user-visible regression inputs now live under one unified samples/main_process tree, with metadata-heavy and asset-heavy subcases co-located under the same format roots. Each format package now keeps its checked Markdown and exact CLI metadata expectations under samples/main_process/<format>/expected/. A checked-in samples/real_world corpus now complements the smaller feature-focused samples/main_process set with complex-only scenario files across DOCX, PPTX, XLSX, PDF, HTML, ZIP, and EPUB. The default ./samples/check.sh chain runs the full real-world set, and ./samples/check.sh --real-world --tags complex remains available for focused reruns.

    Detailed validation counts, sample matrices, metadata/assets checks, benchmark smoke counts, batch profile results, and MarkItDown comparison runs are tracked in docs/validation-and-benchmark-summary.md.

    #Documentation

    #Non-goals

    markitdown-mb does not aim to be:

    • a Word, PowerPoint, or PDF visual layout engine
    • a browser-grade HTML renderer
    • an OCR-first default converter
    • a DRM/CSS/JS/remote-fetch pipeline
    • a full Excel formula engine
    • a benchmark claim beyond the checked-in corpora

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io