moon_repomix

    Generate AI-ready repository context with native AST compression, Git diffs/history, HTTPS inputs and exact o200k token counts.

    repomix
    cli
    context
    native
    Download zip
    Author
    Version
    0.5.1
    License
    MIT
    Last updated
    1 hour ago
    Downloads
    68

    #moon-repomix

    把本地目录或公开 HTTPS 仓库整理成 XML、Markdown、JSON 或纯文本,供 AI 阅读、代码评审和问题追踪。使用 MoonBit native 实现,提供 CLI、SDK 和本地浏览器工作台;解析器和分词资源随源码或安装包提供,运行时不需要 Node.js。

    当前源码版本为 v0.5.1,使用 MoonBit 0.10.17+d971670cd 构建;安装包和 SDK 的可用版本分别见 Releases 与 Mooncakes。v0.5 系列新增完整实现读取、累计预算补读和任务上下文规划;原有 CLI、工作台、混合全文/骨架及精确分卷用法保持一致。完整变化见 Changelog,升级步骤见 迁移指南。

    #安装与第一次运行

    从 v0.5.1 Release 下载平台归档,用附带的 SHA256SUMS 校验。解压后保留 bin、share 和 LICENSES 的相对位置,在解压目录运行:

    ./bin/moon-repomix /path/to/project

    Windows PowerShell:

    .\bin\moon-repomix.exe "C:\path\to\project"

    结果写到启动目录的 repomix-output.xml,终端显示文件数和精确 token 数。也可将 bin 加入 PATH,在要处理的项目目录直接运行 moon-repomix。归档运行不需要 MoonBit;本地打包和骨架压缩不需要 Git,差异、历史和远程获取需要系统 Git。

    #从源码运行

    从源码构建需要固定的 MoonBit 0.10.17+d971670cd 和平台 C 工具链,见 工具链。在仓库根目录运行:

    moon update moon run src/cmd/moon-repomix --release -- examples/demo -f md -o -

    终端显示示例源码和目录结构;正文写 stdout,统计与诊断写 stderr。项目默认使用 native,标准源码构建自动查找 assets/ 资源。要将当前源码与资源一起安装到本地前缀:

    moon run --target native scripts/install.mbtx "$HOME/.local"

    Windows PowerShell 使用 moon run --target native scripts/install.mbtx "$env:LOCALAPPDATA\moon-repomix"。安装只需执行一次,将前缀的 bin 加入 PATH 后,日常使用下面的 moon-repomix 命令。移动安装目录时一并保留 bin、share 和 LICENSES。

    #常用命令

    # 当前目录,默认输出 repomix-output.xml moon-repomix # 导出 Markdown;加 --compress 提取骨架 moon-repomix ./project -f md -o context.md moon-repomix ./project --compress -f md -o structure.md # 只选源码和 README,导出 JSON moon-repomix ./project --include 'src/**,README.md' -f json -o context.json # 检查单文档预算,或按每卷硬上限分卷 moon-repomix ./project --budget 12000 moon-repomix ./project --compress --split 12000 -f md -o review-parts

    输入目录默认是当前目录。格式用 -f 指定:xml、md、json、txt;未指定时使用配置值或 XML。-o - 将正文写 stdout。--budget 超限仍保留完整输出并退出 1;--split 限制每卷的最终 token 数,-o 此时是必须不存在的新目录,索引另外计数。

    用 moon-repomix --help 查看常用任务,--help-all 查看全部参数。已有 --style、--token-budget、--split-output 继续可用;JSON 字段名称保持不变。Git 差异、远程仓库、stdin、保存常用配置等用法见 CLI 按任务速查,关键文件保留全文和自动化组合见 使用场景。

    输出和显式配置路径相对启动目录,自动配置位于输入根。普通输出文件会原子替换已有同名文件;失败时保留旧输出。分卷目标必须不存在。详见 配置 与 文件选择。

    #浏览器工作台

    工作台使用原生 HTML/CSS/JavaScript,通过本地 HTTP 接口调用 MoonBit native 引擎。目录浏览、源码读取、安全扫描、AST 压缩和最终 JSON 的精确 token 计数均由 Native 服务完成;浏览器负责配置、查看与下载。

    编辑器工作台:选择目录、生成上下文、切换原文与结构输出、检查预算

    安装 v0.5.1 后,在要处理的项目目录运行:

    moon-repomix --webui

    也可在本仓库根目录直接运行 moon run src/cmd/moon-repomix --release -- --webui。两种方式均默认处理当前目录,自动定位词表和页面资源;指定其他项目时使用 moon-repomix ./project --webui。

    打开 http://127.0.0.1:8765,在右侧选择授权目录、全文/骨架/混合策略及文件规则,点击“生成上下文”;左侧浏览结果文件,中间切换原文、处理后正文或完整 JSON,底部复制 CLI 或下载原始 JSON。

    修改参数会将现有结果标为旧结果,下一次生成成功后才替换;预算超限仍保留完整输出。动图来自真实 Native 服务操作,三种模式的浏览器下载已与相同参数 CLI 输出逐字节核对。端口设置、输入限额与服务边界见 工作台指南。

    #输出、计数与安全

    • 默认保留完整正文;--compress 提取语法骨架,保留声明、类型、注释与结构。支持 MoonBit(含 .mbti)、JavaScript/TypeScript(含 JSX/TSX)、Python、Go、Rust、Java;其他语言或无精确 token 收益时保留原文。范围见 仓库上下文。
    • 读取有效 UTF-8 普通文件,去除 UTF-8 BOM 并保留换行。默认单文件上限 50 MiB;另有 20,000 个过滤后普通候选、200 MiB 原始读取的总量限制。链接、二进制、无效 UTF-8 等跳过原因见 错误指南。
    • 默认扫描压缩前原文,命中时排除整个文件、Git 补丁或提交;模板命中则停止整个打包。扫描是启发式检测,分享前仍应审阅结果,详见 安全边界 和 检测规则。
    • 对完整渲染字符串精确计算 o200k_base tokens,包含格式包装、模板与显示装饰。文件排名和目录统计不能代替最终预算;分卷的独立索引开销见 分卷。

    打包与计数在内存中完成,token 预算不是内存配额。v0.4.1 在固定 35 MB TypeScript 大库的五轮 CLI 对照中,macOS 原文/完整元数据压缩的耗时中位数为 1.05 / 1.43 秒,原版为 1.37 / 1.62 秒;Linux 分别为 1.82 / 2.40 秒,原版为 2.62 / 3.59 秒。平台、内存代价、Linux 原文回退、全部样本及常驻 MCP 结果见 性能测量。

    #SDK

    SDK 模块为 Delthin/moon_repomix。以 0.5.1 为例,在调用项目的 moon.mod 中声明依赖、在 moon.pkg 导入:

    import {
    "Delthin/moon_repomix" @repomix,
    }

    在 async fn 中加载资源并打包:

    let runtime = @repomix.Runtime::load("/absolute/share/moon-repomix/o200k_base.tiktoken")
    let options = @repomix.options("/absolute/project", style=@repomix.Style::Json, compress=true)
    let result = @repomix.pack(options, runtime)
    // result.text 是完整文档,result.tokens 是最终 token 数。
    // result.document 提供规范模型;files / skipped / findings / security / budget 提供处理结果。

    词表同目录须有 tokenizer-unicode-16.0.0.txt,加载时校验两份资源的 SHA-256。Runtime 可复用于多次顺序打包;SDK 返回结果,不写输出文件。pack_volumes、explain_selection、选项与结构字段见 迁移 和 生成接口。

    v0.5.0 新增两种 SDK / MCP 读取方式:

    • 按实现读取与补读:读取完整函数、指定行或符号目录,按需补充依赖;MCP 在累计预算内允许最多三轮成功读取。
    • 任务上下文预算:按必选文件与候选优先级选择全文或骨架,生成多任务共享文档及独立审计。

    #文档与贡献

    贡献指南 说明定点测试、格式与接口更新、资源许可;工具链 说明本地构建和归档。演示指南 提供可保留产物的三步演示。三平台 CI 覆盖 macOS arm64、Linux x64 和 Windows x64。

    #来源与许可

    项目参考 Repomix 的仓库上下文生成行为,采用 MIT 许可。第三方代码及资源告知保存在 LICENSES/,解析器固定版本和本地修改见 来源清单。

    Finding

    Locations refer to the original disk UTF-8 bytes, including a stripped BOM.

    Partition

    Total tokens include each independent volume and the independent index.

    PartitionError

    Callback errors propagate unchanged; these errors describe partition failures.

    Section

    Additional untrusted repository context, such as a diff or commit message.

    Skipped

    Path is root-relative. Reasons are documented in docs/file-selection.md.

    UnitIdentity

    Identity in the input document, independent of section titles.

    Volume

    text is exactly the rendered string passed to the counter.

    CodeContextError

    pub(all) suberror CodeContextError {
    InvalidCodeSpec
    }

    A malformed code-reading request, rejected before filesystem access.

    PackError

    pub(all) suberror PackError {
    ConfigurationError
    ContextTextError(String)
    ContextSecurityFinding(String)
    SelectionError
    InputLimitExceeded(kind~ : String, limit~ : Int64)
    ScanError
    ResourceError
    GitError
    RemoteError
    }

    TaskContextError

    pub(all) suberror TaskContextError {
    InvalidTaskSpec
    }

    Invalid task descriptions are rejected before filesystem access.

    BudgetStatus

    pub(all) enum BudgetStatus {
    NotSet
    Within(Int)
    Exceeded(Int)
    }

    CodeContextPlan

    pub struct CodeContextPlan {
    status : String
    text : String?
    tokens : Int
    bytes : Int
    required_tokens : Int?
    audit : Json
    security : SecurityStatus
    }

    A complete, exactly counted JSON payload, or a body-free failure audit. tokens includes the embedded audit; protocol envelopes are not counted.

    CodeContextSpec

    pub struct CodeContextSpec {
    max_tokens : Int
    // private fields
    }

    Immutable validated request for read_code_context.

    CodeContextSpec::from_json

    fn CodeContextSpec::from_json(value : Json) -> CodeContextSpec raise CodeContextError

    Each path selects exactly one of symbols, inclusive ranges, full:true, or outline:true. includeDependencies is available only with symbols. Unknown fields, duplicate paths, and non-canonical relative paths fail.

    test "code context request example" {
    let spec = @moon_repomix.CodeContextSpec::from_json({
    "maxTokens": 4000,
    "requests": [{ "path": "lib/response.js", "symbols": ["res.render"] }],
    })
    assert_eq(spec.max_tokens, 4000)
    }

    FileStats

    pub(all) struct FileStats {
    path : String
    disk_bytes : Int
    tokens : Int
    original_tokens : Int
    }

    Body counts precede presentation line prefixes and format escaping. PackResult.tokens counts the complete rendered document instead.

    PackResult

    pub struct PackResult {
    document :
    Document

    text : String
    bytes : Int
    tokens : Int
    files : Array[FileStats]
    skipped : Array[
    Skipped
    ]
    findings : Array[
    Finding
    ]
    security : SecurityStatus
    budget : BudgetStatus
    }

    Excluded files retain only metadata; their contents are absent.

    PathExplanation

    pub struct PathExplanation {
    path : String
    eligible : Bool
    status : String
    stage : String
    reason : String
    origin :
    RuleOrigin
    ?
    finding_count : Int
    finding_rules : Array[String]
    }

    One queried path: eligibility precedes source decoding and safety checks.

    Runtime

    pub struct Runtime {
    encoder :
    Encoder

    }

    Validated resources shared by sequential pack operations.

    Runtime::load

    async fn Runtime::load(tokenizer_path : String) -> Runtime

    SecurityStatus

    pub(all) enum SecurityStatus {
    Passed
    Excluded
    Disabled
    }

    SelectionReport

    pub struct SelectionReport {
    paths : Array[PathExplanation]
    security : SecurityStatus
    style :
    Style

    candidate_count : Int
    accepted_count : Int
    read_bytes : Int64
    max_files : Int
    max_total_size : Int64
    }

    A bounded query, not a promise that an entire repository fits its quotas.

    SelectionReport::to_json

    fn SelectionReport::to_json(self : SelectionReport) -> Json

    Machine-safe projection: no file bodies, rule text or absolute rule paths.

    TaskContextPlan

    pub struct TaskContextPlan {
    status : String
    text : String?
    tokens : Int
    bytes : Int
    audit : Json
    audit_text : String
    audit_tokens : Int
    audit_bytes : Int
    security : SecurityStatus
    }

    One shared model input plus a separately counted, body-free audit. Only complete and with_omissions contain text; failures cannot be committed.

    TaskContextSpec

    pub struct TaskContextSpec {
    max_tokens : Int
    // private fields
    }

    A validated, bounded task set. Construct with TaskContextSpec::from_json. The serialized description is retained privately to prevent mutable JSON or arrays supplied by a caller from changing a validated request.

    TaskContextSpec::from_json

    fn TaskContextSpec::from_json(value : Json) -> TaskContextSpec raise TaskContextError

    Parse a strict task description without reading any files. Required entries accept full or skeleton; optional entries also accept either and a priority from 0 to 100 (default 0). Unknown fields are rejected.

    VolumePackResult

    pub struct VolumePackResult {
    partition :
    Partition

    file_names : Map[String, String]
    files : Array[FileStats]
    skipped : Array[
    Skipped
    ]
    findings : Array[
    Finding
    ]
    security : SecurityStatus
    }

    Prepared, scanned input split into separately bounded documents and an index.

    explain_selection

    async fn explain_selection(options :
    Options
    , paths : Array[String]) -> SelectionReport

    Inspect only requested local paths. Uses the same selection and source safety stages as pack, without tokenizer loading, transforms, templates or Git export. Does not write files or alter any generated context.

    options

    fn options(directory : String, style? :
    Style
    , split_output? :
    SplitOutput
    , compress? : Bool, keep_full_patterns? : Array[String], header_text? : String, instruction_file_path? : String?, full_metadata? : Bool, file_summary? : Bool, directory_structure? : Bool, include_files? : Bool, git_sections? : Bool, metadata? : Bool, show_line_numbers? : Bool, output_file_path_style? :
    PathStyle
    , file_paths? : Array[String]?, include_diffs? : Bool, include_logs? : Bool, include_logs_count? : Int, remote? : String?, remote_branch? : String?, max_files? : Int, max_total_size? : Int64) ->
    Options
    raise

    Validated options for one directory, with the selected format and scanning enabled.

    Use options("/absolute/project", style=Json) to select JSON and its default filename.

    pack

    async fn pack(options :
    Options
    , runtime : Runtime) -> PackResult

    Build and count a complete document. Remote inputs are fetched and cleaned here. The caller decides where to write the result.

    pack_volumes

    async fn pack_volumes(options :
    Options
    , runtime : Runtime, max_tokens~ : Int) -> VolumePackResult

    Prepare one immutable context, then split it at file/section boundaries. max_tokens bounds every complete rendered volume; index_tokens is separate. Options.token_budget is the single-document pack budget and is not applied here. Returned strings are the exact strings counted; this function writes no files.

    plan_task_context

    async fn plan_task_context(options :
    Options
    , runtime : Runtime, spec : TaskContextSpec) -> TaskContextPlan

    Build one exact-token task context from literal paths under the host's existing selection and security rules. Optional files are read lazily in descending priority. This local-only API rejects remote/Git/split-output options and hidden file bodies; pack and pack_volumes are unchanged.

    The cap covers the returned text, not audit or the caller's model messages. Reserve those costs before choosing maxTokens. Read limits retain the existing reader's single excess-byte probe.

    read_code_context

    async fn read_code_context(options :
    Options
    , runtime : Runtime, spec : CodeContextSpec) -> CodeContextPlan

    Read explicitly scoped symbols, source ranges, outlines, or full files. Selection and security are applied before indexing or slicing. All requested entries must be available and fit the complete JSON payload cap; no source body is returned on failure. Dependency expansion is lexical and same-file, not a semantic cross-file call graph. Use another request for missing code.

    version

    fn version() -> String

    Version of the native CLI and library.

    Powered by MoonBit

    Site sourceReport issueModulesSkillsBuild queueStatistics

    © 2026 mooncakes.io