🥮 mooncakes.io

    agentcanary

    Deterministic, artifact-aware policy replay and counterfactual regression testing for AI tool traces.

    agent
    policy
    replay
    security
    testing
    Download zip
    Version
    0.1.1
    License
    MIT
    Repository
    Last updated
    17 days ago
    Downloads
    33

    No dependencies

    #AgentCanary

    CI Mooncakes

    AI 工具调用的离线策略回放与回归测试引擎。 核心使用 MoonBit，无第三方 Mooncakes 依赖，同一套逻辑运行于 Wasm、Wasm GC、JavaScript 和 Native 后端。

    一份客户资料被读取、摘要、再上传：只检查上传调用本身，看不到资料来自哪里。AgentCanary 沿显式 artifact 依赖传播敏感标签，给出来源事件；同时检查审批复用、累计费用、窗口频率和前置步骤。策略升级时，对同一观察记录比较两套策略，确认新增拦截与误拦风险。

    在线实验室 · English · 严格语义 · 输入格式 · 可执行 API 示例 · 参赛申报书

    交互实验室

    #五分钟运行

    安装 MoonBit 工具链 0.10.16 或更高版本（旧版可运行 moon upgrade）；CLI 和浏览器演示还需 Node.js 22 或更高版本。无需 npm install、API Key、模型或云服务。

    git clone https://github.com/zhangbowen2006/AgentCanary.git
cd AgentCanary
moon check --deny-warn
moon test --deny-warn
moon run cmd/demo
node scripts/build.mjs
node bin/agentcanary.mjs audit --policy examples/policy.json --trace examples/incident.jsonl
node bin/agentcanary.mjs test examples/regressions.json
node scripts/serve.mjs

    打开 http://127.0.0.1:4173。演示提供三类风险、时间线、来源链、策略开关、预算调整、基线对比、JSON 编辑、记录导入和报告导出；导入内容只在当前浏览器处理。开始运行后不需要外部网络。

    样例共 10 条观察事件、9 次工具调用：6 次放行、3 次拒绝。拒绝分别为敏感数据外发、单次审批复用、模型累计成本 1100 超过 1000。公开资料发布仍放行。全部样例均为人工构造的虚构记录，未执行实际删除或上传。

    #在 MoonBit 项目中复用

    moon add zhangbowen2006/agentcanary@0.1.1

    在调用包的 moon.pkg 中添加：

    import {
      "zhangbowen2006/agentcanary" @canary,
    }

    通过 parse_policy、parse_trace、evaluate、compare、run_suite 获得类型化结果；字符串接口为 evaluate_json、compare_json、run_suite_json。具体示例在 API.mbt.md，这些示例也参与编译和测试。生成的 pkg.generated.mbti 列出完整公共 API。

    #用作回归门禁

    # 有拒绝判定时退出 1；普通 audit 默认只生成报告。
node bin/agentcanary.mjs audit --policy examples/policy.json --trace examples/incident.jsonl --fail-on-deny

# 策略收紧导致新增拒绝时退出 1；也可以使用 --fail-on-new-allow。
node bin/agentcanary.mjs compare --baseline examples/baseline.json --candidate examples/policy.json --trace examples/incident.jsonl --fail-on-new-deny

# 自包含用例集：每个调用均需断言判定和完整的违规码集合。
node bin/agentcanary.mjs test examples/regressions.json --format json

    audit 和 compare 的 JSONL 记录可用 --trace - 从 stdin 输入。三条命令均支持 --format json|text。退出码：0 通过，1 门禁或用例不通过，2 输入、文件或环境错误。

    #规则与边界

    规则作用
    allow / deny工具和资源的锚定 glob 匹配；deny 优先
    block_labels检查本次声明和上游继承的敏感标签，附来源事件
    require_approval同会话、精确工具和资源、到期失效、单次使用、可撤销
    budget每会话累计或滑动窗口的整数微积分预算
    rate每会话滑动窗口内的调用次数
    require_before在本会话及可选窗口内已成功完成的前置工具

    这是对已观察历史的离线分析。被策略判为拒绝的历史调用仍参与费用和数据流计算；比较结果不模拟拦截之后 Agent 会产生的全新行为。采集器必须真实、完整地记录 artifact 依赖与敏感标签，并独立认证审批来源。引擎不执行工具，不检测自然语言提示注入，不推断未标注的敏感内容，也不提供操作系统沙箱。详见 语义与信任边界。

    报告默认省略资源路径，不接受任意原始 payload 字段；事件 ID、工具名和标签仍可能包含调用方写入的信息，公开报告前应自行审查。

    #验证与工程证据

    moon check --target all --deny-warn
moon test --target all --deny-warn
moon build --target all --deny-warn
moon fmt --check
moon info
node scripts/build.mjs
node --test tests/cli.test.mjs
node scripts/benchmark.mjs

    Native 需要系统 C 编译环境；Windows 用户可用 MSVC Developer PowerShell。只使用浏览器、CLI 或默认 Wasm GC 后端无需 C 编译器。CI 在 Linux、macOS、Windows 检查全部四个后端，并验证生成接口、CLI 和回归用例。基准说明 记录测量方法及限制，不将本机数据泛化为所有平台性能保证。

    项目为 zhangbowen2006 的十月独立新项目；无既有参赛项目核心代码复用、无开源项目移植。核心、测试、合成样例和演示有 AI 辅助，范围、来源、技术解释和维护事项见 PROVENANCE.md。许可证：MIT。

    InputError

    source
    pub(all) suberror InputError {
      InputError(String)
    } derive(Eq)

    Structured diagnostic. Messages contain field paths, never raw input values.

    InputError::equal

    source
    fn InputError::equal(InputError, InputError) -> Bool

    InputError::not_equal

    source
    fn InputError::not_equal(x : InputError, y : InputError) -> Bool

    CaseResult

    source
    pub(all) struct CaseResult {
      name : String
      passed : Bool
      calls : Int
      mismatches : Array[Mismatch]
    } derive(Eq, ToJson)

    CaseResult::equal

    source
    fn CaseResult::equal(CaseResult, CaseResult) -> Bool

    CaseResult::not_equal

    source
    fn CaseResult::not_equal(x : CaseResult, y : CaseResult) -> Bool

    CaseResult::to_json

    source
    fn CaseResult::to_json(CaseResult) -> Json

    Change

    source
    pub(all) struct Change {
      event_id : String
      session : String
      before : String
      after : String
      before_rules : Array[String]
      after_rules : Array[String]
      before_findings : Array[Finding]
      after_findings : Array[Finding]
    } derive(Eq, ToJson)

    Change::equal

    source
    fn Change::equal(Change, Change) -> Bool

    Change::not_equal

    source
    fn Change::not_equal(x : Change, y : Change) -> Bool

    Change::to_json

    source
    fn Change::to_json(Change) -> Json

    Comparison

    source
    pub(all) struct Comparison {
      schema_version : Int
      baseline : String
      candidate : String
      newly_denied : Int
      newly_allowed : Int
      explanation_changes : Int
      changes : Array[Change]
    } derive(Eq, ToJson)

    Comparison::equal

    source
    fn Comparison::equal(Comparison, Comparison) -> Bool

    Comparison::not_equal

    source
    fn Comparison::not_equal(x : Comparison, y : Comparison) -> Bool

    Comparison::to_json

    source
    fn Comparison::to_json(Comparison) -> Json

    Decision

    source
    pub(all) struct Decision {
      event_id : String
      session : String
      seq : Int
      at_ms : Int
      tool : String
      decision : String
      findings : Array[Finding]
      taints : Array[Taint]
    } derive(Eq, ToJson)

    Decision::equal

    source
    fn Decision::equal(Decision, Decision) -> Bool

    Decision::not_equal

    source
    fn Decision::not_equal(x : Decision, y : Decision) -> Bool

    Decision::to_json

    source
    fn Decision::to_json(Decision) -> Json

    Event

    source
    pub(all) struct Event {
      id : String
      session : String
      seq : Int
      at_ms : Int
      kind : String
      tool : String
      resource : String
      inputs : Array[String]
      output : String?
      labels : Array[String]
      cost : Int
      approval : String?
      expires_ms : Int?
      outcome : String
    } derive(Eq, ToJson)

    A collector-supplied event. Times are nonnegative, relative milliseconds. Costs use integer microcredits, never floating-point currency.

    Event::equal

    source
    fn Event::equal(Event, Event) -> Bool

    Event::not_equal

    source
    fn Event::not_equal(x : Event, y : Event) -> Bool

    Event::to_json

    source
    fn Event::to_json(Event) -> Json

    Finding

    source
    pub(all) struct Finding {
      rule_id : String
      code : String
      message : String
      evidence : Array[String]
    } derive(Eq, ToJson)

    Finding::equal

    source
    fn Finding::equal(Finding, Finding) -> Bool

    Finding::not_equal

    source
    fn Finding::not_equal(x : Finding, y : Finding) -> Bool

    Finding::to_json

    source
    fn Finding::to_json(Finding) -> Json

    Mismatch

    source
    pub(all) struct Mismatch {
      event_id : String
      expected : String
      actual : String
      expected_codes : Array[String]
      actual_codes : Array[String]
    } derive(Eq, ToJson)

    Mismatch::equal

    source
    fn Mismatch::equal(Mismatch, Mismatch) -> Bool

    Mismatch::not_equal

    source
    fn Mismatch::not_equal(x : Mismatch, y : Mismatch) -> Bool

    Mismatch::to_json

    source
    fn Mismatch::to_json(Mismatch) -> Json

    Policy

    source
    pub(all) struct Policy {
      name : String
      default_decision : String
      rules : Array[Rule]
    } derive(Eq)

    impl ToJson for Policy

    Policy::equal

    source
    fn Policy::equal(Policy, Policy) -> Bool

    Policy::not_equal

    source
    fn Policy::not_equal(x : Policy, y : Policy) -> Bool

    Policy::to_json

    source
    fn Policy::to_json(self : Policy) -> Json

    Report

    source
    pub(all) struct Report {
      schema_version : Int
      policy : String
      events : Int
      calls : Int
      allowed : Int
      denied : Int
      observed_cost : Int64
      decisions : Array[Decision]
    } derive(Eq, ToJson)

    Report::equal

    source
    fn Report::equal(Report, Report) -> Bool

    Report::not_equal

    source
    fn Report::not_equal(x : Report, y : Report) -> Bool

    Report::to_json

    source
    fn Report::to_json(Report) -> Json

    Rule

    source
    pub(all) struct Rule {
      id : String
      kind : String
      tool : String
      resource : String
      labels : Array[String]
      limit : Int?
      window_ms : Int?
      prerequisite : String?
    } derive(Eq)

    Only fields used by the selected rule kind may be supplied in JSON.
    impl ToJson for Rule

    Rule::equal

    source
    fn Rule::equal(Rule, Rule) -> Bool

    Rule::not_equal

    source
    fn Rule::not_equal(x : Rule, y : Rule) -> Bool

    Rule::to_json

    source
    fn Rule::to_json(self : Rule) -> Json

    SuiteReport

    source
    pub(all) struct SuiteReport {
      schema_version : Int
      policy : String
      cases : Int
      passed : Int
      failed : Int
      results : Array[CaseResult]
    } derive(Eq, ToJson)

    SuiteReport::equal

    source
    fn SuiteReport::equal(SuiteReport, SuiteReport) -> Bool

    SuiteReport::not_equal

    source
    fn SuiteReport::not_equal(x : SuiteReport, y : SuiteReport) -> Bool

    SuiteReport::to_json

    source
    fn SuiteReport::to_json(SuiteReport) -> Json

    Taint

    source
    pub(all) struct Taint {
      label : String
      sources : Array[String]
    } derive(Eq, ToJson)

    Source event IDs explaining one sensitivity label.

    Taint::equal

    source
    fn Taint::equal(Taint, Taint) -> Bool

    Taint::not_equal

    source
    fn Taint::not_equal(x : Taint, y : Taint) -> Bool

    Taint::to_json

    source
    fn Taint::to_json(Taint) -> Json

    compare

    source
    fn compare(baseline : Policy, candidate : Policy, events : Array[Event]) -> Comparison raise InputError

    Compare both policies against exactly the same observed history. Explanation changes include changed error codes/evidence, even with the same verdict.

    compare_json

    source
    fn compare_json(baseline : String, candidate : String, trace : String) -> String raise InputError

    evaluate

    source
    fn evaluate(policy : Policy, events : Array[Event]) -> Report raise InputError

    Replay observed calls, including calls a tested policy would deny. This preserves actual artifact lineage and costs; it is not an alternate-world simulator.

    evaluate_json

    source
    fn evaluate_json(policy : String, trace : String) -> String raise InputError

    Parse, validate and evaluate, with deterministic JSON output.

    glob_matches

    source
    fn glob_matches(pattern : String, value : String) -> Bool

    Anchored glob matching: '*' spans zero or more UTF-16 units, '?' one unit. Backslash is an ordinary character; no regex, path or URL normalization occurs. A linear-space greedy matcher avoids recursive wildcard expansion.

    parse_policy

    source
    fn parse_policy(source : String) -> Policy raise InputError

    parse_trace

    source
    fn parse_trace(source : String) -> Array[Event] raise InputError

    Accept a JSON array or JSONL. Blank JSONL lines are ignored; comments are errors.

    run_suite

    source
    fn run_suite(source : String) -> SuiteReport raise InputError

    Every observed call needs exactly one expectation. Missing assertions fail.

    run_suite_json

    source
    fn run_suite_json(source : String) -> String raise InputError

    Source Files

    Powered by MoonBit

    Site sourceReport issueModulesSkillsCLIBuild queueStatistics

    © 2026 mooncakes.io