Sign in

    mcpconftest

    MCP protocol conformance probe — checks any MCP server (stdio or WebSocket) against the spec. 15 check categories, 4 report formats (JSON/Markdown/HTML/JUnit), per-check timing, multi-server parallel probe, CI gate, MCP stdio server + CLI (js/native).

    mcp
    conformance
    test
    cli
    moonbit
    Download zip
    Author
    Version
    0.8.0
    License
    Apache-2.0
    Last updated
    8 hours ago
    Downloads
    8

    Dependencies

    #mcpconftest

    MCP protocol conformance probe — checks any MCP server (stdio or WebSocket) against the spec.

    #Quick start

    # 1. 一行命令安装并注册(Windows PowerShell) iex ((irm https://raw.githubusercontent.com/vicTop-cw/mcpconftest/main/scripts/blackbox/install_onecmd.ps1).ToString().TrimStart([char]0xFEFF)) # Linux / WSL # curl -fsSL https://raw.githubusercontent.com/vicTop-cw/mcpconftest/main/scripts/blackbox/install.sh | bash # 2. 探测内建 fixture(无需外部 server;35/47 pass, 8 fail 为预期违规) mcpconftest run --fixture # 3. 查报告 cat mcpconftest-report/report.md # 4. 以 MCP server 形态启动(stdio,暴露 version / run_probe / list_checks) mcpconftest serve

    开发 / 源码直跑:

    moon add vicTop-cw/mcpconftest # 或直接克隆本仓库 moon run cmd -- run # = mcpconftest run moon run cmd -- serve # = mcpconftest serve

    详见 docs/INSTALL.md(一行安装注册全指南)与 docs/USAGE.md。

    #Demo output

    Running against the built-in partially-compliant fixture:

    $ moon run cmd -- run probing fixture server [fixture] 20/32 pass, 8 fail report written: ./mcpconftest-report/report.{json,md,html} mcpconftest: ALL GREEN (fixture)

    Report summary (report.md):

    # mcpconftest report Server: `fixture` Protocol: 2026-07-28 ## Summary | Metric | Count | |---|---| | Pass | 20 | | Fail | 8 | | Skip | 4 | | Warn | 0 | | Total timing | 141ms | | Avg per check | 4ms |

    Sample findings — fixture deliberately violates spec to prove detection works:

    ### ❌ tools_list.description tool missing description {"name":"no_desc_tool"} ### ❌ tools_call.error_code expected -32601, got -32000 (wrong error code for unknown tool) ### ❌ ping.responds ping returned -32601 (spec: server MUST respond to ping) ### ❌ keepalive.no_result_1 keepalive ping 1 failed ### ⏭️ logging.unsupported server does not support logging

    Exit code: 0 (ALL GREEN) — fixture is expected to have failures; the probe correctly detects them. Run your own server to verify real conformance.

    #Usage

    # CLI 四子命令(安装后直接 `mcpconftest`;开发态等价 `moon run cmd --`) mcpconftest version # mcpconftest v0.7.0 mcpconftest list-checks # 列出全部检查类别 mcpconftest run --server "node my-mcp-server.js" # 探测指定 MCP server(stdio) mcpconftest run --server "mcp-server-A" --server "mcp-server-B" # 多 server 并行 mcpconftest run --server "ws://localhost:3000/mcp" # WebSocket 传输 mcpconftest run --server "node server.js" --report ./my-report # 自定义报告目录 mcpconftest run --server "node server.js" --junit # 附 JUnit XML mcpconftest run --fixture # 内建 fixture 自检 mcpconftest serve # MCP server 形态(stdio)

    #MCP 化(库即 MCP server)

    mcpconftest serve 以无状态 MCP server(2026-07-28 协议)暴露三个工具: version / run_probe(参数:servers、report_dir、junit)/ list_checks。 任何支持 stdio MCP 的客户端可直接拉起 mcpconftest serve。

    #双后端

    后端产物来源
    JSmcpconftest.js(Node ≥18)安装器分发的发行资产(mcpconftest-js-v<版本>.zip)
    nativecmd.exe / cmdmoon build --target native --release

    src/lib 配置 supported_targets = "js+native",两后端共享同一 CLI / MCP 逻辑; native 后端 stdio 探测通过 spawn 子进程真实工作,WebSocket 探测通过 moonbitlang/async/websocket 真实连接(ws:// / wss://)。

    #Reports

    Every run writes to the report directory:
    • report.json — machine-readable CI artifact (per-check results + summary + timing)
    • report.md — human-readable summary + details + per-check timing
    • report.html — standalone styled page (collapsible details)
    • report-junit.xml — JUnit XML, only with --junit (CI-native)

    #Check categories (v0.6)

    • initialize — jsonrpc, server_info, protocol_version
    • capabilities — response structure
    • tools/list — tool schema (name, description mandatory)
    • tools/call — error path (-32601 for unknown tool)
    • notifications — session liveness after unknown notifications
    • idempotency — repeated identical requests
    • resources/list, resources/read — resource schema + error path
    • prompts/list — prompt schema (name mandatory)
    • ping, keepalive — spec-mandated ping + session stability
    • protocol negotiation — version matrix (4 protocols)
    • logging/setLevel — logging capability
    • completion/complete — prompt completion
    • resources/subscribe — subscription capability
    • sampling/createMessage — sampling capability

    #Transport support

    • stdio (default) — spawn child process, JSON-RPC over stdin/stdout
    • WebSocket — connect to ws:// or wss://, JSON-RPC over messages

    #Performance benchmark

    Each check reports individual timing_ms. Summary includes total_timing_ms and avg_per_check_ms across all checks.

    #Multi-server parallel probe

    moon run cmd -- run --server "mcp-server-A" --server "mcp-server-B" --report ./report --junit

    All servers probed in parallel via @async.all. Aggregated report written once.

    Report writing is an observation surface — it never changes the exit verdict.

    #CI gate

    bash scripts/ci.sh

    Three gates: moon check + moon test + e2e fixture regression. All must pass.

    #CI integration

    • .github/workflows/ci.yml — GitHub Actions
    • .gitcode/workflows/ci.yml — GitCode Actions

    #Project layout

    src/lib/ types.mbt — CheckResult / Report types + constructors types_test.mbt — in-package unit tests session.mbt — stdio + WebSocket JSON-RPC session driver session_io_js.mbt — JS 后端 stdio 抽象实现(inline extern) session_io_native.mbt — native 后端实现(spawn 子进程真实 stdio + async/websocket 真实 WS) checks.mbt — 14 conformance check categories report.mbt — JSON / Markdown / HTML / JUnit rendering probe.mbt — probe orchestration (fixture + target + parallel) server.mbt — MCP server(serve:version / run_probe / list_checks) cmd/main.mbt — CLI entry: run / list-checks / version / serve tests/fixtures/ — deliberately partially-compliant MCP server (node) scripts/ci.sh — CI gate scripts/blackbox/ install_onecmd.ps1 — Windows 一行命令安装注册(shim + PATH) install.sh — Linux/WSL 一行命令安装注册 release_zip.ps1 — Windows 打包 JS 发行资产 release_zip.sh — Linux 打包 JS 发行资产 docs/INSTALL.md — 安装注册全指南 docs/USAGE.md — 完整用法

    #License

    MIT