A dialect-agnostic LLM IR, one wire mapping per provider protocol, a data-driven provider registry, and a transport a browser can implement
Dependencies
moon add marianoguerra/llmlet registry = @catalog.default_registry()
guard registry.dialect_of("anthropic:claude-sonnet-5") is Some(dialect)
let body = dialect.lower_context(context) // -> provider request JSON
let reply = dialect.lift_message(json) // -> ContextMessage| Package | What it is |
|---|---|
| llm | LlmContext and the messages, blocks, usage and per-turn params it is made of; the Dialect trait; StreamEvent; ModelInfo; the Registry; LlmTransport and RelayTransport |
| llm/anthropic | anthropic-messages |
| llm/openai | openai-responses and openai-completions, plus a probed capability table |
| llm/gemini | google-generative-ai — the one whose endpoint carries the model |
| llm/openrouter | openrouter, which borrows the completions mapping |
| llm/catalog | The five providers as data — Cerebras among them, riding openai-completions with no dialect of its own. Credential-free |
| llm/wire | Reading a key out of an environment, resolving an endpoint with it, and an HTTP transport. The only native library here |
| cmd/smoke | A live check against every provider whose key is set. Native, and never run by just ci |
| Provider | xl | lg | md | sm |
|---|---|---|---|---|
| openai | gpt-6-astra gpt-5.5-pro | gpt-5.6-sol gpt-5.5 | gpt-5.6-terra gpt-5.3-codex | gpt-5.6-luna gpt-5.4-mini |
| anthropic | claude-fable-5-1 claude-fable-5 | claude-opus-5 claude-opus-4-8 | claude-sonnet-5 claude-sonnet-4-6 | claude-haiku-4-5 |
| gemini | — | gemini-3.1-pro-preview gemini-2.5-pro | gemini-3.8-flash gemini-3.7-flash | gemini-3.5-flash-lite gemini-3.1-flash-lite |
| cerebras | — | — | gpt-oss-120b | qwen-3.8-27b gemma-4-31b |
| Family | Models |
|---|---|
| qwen/ | qwen3.8-max-0902 qwen3.8-2.4t-a95b qwen3.7-plus qwen3-coder-next qwen3.8-flash qwen3.7-flash |
| deepseek/ | deepseek-v4-pro-0813 deepseek-v4-pro deepseek-v4.1-flash deepseek-v4-flash-vision-exp |
| moonshotai/ | kimi-k3 kimi-k2.7-code kimi-k2.6 |
| z-ai/ | glm-5.3 glm-5.2 glm-5.3-flash |
| x-ai/ | grok-4.6 grok-4.5 |
| meta/ | muse-spark-1.3 muse-spark-1.2 |
| tencent/ | hy4-preview hy3 |
| thinkingmachines/ | inkling inkling-small |
| minimax/ | minimax-m3 |
| mistralai/ | mistral-medium-3-5 |
| openai/ | gpt-5.4-mini |
let registry = @catalog.default_registry().with_provider({
id: "lmstudio",
name: "LM Studio",
api: "openai-completions",
endpoint: "http://localhost:1234/v1/chat/completions",
auth: NoAuth,
key_env: "",
model_env: "",
default_model: "qwen3-coder",
models: [@llm.ModelInfo::permissive("qwen3-coder")],
headers: { "content-type": "application/json" },
telemetry_name: "lmstudio",
server_address: "localhost",
})let registry = @catalog.default_registry()
.with_model(provider="openrouter", @llm.ModelInfo::permissive("deepseek/deepseek-v4.1-flash"))
.unwrap()pub fn api_spec() -> @llm.ApiSpec {
{ id: "my-api", make: info => MyDialect::make(model=info.id), caps: id => @llm.ModelInfo::permissive(id) }
}just smokecp .env.example .env # then fill in the keys you have
set -a; . ./.env; set +a
``` Each provider gets
its cheapest model, `effort: Off` clamped onto whatever the model's row says is
least, and a reply capped at a few dozen tokens — a full five-provider run is
fractions of a cent.
The two cases are the two halves of a mapping: one plain turn (system prompt,
user turn, text and usage back) and one with a tool declared (tool lowering,
and a `ToolCall` lifted with its arguments parsed back out of the JSON string
they travel as). Both go through `stream_init`/`stream_feed`/`stream_finish`,
which is the path a real caller drives.
It lives in `cmd/smoke` as an executable rather than a `_test.mbt`, because
`moon test` runs everything it finds and this costs money and needs
credentials. `just ci` does not run it.
## Two things that look odd and are not
**`stream_feed` returns events, not a message.** Providers differ in how a
response is *framed* — event names, delta shapes, where usage and the stop
reason arrive — and decoding that anywhere but inside the dialect would mean
adding a provider in two places. `lift_message` is the non-streaming special
case of `stream_init ▸ stream_feed ▸ stream_finish`.
**`LlmTransport` is in continuations, not `async`.** A browser cannot have
async: `moonbitlang/async`'s event loop is unimplemented for wasm-gc, and an
async function cannot be called from the plain exported functions a page starts
from. A trait only a server could implement would not be a seam, so the native
side parks on a semaphore instead.pub(open) trait Dialect {
fn id(Self) -> String
fn model(Self) -> String = _
fn lower_context(Self, LlmContext) -> Json
fn lift_message(Self, Json) -> ContextMessage raise DialectError
fn stream_init(Self) -> StreamState = _
fn stream_feed(Self, StreamState, chunk~ : String) -> (StreamState, Array[StreamEvent]) raise DialectError = _
fn stream_finish(Self, StreamState) -> ContextMessage raise DialectError = _
}impl Show for DialectErrorpub(all) enum ContextMessage {
UserMsg(content~ : Array[UserBlock])
AssistantMsg(content~ : Array[AssistantBlock], stop~ : StopReason, model~ : String, response_id~ : String?, usage~ : Usage?, error~ : String?)
ToolResultMsg(tool_call_id~ : String, tool_name~ : String, content~ : Array[UserBlock], is_error~ : Bool)
} derive(ToJson, Debug)pub(all) struct LlmParams {
effort : ThinkingLevel?
pro_mode : Bool
reasoning_context : ReasoningContext?
reasoning_summary : Bool
max_output_tokens : Int?
cache_key : String?
cache_retention : CacheRetention
} derive(ToJson, Debug)pub(all) struct ModelInfo {
id : String
api : String?
reasoning : Bool
thinking_levels : Array[(ThinkingLevel, String)]
supports_pro_mode : Bool
reasoning_contexts : Array[ReasoningContext]
input : Array[Modality]
context_window : Int
max_output_tokens : Int?
cost : Cost?
} derive(ToJson, Debug)pub(all) struct RelayTransport {
post : (String, String, String, (String) -> Unit, (String) -> Unit) -> Unit
stop : (String) -> Unit
}impl LlmTransport for RelayTransportfn send(self : RelayTransport, handle~ : String, target~ : String, body~ : Json, headers~ : Map[String, String], on_chunk~ : (String) -> Unit, on_done~ : () -> Unit, on_error~ : (String) -> Unit) -> Unitfn RelayTransport::make(post~ : (String, String, String, (String) -> Unit, (String) -> Unit) -> Unit, stop? : (String) -> Unit) -> RelayTransportfn RelayTransport::send(self : RelayTransport, handle~ : String, target~ : String, body~ : Json, headers~ : Map[String, String], on_chunk~ : (String) -> Unit, on_done~ : () -> Unit, on_error~ : (String) -> Unit) -> Unitpub(all) enum StreamEvent {
Start(model~ : String)
TextStart(content_index~ : Int)
TextDelta(content_index~ : Int, delta~ : String)
TextEnd(content_index~ : Int)
ThinkingStart(content_index~ : Int)
ThinkingDelta(content_index~ : Int, delta~ : String)
ThinkingEnd(content_index~ : Int)
ToolCallStart(content_index~ : Int, id~ : String, name~ : String)
ToolCallDelta(content_index~ : Int, delta~ : String)
ToolCallEnd(content_index~ : Int)
Done(stop~ : StopReason, usage~ : Usage?)
Error(message~ : String)
} derive(Eq, ToJson, Debug)Install
Download zipA dialect-agnostic LLM IR, one wire mapping per provider protocol, a data-driven provider registry, and a transport a browser can implement
Dependencies