#moonmuse

let provider = @openai.Client::new(api_key="sk-...")
let reply = provider.chat(
  @moonmuse.Request::new("gpt-4o-mini", [@moonmuse.Message::user("Why is the sky blue?")]),
)
println(reply.text())

#Install

moon add moonbitstack/moonmuse

#The two contracts

#Packages

#Chat

let request = @moonmuse.Request::new(
  "claude-sonnet-4-20250514",
  [
    @moonmuse.Message::system("Answer in one sentence."),
    @moonmuse.Message::user("Why is the sky blue?"),
  ],
  temperature=0.2,
  max_tokens=256,
  stop=["\n\n"],
)
let reply = provider.chat(request)
let (text, usage) = (reply.text(), reply.usage.total())

#Streaming

let reply = provider.stream(request, delta => {
  match delta {
    Text(chunk) => print(chunk)
    CallStart(_, name~) => println("\ncall " + name)
    CallArgs(_, chunk~) => print(chunk)
    Done(_) => println("")
  }
})

#Tools

let tools = @toolbox.Toolbox::new()
tools.add(@moonmuse.Tool::new("add", "Add two integers"), args => {
  (json_int(args, "a").unwrap_or(0) + json_int(args, "b").unwrap_or(0)).to_string()
})
let reply = @toolbox.run(provider, request, tools, limit=10)

#Layers

let gateway = @layers.cached(
  @layers.retry(@layers.fallback([primary, secondary])),
  @layers.Memory::new(limit=512),
)

#Structured output

let answer = @structured.ask(
  provider,
  request,
  name="weather",
  schema=my_schema,
  decode=json => json_int(json, "celsius"),
)

#Serve

let app = @serve.serve(
  provider,
  dialects=[@openai.dialect, @anthropic.dialect, @gemini.dialect, @ollama.dialect],
  keys=@serve.StaticKeys::new(["sk-local"]),
)

#Local models

let model = @local.Local::new(
  engine,                            // your inference engine
  template=@local.llama3(),          // or @local.chatml()
  calls=@local.Tagged,               // how it writes a tool call
  model="qwen2.5-7b",
)
let reply = model.chat(request)      // a Provider like any other

#Model catalogue

let info = @models.find("gpt-4o-mini")
let cost = @models.estimate_cost("gpt-4o-mini", reply.usage)

#Consistency

• Wire round trips. For each of the four formats, a request goes out through the dialect, is parsed back, and must equal the original — and a reply becomes a body and reads back the same. The fixtures are the shapes the vendors document.
• Recorded streams. A recorded OpenAI chat stream, Anthropic event stream and Ollama newline stream are replayed through the reader; the assembled reply must match the non-streamed one. The OpenAI stream is also cut at every byte offset, to prove the reader never depends on where the transport splits a frame.
• The stream writer against the reader. The frames a gateway writes are read back by the client reader, and must produce the same deltas.
• A real server. The Ollama client is exercised against a live server in CI: chat, a tool declaration, and embeddings. Those tests run when MOONMUSE_LIVE is set, and fail loudly if it is set and the server is missing.
• Transport. moonmuse/transport maps a Call onto moonfetch, including status and retry-after; a non-JSON error body is reported as a status, not as a decode failure.
• Official client libraries. scripts/interop runs the gateway on port 12000 and drives it with openai, anthropic and ollama — the real SDKs, not hand-written requests. CI runs chat, streaming, tool calls, embeddings, model lists and an error case.
• The official field description. scripts/openapi/gen_fields.py reads openai-openapi at a pinned commit and writes the field table the tests check bodies against (21 schemas, 182 fields). A body that carries a key the description does not have fails the build, and a stale table fails CI before it can pass a wrong body.
• LangChain standard tests. The cases a provider can carry are transcribed in src/serve/langchain_wbtest.mbt, keeping the original names: conversation and double-message histories, usage with and without streaming, stop sequences, model override, tool calling with and without arguments, tool choice, tool-result and error histories, image inputs, Unicode tool calls, and sequential calls. The cases that test LangChain's own binding, event graph or timing layer are listed there with why they do not apply.

• Streamed answers from a client. moonfetch has no streamed response yet, so Provider::stream on a client fetches the whole answer and replays it as deltas: the deltas are correct, the timing is not. The Dialect side — reading a real stream and writing one — is complete and tested.
• /v1/messages/count_tokens answers Unsupported: counting tokens needs a tokenizer, and this library does not ship one.
• The LangChain standard-tests transcription and a scripted comparison of @openai fields against openai-openapi are still owed.
• moonmuse/local ships the template, the call parser and the engine contract; it does not ship an inference engine, so a model has to be wired in.

#Targets

#License