#moonmuse

let provider : &@moonmuse.Provider = @openai.Client::new(api_key="sk-test")
let request = @moonmuse.Request::new(
  "gpt-4o-mini",
  [@moonmuse.Message::user("Why is the sky blue?")],
  temperature=0.2,
  max_tokens=256,
)
let reply = provider.chat(request)
let answer = reply.text()

#Install

moon add moonbitstack/moonmuse

#The two contracts

#Packages

#Chat

let with_system = @moonmuse.Request::new(
  "claude-sonnet-4-20250514",
  [
    @moonmuse.Message::system("Answer in one sentence."),
    @moonmuse.Message::user("Why is the sky blue?"),
  ],
  stop=["\n\n"],
)
let priced = @models.estimate_cost("claude-sonnet-4-20250514", reply.usage)

#Streaming

let streamed = provider.stream(request, delta => {
  match delta {
    @moonmuse.Text(chunk) => println(chunk)
    @moonmuse.CallStart(name~, ..) => println("call " + name)
    @moonmuse.CallArgs(chunk~, ..) => println(chunk)
    @moonmuse.Done(_) => println("")
  }
})

#Tools

let tools = @toolbox.Toolbox::new()
tools.add(@moonmuse.Tool::new("add", "Add two integers"), args => {
  let a = @moonmuse.json_int(args, "a").unwrap_or(0)
  let b = @moonmuse.json_int(args, "b").unwrap_or(0)
  (a + b).to_string()
})
let looped = @toolbox.run(provider, request, tools, limit=10)

#Layers

let secondary : &@moonmuse.Provider = @ollama.Client::new()
let gateway = @layers.cached(
  @layers.retry(@layers.fallback([provider, secondary])),
  @layers.Memory::new(limit=512),
)
let served = gateway.chat(request)

#Structured output

let weather = @moonmuse.parse_json_lossy(
  "{\"type\":\"object\",\"properties\":{\"celsius\":{\"type\":\"number\"}}}",
)
let celsius = @structured.ask(
  provider,
  request,
  name="weather",
  schema=weather,
  decode=json => @moonmuse.json_int(json, "celsius"),
)

#Serve

let app = @serve.serve(
  provider,
  dialects=[@openai.dialect, @anthropic.dialect, @gemini.dialect, @ollama.dialect],
  keys=@serve.StaticKeys::new(["sk-local"]),
)

let ranks = @vocab.Ranks::from_tiktoken(vocabulary_text)   // a tiktoken table, read once
let tokenizer = @moontoken.Tokenizer::cl100k(ranks~)
let counted = @serve.serve(provider, dialects=[@anthropic.dialect], counter=@count.Tokens::new(tokenizer~))

#Local models

struct Echo {
}

impl @local.Engine for Echo with fn generate(_self, prompt, _limit) {
  prompt
}

pub extend Echo with @local.Engine::{generate}

let local : &@moonmuse.Provider = @local.Local::new(
  Echo::{},
  template=@local.llama3(),
  calls=@local.Tagged,
  model="qwen2.5-7b",
)
let answered = local.chat(request)

#Observation

let recorder = @trace.Recorder::new()
let rand = width => Bytes::make(width, b'\x01')
let tracer = @trace.Tracer::new(
  rand~,
  sink=recorder,
  sampler=@trace.parent_based(root=@trace.always),
)
let observed = @otel.traced(provider, tracer)
let measured = @otel.metered(provider, @moonp8s.Registry::new())

#Binary endpoints

let client = @openai.Client::new(api_key="sk-test")
let audio = client.speech("Read this aloud.")
let said = client.transcribe(audio, filename="out.mp3")

#Consistency

• Wire round trips. For each of the four formats, a request goes out through the dialect, is parsed back, and must equal the original — and a reply becomes a body and reads back the same. The fixtures are the shapes the vendors document.
• Recorded streams. A recorded OpenAI chat stream, Anthropic event stream and Ollama newline stream are replayed through the reader; the assembled reply must match the non-streamed one. The OpenAI stream is also cut at every byte offset, to prove the reader never depends on where the transport splits a frame.
• A stream off a socket. A server writes one frame and withholds the next until the caller has it, which is the only way to show that a frame arrives before the body ends; another cuts a character across two writes, so a decoder working per read would show the break.
• One answer, four formats. An Ollama-format streaming request is served in all four formats and each answer is read back with its own reader: the pieces must be the same pieces, in the same order, and a tool call split across frames must arrive as one call.
• The stream writer against the reader. The frames a gateway writes are read back by the client reader, and must produce the same deltas.
• A real server. The Ollama client is exercised against a live server in CI: chat, a tool declaration, and embeddings. Those tests run when MOONMUSE_LIVE is set, and fail loudly if it is set and the server is missing.
• Official client libraries. scripts/interop runs the gateway on port 12000 and drives it with openai, anthropic, ollama and google-genai — the real SDKs, not hand-written requests. CI runs chat, streaming, tool calls, embeddings, model lists and an error case.
• The official field description. scripts/openapi/gen_fields.py reads openai-openapi at a pinned commit and writes the field table the tests check bodies against (21 schemas, 182 fields). A body that carries a key the description does not have fails the build, and a stale table fails CI before it can pass a wrong body.
• The semantic conventions. A span the library emits is sent over OTLP to the conventions project's own checker, which matches every attribute against the GenAI registry. An attribute that is unknown, misspelled or retired fails the build; the run is scripts/otel-live-check.sh.
• LangChain standard tests. The cases a provider can carry are transcribed in src/serve/langchain_wbtest.mbt, keeping the original names: conversation and double-message histories, usage with and without streaming, stop sequences, model override, tool calling with and without arguments, tool choice, tool-result and error histories, image inputs, Unicode tool calls, and sequential calls. The cases that test LangChain's own binding, event graph or timing layer are listed there with why they do not apply.

• moonmuse/local ships the template, the call parser and the engine contract; it does not ship an inference engine, so a model has to be wired in.
• Token counts are the vocabulary's count plus a framing amount the caller sets. The endpoint's own number is the authority; this one is for budgeting before the call.

#Targets

#License