🥮 mooncakes.io

    openai

    Thin OpenAI API client for MoonBit built on gaato/http and gaato/mttp: models, embeddings, responses (buffered and streaming).

    openai
    llm
    sdk
    responses
    embeddings
    Download zip
    Author
    Version
    0.7.0
    License
    Apache-2.0
    Repository
    Last updated
    9 hours ago
    Downloads
    200

    #gaato/openai

    An unofficial OpenAI API client for MoonBit: models, embeddings, Responses and Chat Completions, buffered and streaming, with tool calling. Compatible servers are selected with base_url. It uses gaato/mttp, with a caller-supplied Backend and Clock. The facade itself requires no async runtime.

    Status: unofficial and experimental. Minor releases may contain breaking changes; the changelog lists each one.

    #Install

    moon add gaato/openai
moon add gaato/http
moon add gaato/mttp
moon add gaato/http-async

    #Usage

    With gaato/http-async/backend imported as @backend and gaato/http-async/runtime as @runtime, gaato/http-async/codings as @codings and gaato/http/h1client as @h1client, create the adapters. Opt in to the exact experimental packages moonbitlang/core/async and gaato/http/h1client in the calling package's moon.pkg:

    ///|
    let backend = @codings.Decompress::new(
      @backend.AsyncBackend::new(config={
        ..@h1client.ClientConfig::default(),
        connect_timeout_ms: Some(5000),
      }),
    )

    ///|
    let clock = @runtime.AsyncClock::new()

    ///|
    let timer = @runtime.ClockTimer::new()

    Then pass them to the client and send a request:

    ///|
    /// Asks `model` one question and returns its answer.
    pub async fn ask(
      backend : &@http.Backend,
      clock : &@rt.Clock,
      timer : &@rt.Timer,
      api_key~ : String,
      model~ : String,
    ) -> Result[String, @openai.OpenAIFailure] {
      let openai = @openai.OpenAI::new(api_key~, backend, clock, timer~)
      let reply = openai.chat_completion(
        @openai.ChatRequest::new(model~, messages=[
          @openai.ChatMessage::user("Hello"),
        ]),
      )
      match reply {
        Ok(reply) => Ok(reply.text)
        Err(error) => Err(error)
      }
    }

    Buffered responses are limited to 64 MiB by default. To change the limit, construct an mttp Client with max_body_bytes and pass it to from_client. Streaming reads incrementally, with a 16 MiB per-event limit. Limit failures raise TransportError(TooLarge, ...). HTTP and decode failures return Err(Response(ResponseFailure)), preserving the received head and bounded diagnostic body. Input checks return Err(Validation(message)). api_error extracts a provider error envelope from an OpenAIFailure value. Transport failures and unknown callback exceptions remain raised errors; cancellation remains a signal.

    #Packages

    PackageWhat it is
    gaato/openaiThe OpenAI client and its request, response and event types
    gaato/openai/genTypes and operations generated from the vendored OpenAPI spec

    #Operations

    The client has list_models, retrieve_model, create_embeddings, create_response, stream_response, chat_completion and stream_chat_completion. A request's extra map adds JSON fields the typed request does not have.

    #Streaming

    stream_response and stream_chat_completion call a handler for each event as it arrives:

    ///|
    /// Streams a response, passing each piece of text to `on_text` as it arrives.
    pub async fn stream_text(
      openai : @openai.OpenAI,
      model~ : String,
      on_text : (String) -> Unit,
    ) -> Result[Unit, @openai.OpenAIFailure] {
      openai.stream_response(
        @openai.ResponseRequest::new(model~, input="Write a haiku about rivers."),
        event => if event is OutputTextDelta(text) { on_text(text) },
      )
    }

    ToolCallAccumulator reassembles streamed tool-call arguments. It grows with what you feed it, so cap a stream by its events or bytes, and set a token limit on the request.

    #OpenAI-compatible servers

    Pass base_url for a server that speaks the OpenAI API. Servers that implement only Chat Completions take ChatRequest with chat_completion or stream_chat_completion.

    #Retries

    Retries follow the official Python SDK: two retries of 408, 409, 429 and every 5xx, and of a timeout or network failure, for every method. That includes POST, so a generation can run, and be billed, twice; pass retry=@client.RetryPolicy::conservative() to retry a POST only when a failure proves nothing was sent, unless replay safety is explicit. The server's x-should-retry is obeyed, waits back off from 0.5 s to 8 s, and a Retry-After over two minutes returns the last failure instead of waiting.

    #Timeouts

    Each attempt defaults to 600 s. A configured total_timeout_ms requires a timer and covers admission, retries, waits and streaming callbacks. Expiry initiates cancellation; necessary cleanup may finish afterwards. Backend phase deadlines can finish a call earlier. The example sets a 5-second connect deadline; the SDK passes the attempt budget in SendOptions. Streaming callbacks run after the attempt has been selected and are not retried after delivery starts. Close the backend when its owner shuts down.

    #Links

    #License

    Apache-2.0

    ApiErrorBody

    source
    pub(all) struct ApiErrorBody {
      message : String
      type_ : String?
      param : String?
      code : String?
    } derive(Eq, 
    Debug    )

    The error body returned by the OpenAI API.

    test {
      let error : @openai.ApiErrorBody = {
        message: "invalid request",
        type_: Some("invalid_request_error"),
        param: None,
        code: None,
      }
      assert_eq(error.message, "invalid request")
    }

    ApiErrorBody::equal

    source

    Compares API error bodies field by field.

    ApiErrorBody::not_equal

    source
    fn ApiErrorBody::not_equal(x : ApiErrorBody, y : ApiErrorBody) -> Bool

    Compares API error bodies field by field.

    ApiErrorBody::to_repr

    source

    Debug representation of an API error body.

    ChatCompletion

    source
    pub(all) struct ChatCompletion {
      id : String
      model : String
      text : String
      finish_reason : FinishReason?
      usage : Usage?
      tool_calls : Array[ToolCall]
      raw : 
    Json
    } derive(Eq, 
    Debug    )

    A buffered chat completion together with its original JSON value.

    ChatCompletion::equal

    source

    Compares chat completions field by field.

    ChatCompletion::not_equal

    source
    fn ChatCompletion::not_equal(x : ChatCompletion, y : ChatCompletion) -> Bool

    Compares chat completions field by field.

    ChatCompletion::to_repr

    source

    Debug representation of a chat completion.

    ChatEvent

    source
    pub(all) enum ChatEvent {
      Delta(String)
      ReasoningDelta(String)
      ToolCallDelta(index~ : Int, id~ : String?, name~ : String?, arguments~ : String)
      Finished(FinishReason)
      Usage(Usage)
      Other(String, 
    Json    )
    } derive(Eq, 
    Debug    )

    One event from a streaming chat completion. Unknown chunks retain raw JSON.

    ChatEvent::equal

    source
    fn ChatEvent::equal(ChatEvent, ChatEvent) -> Bool

    Compares chat events, including retained unknown JSON.

    ChatEvent::not_equal

    source
    fn ChatEvent::not_equal(x : ChatEvent, y : ChatEvent) -> Bool

    Compares chat events, including retained unknown JSON.

    ChatEvent::to_repr

    source

    Debug representation of a chat event.

    ChatMessage

    source
    pub(all) struct ChatMessage {
      role : String
      content : String
      name : String?
      tool_calls : Array[ToolCall]?
      tool_call_id : String?
    } derive(Eq, 
    Debug    )

    One text message sent to the Chat Completions API.

    test {
      let message = @openai.ChatMessage::user("hello")
      assert_eq(message.role, "user")
      assert_eq(message.content, "hello")
    }

    ChatMessage::assistant

    source
    fn ChatMessage::assistant(content : String) -> ChatMessage

    Creates an assistant chat message.

    test {
      assert_eq(@openai.ChatMessage::assistant("hello").role, "assistant")
    }

    ChatMessage::assistant_tool_calls

    source
    fn ChatMessage::assistant_tool_calls(tool_calls : Array[ToolCall], content? : String) -> ChatMessage

    Creates an assistant message containing function tool calls. When content is absent it is encoded without a content field.

    test {
      let message = @openai.ChatMessage::assistant_tool_calls([
        { id: "call_1", name: "lookup", arguments: "{}", },
      ])
      assert_eq(message.tool_calls.unwrap().length(), 1)
    }

    ChatMessage::equal

    source
    fn ChatMessage::equal(ChatMessage, ChatMessage) -> Bool

    Compares chat messages field by field.

    ChatMessage::not_equal

    source
    fn ChatMessage::not_equal(x : ChatMessage, y : ChatMessage) -> Bool

    Compares chat messages field by field.

    ChatMessage::system

    source
    fn ChatMessage::system(content : String) -> ChatMessage

    Creates a system chat message.

    test {
      assert_eq(@openai.ChatMessage::system("Be concise").role, "system")
    }

    ChatMessage::to_repr

    source

    Debug representation of a chat message.

    ChatMessage::tool

    source
    fn ChatMessage::tool(tool_call_id~ : String, content~ : String) -> ChatMessage

    Creates a tool result message associated with one model tool call.

    test {
      let message = @openai.ChatMessage::tool(tool_call_id="call_1", content="done")
      assert_eq(message.tool_call_id, Some("call_1"))
    }

    ChatMessage::user

    source
    fn ChatMessage::user(content : String) -> ChatMessage

    Creates a user chat message.

    test {
      assert_eq(@openai.ChatMessage::user("hello").role, "user")
    }

    ChatRequest

    source
    pub(all) struct ChatRequest {
      model : String
      messages : Array[ChatMessage]
      max_tokens : Int?
      temperature : Double?
      reasoning_effort : String?
      tools : Array[ToolDef]?
      tool_choice : 
    Json    ?
      extra : Map[String, 
    Json    ]
    } derive(Eq, 
    Debug    )

    Input accepted by buffered and streaming Chat Completions operations.

    ChatRequest::equal

    source
    fn ChatRequest::equal(ChatRequest, ChatRequest) -> Bool

    Compares chat requests field by field.

    ChatRequest::new

    source
    fn ChatRequest::new(model~ : String, messages~ : Array[ChatMessage], max_tokens? : Int, temperature? : Double, reasoning_effort? : String, tools? : Array[ToolDef], tool_choice? : 
    Json    , extra? : Map[String, 
    Json    ]) -> ChatRequest

    Creates a chat request and snapshots mutable inputs.

    test {
      let request = @openai.ChatRequest::new(model="gpt-4o-mini", messages=[
        @openai.ChatMessage::user("hello"),
      ])
      assert_eq(request.messages.length(), 1)
    }

    ChatRequest::not_equal

    source
    fn ChatRequest::not_equal(x : ChatRequest, y : ChatRequest) -> Bool

    Compares chat requests field by field.

    ChatRequest::to_repr

    source

    Debug representation of a chat request.

    Embedding

    source
    pub(all) struct Embedding {
      index : Int
      embedding : Array[Double]
    } derive(Eq, 
    Debug    )

    One embedding vector and its position in the result list.

    Embedding::equal

    source
    fn Embedding::equal(Embedding, Embedding) -> Bool

    Compares embedding values field by field.

    Embedding::not_equal

    source
    fn Embedding::not_equal(x : Embedding, y : Embedding) -> Bool

    Compares embedding values field by field.

    Embedding::to_repr

    source

    Debug representation of an embedding value.

    EmbeddingRequest

    source
    pub(all) struct EmbeddingRequest {
      model : String
      input : Array[String]
      dimensions : Int?
      user : String?
    } derive(Eq, 
    Debug    )

    Input accepted by the embeddings operation.

    EmbeddingRequest::equal

    source

    Compares embedding requests field by field.

    EmbeddingRequest::new

    source
    fn EmbeddingRequest::new(model~ : String, input~ : Array[String], dimensions? : Int, user? : String) -> EmbeddingRequest

    Creates an embedding request and snapshots its input array.

    test {
      let request = @openai.EmbeddingRequest::new(model="text-embedding-3-small", input=[
        "hello",
      ])
      assert_eq(request.input, ["hello"])
    }

    EmbeddingRequest::not_equal

    source
    fn EmbeddingRequest::not_equal(x : EmbeddingRequest, y : EmbeddingRequest) -> Bool

    Compares embedding requests field by field.

    EmbeddingRequest::to_repr

    source

    Debug representation of an embedding request.

    EmbeddingResponse

    source
    pub(all) struct EmbeddingResponse {
      data : Array[Embedding]
      model : String
      prompt_tokens : Int
      total_tokens : Int
    } derive(Eq, 
    Debug    )

    The flattened result of an embeddings operation.

    EmbeddingResponse::equal

    source

    Compares embedding responses field by field.

    EmbeddingResponse::not_equal

    source
    fn EmbeddingResponse::not_equal(x : EmbeddingResponse, y : EmbeddingResponse) -> Bool

    Compares embedding responses field by field.

    EmbeddingResponse::to_repr

    source

    Debug representation of an embedding response.

    FinishReason

    source
    pub(all) enum FinishReason {
      Stop
      Length
      ToolCalls
      ContentFilter
      Unknown(String)
    } derive(Eq, 
    Debug    )

    Why a chat completion stopped. Unknown values are retained.

    FinishReason::equal

    source

    Compares finish reasons, including unknown raw values.

    FinishReason::not_equal

    source
    fn FinishReason::not_equal(x : FinishReason, y : FinishReason) -> Bool

    Compares finish reasons, including unknown raw values.

    FinishReason::to_repr

    source

    Debug representation of a finish reason.

    Model

    source
    pub(all) struct Model {
      id : String
      created : Int64?
      owned_by : String?
    } derive(Eq, 
    Debug    )

    Basic information about an OpenAI model.

    Model::equal

    source
    fn Model::equal(Model, Model) -> Bool

    Compares model information field by field.

    Model::not_equal

    source
    fn Model::not_equal(x : Model, y : Model) -> Bool

    Compares model information field by field.

    Model::to_repr

    source

    Debug representation of model information.

    OpenAI

    source
    pub struct OpenAI {
      // private fields
    }

    A thin OpenAI client backed by the runtime-free mttp Client.
    impl Debug for OpenAI

    OpenAI::chat_completion

    source
    async fn OpenAI::chat_completion(self : OpenAI, request : ChatRequest) -> Result[ChatCompletion, OpenAIFailure]

    Creates and buffers one Chat Completions response.

    OpenAI::create_embeddings

    source
    async fn OpenAI::create_embeddings(self : OpenAI, request : EmbeddingRequest) -> Result[EmbeddingResponse, OpenAIFailure]

    Creates embeddings for every input string in the request.

    OpenAI::create_response

    source
    async fn OpenAI::create_response(self : OpenAI, request : ResponseRequest) -> Result[Response, OpenAIFailure]

    Creates and buffers one model response.

    OpenAI::from_client

    source
    fn OpenAI::from_client(client : 
    Client    ) -> OpenAI

    Wraps a preconfigured runtime client for compatible APIs, Azure, and tests.

    OpenAI::list_models

    source
    async fn OpenAI::list_models(self : OpenAI) -> Result[Array[Model], OpenAIFailure]

    Lists the models visible to the configured account.

    OpenAI::new

    source
    fn OpenAI::new(api_key~ : String, backend : &
    Backend    , clock : &
    Clock    , base_url? : String, organization? : String, project? : String, retry? : 
    RetryPolicy    , timer? : &
    Timer    , timeout_ms? : Int, max_body_bytes? : Int, total_timeout_ms? : Int) -> OpenAI

    Creates an OpenAI client using bearer authentication and optional tenant headers.

    retry defaults to the official SDK's behaviour (openai-python, _base_client.py): an SDK-local policy with two retries of 408, 409, 429 and every 5xx, and of a timeout or network failure, for every method including POST, so a generation can run twice; x-should-retry is obeyed, and a Retry-After over two minutes is not waited for. Pass retry=@client.RetryPolicy::conservative() to retry a POST only on failures that prove nothing was sent, unless explicitly declared replay-safe.

    Each attempt is limited to timeout_ms (600 s, the official default) and, when total_timeout_ms is given, the whole call, retries included, to that. A total deadline requires a supplied Timer and includes admission, waits and stream callbacks. Backend phase deadlines can finish attempts earlier.

    OpenAI::retrieve_model

    source
    async fn OpenAI::retrieve_model(self : OpenAI, model : String) -> Result[Model, OpenAIFailure]

    Retrieves one model by identifier.

    OpenAI::stream_chat_completion

    source
    async fn OpenAI::stream_chat_completion(self : OpenAI, request : ChatRequest, f : async (ChatEvent) -> Unit) -> Result[Unit, OpenAIFailure]

    Dispatches Chat Completions events in wire order, including final usage. EOF, [DONE], decoding failure and callback failure close the stream.

    OpenAI::stream_response

    source
    async fn OpenAI::stream_response(self : OpenAI, request : ResponseRequest, f : async (ResponseEvent) -> Unit) -> Result[Unit, OpenAIFailure]

    Dispatches Responses events in wire order, closing on EOF, [DONE] or failure. User callback failures happen outside the retry attempt.

    OpenAI::to_repr

    source

    Redacted debug representation of an OpenAI client.

    OpenAIFailure

    source
    pub(all) enum OpenAIFailure {
      Response(
    ResponseFailure    )
      Validation(String)
    } derive(
    Debug    )

    An expected SDK failure. TransportError and unknown user errors remain raised.

    OpenAIFailure::to_repr

    source

    Formats an expected failure with its original response metadata.

    Response

    source
    pub(all) struct Response {
      id : String
      status : ResponseStatus
      model : String
      output_text : String
      usage : Usage?
      tool_calls : Array[ToolCall]
      raw : 
    Json
    } derive(Eq, 
    Debug    )

    A decoded response together with its original JSON value.

    Response::equal

    source
    fn Response::equal(Response, Response) -> Bool

    Compares decoded responses field by field.

    Response::not_equal

    source
    fn Response::not_equal(x : Response, y : Response) -> Bool

    Compares decoded responses field by field.

    Response::to_repr

    source

    Debug representation of a decoded response.

    ResponseEvent

    source
    pub(all) enum ResponseEvent {
      Created(Response)
      InProgress(Response)
      OutputTextDelta(String)
      OutputTextDone(String)
      FunctionCallArgumentsDelta(item_id~ : String, delta~ : String)
      FunctionCallArgumentsDone(item_id~ : String, arguments~ : String)
      Completed(Response)
      Failed(Response)
      Incomplete(Response)
      Error(ApiErrorBody)
      Other(String, 
    Json    )
    } derive(Eq, 
    Debug    )

    One event from a streaming response. Unknown event types retain their raw JSON.

    ResponseEvent::equal

    source

    Compares response events, including retained unknown JSON.

    ResponseEvent::not_equal

    source
    fn ResponseEvent::not_equal(x : ResponseEvent, y : ResponseEvent) -> Bool

    Compares response events, including retained unknown JSON.

    ResponseEvent::to_repr

    source

    Debug representation of a response event.

    ResponseRequest

    source
    pub(all) struct ResponseRequest {
      model : String
      input : String
      instructions : String?
      max_output_tokens : Int?
      temperature : Double?
      previous_response_id : String?
      tools : Array[ToolDef]?
      tool_choice : 
    Json    ?
      extra : Map[String, 
    Json    ]
      // private fields
    } derive(Eq, 
    Debug    )

    Input accepted by buffered and streaming response creation.

    ResponseRequest::equal

    source

    Compares response requests field by field.

    ResponseRequest::new

    source
    fn ResponseRequest::new(model~ : String, input~ : String, instructions? : String, max_output_tokens? : Int, temperature? : Double, previous_response_id? : String, tools? : Array[ToolDef], tool_choice? : 
    Json    , extra? : Map[String, 
    Json    ]) -> ResponseRequest

    Creates a response request and snapshots the extra parameter map.

    test {
      let request = @openai.ResponseRequest::new(model="gpt-4.1", input="hello")
      assert_eq(request.extra, {})
    }

    ResponseRequest::not_equal

    source
    fn ResponseRequest::not_equal(x : ResponseRequest, y : ResponseRequest) -> Bool

    Compares response requests field by field.

    ResponseRequest::to_repr

    source

    Debug representation of a response request.

    ResponseRequest::with_tool_outputs

    source
    fn ResponseRequest::with_tool_outputs(self : ResponseRequest, outputs : Array[(String, String)]) -> ResponseRequest

    Returns a request with function outputs appended to its input items.

    test {
      let request = @openai.ResponseRequest::new(model="gpt-4.1", input="continue").with_tool_outputs([
          ("call_1", "done"),
        ],
      )
      assert_eq(request.input, "continue")
    }

    ResponseStatus

    source
    pub(all) enum ResponseStatus {
      Completed
      Failed
      InProgress
      Incomplete
      Cancelled
      Queued
      Unknown(String)
    } derive(Eq, 
    Debug    )

    A response lifecycle status. Unknown values are retained for forward compatibility.

    ResponseStatus::equal

    source

    Compares response statuses, including unknown raw values.

    ResponseStatus::not_equal

    source
    fn ResponseStatus::not_equal(x : ResponseStatus, y : ResponseStatus) -> Bool

    Compares response statuses, including unknown raw values.

    ResponseStatus::to_repr

    source

    Debug representation of a response status.

    ToolCall

    source
    pub(all) struct ToolCall {
      id : String
      name : String
      arguments : String
    } derive(Eq, 
    Debug    )

    A function call produced by a model. Arguments remain raw text until parsed.

    ToolCall::arguments_json

    source

    Parses the model-produced argument string as JSON on demand.

    test {
      let call : @openai.ToolCall = {
        id: "call_1",
        name: "lookup",
        arguments: "{\"id\":1}",
      }
      assert_eq(call.arguments_json(), { "id": 1 })
    }

    ToolCall::equal

    source
    fn ToolCall::equal(ToolCall, ToolCall) -> Bool

    Compares tool calls field by field.

    ToolCall::not_equal

    source
    fn ToolCall::not_equal(x : ToolCall, y : ToolCall) -> Bool

    Compares tool calls field by field.

    ToolCall::to_repr

    source

    Debug representation of a tool call.

    ToolCallAccumulator

    source
    pub struct ToolCallAccumulator {
      // private fields
    }

    Incrementally reconstructs tool calls from streamed chat deltas.

    test {
      let calls = @openai.ToolCallAccumulator::new()
      calls.feed(index=0, id="call_1", name="lookup", arguments="{}")
      assert_eq(calls.finish()[0].arguments, "{}")
    }

    ToolCallAccumulator::feed

    source
    fn ToolCallAccumulator::feed(self : ToolCallAccumulator, index~ : Int, id? : String, name? : String, arguments~ : String) -> Unit

    Adds one tool-call delta. The first supplied id and name for an index win.

    ToolCallAccumulator::finish

    source

    Returns reconstructed calls ordered by their stream index. Missing ids or names are returned as empty strings.

    ToolCallAccumulator::new

    source

    Creates an empty streamed tool-call accumulator.

    ToolDef

    source
    pub(all) struct ToolDef {
      name : String
      description : String?
      parameters : 
    Json
      strict : Bool?
    } derive(Eq, 
    Debug    )

    A function tool definition shared by Chat Completions and Responses.

    test {
      let tool = @openai.ToolDef::new(name="lookup", parameters={ "type": "object" })
      assert_eq(tool.name, "lookup")
    }

    ToolDef::equal

    source
    fn ToolDef::equal(ToolDef, ToolDef) -> Bool

    Compares tool definitions field by field.

    ToolDef::new

    source
    fn ToolDef::new(name~ : String, parameters~ : 
    Json    , description? : String, strict? : Bool) -> ToolDef

    Creates a function tool definition with a JSON Schema parameter object.

    ToolDef::not_equal

    source
    fn ToolDef::not_equal(x : ToolDef, y : ToolDef) -> Bool

    Compares tool definitions field by field.

    ToolDef::to_repr

    source

    Debug representation of a tool definition.

    Usage

    source
    pub(all) struct Usage {
      input_tokens : Int
      output_tokens : Int
      total_tokens : Int
    } derive(Eq, 
    Debug    )

    Token usage reported by a response.

    Usage::equal

    source
    fn Usage::equal(Usage, Usage) -> Bool

    Compares response usage field by field.

    Usage::not_equal

    source
    fn Usage::not_equal(x : Usage, y : Usage) -> Bool

    Compares response usage field by field.

    Usage::to_repr

    source

    Debug representation of response usage.

    api_error

    source
    fn api_error(error : OpenAIFailure) -> ApiErrorBody?

    Extracts a standard OpenAI error body from an HTTP status failure. Returns None for transport, decode, configuration, and malformed error bodies.

    Source Files