moonllm
Type-safe, OpenAI-compatible LLM client for MoonBit: chat completions, streaming (SSE), tool calling, and multimodal input.
llm
openai
ai
sdk
http-client
moon add DC-Z-lab/moonllm@0.1.0
Download zipAuthor
Version
0.1.0
License
Apache-2.0
Repository
Last updated
last month
Downloads
378
Dependencies
- oboard/mio0.5.2
- moonbitlang/async0.20.1
README#DC-Z-lab/moonllm
#LLMError
pub(all) suberror LLMError {
Transport(String)
ApiError(code~ : Int, message~ : String)
Decode(String)
Stream(String)
} derive(Debug)#AuthScheme
How the API key is presented to the server.
#Backoff
The backoff strategy used between retries.
#Batch
pub(all) struct Batch {
id : String
status : String
input_file_id : String
output_file_id : String?
error_file_id : String?
counts : BatchCounts
} derive(Debug)#BatchCounts
Per-request counts for a batch job.
#BatchRequest
pub(all) struct BatchRequest {
input_file_id : String
endpoint : String
completion_window : String
}A request to create a batch job (POST /batches).
impl ToJson for BatchRequest
#BatchRequest::new
fn BatchRequest::new(input_file_id : String, endpoint? : String, completion_window? : String) -> BatchRequestCreate a batch request. endpoint is the API path the batch runs against
(e.g. /v1/chat/completions); completion_window is typically "24h".
#ChatRequest
pub(all) struct ChatRequest {
model : String
messages : Array[Message]
temperature : Double?
max_tokens : Int?
top_p : Double?
stop : Array[String]?
tools : Array[Tool]?
tool_choice : ToolChoice?
frequency_penalty : Double?
presence_penalty : Double?
seed : Int?
n : Int?
response_format : ResponseFormat?
logit_bias : Map[String, Double]?
user : String?
stream : Bool
}A chat completion request. Build with ChatRequest::new and the
chainable setters, or construct the struct directly.
impl ToJson for ChatRequest#ChatRequest::frequency_penalty
#ChatRequest::json_mode
Shortcut: request a JSON-object response ("JSON mode").
#ChatRequest::logit_bias
#ChatRequest::max_tokens
#ChatRequest::new
Create a request for model with the given messages.
#ChatRequest::presence_penalty
#ChatRequest::response_format
#ChatRequest::temperature
#ChatRequest::tool_choice
#ChatRequest::validate
Validate the request parameters, returning an error message if any value
is out of its documented range. Returns None when the request is valid.
This is a best-effort local check against the OpenAI-documented ranges;
it does not guarantee the server will accept the request.
#ChatResponse
A non-streaming chat completion response.
impl FromJson for ChatResponse#ChatResponse::finish_reason
The finish reason of the first choice, if any.
#ChatResponse::first
The first choice, if the response contains any.
#ChatResponse::has_tool_calls
Whether the first choice requested one or more tool calls.
#ChatResponse::text
Convenience: the text content of the first choice, if any.
#ChatResponse::tool_calls
The tool calls requested by the first choice, if any.
#ChatResponse::total_tokens
The total tokens used, or 0 if usage was not reported.
#Choice
A single choice in a chat completion response.
#Choice::truncated
Whether this choice was cut off because it hit the token limit.
#Client
pub struct Client {
api_key : String
base_url : String
timeout_ms : Int
extra_headers : Map[String, String]
auth : AuthScheme
}An LLM client bound to an OpenAI-compatible endpoint.
Construct with Client::new, optionally overriding base_url.
The default base_url targets the OpenAI public API; point it at any
OpenAI-compatible gateway (Azure, local vLLM, router services, ...).
#Client::batch
Retrieve a batch job's status (GET /batches/{id}).
#Client::cancel_batch
Cancel a batch job (POST /batches/{id}/cancel).
#Client::cancel_fine_tune
Cancel a fine-tuning job (POST /fine_tuning/jobs/{id}/cancel).
#Client::chat
Perform a non-streaming chat completion.
Raises LLMError on transport failure, non-2xx status, or decode failure.
#Client::chat_anthropic
async fn Client::chat_anthropic(self : Client, request : ChatRequest) -> ChatResponse raise LLMErrorPerform a chat completion against an Anthropic Messages endpoint.
This assumes the client's base_url targets an Anthropic-compatible host
(default https://api.anthropic.com/v1). Note that Anthropic requires the
x-api-key and anthropic-version headers; configure these via a client
whose auth is set accordingly, or use this against a proxy that injects
them. The request/response translation is handled here.
#Client::chat_stream
async fn Client::chat_stream(self : Client, request : ChatRequest, on_delta : (String) -> Unit) -> String raise LLMErrorPerform a streaming chat completion (text only).
on_delta is invoked for each incremental text fragment as it arrives.
Returns the fully accumulated assistant text once the stream completes.
Raises LLMError on transport failure, non-2xx status, or decode failure.
#Client::chat_stream_full
async fn Client::chat_stream_full(self : Client, request : ChatRequest, on_chunk : (StreamChunk) -> Unit) -> StreamResult raise LLMErrorPerform a streaming chat completion, invoking on_chunk for every parsed
StreamChunk as it arrives, and returning the fully assembled result
(text + tool calls) once the stream completes.
This is the low-level streaming entry point. For the common
text-only case, prefer chat_stream.
Raises LLMError on transport failure, non-2xx status, or decode failure.
#Client::chat_with_policy
async fn Client::chat_with_policy(self : Client, request : ChatRequest, policy : RetryPolicy) -> ChatResponse raise LLMErrorPerform a chat completion with retries governed by an explicit policy.
On a retryable ApiError, if the policy honors Retry-After and the
error message contains a parseable hint, that delay is preferred over the
computed backoff.
#Client::chat_with_retry
async fn Client::chat_with_retry(self : Client, request : ChatRequest, max_retries? : Int, base_delay_ms? : Int) -> ChatResponse raise LLMErrorPerform a chat completion with automatic retries on transient failures.
Retries up to max_retries times with exponential backoff starting at
base_delay_ms. Non-retryable errors (4xx other than 429, decode errors)
are raised immediately.
#Client::completion
async fn Client::completion(self : Client, request : CompletionRequest) -> CompletionResponse raise LLMErrorPerform a legacy text completion.
#Client::create_batch
Create a batch job (POST /batches).
#Client::create_fine_tune
async fn Client::create_fine_tune(self : Client, request : FineTuneRequest) -> FineTuneJob raise LLMErrorCreate a fine-tuning job.
#Client::delete_file
Delete a file (DELETE /files/{id}).
#Client::delete_json
Perform a DELETE request to path and return the parsed response JSON.
#Client::embeddings
async fn Client::embeddings(self : Client, request : EmbeddingRequest) -> EmbeddingResponse raise LLMErrorPerform an embeddings request.
#Client::file
Retrieve metadata for a single file (GET /files/{id}).
#Client::files
List the files stored at the endpoint (GET /files).
#Client::fine_tune
Retrieve a fine-tuning job (GET /fine_tuning/jobs/{id}).
#Client::fine_tunes
List fine-tuning jobs (GET /fine_tuning/jobs).
#Client::generate_image
async fn Client::generate_image(self : Client, request : ImageRequest) -> ImageResponse raise LLMErrorGenerate one or more images from a text prompt.
#Client::get_json
Perform a GET request to path and return the parsed response JSON.
#Client::model
Retrieve metadata for a single model (GET /models/{id}).
#Client::models
List the models available at the endpoint (GET /models).
#Client::moderations
async fn Client::moderations(self : Client, request : ModerationRequest) -> ModerationResponse raise LLMErrorPerform a moderation request.
#Client::new
Create a client with the given API key.
- base_url defaults to https://api.openai.com/v1.
- timeout_ms defaults to 60_000.
#Client::post_json
Perform a POST with a JSON body to path and return the parsed response
JSON. Shared by all non-streaming endpoints. Handles transport failures,
non-2xx status, and JSON parse errors uniformly.
#Client::respond
Perform a request against the Responses API (POST /responses).
#Client::speech
Synthesize speech from text. Returns the raw audio bytes.
#ClientBuilder
pub struct ClientBuilder {
api_key : String
base_url : String
timeout_ms : Int
headers : Map[String, String]
auth : AuthScheme
}A builder for Client, for when you need more than the key and base URL:
custom headers, an organization id, a non-bearer auth scheme, or a
specific timeout.
#ClientBuilder::anthropic
Configure the builder for the Anthropic Messages API: x-api-key auth and
the required anthropic-version header.
#ClientBuilder::api_key
#ClientBuilder::auth
Set the authentication scheme.
#ClientBuilder::base_url
#ClientBuilder::build
Finalize the builder into a Client.
#ClientBuilder::header
Add a header sent with every request.
#ClientBuilder::organization
Set the OpenAI-Organization header.
#ClientBuilder::timeout_ms
#CompletionRequest
pub(all) struct CompletionRequest {
model : String
prompt : String
max_tokens : Int?
temperature : Double?
top_p : Double?
stop : Array[String]?
suffix : String?
}A request to the legacy /completions (text completion) endpoint.
This is the pre-chat completion API, still supported by some models and
many OpenAI-compatible gateways for raw prompt completion.
impl ToJson for CompletionRequest#CompletionRequest::max_tokens
#CompletionRequest::new
Create a completion request.
#CompletionRequest::stop
#CompletionRequest::temperature
#CompletionResponse
pub(all) struct CompletionResponse {
id : String
model : String
choices : Array[CompletionChoice]
usage : Usage?
} derive(Debug)A response from the legacy /completions endpoint.
impl FromJson for CompletionResponse#CompletionResponse::text
The text of the first completion choice, if any.
#Content
The content of a message: either a single text string or a list of parts.
#ContentPart
A single part of a multimodal message content.
OpenAI-compatible: a message's content may be a plain string or an array
of content parts. Text and ImageUrl cover the common cases.
#ContentPart::image_base64
An image content part from raw base64 data, wrapped as a data: URI.
mime is e.g. "image/png" or "image/jpeg".
#ContentPart::image_url
An image content part from a URL (https://... or a data: URI).
#Conversation
A mutable conversation: an ordered list of messages plus an optional
system prompt, with helpers for appending turns, estimating token usage,
and trimming history to fit a budget.
This is a convenience layer over the raw Array[Message]; it does not
itself call the API — turn it into a request with to_request.
#Conversation::add_response
Append the assistant message from a ChatResponse (the first choice), so
the reply becomes part of the ongoing history. Returns the appended text.
#Conversation::assistant
Append an assistant message.
#Conversation::clear
Remove all messages (keeping the system prompt).
#Conversation::estimate_tokens
Estimate the total token count of the conversation (system + all messages).
#Conversation::length
The number of messages currently in the history (excluding the system
prompt).
#Conversation::new
Create an empty conversation, optionally with a system prompt.
#Conversation::push
Append an arbitrary message.
#Conversation::set_system
Set (or replace) the system prompt.
#Conversation::to_messages
Snapshot the full message list including the system prompt (if any) as the
leading system message.
#Conversation::to_request
Build a ChatRequest for model from the conversation's current state.
#Conversation::tool_result
Append a tool-result message.
#Conversation::trim_to_budget
Trim the oldest messages until the estimated token count fits within
max_tokens, preserving the system prompt and the most recent messages.
Returns the number of messages dropped.
#Conversation::user
Append a user message.
#Embedding
#EmbeddingInput
The input to an embeddings request: one or more strings to embed.
impl ToJson for EmbeddingInput#EmbeddingRequest
pub(all) struct EmbeddingRequest {
model : String
input : EmbeddingInput
dimensions : Int?
user : String?
encoding_format : String?
}A request to the /embeddings endpoint.
impl ToJson for EmbeddingRequest#EmbeddingRequest::dimensions
Set the target dimensionality of the returned embeddings.
#EmbeddingRequest::new
Create an embeddings request for a single text.
#EmbeddingRequest::of_batch
Convenience: build a request embedding a batch of strings.
#EmbeddingRequest::of_text
Convenience: build a request embedding a single string.
#EmbeddingRequest::user
Set the end-user identifier.
#EmbeddingResponse
The response from the /embeddings endpoint.
impl FromJson for EmbeddingResponse#EmbeddingResponse::vector
The first embedding vector, if any.
#FileList
#FileObject
pub(all) struct FileObject {
id : String
bytes : Int64
created_at : Int64
filename : String
purpose : String
} derive(Debug)Metadata for a file stored at the endpoint (/files).
impl FromJson for FileObject#FineTuneJob
pub(all) struct FineTuneJob {
id : String
model : String
status : String
training_file : String
fine_tuned_model : String?
} derive(Debug)A fine-tuning job object.
impl FromJson for FineTuneJob#FineTuneJob::is_done
Whether the job has finished (successfully or not).
#FineTuneRequest
pub(all) struct FineTuneRequest {
model : String
training_file : String
validation_file : String?
suffix : String?
n_epochs : Int?
}A request to create a fine-tuning job (POST /fine_tuning/jobs).
impl ToJson for FineTuneRequest#FineTuneRequest::n_epochs
Set the number of training epochs.
#FineTuneRequest::new
Create a fine-tuning request from a base model and a training file id.
#FineTuneRequest::suffix
Set a suffix for the resulting model name.
#FineTuneRequest::validation_file
Set the validation file id.
#FunctionCall
A function invocation requested by the model.
#GeneratedImage
pub(all) struct GeneratedImage {
url : String?
b64_json : String?
revised_prompt : String?
} derive(Debug)A single generated image: either a URL or base64-encoded data.
impl FromJson for GeneratedImage#ImageRequest
pub(all) struct ImageRequest {
prompt : String
model : String?
n : Int?
size : String?
quality : String?
response_format : String?
}A request to the /images/generations endpoint.
impl ToJson for ImageRequest#ImageRequest::new
Create an image generation request from a prompt.
#ImageRequest::quality
#ImageRequest::size
Set the output size, e.g. "1024x1024".
#ImageResponse
A response from the /images/generations endpoint.
impl FromJson for ImageResponse#ImageResponse::urls
All non-empty image URLs in the response.
#LogLevel
#MemoryLogger
A simple collecting logger that buffers formatted lines in memory — useful
for tests and for surfacing a request trace in a UI. A production app would
swap in its own observer that writes to a real logging backend.
#MemoryLogger::clear
Clear all buffered lines.
#MemoryLogger::count
The number of buffered log lines.
#MemoryLogger::dump
All buffered lines joined by newlines.
#MemoryLogger::has_at_least
Whether any line at level or above was recorded.
#MemoryLogger::log
Log a message at a level (dropped if below the threshold).
#MemoryLogger::log_request
Log a completed request round-trip at an appropriate level.
#MemoryLogger::new
Create a memory logger that keeps entries at or above min_level.
#Message
A chat message.
#Message::has_image
Whether the message contains any image part.
#Message::image_count
Count the image parts in a message.
#Message::system
Convenience constructors for messages.
#Message::tool_result
Build a tool role message carrying the result of a tool call.
#Message::user_parts
Build a multimodal user message from content parts (text and/or images).
#MessageBuilder
Fluent builder for a multimodal message, so callers can assemble mixed
text/image content without manually constructing the Content enum.
Example:
let msg = MessageBuilder::user()
.text("What is in these images?")
.image_url("https://a/1.png")
.image_url("https://a/2.png")
.build()#MessageBuilder::assistant
Start building an assistant message.
#MessageBuilder::build
Finalize into a Message.
If the builder holds exactly one text part and no name, it collapses to a
plain string content (the common case); otherwise it emits a parts array.
#MessageBuilder::image_base64
fn MessageBuilder::image_base64(self : MessageBuilder, mime : String, data : String) -> MessageBuilderAppend a base64 image part.
#MessageBuilder::image_url
Append an image-URL part.
#MessageBuilder::name
Set the optional author name.
#MessageBuilder::new
Start building a message with the given role.
#MessageBuilder::part_count
The number of content parts accumulated so far.
#MessageBuilder::system
Start building a system message.
#MessageBuilder::text
Append a text part.
#MessageBuilder::user
Start building a user message.
#Model
pub(all) struct Model {
id : String
object : String
created : Int64
owned_by : String
} derive(Debug)#ModelList
#ModerationRequest
A request to the /moderations endpoint.
impl ToJson for ModerationRequest#ModerationRequest::model
Set the moderation model explicitly.
#ModerationRequest::of_batch
Create a moderation request for a batch of texts.
#ModerationRequest::of_text
Create a moderation request for a single text.
#ModerationResponse
pub(all) struct ModerationResponse {
model : String
results : Array[ModerationResult]
} derive(Debug)The response from the /moderations endpoint.
impl FromJson for ModerationResponse#ModerationResponse::any_flagged
Whether any input in the batch was flagged.
#ModerationResult
A single moderation result: whether the input was flagged, and which
categories were triggered.
impl FromJson for ModerationResult#ModerationResult::flagged_categories
The categories that were flagged, in no particular order.
#RequestLog
pub(all) struct RequestLog {
path : String
request_body : String
status : Int
response_body : String
ok : Bool
}A record describing a single request/response round-trip, passed to an
observer for logging, metrics, or debugging.
#ResponseRequest
pub(all) struct ResponseRequest {
model : String
input : String
instructions : String?
max_output_tokens : Int?
temperature : Double?
}A minimal wrapper for OpenAI's Responses API (/responses), a newer
unified endpoint that accepts a single input (string or message list)
and returns an output array of items.
This covers the common text-in/text-out case; the full Responses API has a
much larger surface (tools, state, streaming) that a real client would
extend over time.
impl ToJson for ResponseRequest#ResponseRequest::instructions
Set top-level instructions (akin to a system prompt).
#ResponseRequest::max_output_tokens
#ResponseRequest::new
Create a Responses request with a plain-text input.
#ResponseRequest::temperature
#RetryPolicy
A retry policy: how many times to retry and how long to wait between tries.
#RetryPolicy::default
A sensible default policy: 3 retries with exponential backoff from 500ms,
capped at 8s, honoring Retry-After.
#RetryPolicy::delay_for
Compute the delay in milliseconds before the given zero-based attempt
(0 = the wait before the first retry).
#RetryPolicy::new
fn RetryPolicy::new(max_retries : Int, backoff : Backoff, respect_retry_after? : Bool) -> RetryPolicyConstruct a custom retry policy.
#Role
#SSEEvent
A fully-framed Server-Sent Event.
Per the SSE spec, an event is terminated by a blank line and may carry
multiple data: lines (concatenated with newlines), an event: type,
an id:, and a retry: value.
#SSEParser
pub struct SSEParser {
event : String?
data : StringBuilder
id : String?
retry : Int?
has_data : Bool
}A stateful Server-Sent Events framer.
Feed it lines (without worrying about trailing CR/LF) via push_line;
it returns a complete SSEEvent whenever a blank line terminates the
current event. This correctly handles multi-line data: payloads, which
the naive one-line-per-event approach cannot.
#Schema
pub(all) enum Schema {
StrSchema(description~ : String?)
NumSchema(description~ : String?)
IntSchema(description~ : String?)
BoolSchema(description~ : String?)
EnumSchema(values~ : Array[String], description~ : String?)
ArraySchema(items~ : Schema, description~ : String?)
ObjectSchema(fields~ : Array[(String, Schema)], required~ : Array[String], description~ : String?)
}A small, type-safe builder for JSON Schema fragments.
LLM tool/function parameters are described with JSON Schema. Writing that
schema by hand as a raw JSON string is error-prone; Schema lets you build
it with typed constructors and render it to a Json value via to_json.
Example:
let params = Schema::object(
fields=[
("city", Schema::string(description="City name")),
("units", Schema::enum_(["celsius", "fahrenheit"])),
],
required=["city"],
)#Schema::array
An array schema over items.
#Schema::enum_
A string-enum schema.
#Schema::object
An object schema. fields maps property names to sub-schemas; required
lists the names that must be present.
#SpeechRequest
pub(all) struct SpeechRequest {
model : String
input : String
voice : String
response_format : String?
speed : Double?
}A request to the /audio/speech (text-to-speech) endpoint.
impl ToJson for SpeechRequest#SpeechRequest::new
Create a text-to-speech request.
#SpeechRequest::response_format
Set the audio output format, e.g. "mp3", "wav", "opus".
#SpeechRequest::speed
Set the playback speed (0.25–4.0).
#StreamChunk
pub(all) struct StreamChunk {
content : String?
tool_calls : Array[ToolCallDelta]
finish_reason : String?
} derive(Eq, Debug)A parsed delta from a streaming chat completion chunk.
#StreamResult
The complete result of a streaming chat completion.
#TokenizerHint
A family of estimation heuristics with different characters-per-token
ratios, since different model families tokenize at different densities.
#Tool
A tool (function) definition advertised to the model.
#ToolCall
#ToolCallAccumulator
Accumulates streaming ToolCallDelta fragments into complete ToolCalls.
Feed each chunk's tool_calls via add; call finish to obtain the
assembled list once the stream completes.
#ToolCallAccumulator::add
Fold a chunk's tool-call fragments into the accumulator.
#ToolCallAccumulator::finish
Assemble the accumulated fragments into complete tool calls, in the order
they first appeared.
#ToolCallAccumulator::new
Create an empty accumulator.
#ToolCallDelta
An incremental tool-call fragment within a streaming chunk.
During streaming, a tool call is delivered piecewise: the index
identifies which call it belongs to, and any of id / name /
arguments may be a partial fragment to be concatenated.
#ToolChoice
How the model should choose among the provided tools.
impl ToJson for ToolChoice#TranscriptionResponse
pub(all) struct TranscriptionResponse {
text : String
language : String?
duration : Double?
} derive(Debug)A response from the /audio/transcriptions endpoint.
impl FromJson for TranscriptionResponse#Usage
#UsageTracker
pub struct UsageTracker {
calls : Int
prompt_tokens : Int
completion_tokens : Int
prompt_price_per_1k : Double
completion_price_per_1k : Double
}Aggregates token usage and call counts across many requests, so an
application can report cumulative consumption (and estimated cost).
#UsageTracker::calls
The number of calls recorded.
#UsageTracker::cost
The estimated cost so far, in dollars, using the configured prices.
#UsageTracker::new
fn UsageTracker::new(prompt_price_per_1k? : Double, completion_price_per_1k? : Double) -> UsageTrackerCreate a tracker. Prices default to 0 (cost reported as 0 until set).
#UsageTracker::record
Record a usage entry (e.g. from a ChatResponse.usage).
#UsageTracker::record_response
Record the usage from a chat response, if it reported any.
#UsageTracker::reset
Reset all counters to zero (prices are preserved).
#UsageTracker::snapshot
A Usage snapshot of the accumulated totals.
#UsageTracker::summary
A one-line human-readable summary of the accumulated usage.
#UsageTracker::total_tokens
The total tokens across all recorded calls.
#anthropic_request_body
Build an Anthropic Messages request body from a common ChatRequest.
System messages are hoisted into the top-level system field; all other
messages are mapped to Anthropic's {role, content} shape. Anthropic
requires max_tokens, so a default of 1024 is used when unset.
#build_query
Build a query string (without the leading ?) from key/value pairs,
percent-encoding both keys and values. Returns an empty string when there
are no parameters.
#cosine_similarity
Compute the cosine similarity between two equal-length vectors.
Returns 0.0 if either vector is empty or their lengths differ.
#estimate_cost
fn estimate_cost(prompt_tokens : Int, completion_tokens : Int, prompt_price_per_1k~ : Double, completion_price_per_1k~ : Double) -> DoubleApproximate a US-dollar cost given token counts and per-1K-token prices.
#estimate_message_tokens
Estimate the token count of a single message, including a small overhead
per message for role/formatting (mirrors OpenAI's ~4 tokens/message).
#estimate_messages_tokens
Estimate the total prompt tokens for a list of messages under a hint,
including a per-message overhead (role + formatting).
#estimate_tokens
fn estimate_tokens(text : String) -> IntA rough token-count estimate for a piece of text.
Uses the widely-cited heuristic of ~4 characters per token. This is an
approximation for budgeting only, not an exact tokenizer.
#estimate_tokens_by_words
fn estimate_tokens_by_words(text : String) -> IntA word-boundary based estimate: counts whitespace-separated words plus a
surcharge for punctuation-heavy text. Sometimes closer than the pure
char-ratio for natural language.
#estimate_tokens_with
Estimate the token count of text under a given hint.
#gemini_request_body
Build a Gemini generateContent request body from a ChatRequest.
#is_retryable_error
Whether an error is retryable under this policy.
#is_valid_speech_format
fn is_valid_speech_format(fmt : String) -> BoolWhether fmt is a recognized speech output format.
#join_url
fn join_url(base : String, path : String) -> StringJoin a base URL and a path, collapsing a duplicated / at the boundary and
inserting one if neither side provides it.
#merge_consecutive
Merge consecutive messages from the same role into a single message,
joining their text with a blank line. Some APIs reject two user messages
in a row; this collapses them.
#parse_anthropic_event
Parse one Anthropic streaming event, given its event: type and data:
JSON payload, into a common StreamChunk.
Returns None for events that carry no usable delta (message_start,
content_block_start/stop, ping, etc.). Raises LLMError::Decode on
malformed JSON.
#parse_anthropic_response
Parse an Anthropic Messages response into the common ChatResponse.
The content blocks of type text are concatenated into a single
assistant message; usage is mapped from input_tokens/output_tokens.
#parse_anthropic_tool_calls
Parse the tool_use content blocks from a complete (non-streaming)
Anthropic response into common ToolCalls.
Anthropic represents tool calls as {"type":"tool_use","id":...,"name":...,"input":{...}} blocks in the response content array; the
input object is re-serialized into the arguments JSON string to match
the OpenAI shape.
#parse_gemini_response
Parse a Gemini generateContent response into the common ChatResponse.
#parse_retry_after
fn parse_retry_after(value : String) -> Int?Parse a Retry-After header value into milliseconds.
Supports the delta-seconds form (e.g. "5" → 5000ms). The HTTP-date form
is not supported and yields None.
#parse_sse_line
fn parse_sse_line(line : String) -> (String, String)?Strip a single leading SSE field prefix (data: / event: ...) and the
optional single space after the colon, returning the field value.
Returns None for comment lines (starting with :) and blank lines.
#parse_stream_data
Parse one SSE data: payload (the JSON after data: ) into a StreamChunk.
Returns None for payloads that carry no usable delta (e.g. role-only
opening chunks). Raises LLMError::Decode on malformed JSON.
#parse_transcription
Parse a transcription response from raw JSON text (as returned by the API).
The transcription endpoint itself is multipart/form-data for the audio
upload, which is out of scope for the JSON client; this helper covers the
response side so callers using a multipart layer can decode the result.
#percent_encode
fn percent_encode(s : String) -> StringPercent-encode a string for use in a URL query value. Unreserved
characters (A-Z a-z 0-9 - _ . ~) pass through; everything else is encoded
as %XX using UTF-8 bytes.
#render_transcript
Render a list of messages into a single plain-text transcript, one line
per message prefixed by the role — handy for logging or debugging.
#speech_formats
The supported audio output formats for speech synthesis.
#truncate_to_tokens
Truncate text to at most max_tokens (estimated), returning the kept
prefix. Useful for clamping a single oversized message.
Source Files
- anthropic.mbt
- anthropic_stream.mbt
- audio.mbt
- batch.mbt
- client.mbt
- completions.mbt
- conversation.mbt
- embeddings.mbt
- errors.mbt
- files.mbt
- fine_tuning.mbt
- gemini.mbt
- images.mbt
- message_builder.mbt
- models.mbt
- moderations.mbt
- observer.mbt
- request.mbt
- responses.mbt
- retry.mbt
- schema.mbt
- sse.mbt
- stream.mbt
- tokenizer.mbt
- types.mbt
- url.mbt
- usage_tracker.mbt
Type-safe, OpenAI-compatible LLM client for MoonBit: chat completions, streaming (SSE), tool calling, and multimodal input.
llmopenaiaisdkhttp-client
0.1.0
Apache-2.0
Updated last month
378 downloadsDependencies
- oboard/mio0.5.2
- moonbitlang/async0.20.1