llm

    Pure MoonBit LLM client library

    llm
    ai
    client
    Download zip
    Author
    Version
    0.3.3
    License
    Apache-2.0
    Last updated
    3 hours ago
    Downloads
    11

    #MoonBit LLM client

    totto2727/llm is a compatibility fork of mizchi/llm for current MoonBit releases. It provides shared request and response types, streaming events, and provider adapters.

    After the version is published on Mooncakes, add totto2727/llm@0.3.3 to moon.mod and import totto2727/llm in moon.pkg. The default package alias remains @llm.

    #Runtimes

    OpenAI-compatible and Anthropic providers use moonbitlang/async/http to stream responses on JavaScript, native, and Wasm1. For JavaScript, use Node.js with Fetch or a browser with a CORS-compatible endpoint. For Wasm1, use a host that provides MoonBit async socket support, such as moonrun. Wasm GC is not supported by this version of the async HTTP library. Claude Code requires a host that can launch the installed claude executable and is not available in browsers.

    Provider::stream, collect, collect_text, and the agent loops are async and must be called from an async context. Provider trait implementations still use impl Provider for Type with fn stream; the trait supplies the async effect.

    See upstream differences for the upstream revision, compatibility changes, and release procedure.

    SchemaBuilder

    SchemaBuilder helps construct JSON Schema for tool input_schema.

    ToolChoice

    ToolChoice controls how the model selects tools.

    ToolDef

    ToolDef defines a tool that can be called by the model.

    ToolExecuteError

    ToolRegistry

    ToolRegistry manages available tools and their handlers.

    Provider

    pub(open) trait Provider {
    async fn stream(Self, Array[Message], Array[
    ToolDef
    ], StreamHandler) -> Unit
    fn name(Self) -> String
    }

    Provider is the core trait for LLM providers.

    AgentEvent

    pub(all) enum AgentEvent {
    Stream(StreamEvent)
    ToolExecute(name~ : String, input~ :
    Json
    )
    ToolResult(name~ : String, result~ : String, is_error~ : Bool)
    StepComplete(step~ : Int)
    Done(reason~ : String)
    }

    AgentEvent represents events emitted during agent execution.

    BoxedProvider

    pub struct BoxedProvider {
    inner : &Provider
    }

    BoxedProvider wraps a Provider for dynamic dispatch.

    BoxedProvider::name

    fn BoxedProvider::name(self : BoxedProvider) -> String

    BoxedProvider::new

    fn BoxedProvider::new(provider : &Provider) -> BoxedProvider

    BoxedProvider::stream

    async fn BoxedProvider::stream(self : BoxedProvider, messages : Array[Message], tools : Array[
    ToolDef
    ], handler : StreamHandler) -> Unit

    CollectResult

    pub(all) struct CollectResult {
    text : String
    tool_calls : Array[ToolCall]
    finish_reason : FinishReason
    usage : Usage?
    }

    CollectResult holds the full result of a collected stream.

    ContentBlock

    pub(all) enum ContentBlock {
    Text(String)
    ToolUse(id~ : String, name~ : String, input~ :
    Json
    )
    ToolResult(tool_use_id~ : String, content~ : String, is_error~ : Bool)
    }

    ContentBlock represents typed content within a message.

    ContentBlock::to_anthropic_json

    Convert a ContentBlock to JSON (Anthropic format).

    ContentBlock::to_openai_json

    Convert a ContentBlock to JSON (OpenAI format).

    FinishReason

    pub(all) enum FinishReason {
    Stop
    ToolUse
    Length
    ContentFilter
    Error
    Unknown(String)
    } derive(
    Debug
    )

    FinishReason indicates why the model stopped generating.

    FinishReason::from_string

    fn FinishReason::from_string(s : String) -> FinishReason

    JsonChatLoopResult

    pub(all) struct JsonChatLoopResult {
    final_content : String
    messages : Array[
    Json
    ]
    iterations : Int
    }

    JsonChatLoopResult is the final result of run_json_chat_loop.

    JsonChatResponse

    pub(all) struct JsonChatResponse {
    content : String
    tool_calls : Array[JsonChatToolCall]
    reasoning_content : String?
    }

    JsonChatResponse is the per-step response from a model callback.

    JsonChatResponse::new

    fn JsonChatResponse::new(content : String, tool_calls : Array[JsonChatToolCall], reasoning_content : String?) -> JsonChatResponse

    JsonChatToolCall

    pub(all) struct JsonChatToolCall {
    id : String
    name : String
    arguments :
    Json

    }

    JsonChatToolCall represents a function call in JSON chat loop.

    JsonChatToolCall::new

    fn JsonChatToolCall::new(id : String, name : String, arguments :
    Json
    ) -> JsonChatToolCall

    Message

    pub(all) struct Message {
    role : Role
    content : Array[ContentBlock]
    }

    Message represents a single message in a conversation.

    Message::assistant

    fn Message::assistant(text : String) -> Message

    Create an assistant text message.

    Message::estimate_tokens

    fn Message::estimate_tokens(self : Message) -> Int

    Estimate token count for a message (rough: ~4 chars per token).

    Message::get_text

    fn Message::get_text(self : Message) -> String

    Get the text content of a message (concatenated from all Text blocks).

    Message::system

    fn Message::system(text : String) -> Message

    Create a system text message.

    Message::text

    fn Message::text(role : Role, text : String) -> Message

    Create a simple text message.

    Message::to_anthropic_json

    fn Message::to_anthropic_json(self : Message) ->
    Json

    Convert a Message to Anthropic JSON format.

    Message::to_openai_json

    fn Message::to_openai_json(self : Message) ->
    Json

    Convert a Message to OpenAI JSON format.

    Message::user

    fn Message::user(text : String) -> Message

    Create a user text message.

    MockProvider

    pub struct MockProvider {
    responses : Array[Array[StreamEvent]]
    call_count : Int
    }

    MockProvider::boxed

    MockProvider::new

    fn MockProvider::new(responses : Array[Array[StreamEvent]]) -> MockProvider

    ProviderConfig

    pub(all) struct ProviderConfig {
    api_key : String
    model : String
    max_tokens : Int
    system_prompt : String
    base_url : String
    timeout_sec : Int
    max_retries : Int
    retry_delay_ms : Int
    }

    ProviderConfig holds common configuration for providers.

    ProviderConfig::new

    fn ProviderConfig::new(api_key : String, model : String, base_url : String, max_tokens? : Int, system_prompt? : String, timeout_sec? : Int, max_retries? : Int, retry_delay_ms? : Int) -> ProviderConfig

    Create a ProviderConfig with defaults.

    Role

    pub(all) enum Role {
    User
    Assistant
    System
    Tool
    } derive(Eq,
    Debug
    )

    Role represents the sender of a message in a conversation.
    impl Show for Role

    Role::from_string

    fn Role::from_string(s : String) -> Role

    Role::to_string

    fn Role::to_string(self : Role) -> String

    SkillEntry

    pub(all) struct SkillEntry {
    name : String
    path : String
    source : String
    content : String
    }

    SkillEntry is a raw skill definition with content.

    SkillEntry::new

    fn SkillEntry::new(name : String, path : String, source : String, content : String) -> SkillEntry

    SkillInfo

    pub(all) struct SkillInfo {
    name : String
    path : String
    source : String
    available : Bool
    }

    SkillInfo is a projected skill metadata view for listings.

    SkillInfo::new

    fn SkillInfo::new(name : String, path : String, source : String, available : Bool) -> SkillInfo

    SkillsLoader

    pub struct SkillsLoader {
    workspace_skills : Map[String, SkillEntry]
    builtin_skills : Map[String, SkillEntry]
    }

    SkillsLoader::build_skills_summary

    fn SkillsLoader::build_skills_summary(self : SkillsLoader, bins? : Map[String, Bool], env? : Map[String, String]) -> String

    SkillsLoader::get_always_skills

    fn SkillsLoader::get_always_skills(self : SkillsLoader, bins? : Map[String, Bool], env? : Map[String, String]) -> Array[String]

    SkillsLoader::get_skill_description

    fn SkillsLoader::get_skill_description(self : SkillsLoader, name : String) -> String

    SkillsLoader::get_skill_metadata

    fn SkillsLoader::get_skill_metadata(self : SkillsLoader, name : String) -> Map[String, String]?

    SkillsLoader::list_skills

    fn SkillsLoader::list_skills(self : SkillsLoader, filter_unavailable? : Bool, bins? : Map[String, Bool], env? : Map[String, String]) -> Array[SkillInfo]

    SkillsLoader::load_skill

    fn SkillsLoader::load_skill(self : SkillsLoader, name : String) -> String?

    SkillsLoader::load_skills_for_context

    fn SkillsLoader::load_skills_for_context(self : SkillsLoader, skill_names : Array[String]) -> String

    SkillsLoader::new

    SkillsLoader::register_builtin_skill

    fn SkillsLoader::register_builtin_skill(self : SkillsLoader, name : String, content : String, path? : String) -> Unit

    SkillsLoader::register_workspace_skill

    fn SkillsLoader::register_workspace_skill(self : SkillsLoader, name : String, content : String, path? : String) -> Unit

    StopCondition

    pub(all) enum StopCondition {
    MaxSteps(Int)
    OnFinish
    }

    StopCondition controls when the agent loop should stop.

    StreamEvent

    pub(all) enum StreamEvent {
    TextDelta(String)
    ToolCallStart(id~ : String, name~ : String)
    ToolCallDelta(id~ : String, input_delta~ : String)
    ToolCallEnd(id~ : String, name~ : String, input~ :
    Json
    )
    MessageStart
    MessageEnd(finish_reason~ : FinishReason, usage~ : Usage?)
    Error(String)
    }

    StreamEvent represents events during streaming generation.
    impl Show for StreamEvent

    StreamHandler

    pub(all) struct StreamHandler {
    on_event : (StreamEvent) -> Unit
    }

    StreamHandler receives streaming events.

    ToolCall

    pub(all) struct ToolCall {
    id : String
    name : String
    input :
    Json

    }

    ToolCall represents a tool invocation from the model.

    Usage

    pub(all) struct Usage {
    input_tokens : Int
    output_tokens : Int
    } derive(
    Debug
    )

    Usage tracks token consumption.
    impl Show for Usage

    collect

    async fn collect(provider : &Provider, messages : Array[Message], tools? : Array[
    ToolDef
    ]) -> CollectResult

    Collect full response including text, tool calls, finish reason, and usage.

    collect_text

    async fn collect_text(provider : &Provider, messages : Array[Message], tools? : Array[
    ToolDef
    ]) -> String

    Collect all text from a streaming provider call into a single string.

    default_anthropic_model

    let default_anthropic_model : String

    Default models for each provider.

    default_ollama_model

    let default_ollama_model : String

    default_openai_model

    let default_openai_model : String

    default_openrouter_model

    let default_openrouter_model : String

    json_get_array

    fn json_get_array(j :
    Json
    , field : String) -> Array[
    Json
    ]?

    Helper to get a JSON array field.

    json_get_int

    fn json_get_int(j :
    Json
    , field : String) -> Int?

    Helper to get an int field from a JSON object.

    json_get_object

    fn json_get_object(j :
    Json
    , field : String) ->
    Json
    ?

    Helper to get a JSON object field.

    json_get_string

    fn json_get_string(j :
    Json
    , field : String) -> String?

    Helper to get a string field from a JSON object.

    parse_skill_frontmatter

    fn parse_skill_frontmatter(content : String) -> (Map[String, String]?, String)

    Parse a markdown skill file frontmatter and return (metadata, body).

    run_agent

    async fn run_agent(provider : BoxedProvider, registry :
    ToolRegistry
    , messages : Array[Message], stop : StopCondition, on_event : (AgentEvent) -> Unit) -> Unit

    run_agent executes a tool loop with the given provider and tools.

    run_agent_cancellable

    async fn run_agent_cancellable(provider : BoxedProvider, registry :
    ToolRegistry
    , messages : Array[Message], stop : StopCondition, should_cancel : () -> Bool, on_event : (AgentEvent) -> Unit) -> Unit

    run_json_chat_loop

    async fn run_json_chat_loop(messages : Array[
    Json
    ], tools : Array[
    Json
    ], max_iterations : Int, chat : async (Array[
    Json
    ], Array[
    Json
    ]) -> JsonChatResponse, execute_tool : (String,
    Json
    ) -> String, fallback_content? : String) -> JsonChatLoopResult

    skill_metadata_description

    fn skill_metadata_description(metadata : Map[String, String]?, fallback : String) -> String

    Resolve human-readable skill description from metadata.

    skill_metadata_is_always

    fn skill_metadata_is_always(metadata : Map[String, String]?, scope_key? : String) -> Bool

    always may be specified either at root frontmatter or namespaced metadata.

    skill_metadata_is_available

    fn skill_metadata_is_available(metadata : Map[String, String]?, bins? : Map[String, Bool], env? : Map[String, String], scope_key? : String) -> Bool

    Evaluate whether a skill is runnable in the current environment.

    skill_metadata_missing_requirements

    fn skill_metadata_missing_requirements(metadata : Map[String, String]?, bins? : Map[String, Bool], env? : Map[String, String], scope_key? : String) -> String

    Collect missing requirement text (e.g. CLI: tmux, ENV: OPENAI_API_KEY).

    strip_skill_frontmatter

    fn strip_skill_frontmatter(content : String) -> String

    Remove frontmatter and return trimmed skill markdown body.

    truncate_messages

    fn truncate_messages(messages : Array[Message], max_tokens : Int) -> Array[Message]

    Truncate messages to fit within a token budget. Keeps system messages and the last N messages within budget.

    validate_params

    fn validate_params(input :
    Json
    , schema :
    Json
    ) -> Array[String]

    validate_tool_input

    fn validate_tool_input(input :
    Json
    , schema :
    Json
    ) -> Array[String]