vector

    A lightweight, zero-dependency vector similarity search and index engine (Vector Database) in MoonBit.

    vector-search
    similarity
    machine-learning
    knn
    vector-database
    Download zip
    Author
    Version
    0.1.3
    License
    Apache-2.0
    Last updated
    2 months ago
    Downloads
    53

    No dependencies

    #MoonVector Similarity Search Library (MoonVector)

    CI License

    A pure MoonBit, zero-dependency high-dimensional Vector Similarity Search and Indexing Engine (Vector Database). It is designed to support client-side and server-side retrieval (Wasm, JS, Native), making it ideal for modern LLM applications such as Retrieval-Augmented Generation (RAG), embedding similarity search, and semantic classification.

    MoonVector implements multiple indexing algorithms (Flat, IVF-Flat, KD-Tree, LSH) alongside K-Means clustering, metadata filtering, and JSON serialization.


    #Features

    • Zero Dependencies: Hand-crafted mathematical primitives and data structures in pure MoonBit.
    • Multiple Distance Metrics: Cosine Similarity, L2 Euclidean Distance, L1 Manhattan Distance, and Dot Product.
    • Advanced Spatial Indexing:
      • FlatIndex: Baseline linear exact search.
      • IvfIndex (Inverted File Index): Clustered space partitions using K-Means. Limits query scanning to the closest centroids, optimizing retrieval speeds for high-dimensional vectors.
      • KdTreeIndex: Spatial-partitioning binary tree for low-to-medium dimensional vectors, implementing exact nearest neighbor search with hyper-plane bounding pruning. Supports Euclidean (L2) and Manhattan (L1) metrics.
      • LshIndex (Locality Sensitive Hashing): Random projection hashing for fast approximate Cosine similarity search.
    • Metadata Tag Filtering: Prunes search targets on-the-fly using key-value tag constraints.
    • JSON Serialization: Built-in, zero-dependency parser for importing/exporting Document records to/from JSON.
    • Index Analyzer: Retrieve structural analytics (IndexStats) such as tree depths, centroid count, and document size.
    • Wasm & Native Optimized: Compiles cleanly without target-sensitive warnings.


    #Installation

    Add this package to your MoonBit dependencies:

    moon add ywz1314/vector

    Or manually declare it in your moon.mod dependencies:

    import {
    "ywz1314/vector@0.1.3",
    }

    #Repositories


    #Quick Start

    import "ywz1314/vector" as @vector

    fn main {
    // Create a Flat index
    let index = @vector.FlatIndex::new()

    // Insert documents with 3-dimensional embeddings and tags
    index.add(@vector.Document::new("doc_1", [1.0, 2.0, 3.0], [("category", "ml")]))
    index.add(@vector.Document::new("doc_2", [4.0, 5.0, 6.0], [("category", "security")]))

    // Search Top 2 nearest neighbors using L2 Euclidean Distance
    let query = [1.2, 2.2, 3.2]
    let results = index.search(query, 2, @vector.Euclidean, [])

    for i = 0; i < results.length(); i = i + 1 {
    println("Rank " + (i + 1).to_string() + ": " + results[i].id + ", Score: " + results[i].score.to_string())
    }
    }

    #2. High-Dimensional IVF-Flat Cluster Searching

    import "ywz1314/vector" as @vector

    fn main {
    // IVF Index with K=2 centroids evaluated under Cosine Metric
    let index = @vector.IvfIndex::new(2, @vector.Cosine)

    let docs = [
    @vector.Document::new("doc_1", [1.0, 2.0, 3.0], [("category", "ml")]),
    @vector.Document::new("doc_2", [1.5, 2.5, 3.5], [("category", "ml")]),
    @vector.Document::new("doc_3", [5.0, 5.0, 5.0], [("category", "security")]),
    ]

    // Build cluster index using K-Means
    index.build(docs)

    // Search the closest 1 cluster centroid (nprobe=1)
    let query = [1.1, 2.1, 3.1]
    let results = index.search(query, 2, 1, [])
    }

    #3. Spatial Partitioning with KD-Trees

    import "ywz1314/vector" as @vector

    fn main {
    let index = @vector.KdTreeIndex::new(@vector.Euclidean)

    let docs = [
    @vector.Document::new("doc_1", [1.0, 2.0], []),
    @vector.Document::new("doc_2", [3.0, 4.0], []),
    ]

    index.build(docs)

    // Search nearest neighbor
    let query = [1.1, 2.1]
    let results = index.search(query, 1, [])
    }

    #4. Approximate Cosine Search using LSH

    import "ywz1314/vector" as @vector

    fn main {
    // Set up LSH index with 4 random projection planes for 3D vectors
    let index = @vector.LshIndex::new(4, 3)

    index.add(@vector.Document::new("doc_1", [1.0, 2.0, 3.0], []))
    index.add(@vector.Document::new("doc_2", [-1.0, -2.0, -3.0], []))

    // Retrieve candidate neighbors sharing the same projection hash
    let query = [1.1, 2.1, 3.1]
    let results = index.search(query, 1, [])
    }

    #5. Document JSON Serialization

    import "ywz1314/vector" as @vector

    fn main {
    let doc = @vector.Document::new("doc_1", [1.25, -2.5], [("tag", "a")])
    let json = doc.to_json()
    println("JSON: " + json)

    // Deserialization
    let parsed = @vector.Document::from_json(json)
    println("Parsed Document ID: " + parsed.id)
    }


    #API Reference

    #1. Distance Metrics

    • cosine_similarity(v1: Array[Double], v2: Array[Double]) -> Double raise VectorError
    • euclidean_distance(v1: Array[Double], v2: Array[Double]) -> Double raise VectorError
    • manhattan_distance(v1: Array[Double], v2: Array[Double]) -> Double raise VectorError
    • dot_product(v1: Array[Double], v2: Array[Double]) -> Double raise VectorError

    #2. Unsupervised Clustering

    • kmeans(vectors: Array[Array[Double]], k: Int, max_iters: Int) -> Array[Array[Double]] raise VectorError

    #3. Indexing Classes

    • FlatIndex::new() -> FlatIndex
      • add(self: FlatIndex, doc: Document) -> Unit
      • search(self: FlatIndex, query: Array[Double], top_k: Int, metric: DistanceMetric, filters: Array[(String, String)]) -> Array[SearchResult] raise VectorError
      • stats(self: FlatIndex) -> IndexStats

    • IvfIndex::new(k: Int, metric: DistanceMetric) -> IvfIndex
      • build(self: IvfIndex, docs: Array[Document]) -> Unit raise VectorError
      • search(self: IvfIndex, query: Array[Double], top_k: Int, nprobe: Int, filters: Array[(String, String)]) -> Array[SearchResult] raise VectorError
      • stats(self: IvfIndex) -> IndexStats

    • KdTreeIndex::new(metric: DistanceMetric) -> KdTreeIndex
      • build(self: KdTreeIndex, docs: Array[Document]) -> Unit
      • search(self: KdTreeIndex, query: Array[Double], top_k: Int, filters: Array[(String, String)]) -> Array[SearchResult] raise VectorError
      • stats(self: KdTreeIndex) -> IndexStats

    • LshIndex::new(num_planes: Int, dim: Int) -> LshIndex
      • add(self: LshIndex, doc: Document) -> Unit raise VectorError
      • search(self: LshIndex, query: Array[Double], top_k: Int, filters: Array[(String, String)]) -> Array[SearchResult] raise VectorError

    #Data-quality and application helpers

    • validate_documents(docs) rejects empty corpora, blank/duplicate ids, empty vectors, and mixed dimensions before indexing.
    • normalize_l2 and normalize_documents provide explicit embedding normalization without mutating the input arrays.
    • VectorCollection supports validated upsert, remove, replace_all, rebuild, exact search, and batch search for in-memory applications.
    • search_batch, recall_at_k, mean_recall, evaluate_ivf, and evaluate_lsh make approximate-index evaluation reproducible.
    • paginate_results, merge_results, diversify_results, inspect_corpus, and summarize_workload cover common service-layer and monitoring needs.

    #Complete application workflows

    The repository also includes reusable application-level paths rather than only isolated algorithms:

    • KnowledgeBase manages validated upserts, deletes, revisions, exact search, approximate index fallback, batch requests, health reports, and tenant-safe metadata expressions.
    • MetadataFilter and FilterExpression support equality, inequality, prefix, substring, membership, AND, OR, and NOT policies.
    • hybrid_search combines embedding similarity with text metadata overlap; compose_retrieval_context produces bounded RAG context with source ids and citations.
    • run_rag_scenario, run_catalog_scenario, and run_batch_scenario are deterministic application fixtures covering multi-tenant retrieval, filtered recommendations, and batch workload telemetry.
    • Ranking quality helpers include precision@k, recall curves, MAP, NDCG, and coverage so approximate retrieval can be evaluated against Flat.


    #Testing

    Run the full whitebox and blackbox test suite with:

    moon test

    The current local suite contains 21 tests covering metrics, clustering, all index types, JSON round-trips, filtering, malformed input, collection lifecycle, batch search, benchmark recall, multi-tenant RAG, catalog recommendation, and application-level retrieval policies. Expected output:
    Total tests: 21, passed: 21, failed: 0.

    For the strict CI-equivalent check, run:

    moon fmt --check moon check --deny-warn moon test --deny-warn moon build moon info git diff --exit-code

    #Reproducible benchmark

    The repository includes a deterministic 48-document, 8-dimensional corpus and six query vectors in benchmark.mbt. It is intentionally synthetic but is structured into four semantic topic bands so that cluster locality, metadata filtering, and approximate recall can be checked without downloading private or copyrighted data.

    The public helpers are benchmark_corpus, benchmark_queries, evaluate_ivf, and evaluate_lsh. The test suite compares IVF and LSH against the exact FlatIndex baseline using recall@k. This gives a repeatable quality signal rather than a claim based on a single hand-picked example.

    #Boundary and data-quality behavior

    • Empty vectors are rejected by metric, normalization, and corpus-validation APIs.
    • Mismatched dimensions raise VectorError::DimensionMismatch.
    • Duplicate or blank document ids are rejected by validate_documents and reported by inspect_corpus.
    • top_k <= 0 returns no results; requesting more results than available is safely truncated.
    • Empty filters match every document. Multiple filters use AND semantics.
    • Document::from_json rejects missing required fields; this lightweight serializer expects string values without embedded unescaped quotes.
    • IVF nprobe is bounded to the number of clusters. LSH is approximate and must be evaluated against Flat when a recall target matters.

    #Application APIs

    VectorCollection provides validated upsert/remove/replace/rebuild lifecycle operations for applications receiving documents over time. QueryPlan and recommend_strategy make the choice between exact Flat, IVF, KD-Tree, and LSH explicit. ResultPage, merge_results, diversify_results, and SearchTelemetry support common service-layer concerns without coupling the library to a network framework.

    #Original work and open-source compliance

    MoonVector is an original MoonBit implementation. It is not a line-by-line port and has no vendored third-party source, generated source, private code, or external benchmark dataset. The repository depends only on MoonBit's standard library and is released under Apache-2.0. The deterministic benchmark fixture is authored in benchmark.mbt; it contains no personal or proprietary data.

    See docs/ARCHITECTURE.md for package boundaries, algorithm trade-offs, correctness guarantees, and known limitations.

    #OSC 2026 Submission Notes

    • Repository layout: package root, CLI entrypoint in cmd/main, whitebox and blackbox tests, docs, and package metadata are all present.
    • License: Apache-2.0.
    • Validation: CI now checks moon check, moon fmt --deny-warn equivalent formatting enforcement, moon info --deny-warn interface generation, and moon test.
    • Toolchain target: MoonBit toolchain 0.10.3-compatible workflow.
    • Source scale: the project includes multiple MoonBit source modules, tests, and documentation artifacts rather than a minimal stub.


    #License

    This project is licensed under the Apache 2.0 License - see the LICENSE file for details.

    VectorError

    pub suberror VectorError {
    DimensionMismatch(String)
    EmptyVector
    InvalidK
    ClusterError(String)
    IndexError(String)
    }

    impl Show for VectorError

    BenchmarkReport

    pub(all) struct BenchmarkReport {
    queries : Int
    corpus : Int
    top_k : Int
    recall : Double
    index_name : String
    }

    A reproducible evaluation summary for an approximate index.

    ContextChunk

    pub(all) struct ContextChunk {
    id : String
    text : String
    score : Double
    metadata : Array[(String, String)]
    }

    Text chunk selected for a retrieval-augmented generation prompt.

    DistanceMetric

    pub(all) enum DistanceMetric {
    Cosine
    Euclidean
    Manhattan
    DotProduct
    }

    Document

    pub(all) struct Document {
    id : String
    vector : Array[Double]
    metadata : Array[(String, String)]
    }

    Document::from_json

    fn Document::from_json(json : String) -> Document raise VectorError

    Document::new

    fn Document::new(id : String, vector : Array[Double], metadata : Array[(String, String)]) -> Document

    Document::to_json

    fn Document::to_json(self : Document) -> String

    FilterExpression

    pub(all) enum FilterExpression {
    MatchAll
    MatchNone
    AllOf(Array[MetadataFilter])
    AnyOf(Array[MetadataFilter])
    Not(MetadataFilter)
    }

    A small composable expression for common metadata policies.

    FilterReport

    pub(all) struct FilterReport {
    total : Int
    matched : Int
    selectivity : Double
    }

    A deterministic report for evaluating filter selectivity.

    FlatIndex

    pub struct FlatIndex {
    documents : Array[Document]
    }

    FlatIndex::add

    fn FlatIndex::add(self : FlatIndex, doc : Document) -> Unit

    FlatIndex::new

    fn FlatIndex::new() -> FlatIndex

    FlatIndex::search

    fn FlatIndex::search(self : FlatIndex, query : Array[Double], top_k : Int, metric : DistanceMetric, filters : Array[(String, String)]) -> Array[SearchResult] raise VectorError

    FlatIndex::search_batch

    fn FlatIndex::search_batch(self : FlatIndex, queries : Array[Array[Double]], top_k : Int, metric : DistanceMetric, filters : Array[(String, String)]) -> Array[Array[SearchResult]] raise VectorError

    Run the same query against a flat index and return results in input order.

    FlatIndex::stats

    fn FlatIndex::stats(self : FlatIndex) -> IndexStats

    HealthReport

    pub(all) struct HealthReport {
    healthy : Bool
    document_count : Int
    dimension : Int
    duplicate_ids : Int
    empty_ids : Int
    dimension_errors : Int
    message : String
    }

    A machine-readable health snapshot for monitoring an index in production.

    IndexStats

    pub(all) struct IndexStats {
    count : Int
    dim : Int
    extra_info : String
    }

    impl Show for IndexStats

    IndexStrategy

    pub(all) enum IndexStrategy {
    ExactFlat
    ClusteredIvf
    SpatialKdTree
    ApproximateLsh
    }

    Selects an index strategy based on corpus size and caller requirements.

    IvfIndex

    pub struct IvfIndex {
    centroids : Array[Array[Double]]
    inverted_lists : Array[Array[Document]]
    k : Int
    metric : DistanceMetric
    }

    IvfIndex::build

    fn IvfIndex::build(self : IvfIndex, docs : Array[Document]) -> Unit raise VectorError

    IvfIndex::new

    fn IvfIndex::new(k : Int, metric : DistanceMetric) -> IvfIndex

    IvfIndex::search

    fn IvfIndex::search(self : IvfIndex, query : Array[Double], top_k : Int, nprobe : Int, filters : Array[(String, String)]) -> Array[SearchResult] raise VectorError

    IvfIndex::search_batch

    fn IvfIndex::search_batch(self : IvfIndex, queries : Array[Array[Double]], top_k : Int, nprobe : Int, filters : Array[(String, String)]) -> Array[Array[SearchResult]] raise VectorError

    Search a batch using IVF-Flat with the same probe budget for each query.

    IvfIndex::stats

    fn IvfIndex::stats(self : IvfIndex) -> IndexStats

    KdNode

    pub enum KdNode {
    Empty
    Node(Int, Document, KdNode, KdNode)
    }

    KdTreeIndex

    pub struct KdTreeIndex {
    root : KdNode
    metric : DistanceMetric
    }

    KdTreeIndex::build

    fn KdTreeIndex::build(self : KdTreeIndex, docs : Array[Document]) -> Unit

    KdTreeIndex::new

    KdTreeIndex::search

    fn KdTreeIndex::search(self : KdTreeIndex, query : Array[Double], top_k : Int, filters : Array[(String, String)]) -> Array[SearchResult] raise VectorError

    KdTreeIndex::stats

    fn KdTreeIndex::stats(self : KdTreeIndex) -> IndexStats

    KnowledgeBase

    pub struct KnowledgeBase {
    collection : VectorCollection
    ivf : IvfIndex?
    lsh : LshIndex?
    revision : Int
    }

    Mutable in-memory knowledge base with exact and approximate retrieval paths.

    KnowledgeBase::clear

    fn KnowledgeBase::clear(self : KnowledgeBase) -> Unit

    Delete all documents and reset index state.

    KnowledgeBase::documents

    fn KnowledgeBase::documents(self : KnowledgeBase) -> Array[Document]

    Return a stable document snapshot for export or diagnostics.

    KnowledgeBase::health

    Return a health report for readiness probes.

    KnowledgeBase::length

    fn KnowledgeBase::length(self : KnowledgeBase) -> Int

    Return the number of indexed documents.

    KnowledgeBase::metadata_values

    fn KnowledgeBase::metadata_values(self : KnowledgeBase, key : String) -> Array[String]

    Return metadata values present in the knowledge base.

    KnowledgeBase::new

    Create an empty knowledge base.

    KnowledgeBase::remove

    fn KnowledgeBase::remove(self : KnowledgeBase, id : String) -> Bool

    Remove one document and report whether it existed.

    KnowledgeBase::replace_all

    fn KnowledgeBase::replace_all(self : KnowledgeBase, docs : Array[Document]) -> Unit raise VectorError

    Replace the complete corpus after validation.

    KnowledgeBase::retrieve

    Execute one retrieval request.

    KnowledgeBase::retrieve_batch

    fn KnowledgeBase::retrieve_batch(self : KnowledgeBase, requests : Array[RetrievalOptions]) -> Array[RetrievalResponse] raise VectorError

    Execute a batch of requests in input order.

    KnowledgeBase::revision

    fn KnowledgeBase::revision(self : KnowledgeBase) -> Int

    Return a monotonically increasing data revision.

    KnowledgeBase::search_exact

    fn KnowledgeBase::search_exact(self : KnowledgeBase, query : Array[Double], top_k : Int, metric : DistanceMetric, expression : FilterExpression) -> Array[SearchResult] raise VectorError

    Run exact search over an expression-filtered subset.

    KnowledgeBase::stats

    Return index statistics for the exact collection.

    KnowledgeBase::upsert

    fn KnowledgeBase::upsert(self : KnowledgeBase, doc : Document) -> Unit raise VectorError

    Insert or replace one document and rebuild approximate indexes.

    LshIndex

    pub struct LshIndex {
    planes : Array[Array[Double]]
    buckets : Map[Int, Array[Document]]
    num_planes : Int
    dim : Int
    }

    LshIndex::add

    fn LshIndex::add(self : LshIndex, doc : Document) -> Unit raise VectorError

    LshIndex::new

    fn LshIndex::new(num_planes : Int, dim : Int) -> LshIndex

    LshIndex::search

    fn LshIndex::search(self : LshIndex, query : Array[Double], top_k : Int, filters : Array[(String, String)]) -> Array[SearchResult] raise VectorError

    LshIndex::search_batch

    fn LshIndex::search_batch(self : LshIndex, queries : Array[Array[Double]], top_k : Int, filters : Array[(String, String)]) -> Array[Array[SearchResult]] raise VectorError

    Search a batch with the approximate LSH index.

    MetadataFilter

    pub(all) enum MetadataFilter {
    MetadataEquals(String, String)
    MetadataNotEquals(String, String)
    MetadataPrefix(String, String)
    MetadataContains(String, String)
    MetadataIn(String, Array[String])
    }

    Metadata predicates used by application-facing retrieval APIs.

    QueryPlan

    pub(all) struct QueryPlan {
    query : Array[Double]
    top_k : Int
    metric : DistanceMetric
    filters : Array[(String, String)]
    strategy : IndexStrategy
    nprobe : Int
    }

    Query options shared by application-facing search adapters.

    QueryPlan::exact

    fn QueryPlan::exact(query : Array[Double], top_k : Int, metric : DistanceMetric, filters : Array[(String, String)]) -> QueryPlan

    Create a conservative exact query plan.

    QueryPlan::ivf

    fn QueryPlan::ivf(query : Array[Double], top_k : Int, metric : DistanceMetric, filters : Array[(String, String)], nprobe : Int) -> QueryPlan

    Create an IVF plan with a bounded probe count.

    ResultPage

    pub(all) struct ResultPage {
    results : Array[SearchResult]
    offset : Int
    limit : Int
    total : Int
    has_more : Bool
    }

    A page of ranked search results for API pagination.
    impl Show for ResultPage

    RetrievalContext

    pub(all) struct RetrievalContext {
    chunks : Array[ContextChunk]
    text : String
    source_ids : Array[String]
    total_characters : Int
    }

    Prompt-ready context with source attribution.

    RetrievalOptions

    pub(all) struct RetrievalOptions {
    query : Array[Double]
    top_k : Int
    metric : DistanceMetric
    filters : Array[(String, String)]
    expression : FilterExpression
    strategy : RetrievalStrategy
    nprobe : Int
    }

    Request object shared by knowledge-base and service integrations.

    RetrievalResponse

    pub(all) struct RetrievalResponse {
    results : Array[SearchResult]
    strategy : RetrievalStrategy
    corpus_count : Int
    candidate_count : Int
    returned_count : Int
    filter_description : String
    }

    Result envelope useful for API responses and observability.

    RetrievalStrategy

    pub(all) enum RetrievalStrategy {
    RetrievalExact
    RetrievalIvf
    RetrievalKdTree
    RetrievalLsh
    }

    Index strategy selected for an application retrieval request.

    ScenarioReport

    pub(all) struct ScenarioReport {
    name : String
    documents : Int
    queries : Int
    successful_queries : Int
    average_recall : Double
    context_sources : Int
    passed : Bool
    notes : String
    }

    A reproducible application-level scenario result.

    SearchResult

    pub struct SearchResult {
    id : String
    score : Double
    metadata : Array[(String, String)]
    }

    SearchTelemetry

    pub(all) struct SearchTelemetry {
    queries : Int
    returned : Int
    filtered : Int
    empty_queries : Int
    average_results : Double
    }

    Operational counters for observing a search workload without exposing data.

    VectorCollection

    pub struct VectorCollection {
    documents : Array[Document]
    index : FlatIndex
    }

    A small mutable vector collection that owns ingestion and index rebuilding.

    VectorCollection is intended for applications that receive documents over time. It keeps the public lifecycle explicit: upsert, remove, rebuild, then search. The implementation uses FlatIndex as a correctness-first baseline; callers can export the validated corpus to IVF, KD-Tree, or LSH when scale justifies an approximate or partitioned index.

    VectorCollection::clear

    fn VectorCollection::clear(self : VectorCollection) -> Unit

    Remove every document and reset the collection.

    VectorCollection::documents

    Return a shallow document snapshot in insertion order.

    VectorCollection::get

    fn VectorCollection::get(self : VectorCollection, id : String) -> Document?

    Locate a document by id.

    VectorCollection::is_empty

    fn VectorCollection::is_empty(self : VectorCollection) -> Bool

    Return whether the collection has no documents.

    VectorCollection::length

    fn VectorCollection::length(self : VectorCollection) -> Int

    Number of documents currently stored.

    VectorCollection::new

    Create an empty collection.

    VectorCollection::rebuild

    fn VectorCollection::rebuild(self : VectorCollection) -> Unit

    Rebuild the exact baseline index from the current snapshot.

    VectorCollection::remove

    fn VectorCollection::remove(self : VectorCollection, id : String) -> Bool

    Remove a document. Returns whether an item was removed.

    VectorCollection::replace_all

    fn VectorCollection::replace_all(self : VectorCollection, docs : Array[Document]) -> Unit raise VectorError

    Replace all data after validating dimensions and ids.

    VectorCollection::search

    fn VectorCollection::search(self : VectorCollection, query : Array[Double], top_k : Int, metric : DistanceMetric, filters : Array[(String, String)]) -> Array[SearchResult] raise VectorError

    Search the current collection using the exact baseline.

    VectorCollection::search_batch

    fn VectorCollection::search_batch(self : VectorCollection, queries : Array[Array[Double]], top_k : Int, metric : DistanceMetric, filters : Array[(String, String)]) -> Array[Array[SearchResult]] raise VectorError

    Search multiple queries using the exact baseline.

    VectorCollection::stats

    Return basic index statistics for observability.

    VectorCollection::upsert

    fn VectorCollection::upsert(self : VectorCollection, doc : Document) -> Unit raise VectorError

    Add a document or replace the existing document with the same id.

    VectorSummary

    pub(all) struct VectorSummary {
    count : Int
    dimension : Int
    min_norm : Double
    max_norm : Double
    mean_norm : Double
    mean_vector : Array[Double]
    }

    Descriptive statistics for a vector corpus.

    annotate_documents

    fn annotate_documents(docs : Array[Document], key : String, value : String) -> Array[Document]

    Return a copy of a corpus with one metadata pair appended to every document.

    annotate_ordinals

    fn annotate_ordinals(docs : Array[Document], key : String) -> Array[Document]

    Add a deterministic ordinal metadata field to a corpus.

    argmax

    fn argmax(vector : Array[Double]) -> Int?

    Return the index of the largest coordinate, or no value for an empty vector.

    argmin

    fn argmin(vector : Array[Double]) -> Int?

    Return the index of the smallest coordinate, or no value for an empty vector.

    average_pair_similarity

    fn average_pair_similarity(vectors : Array[Array[Double]]) -> Double raise VectorError

    Calculate the average cosine similarity among all distinct pairs.

    average_precision

    fn average_precision(results : Array[SearchResult], relevant : Array[SearchResult]) -> Double

    Compute average precision for a ranked result list.

    benchmark_corpus

    fn benchmark_corpus() -> Array[Document]

    Build a deterministic corpus shaped like a small semantic embedding set. The four topics make metadata filtering and cluster locality observable.

    benchmark_queries

    fn benchmark_queries() -> Array[Array[Double]]

    Return queries selected from different semantic regions of the corpus.

    build_flat_index

    fn build_flat_index(docs : Array[Document]) -> FlatIndex raise VectorError

    Build a validated exact Flat index from a complete corpus.

    build_ivf_index

    fn build_ivf_index(docs : Array[Document], k : Int, metric : DistanceMetric) -> IvfIndex raise VectorError

    Build a validated IVF-Flat index.

    build_kd_tree_index

    fn build_kd_tree_index(docs : Array[Document], metric : DistanceMetric) -> KdTreeIndex raise VectorError

    Build a KD-Tree while checking corpus dimensions first.

    build_lsh_index

    fn build_lsh_index(docs : Array[Document], num_planes : Int) -> LshIndex raise VectorError

    Build an LSH index with deterministic planes.

    build_rag_context

    fn build_rag_context(docs : Array[Document], query_vector : Array[Double], query_text : String, top_k : Int, semantic_weight : Double, character_budget : Int) -> RetrievalContext raise VectorError

    Build a prompt-ready RAG context in one call.

    calculate_distance

    fn calculate_distance(v1 : Array[Double], v2 : Array[Double], metric : DistanceMetric) -> Double raise VectorError

    cap_query_plan

    fn cap_query_plan(plan : QueryPlan, maximum_k : Int) -> QueryPlan

    Return a plan with a stricter top-k bound, useful for public API limits.

    catalog_scenario_documents

    fn catalog_scenario_documents() -> Array[Document]

    Product catalog documents for filtered recommendation tests.

    center_vectors

    fn center_vectors(vectors : Array[Array[Double]]) -> Array[Array[Double]] raise VectorError

    Center each vector by subtracting the corpus mean.

    compose_retrieval_context

    fn compose_retrieval_context(results : Array[SearchResult], docs : Array[Document], character_budget : Int, separator : String) -> RetrievalContext

    Build a bounded context bundle from ranked results.

    context_citations

    fn context_citations(context : RetrievalContext) -> String

    Return a compact source citation string for a context bundle.

    coordinate_variance

    fn coordinate_variance(vectors : Array[Array[Double]]) -> Array[Double] raise VectorError

    Calculate the population variance for every coordinate.

    cosine_distance

    fn cosine_distance(a : Array[Double], b : Array[Double]) -> Double raise VectorError

    Convert cosine similarity to cosine distance in [0, 2] for finite vectors.

    cosine_similarity

    fn cosine_similarity(v1 : Array[Double], v2 : Array[Double]) -> Double raise VectorError

    count_matching_documents

    fn count_matching_documents(docs : Array[Document], expression : FilterExpression) -> Int

    Count how many records satisfy an expression.

    coverage

    fn coverage(results : Array[SearchResult], relevant : Array[SearchResult]) -> Double

    Measure how much of the relevance set appears anywhere in a result list.

    dcg_at_k

    fn dcg_at_k(results : Array[SearchResult], relevant : Array[SearchResult], k : Int) -> Double

    Compute discounted cumulative gain at k.

    deduplicate_documents

    fn deduplicate_documents(docs : Array[Document]) -> Array[Document]

    Remove duplicate ids, keeping the first document for each id.

    describe_filter

    fn describe_filter(filter : MetadataFilter) -> String

    Produce a stable human-readable description for audit logs.

    describe_filter_expression

    fn describe_filter_expression(expression : FilterExpression) -> String

    Produce a stable human-readable description for an expression.

    dimension_counts

    fn dimension_counts(docs : Array[Document]) -> Array[(Int, Int)]

    Return the distribution of vector dimensions in a possibly malformed corpus.

    distinct_document_count

    fn distinct_document_count(docs : Array[Document]) -> Int

    Return the number of distinct ids in a corpus.

    diversify_results

    fn diversify_results(results : Array[SearchResult], metadata_key : String, per_value : Int) -> Array[SearchResult]

    Keep at most one result per metadata value, useful for diversified feeds.

    document_centroid

    fn document_centroid(docs : Array[Document]) -> Array[Double] raise VectorError

    Compute a centroid from selected documents.

    document_for_result

    fn document_for_result(result : SearchResult, docs : Array[Document]) -> Document?

    Find a document for a ranked result id.

    document_norms

    fn document_norms(docs : Array[Document]) -> Array[Double] raise VectorError

    Return the vector norms in document order.

    document_text

    fn document_text(doc : Document) -> String

    Read the conventional text/content/title metadata fields from a document.

    document_vectors

    fn document_vectors(docs : Array[Document]) -> Array[Array[Double]]

    Return all vectors as independent arrays for numerical processing.

    dot_product

    fn dot_product(v1 : Array[Double], v2 : Array[Double]) -> Double raise VectorError

    euclidean_distance

    fn euclidean_distance(v1 : Array[Double], v2 : Array[Double]) -> Double raise VectorError

    evaluate_ivf

    fn evaluate_ivf(docs : Array[Document], queries : Array[Array[Double]], k : Int, nprobe : Int) -> BenchmarkReport raise VectorError

    Evaluate IVF recall against the exact Flat baseline.

    evaluate_lsh

    fn evaluate_lsh(docs : Array[Document], queries : Array[Array[Double]], k : Int) -> BenchmarkReport raise VectorError

    Evaluate deterministic LSH recall against the exact Flat baseline.

    execute_exact_plan

    fn execute_exact_plan(index : FlatIndex, plan : QueryPlan) -> Array[SearchResult] raise VectorError

    Execute a plan with the exact Flat index.

    execute_ivf_plan

    fn execute_ivf_plan(index : IvfIndex, plan : QueryPlan) -> Array[SearchResult] raise VectorError

    Execute a plan with an already-built IVF index.

    execute_kd_plan

    fn execute_kd_plan(index : KdTreeIndex, plan : QueryPlan) -> Array[SearchResult] raise VectorError

    Execute a plan with an already-built KD-Tree index.

    execute_lsh_plan

    fn execute_lsh_plan(index : LshIndex, plan : QueryPlan) -> Array[SearchResult] raise VectorError

    Execute a plan with an already-built LSH index.

    explain_strategy

    fn explain_strategy(strategy : IndexStrategy) -> String

    Explain the recommendation in user-facing language.

    filter_documents

    fn filter_documents(docs : Array[Document], filters : Array[(String, String)]) -> Array[Document]

    Return documents whose metadata contains all requested pairs.

    filter_documents_expression

    fn filter_documents_expression(docs : Array[Document], expression : FilterExpression) -> Array[Document]

    Filter documents without changing the input order or records.

    filter_id_prefix

    fn filter_id_prefix(docs : Array[Document], prefix : String) -> Array[Document]

    Return documents whose ids start with a prefix.

    filter_results_expression

    fn filter_results_expression(results : Array[SearchResult], expression : FilterExpression) -> Array[SearchResult]

    Filter ranked results after retrieval while preserving ranking order.

    filtered_recall

    fn filtered_recall(approximate : Array[SearchResult], exact : Array[SearchResult], k : Int) -> Double

    Return the fraction of approximate results that also satisfy an exact filter.

    filters_are_valid

    fn filters_are_valid(filters : Array[(String, String)]) -> Bool

    Return whether all filters are syntactically non-empty.

    find_result

    fn find_result(results : Array[SearchResult], id : String) -> SearchResult?

    Find the first result with the requested document id.

    first_result

    fn first_result(results : Array[SearchResult]) -> SearchResult?

    Select the highest-scoring result from a non-empty result list.

    higher_score_is_better

    fn higher_score_is_better(metric : DistanceMetric) -> Bool

    Return whether larger scores are better for a metric.

    hybrid_rerank

    fn hybrid_rerank(query_text : String, semantic_results : Array[SearchResult], docs : Array[Document], semantic_weight : Double) -> Array[SearchResult]

    Rerank semantic results using document text overlap.

    hybrid_score

    fn hybrid_score(semantic_score : Double, lexical_score : Double, semantic_weight : Double) -> Double

    Combine semantic and lexical relevance scores.
    fn hybrid_search(docs : Array[Document], query_vector : Array[Double], query_text : String, top_k : Int, semantic_weight : Double, filters : Array[(String, String)]) -> Array[SearchResult] raise VectorError

    Run a Flat semantic search followed by deterministic hybrid reranking.

    inspect_corpus

    fn inspect_corpus(docs : Array[Document]) -> HealthReport

    Inspect a corpus without raising, so it can be used by readiness probes.

    is_zero_vector

    fn is_zero_vector(vector : Array[Double]) -> Bool

    Return whether every coordinate is exactly zero.

    kmeans

    fn kmeans(vectors : Array[Array[Double]], k : Int, max_iters : Int) -> Array[Array[Double]] raise VectorError

    knowledge_base_scenario_documents

    fn knowledge_base_scenario_documents() -> Array[Document]

    Documents representing a small multi-tenant knowledge base.

    largest_score_gap

    fn largest_score_gap(results : Array[SearchResult]) -> Double

    Compute the largest score gap between adjacent results.

    lexical_match_count

    fn lexical_match_count(query : String, doc : Document) -> Int

    Count query terms that occur in a document's textual metadata.

    lexical_overlap

    fn lexical_overlap(query : String, doc : Document) -> Double

    Return lexical overlap in the range [0, 1].

    manhattan_distance

    fn manhattan_distance(v1 : Array[Double], v2 : Array[Double]) -> Double raise VectorError

    matches_filter_expression

    fn matches_filter_expression(metadata : Array[(String, String)], expression : FilterExpression) -> Bool

    Evaluate a composed metadata expression.

    matches_metadata_filter

    fn matches_metadata_filter(metadata : Array[(String, String)], filter : MetadataFilter) -> Bool

    Return whether one metadata predicate matches a metadata array.

    mean_average_precision

    fn mean_average_precision(approximate : Array[Array[SearchResult]], exact : Array[Array[SearchResult]]) -> Double

    Compute mean average precision for corresponding query batches.

    mean_coverage

    fn mean_coverage(approximate : Array[Array[SearchResult]], exact : Array[Array[SearchResult]]) -> Double

    Return an aggregate coverage score for corresponding query batches.

    mean_recall

    fn mean_recall(approximate : Array[Array[SearchResult]], exact : Array[Array[SearchResult]], k : Int) -> Double

    Calculate the mean recall of corresponding query batches.

    measure_filter

    fn measure_filter(docs : Array[Document], filters : Array[(String, String)]) -> FilterReport

    Measure how many documents survive a metadata filter.

    meets_recall_target

    fn meets_recall_target(approximate : Array[Array[SearchResult]], exact : Array[Array[SearchResult]], k : Int, target : Double) -> Bool

    Determine whether an index appears useful for a requested recall target.

    merge_results

    fn merge_results(lists : Array[Array[SearchResult]], metric : DistanceMetric) -> Array[SearchResult]

    Merge result lists by id, retaining the best score for each document.

    metadata_contains

    fn metadata_contains(key : String, fragment : String) -> MetadataFilter

    Construct a substring predicate.

    metadata_counts

    fn metadata_counts(docs : Array[Document], key : String) -> Array[(String, Int)]

    Count documents in each metadata category.

    metadata_coverage

    fn metadata_coverage(docs : Array[Document], key : String) -> Double

    Return the fraction of documents carrying a metadata key.

    metadata_equals

    fn metadata_equals(key : String, value : String) -> MetadataFilter

    Construct an equality predicate.

    metadata_in

    fn metadata_in(key : String, values : Array[String]) -> MetadataFilter

    Construct a membership predicate.

    metadata_keys

    fn metadata_keys(docs : Array[Document]) -> Array[String]

    Return a stable list of all metadata keys in first-seen order.

    metadata_not_equals

    fn metadata_not_equals(key : String, value : String) -> MetadataFilter

    Construct a negative equality predicate.

    metadata_prefix

    fn metadata_prefix(key : String, prefix : String) -> MetadataFilter

    Construct a string-prefix predicate.

    metadata_values

    fn metadata_values(docs : Array[Document], key : String) -> Array[String]

    Return all metadata values for a key, without duplicates.

    ndcg_at_k

    fn ndcg_at_k(results : Array[SearchResult], relevant : Array[SearchResult], k : Int) -> Double

    Compute normalized discounted cumulative gain at k.

    nearest_distance

    fn nearest_distance(query : Array[Double], docs : Array[Document], metric : DistanceMetric) -> Double? raise VectorError

    Return the nearest document distance for a query, or no value for empty data.

    normalize_documents

    fn normalize_documents(docs : Array[Document]) -> Array[Document] raise VectorError

    Normalize every document while preserving ids and metadata.

    normalize_l2

    fn normalize_l2(vector : Array[Double]) -> Array[Double] raise VectorError

    Return a copy of vector with unit L2 norm.

    normalize_scores

    fn normalize_scores(results : Array[SearchResult], metric : DistanceMetric) -> Array[SearchResult]

    Return a score-normalized copy in the range [0, 1].

    paginate_results

    fn paginate_results(results : Array[SearchResult], offset : Int, limit : Int) -> ResultPage

    Convert a ranked result list into a bounded page.

    pairwise_distances

    fn pairwise_distances(vectors : Array[Array[Double]], metric : DistanceMetric) -> Array[Array[Double]] raise VectorError

    Return a matrix of pairwise distances for small diagnostic datasets.

    precision_at_k

    fn precision_at_k(results : Array[SearchResult], relevant : Array[SearchResult], k : Int) -> Double

    Compute precision@k for an approximate ranking against a relevance set.

    query_is_compatible

    fn query_is_compatible(query : Array[Double], docs : Array[Document]) -> Bool

    Check that a query can be served by the corpus dimension.

    query_terms

    fn query_terms(query : String) -> Array[String]

    Split a query into normalized non-empty tokens.

    recall_at_k

    fn recall_at_k(approximate : Array[SearchResult], exact : Array[SearchResult], k : Int) -> Double

    Calculate recall@k between an approximate result list and an exact list.

    recall_curve

    fn recall_curve(approximate : Array[SearchResult], exact : Array[SearchResult], cutoffs : Array[Int]) -> Array[(Int, Double)]

    Count exact matches at every requested cutoff.

    recommend_strategy

    fn recommend_strategy(corpus_size : Int, dimension : Int, exact_required : Bool) -> IndexStrategy

    Recommend a strategy using simple, explainable defaults.

    relevant_ids

    fn relevant_ids(results : Array[SearchResult]) -> Map[String, Bool]

    Return ids for a relevance judgement set.

    remove_matching_documents

    fn remove_matching_documents(docs : Array[Document], filters : Array[(String, String)]) -> Array[Document]

    Remove all documents that match every filter pair.

    result_ids

    fn result_ids(results : Array[SearchResult]) -> Array[String]

    Return result ids in ranking order.

    result_scores

    fn result_scores(results : Array[SearchResult]) -> Array[Double]

    Return all result scores in ranking order for logging or charting.

    retrieval_options

    fn retrieval_options(query : Array[Double], top_k : Int, metric : DistanceMetric) -> RetrievalOptions

    Construct a default exact retrieval request.

    retrieval_with_expression

    fn retrieval_with_expression(options : RetrievalOptions, expression : FilterExpression) -> RetrievalOptions

    Return a copy of a request with a composed metadata expression.

    retrieval_with_filters

    fn retrieval_with_filters(options : RetrievalOptions, filters : Array[(String, String)]) -> RetrievalOptions

    Return a copy of a request with key/value AND filters.

    retrieval_with_strategy

    fn retrieval_with_strategy(options : RetrievalOptions, strategy : RetrievalStrategy, nprobe : Int) -> RetrievalOptions

    Return a copy of a request using an approximate strategy.

    retrieved_relevant_count

    fn retrieved_relevant_count(approximate : Array[SearchResult], relevant : Array[SearchResult]) -> Int

    Count distinct relevant documents retrieved by an approximate batch.

    run_all_application_scenarios

    fn run_all_application_scenarios() -> Array[ScenarioReport] raise VectorError

    Run all included application scenarios.

    run_batch_scenario

    fn run_batch_scenario() -> ScenarioReport raise VectorError

    Exercise batch retrieval and recall reporting for another application path.

    run_catalog_scenario

    fn run_catalog_scenario() -> ScenarioReport raise VectorError

    Exercise category and price filters in a catalog search.

    run_rag_scenario

    fn run_rag_scenario() -> ScenarioReport raise VectorError

    Exercise a tenant-aware RAG workflow with source attribution.

    run_retrieval_scenario

    fn run_retrieval_scenario(name : String, docs : Array[Document], queries : Array[Array[Double]], top_k : Int) -> ScenarioReport raise VectorError

    Run a retrieval regression scenario against the exact Flat baseline.

    sanitize_query_plan

    fn sanitize_query_plan(plan : QueryPlan) -> QueryPlan

    Return a plan with filters that match the documented empty-value policy.

    scale_documents

    fn scale_documents(docs : Array[Document], scalar : Double) -> Array[Document] raise VectorError

    Return a copy with all vectors transformed by a scalar.

    score_is_better

    fn score_is_better(candidate : Double, current : Double, metric : DistanceMetric) -> Bool

    Compare two scores according to their metric.

    search_documents

    fn search_documents(docs : Array[Document], plan : QueryPlan) -> Array[SearchResult] raise VectorError

    Build an exact index and execute a plan against a corpus.

    search_with_threshold

    fn search_with_threshold(docs : Array[Document], query : Array[Double], metric : DistanceMetric, threshold : Double) -> Array[SearchResult] raise VectorError

    Keep documents within a distance/ranking threshold.

    select_document_ids

    fn select_document_ids(docs : Array[Document], ids : Array[String]) -> Array[Document]

    Copy documents while selecting a stable subset of ids.

    set_metadata

    fn set_metadata(doc : Document, key : String, value : String) -> Document

    Apply a single metadata annotation without mutating input documents.

    slice_documents

    fn slice_documents(docs : Array[Document], start : Int, end : Int) -> Array[Document]

    Copy a corpus while retaining only documents with ids in a half-open range.

    split_documents

    fn split_documents(docs : Array[Document], prefix_count : Int) -> (Array[Document], Array[Document])

    Split a corpus into a deterministic prefix and suffix.

    squared_euclidean_distance

    fn squared_euclidean_distance(a : Array[Double], b : Array[Double]) -> Double raise VectorError

    Compute squared L2 distance without the final square root.

    summarize_vectors

    fn summarize_vectors(vectors : Array[Array[Double]]) -> VectorSummary raise VectorError

    Calculate corpus-level norms and coordinate means.

    summarize_workload

    fn summarize_workload(batches : Array[Array[SearchResult]], requested_top_k : Int) -> SearchTelemetry

    Aggregate result lists into a privacy-preserving workload summary.

    unique_results

    fn unique_results(results : Array[SearchResult]) -> Array[SearchResult]

    Remove duplicate ids while preserving the first occurrence.

    validate_documents

    fn validate_documents(docs : Array[Document]) -> Unit raise VectorError

    Validate a corpus before it is handed to an index.

    The check is deliberately strict: vector databases should reject malformed data at ingestion time instead of producing incomplete or misleading search results later.

    vector_add

    fn vector_add(a : Array[Double], b : Array[Double]) -> Array[Double] raise VectorError

    Add two vectors coordinate by coordinate.

    vector_clip

    fn vector_clip(vector : Array[Double], low : Double, high : Double) -> Array[Double]

    Clamp every coordinate to an inclusive interval.

    vector_fingerprint

    fn vector_fingerprint(vector : Array[Double]) -> String

    Return a compact deterministic fingerprint of a vector for cache keys.

    vector_mean

    fn vector_mean(vectors : Array[Array[Double]]) -> Array[Double] raise VectorError

    Compute the arithmetic mean of a non-empty vector set.

    vector_norm

    fn vector_norm(vector : Array[Double]) -> Double raise VectorError

    Return a vector's L2 norm without changing it.

    vector_scale

    fn vector_scale(vector : Array[Double], scalar : Double) -> Array[Double] raise VectorError

    Scale a vector by a scalar.

    vector_subtract

    fn vector_subtract(a : Array[Double], b : Array[Double]) -> Array[Double] raise VectorError

    Subtract one vector from another.

    vector_sum

    fn vector_sum(vectors : Array[Array[Double]]) -> Array[Double] raise VectorError

    Sum coordinates across a non-empty vector set.

    within_cluster_sse

    fn within_cluster_sse(docs : Array[Document], centroid : Array[Double]) -> Double raise VectorError

    Compute the sum of squared coordinate error around a centroid.

    workload_delta

    fn workload_delta(a : SearchTelemetry, b : SearchTelemetry) -> Double

    Compare two workloads by average result count.

    workload_is_stable

    fn workload_is_stable(report : SearchTelemetry, minimum_queries : Int) -> Bool

    Return whether a workload has enough successful queries for a stable sample.