🥮 mooncakes.io

    moonkart

    A cache for MoonBit: bounded by count or weight, with expiry, loaders and eviction listeners.

    cache
    caching
    lru
    tinylfu
    w-tinylfu
    s3fifo
    arc
    s3-fifo
    trace
    benchmark
    hit-rate
    eviction-policy
    ttl
    tti
    expiry
    eviction
    loader
    concurrent
    disk-cache
    caffeine
    moka
    moonbit
    Download zip
    Version
    0.1.0
    License
    Apache-2.0
    Repository
    Last updated
    15 hours ago
    Downloads
    9

    #moonkart

    A cache for MoonBit. It is bounded by entry count or by total weight, it can expire entries on a schedule you control, it computes missing values on demand, and it tells you what left the cache and why. The eviction strategy is erased behind an open contract, so every constructor returns the same Cache[K, V] type and a third-party policy plugs in without changing it.

    let cache = @moonkart.lru_cache(cap=1000)
    cache.set("answer", 42)

    match cache.get("answer") {
      Some(value) => println("hit: \{value}")
      None => println("miss")
    }

    #Install

    moon add moonbitstack/moonkart

    Then import it in a package:

    import {
      "moonbitstack/moonkart",
    }

    #Choosing an eviction policy

    Every constructor returns a Cache[K, V]. The policy decides which entry leaves when the cache is full; it is hidden behind the Policy contract, so the cache type does not change when you switch strategies.

    ConstructorPolicyGood at
    tinylfu_cacheW-TinyLFU (frequency + recency)general workloads; keeps frequently used entries
    lru_cacheleast recently usedsimple, predictable recency
    s3fifo_cachesecond-chance FIFOscan floods and one-hit wonders

    tinylfu_cache, lru_cache and s3fifo_cache share the same knobs; use new(policy, ...) with your own policy if none of these fit.

    #Measured hit rates

    src/bench replays a trace of keys through each policy and reports the hit rate, so a change to a policy can be judged against real access patterns rather than intuition. Numbers below come from the three ARC traces (the request streams published with the ARC paper), replaying every request:

    TraceCapacitylru_caches3fifo_cachetinylfu_cache
    OLTP (914,145 requests)1,8690.41640.42190.4269
    DS1 (2,505,947 requests)101,7640.10870.11090.2411
    S3 (16.4 million requests)524,2880.2526n/a0.4640

    Two checks make these comparable with published results. First, lru_cache on S3 at 524,288 entries returns 0.25258, reproducing the 25.26% that the ARC paper reports for the same trace at the same capacity, which validates both the replay harness and the LRU implementation. Second, the ordering matches the published conclusions: frequency-aware admission wins by a wide margin on the scan-heavy S3 trace, holds a similar lead on DS1, and only edges ahead on OLTP, where recency dominates and frequency has little to add.

    Replay of the full traces is an offline measurement, not a test; CI replays only the first forty thousand requests of OLTP so it needs no downloaded data.

    The frequency sketch is keyed by the runtime's hash seed, so a different seed reshuffles its collisions and moves a TinyLFU figure by a few tenths of a point; the recency policies are unaffected.

    #Capacity

    Bound the cache by entry count with cap, or by total weight with max_weight plus a weigher:

    // At most 500 entries.
    let by_count = @moonkart.lru_cache(cap=500)

    // At most 10 MiB, where the weight is the byte length of the value.
    let by_weight = @moonkart.lru_cache(
      max_weight=10L * 1024L * 1024L,
      weigher=fn(_key, bytes) { bytes.length().to_int64() },
    )

    Without a weigher every entry weighs one, so cap and max_weight mean the same thing.

    #Expiry

    Expiry deadlines are absolute nanosecond timestamps produced by a clock you supply. The cache never reads the wall clock itself, so tests are deterministic and do not sleep.

    // Time-to-live from the moment of writing; reads do not extend it.
    let ttl_cache = @moonkart.lru_cache(cap=1000, expiry=@moonkart.ttl(60_000_000_000L))

    // Time-to-idle: every read restarts the countdown.
    let tti_cache = @moonkart.lru_cache(cap=1000, expiry=@moonkart.tti(30_000_000_000L))

    ttl and tti are shorthands over the Expiry record, whose after_create, after_read and after_update callbacks compute per-entry deadlines. Pass your own clock to move time under test:

    let now : @ref.Ref[Int64] = @ref.Ref(0L)
    let cache = @moonkart.lru_cache(cap=10, expiry=@moonkart.ttl(1_000_000_000L), clock=fn() { now.val })

    #Loading

    get_with runs a loader only on a miss and stores its result. The loader may raise; a raised error is not cached and reaches the caller. try_get_with lets the loader return None for "no value", which is not cached either.

    let user = cache.get_with(user_id, fn(id) raise { fetch_user(id) })

    When the loader cannot fail, wrap it with infallible_loader so the call site needs no error handling:

    let parsed = cache.get_with(text, @moonkart.infallible_loader(fn(t) { parse(t) }))

    #Remembering a function

    #kart.cached is functools.lru_cache for MoonBit. The function stays as it is, and the generator adds a remembering one beside it:

    #kart.cached(capacity="1024", policy="lru", ttl="60s")
    pub fn lookup(id : Int) -> String {
      expensive(id)
    }

    lookup_cached(id) reads through the cache and lookup_cached_clear() empties it. The parameters are the key, several of them a tuple. capacity is 128 when nothing says, policy is tinylfu, lru or s3fifo, ttl and tti are durations such as 250ms, 60s, 5m or 1h, and name renames the function. An async fn goes through the asynchronous cache. A function that raises is refused, since there is nothing to keep when it does.

    The generator wires in as a dev_build rule, in moon.mod:

    rule(name: "cached", command: "moonx moonbitstack/moonkart/cmd/cached -C \"$mod_dir\" \"$input\" -o \"$output\"")

    and in the moon.pkg of the package, which imports moonbitstack/moonkart and moonbitstack/moonkart/policy:

    dev_build(rule: "cached", input: "slow.mbt", output: "slow.g.mbt")
formatter(ignore: [ "slow.g.mbt" ])

    moon run src/examples/03-cached shows it.

    #The asynchronous cache

    @moonkart.concurrent.AsyncCache wraps any cache so that loads can be awaited. Concurrent readers of one key share a single loader; readers of different keys never wait on each other, and no lock is held while a loader runs. It is built on moonbitlang/async, so it shares the runtime your other tasks already use.

    import {
      "moonbitstack/moonkart",
      "moonbitstack/moonkart/concurrent",
    }

    let cache = @concurrent.AsyncCache::new(@moonkart.lru_cache(cap=1000))
    let user = cache.get_with(id, async fn(id) { fetch_user(id) })

    The first task to miss records a per-key in-flight load and awaits it; later tasks asking for the same key await the same result. A loader that raises delivers its error to every waiter and leaves nothing in the cache, so the next call retries. Three concurrent readers of one key call the loader exactly once. concurrent builds on the synchronous core unchanged, so every policy, expiry rule and counter carries over.

    #Hybrid memory and disk (native)

    @moonkart.disk.DiskCache keeps entries in memory and appends every write or delete to a file, so a restart can rebuild the cache from the log. Keys and values are turned into bytes by codecs you supply, so the layer imposes no format of its own.

    import {
      "moonbitstack/moonkart",
      "moonbitstack/moonkart/disk",
    }

    let cache = @moonkart.lru_cache(cap=10_000)
    let disk = @disk.DiskCache::open(
      cache, "/var/cache/app.bin",
      encode_key, decode_key, encode_value, decode_value,
    )
    disk.set(key, value)   // memory, then an appended record
    disk.get(key)          // memory only
    disk.compact()         // rewrite the log from the live entries

    The filesystem in the standard library is asynchronous and native-only, so this package is too. It is target-gated: the other backends compile unaffected, and moon check --target all stays green. Restored entries are replayed through set, so the cache's current capacity and expiry policy apply to them; call compact periodically so the append log does not grow without bound.

    #Invalidation

    cache.invalidate(key)          // one key, returns whether it was present
    cache.invalidate_all()         // everything
    cache.invalidate_if(fn(k, v) { v.is_stale() })   // by predicate, returns how many

    #Events and statistics

    A removal listener is told about every entry that leaves, with the reason: Capacity, Expired, Explicit or Replaced.

    let cache = @moonkart.lru_cache(
      cap=1000,
      removal_listener=fn(key, _value, reason) {
        println("\{key} left: \{reason}")
      },
    )

    cache.stats() returns counters for hits, misses, evictions, evicted weight, successful and failed loads, and the total time spent in loaders.

    #Custom policies

    @moonkart.policy.Policy is an open trait. A policy sees opaque numeric entry ids, weights and the key's hash, never the keys or values. Implement hit, admit, forget and size for your own type and build a cache with new:

    let cache = @moonkart.new(@my_policy.Policy::new())

    #API

    ItemMeaning
    lru_cache, s3fifo_cache, tinylfu_cacheconstructors for the shipped policies
    new(policy, ...)construct with any policy
    new_with_config(policy, config)construct from a Config bundle
    default_config()defaults for every optional knob
    ttl(ns), tti(ns)expiry shorthands
    system_clockthe default clock, from the runtime
    set(key, value)insert or replace
    get(key)read, or None
    contains(key)presence; no hit, no idle refresh
    entry(key), entry_or_null(key)snapshot of an entry
    remove(key), invalidate(key)remove one, returns whether it was present
    invalidate_all(), invalidate_if(pred)remove many
    get_with(key, loader), try_get_with(key, loader)load on miss; the loader may raise
    infallible_loader(loader)adapt a loader that cannot fail
    stats(), size(), policy_size()counters and sizes

    #Upgrading from the 0.1.0 pre-release

    The module has not been published; these are the changes to make when moving code written against the first source drop.

    BeforeAfter
    Cache[K, V, P]Cache[K, V]; the policy is chosen at construction and erased
    cache[Int, String, @policy.Lru]()@moonkart.lru_cache(cap=...)
    get_with(key, fn(k) { ... })get_with(key, fn(k) raise { ... })
    a loader that cannot failwrap it: infallible_loader(fn(k) { ... })
    @policy.Policy::hit(id) / admit(id, w)hit(id, hash) / admit(id, w, hash)

    #Determinism

    Because the clock is a parameter and every deadline is absolute, an expiry test moves a Ref and never sleeps. See src/cache_test.mbt for the suite, and src/examples/ for runnable programs:

    moon run src/examples/tour
moon run src/examples/01-expiry
moon run src/examples/02-loading

    #The gate

    Every commit passes this, with each exit code seen to be zero:

    moon clean && moon fmt && moon check --target all --deny-warn \
  && moon build --target all && moon test --target all

    The generated interface is checked in; moon info --target all && git diff--exit-code must be clean before a release.

    #Licence

    Apache 2.0.

    Clock

    source
    type Clock = () -> Int64

    A clock returns the current time in nanoseconds. The cache never reads the wall clock itself; every timestamp it compares comes from a clock the caller supplies, so expiry and load timing stay deterministic under test.

    RemovalListener

    source
    type RemovalListener[K, V] = (K, V, EvictionReason) -> Unit

    Receives every entry the cache removes, together with the reason it left.

    Weigher

    source
    type Weigher[K, V] = (K, V) -> Int64

    Weighs an entry so the cache can bound total weight instead of entry count. Return a positive number; the cache treats the result as the entry's weight and evicts by weight when the policy's maximum weight is set.

    Cache

    source
    pub struct Cache[K, V] {
      // private fields
    }

    A bounded cache generic over key K and value V. The eviction strategy is erased behind the open Policy contract, so one Cache type serves every strategy; [new] accepts a third-party policy without changing this type.

    The default strategy, chosen by [tinylfu_cache], is W-TinyLFU; [lru_cache] and [s3fifo_cache] provide the other shipped strategies.

    Cache::contains

    source
    fn[K : Eq + Hash, V] Cache::contains(self : Cache[K, V], key : K) -> Bool

    Whether key is present and not expired. Expired entries are purged and report as absent. This is a presence check: it neither refreshes a time-to-idle deadline nor counts a hit.

    Cache::entry

    source
    fn[K : Eq + Hash, V] Cache::entry(self : Cache[K, V], key : K) -> Entry[K, V]

    Returns a snapshot of the entry for key. Panics if the key is absent or expired; prefer entry_or_null when absence is expected.

    Cache::entry_or_null

    source
    fn[K : Eq + Hash, V] Cache::entry_or_null(self : Cache[K, V], key : K) -> Entry[K, V]?

    Returns a snapshot of the entry for key, or None. The snapshot is a copy detached from the cache.

    Cache::get

    source
    fn[K : Eq + Hash, V] Cache::get(self : Cache[K, V], key : K) -> V?

    Returns the value for key, or None if absent or expired. A successful read counts a hit and bumps the entry's recency; an expired or missing read counts a miss.

    Cache::get_with

    source
    fn[K : Eq + Hash, V, E : Error] Cache::get_with(self : Cache[K, V], key : K, loader : (K) -> V raise E) -> V raise E

    Returns the value for key. On a miss, runs loader(key) and stores the result. The loader may raise; a raised error is not cached, is counted as a load failure, and is handed to the caller unchanged.

    Cache::invalidate

    source
    fn[K : Eq + Hash, V] Cache::invalidate(self : Cache[K, V], key : K) -> Bool

    Removes key explicitly (alias of remove). Returns whether it was present.

    Cache::invalidate_all

    source
    fn[K : Eq + Hash, V] Cache::invalidate_all(self : Cache[K, V]) -> Unit

    Removes every entry, reporting each to the removal listener as Explicit.

    Cache::invalidate_if

    source
    fn[K : Eq + Hash, V] Cache::invalidate_if(self : Cache[K, V], pred : (K, V) -> Bool) -> Int64

    Removes every entry for which pred(key, value) is true. Returns the number removed.

    Cache::policy_size

    source
    fn[K, V] Cache::policy_size(self : Cache[K, V]) -> Int

    Number of entries the eviction policy currently tracks (equals size).

    Cache::remove

    source
    fn[K : Eq + Hash, V] Cache::remove(self : Cache[K, V], key : K) -> Bool

    Removes key explicitly. Returns whether it was present.

    Cache::set

    source
    fn[K : Eq + Hash, V] Cache::set(self : Cache[K, V], key : K, value : V) -> Unit

    Inserts or replaces key. A replaced value is reported to the removal listener as Replaced; an insertion calls after_update when the key was already present, otherwise after_create.

    Cache::size

    source
    fn[K, V] Cache::size(self : Cache[K, V]) -> Int64

    Number of entries currently stored.

    Cache::stats

    source
    fn[K, V] Cache::stats(self : Cache[K, V]) -> CacheStats

    A copy of the running counters.

    Cache::to_array

    source
    fn[K, V] Cache::to_array(self : Cache[K, V]) -> Array[(K, V)]

    A snapshot of every live entry as (key, value) pairs, in no particular order. Useful for persistence and for tests.

    Cache::try_get_with

    source
    fn[K : Eq + Hash, V, E : Error] Cache::try_get_with(self : Cache[K, V], key : K, loader : (K) -> V? raise E) -> V? raise E

    Like get_with, but loader may return None to signal "no value"; such a result is not cached and None is returned. A raised error propagates to the caller and is counted as a load failure.

    CacheStats

    source
    pub(all) struct CacheStats {
      hit_count : Int64
      miss_count : Int64
      eviction_count : Int64
      eviction_weight : Int64
      load_success_count : Int64
      load_failure_count : Int64
      total_load_time : Int64
    }

    Counters the cache keeps while it runs. The raw counts are exposed so callers can aggregate them; hit_rate and miss_rate are derived.

    CacheStats::hit_rate

    source
    fn CacheStats::hit_rate(self : CacheStats) -> Double

    Fraction of reads that hit, in 0.0 ..= 1.0. Zero reads reports 0.0 rather than dividing by zero.

    CacheStats::miss_rate

    source
    fn CacheStats::miss_rate(self : CacheStats) -> Double

    Fraction of reads that missed, in 0.0 ..= 1.0.

    Config

    source
    pub struct Config[K, V] {
      weigher : (K, V) -> Int64?
      clock : () -> Int64
      expiry : Expiry[K, V]?
      removal_listener : (K, V, EvictionReason) -> Unit?
    }

    The cross-cutting knobs of a cache, bundled so they can be built once and reused across caches. Capacity lives on the eviction policy, not here.

    Fields are public so a caller can start from [default_config] and change one knob, or build a record by hand.

    Entry

    source
    pub(all) struct Entry[K, V] {
      key : K
      value : V
      weight : Int64
      expire_at : Int64?
    }

    An entry as seen by a reader: its value, weight and absolute expiry deadline (if any).

    EvictionReason

    source
    pub(all) enum EvictionReason {
      Capacity
      Expired
      Explicit
      Replaced
    }

    Why an entry left the cache.

    Expiry

    source
    pub struct Expiry[K, V] {
      after_create : (K, V, Int64) -> Int64?
      after_read : (K, V, Int64, Int64?) -> Int64?
      after_update : (K, V, Int64, Int64?) -> Int64?
    }

    Per-entry lifetime policy. Each callback returns an absolute nanosecond deadline, or None to leave the existing deadline untouched (when the callback does not apply) or to let the entry live until capacity eviction (on create). after_read and after_update receive the current deadline so they can extend or keep it.

    A pure time-to-live from creation uses ttl; a time-to-idle (idle) lifetime uses tti; anything else is expressed with the three callbacks directly.

    default_config

    source
    fn[K, V] default_config() -> Config[K, V]

    A config with every knob at its default: no weigher (each entry weighs one), the system clock, no expiry, no listener.

    infallible_loader

    source
    fn[K, V] infallible_loader(loader : (K) -> V) -> ((K) -> V raise)

    Adapts an infallible loader to the raising loader type [get_with] and [try_get_with] accept. Use it when producing a value cannot fail, so the call site needs no error handling while still sharing one read path.

    lru_cache

    source
    fn[K : Eq + Hash, V] lru_cache(cap? : Int64, max_weight? : Int64, weigher? : (K, V) -> Int64, clock? : () -> Int64, expiry? : Expiry[K, V], removal_listener? : (K, V, EvictionReason) -> Unit) -> Cache[K, V]

    A cache that evicts least-recently-used entries.

    new

    source
    fn[K : Eq + Hash, V] new(policy : &
    Policy    , weigher? : (K, V) -> Int64, clock? : () -> Int64, expiry? : Expiry[K, V], removal_listener? : (K, V, EvictionReason) -> Unit) -> Cache[K, V]

    Builds a cache from a policy and the individual knobs. Capacity belongs to the policy: pass cap or max_weight to the policy preset, not here.

    new_with_config

    source
    fn[K : Eq + Hash, V] new_with_config(policy : &
    Policy    , config : Config[K, V]) -> Cache[K, V]

    Builds a cache from a policy and a [Config] bundle.

    s3fifo_cache

    source
    fn[K : Eq + Hash, V] s3fifo_cache(cap? : Int64, max_weight? : Int64, weigher? : (K, V) -> Int64, clock? : () -> Int64, expiry? : Expiry[K, V], removal_listener? : (K, V, EvictionReason) -> Unit) -> Cache[K, V]

    A cache using the S3-FIFO-inspired second-chance FIFO: entries that were read since they entered get another round before eviction, which resists scan and one-hit-wonder patterns.

    system_clock

    source
    fn system_clock() -> Int64

    The default clock, built from the runtime clock in milliseconds. Callers may pass their own clock to make expiry deterministic.

    tinylfu_cache

    source
    fn[K : Eq + Hash, V] tinylfu_cache(cap? : Int64, max_weight? : Int64, window_ratio? : Double, protected_ratio? : Double, weigher? : (K, V) -> Int64, clock? : () -> Int64, expiry? : Expiry[K, V], removal_listener? : (K, V, EvictionReason) -> Unit) -> Cache[K, V]

    A cache using W-TinyLFU (the default): a small admission window feeds a frequency-guarded segmented LRU, so frequently used entries survive even after they age out of recency.

    tti

    source
    fn[K, V] tti(duration : Int64) -> Expiry[K, V]

    Time-to-idle: the entry expires duration nanoseconds after its most recent read or write.

    ttl

    source
    fn[K, V] ttl(duration : Int64) -> Expiry[K, V]

    Time-to-live measured from creation: the entry expires duration nanoseconds after it is written and is not refreshed by reads.

    Source Files

    Powered by MoonBit

    Site sourceReport issueModulesSkillsCLIBuild queueStatistics

    © 2026 mooncakes.io