🥮 mooncakes.io

    moonpool

    moonpool — resource pools, backoff and flow control for MoonBit. The bookkeeping of holding a limited number of things, of deciding when to try again, and of not going too fast — with the clock and the randomness passed in, so every sequence is reproducible and none of it touches I/O.

    pool
    retry
    backoff
    ratelimit
    moonbit
    Download zip
    Version
    0.2.0
    License
    Apache-2.0
    Repository
    Last updated
    5 days ago
    Downloads
    209

    Dependencies

    #moonpool

    Resource pools, backoff and flow control. Three kinds of bookkeeping that every client and every server writes, and that everyone writes slightly differently.

    // A pool that hands out what it has and says when to make or wait for more.
let pool : @moonpool.Pool[Conn] = @moonpool.Pool::new(
  limits=@moonpool.Limits::new(size=20),
)
match pool.take(now) {
  Ready(conn) => use(conn)
  Stale(conn) => close(conn) // too old to use; take again
  Make => {
    let conn = open()
    pool.made(conn, now)
    use(conn)
  }
  Wait => queue()
}
pool.give(conn, now) // or pool.drop(conn) when it broke

// A retry policy that says how long to wait, and when to stop.
let policy = @moonpool.Backoff::new(attempts=5)
if policy.more(tries) { sleep(policy.delay(tries, random=draw())) }

// A rate limit, a quota and a concurrency limit.
@moonpool.Bucket::new(100.0, burst=200.0).take(now)
@moonpool.Window::new(1000, @moondate.Span::new(days=1L)).take(now)
@moonpool.Gate::new(64).enter()

    #No I/O, and that is the point

    Nothing here opens, closes, sleeps or reads a clock. The pool says what to do with a resource and the caller does it; the backoff says how long and the caller sleeps it; the limiter says whether and the caller answers. Time is passed in as whatever the caller's own clock read, in nanoseconds.

    That is not purity for its own sake. It is what makes every sequence here reproducible: a test drives a pool through an hour in three lines, and a retry schedule with the same draws is the same schedule twice. A library that read the clock itself could only be tested by waiting.

    #The pieces

    What it answers
    Pool[T]Which resource to hand out, whether to make another, which have sat or lived long enough to let go
    BackoffHow long before the next try, and whether there should be one
    BucketWhether this request is within a rate, and how long until it would be
    WindowWhether it is within a quota — a thousand a day is a window, not a rate
    GateWhether there is room right now, which has nothing to do with time

    #The defaults, and where they come from

    Limits: ten live, ten idle, thirty minutes before an idle one is dropped, no maximum age. Ten is HikariCP's and Go's database/sql default; keeping idle equal to size is database/sql's behaviour, so a pool that grew under load does not immediately throw the connections away.

    Backoff: three attempts, 100 ms doubling to a 1 s ceiling, full jitter. Those are gRPC's documented retry defaults, and gRPC sleeps a value drawn uniformly from [0, cap], which is what full jitter is.

    The four jitter strategies are the ones AWS's Exponential Backoff and Jitter names and measures. Full is the default because that article's own measurements put it ahead; Rigid exists because a policy written elsewhere may specify no jitter, and comparing against it needs the undelayed value, which Backoff::ceiling gives.

    #A note on the newest-first order

    A pool hands out the most recently returned resource, not the oldest. A warm connection is likelier to still be good, and handing out the newest lets the oldest age past keep and be evicted — which is how a pool shrinks again after a burst. A queue would keep every connection just warm enough never to be collected.

    #A resource is never lost, and never younger than it is

    take hands an idle resource that is past its age back as Stale rather than skipping it, so the caller closes it whether or not the eviction timer has run. And the pool remembers when each lent resource was made, recognising it by identity on the way back, so give does not reset its age and life retires a connection that has been reused all along. A resource returned without having been registered by made is taken as born when it comes back.

    Both were wrong in 0.1.0: an expired resource that take came across was dropped without the caller ever seeing it, and every give stamped the resource as new.

    #One package, not three

    The plan for this repository had pool, retry and flow as separate packages. They are one, because splitting a package is worth doing when it lets a caller carry one dependency fewer, and none of the three brings a dependency the others do not. Splitting for tidiness is a ceremony; moonlog made the same call for the same reason.

    #Install

    moon add moonbitstack/moonpool

    #Licence

    Apache-2.0. See LICENSE.

    Backoff

    source
    pub(all) struct Backoff {
      base : 
    Span
      factor : Double
      cap : 
    Span
      attempts : Int
      jitter : Jitter
    } derive(Eq, 
    Debug    )

    When to try again, and how long to wait first.

    The preset is [backoff]. Build another with [Backoff::new], or update one in place with { ..backoff, attempts: 5 }.

    Backoff::ceiling

    source
    fn Backoff::ceiling(self : Backoff, retry : Int) -> 
    Span

    The undelayed ceiling before the retry-th retry, the first retry being 1: min(base × factor^(retry-1), cap).

    This is what a jitter-free client sleeps and what every other strategy draws within, so it is public: a caller comparing this library against a policy written elsewhere compares this.

    Backoff::delay

    source
    fn Backoff::delay(self : Backoff, retry : Int, random? : Double, previous? : 
    Span    ) -> 
    Span

    How long to wait before the retry-th retry.

    random is a value in [0, 1) that the caller draws; passing the same one twice gives the same delay twice, which is what makes a retry sequence testable. previous is what the last delay turned out to be, which only Decorrelated reads — it grows from where it landed rather than from the attempt number.

    Backoff::equal

    source
    fn Backoff::equal(Backoff, Backoff) -> Bool

    Backoff::more

    source
    fn Backoff::more(self : Backoff, made : Int) -> Bool

    Whether a call that has made made attempts may make another.

    The first attempt is 1, so a policy of three attempts allows two retries.

    Backoff::new

    source
    fn Backoff::new(base? : 
    Span    , factor? : Double, cap? : 
    Span    , attempts? : Int, jitter? : Jitter) -> Backoff

    A backoff by name, every knob with the preset's value.

    Backoff::not_equal

    source
    fn Backoff::not_equal(x : Backoff, y : Backoff) -> Bool

    Backoff::schedule

    source
    fn Backoff::schedule(self : Backoff, randoms? : ArrayView[Double]) -> Array[
    Span    ]

    Every delay a run of retries would wait, for a caller that wants the whole sequence rather than one step — a test, or a log line explaining a plan.

    randoms supplies the draws; when it runs short the remaining steps use zero, which is the low end of whatever strategy is in force.

    Backoff::to_repr

    source

    Bucket

    source
    pub struct Bucket {
      rate : Double
      burst : Double
      tokens : Double
      at : Int64
    }

    A token bucket: rate tokens accrue per second up to burst, and each request takes one.

    It is the rate limiter that allows a burst after a quiet period, which is what a limit on a human-facing API wants — nobody should be refused for making two requests in the same second after an hour of making none.

    Time is nanoseconds from the caller's clock, as everywhere in this package.

    Bucket::new

    source
    fn Bucket::new(rate : Double, burst? : Double, at? : Int64) -> Bucket

    A bucket that starts full, which is what lets the first burst through.

    rate is tokens per second and burst is the most it will ever hold. A burst smaller than one means nothing ever passes, so it is raised to one.

    Bucket::next

    source
    fn Bucket::next(self : Bucket, now : Int64, n? : Double) -> 
    Span

    How long until n tokens would be available, without taking them.

    Zero when they are available now. This is what a caller sleeps for rather than spinning, and what a Retry-After header is computed from.

    Bucket::take

    source
    fn Bucket::take(self : Bucket, now : Int64, n? : Double) -> Bool

    Whether n tokens are available now, taking them if they are.

    Bucket::tokens

    source
    fn Bucket::tokens(self : Bucket, now : Int64) -> Double

    How many tokens are in the bucket, for a caller that reports its own headroom.

    Gate

    source
    pub struct Gate {
      limit : Int
      held : Int
    }

    A count with a ceiling: the concurrency limit a server puts on itself.

    Unlike a rate, this does not depend on time at all — it is how many things are happening at once. A server past its limit answers rather than queues, which is what keeps a queue from becoming the outage.

    Gate::enter

    source
    fn Gate::enter(self : Gate) -> Bool

    Whether there is room, taking it if there is.

    Gate::held

    source
    fn Gate::held(self : Gate) -> Int

    How many are inside.

    Gate::leave

    source
    fn Gate::leave(self : Gate) -> Unit

    Give the room back.

    Gate::new

    source
    fn Gate::new(limit : Int) -> Gate

    A gate that nothing has entered.

    Jitter

    source
    pub(all) enum Jitter {
      Rigid
      Full
      Equal
      Decorrelated
    } derive(Eq, 
    Debug    )

    How much randomness goes into a delay.

    A retry with no jitter makes every client that failed together try again together, which is how one outage becomes a second one. The three named strategies are the ones AWS's "Exponential Backoff and Jitter" measures:

    Rigidthe computed delay, unchanged. Reproducible, and it synchronises clients
    Fulluniform over [0, cap]. The one that spreads clients furthest, and what gRPC sleeps
    Equalhalf the cap plus uniform over [0, cap/2]. Keeps a floor under the wait
    Decorrelateduniform over [base, previous × 3]. Grows from where it last landed rather than from the attempt number

    Jitter::equal

    source
    fn Jitter::equal(Jitter, Jitter) -> Bool

    Jitter::not_equal

    source
    fn Jitter::not_equal(x : Jitter, y : Jitter) -> Bool

    Jitter::to_repr

    source

    Limits

    source
    pub(all) struct Limits {
      size : Int
      idle : Int
      keep : 
    Span
      life : 
    Span
    } derive(Eq, 
    Debug    )

    The limits a pool holds itself to.

    The preset is [limits]. Build another with [Limits::new], or update one in place with { ..limits, size: 32 }.

    Limits::equal

    source
    fn Limits::equal(Limits, Limits) -> Bool

    Limits::new

    source
    fn Limits::new(size? : Int, idle? : Int, keep? : 
    Span    , life? : 
    Span    ) -> Limits

    Limits by name, every knob with the preset's value.

    Limits::not_equal

    source
    fn Limits::not_equal(x : Limits, y : Limits) -> Bool

    Limits::to_repr

    source

    Pool

    source
    pub struct Pool[T] {
      limits : Limits
      free : Array[(T, Int64, Int64)]
      out : Array[(T, Int64)]
      reserved : Int
    }

    The bookkeeping of holding a limited number of things.

    It does not open, close, or wait — a caller with a runtime does all three. What it knows is how many are live, which are idle, which are lent and since when each was made, and which have sat or lived long enough to be let go.

    Time is passed in as whatever the caller's clock reads, in nanoseconds; the value's origin does not matter because only differences are used. That is what makes a test drive a pool through an hour in three lines.

    A lent resource is recognised on its way back by identity, which is what every connection handle has. That is how its birth survives the loan: a pool that stamped a returned resource as new would never retire one by age, however old.

    Pool::busy

    source
    fn[T] Pool::busy(self : Pool[T]) -> Int

    How many are lent out or being made.

    Pool::close

    source
    fn[T] Pool::close(self : Pool[T]) -> Array[T]

    Everything the pool holds idle, emptied out for the caller to close — what a shutdown does. Lent resources are the borrowers' to return; each comes back through give and can be closed then.

    Pool::drop

    source
    fn[T] Pool::drop(self : Pool[T], item : T) -> Unit

    Say a lent resource is not coming back, because it broke.

    This is what keeps the live count honest: without it a failed connection would hold a slot for ever and the pool would stop making new ones.

    Pool::evict

    source
    fn[T] Pool::evict(self : Pool[T], now : Int64) -> Array[T]

    Every idle resource that has sat longer than keep or lived longer than life, removed from the pool and handed back for the caller to close.

    Called on a timer by whoever owns the runtime. take hands a stale one back too, so a resource is never lost for want of the timer having run.

    Pool::give

    source
    fn[T] Pool::give(self : Pool[T], item : T, now : Int64) -> Bool

    Hand one back, keeping the birth it was lent with.

    false means the pool did not keep it and the caller should close it: there were already enough idle. A resource that is broken goes to [Pool::drop] instead, because a pool that kept it would hand it to the next caller.

    A resource the pool does not recognise is one made after a Make without [Pool::made] being called: it fills the room take reserved and is taken as born now.

    Pool::idle

    source
    fn[T] Pool::idle(self : Pool[T]) -> Int

    How many are sitting idle.

    Pool::live

    source
    fn[T] Pool::live(self : Pool[T]) -> Int

    How many resources exist right now: idle, lent, and being made.

    Pool::made

    source
    fn[T] Pool::made(self : Pool[T], item : T, now : Int64) -> Unit

    Register a resource made after take said Make, as lent and born now.

    Pool::new

    source
    fn[T] Pool::new(limits? : Limits) -> Pool[T]

    An empty pool.

    Pool::take

    source
    fn[T] Pool::take(self : Pool[T], now : Int64) -> Taken[T]

    Take one, or be told to make one, or to wait — or be handed one that is too old to use.

    The most recently returned resource is handed out first. That is not arbitrary: a warm connection is likelier to still be good than a cold one, and handing out the newest lets the oldest age past keep and be evicted, which is how a pool shrinks after a burst.

    Stale is an idle resource past its age. It is already out of the pool and out of the count; the caller closes it and takes again. Handing it back rather than dropping it is the point — a pool that discarded it silently would leak whatever it holds whenever the eviction timer had not run first.

    Pool::unmade

    source
    fn[T] Pool::unmade(self : Pool[T]) -> Unit

    Say the resource take reserved room for could not be made, so the room is free again. Without it a factory that failed once would shrink the pool for good.

    Taken

    source
    pub(all) enum Taken[T] {
      Ready(T)
      Stale(T)
      Make
      Wait
    }

    What a pool answers when asked for a resource.

    It never blocks and never opens anything: waiting and connecting both need a runtime, and this package has none. Make is the pool saying there is room for another, and reserving it; Wait is it saying there is not, and the caller's queue decides what that means. Stale is an idle resource too old to use, handed back so the caller can close it before asking again.

    Window

    source
    pub struct Window {
      allow : Int
      window : 
    Span
      stamps : Array[Int64]
    }

    A sliding window counter: at most allow events in any window long stretch.

    Where a bucket smooths, a window counts. A quota of "a thousand a day" is a window, not a rate, and answering it with a bucket would let a client spend the whole day's allowance in the first minute.

    The stamps of the events in the window are kept, so the count is exact rather than the approximation a two-bucket scheme gives.

    Window::left

    source
    fn Window::left(self : Window, now : Int64) -> Int

    How many events remain in the current window.

    Window::new

    source
    fn Window::new(allow : Int, window : 
    Span    ) -> Window

    An empty window.

    Window::next

    source
    fn Window::next(self : Window, now : Int64) -> 
    Span

    When the window will next have room, which is when its oldest event falls out.

    Window::take

    source
    fn Window::take(self : Window, now : Int64) -> Bool

    Whether an event is within the quota now, counting it if it is.

    backoff

    source
    let backoff : Backoff

    The preset: three attempts, a hundred milliseconds doubling to a one-second ceiling, spread by full jitter.

    Those are gRPC's documented defaults for a retry policy — maxAttempts 3, initialBackoff 0.1s, maxBackoff 1s, backoffMultiplier 2 — and gRPC sleeps a value drawn uniformly from [0, cap], which is full jitter.

    forever

    source

    A span meaning "no limit", which is what zero is taken to mean for the two ages: a resource with no maximum age is never too old.

    limits

    source
    let limits : Limits

    The preset: ten live at once, ten of them allowed to sit idle, discarded after thirty minutes idle and never by age.

    Ten is HikariCP's and database/sql's default pool size, and thirty minutes is database/sql's SetConnMaxIdleTime in the shape most deployments set it. Keeping idle equal to size is database/sql's behaviour too, so a pool that grew under load does not immediately throw the connections away.

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io