🥮 mooncakes.io

    moonward

    Resilience for MoonBit without I/O: circuit breaking, load shedding, rate limits and deadlines.

    resilience
    fault-tolerance
    circuit-breaker
    rate-limit
    rate-limiter
    token-bucket
    sliding-window
    load-shedding
    backoff
    retry
    deadline
    timeout
    throttling
    moonbit
    Download zip
    Version
    0.2.1
    License
    Apache-2.0
    Repository
    Last updated
    2 days ago
    Downloads
    1K

    moonward

    #What it is

    Nine types that answer one question each: should this call go ahead.

    AnswersRefuses when
    BreakerIs this backend likely to work?a draw falls under a refusal fraction built from the last ten seconds of outcomes
    ShedderShould this server take on another call?the CPU is over its threshold and more calls are in flight than the server has shown it can finish
    BucketIs this call within a rate?the token is spent; next says when the next one lands
    SlidingIs it within a quota over any window-long stretch?the count reaches the quota
    PeriodIs it within a quota this period?the count for the open period is spent
    GateIs there room right now?held has reached limit — it never asks what time it is
    BackoffWhen to try again, and how long first?more(attempts) is false
    DeadlineHow much of the budget is left?expired is true
    WindowWhat does the rolling window hold?never; it is the array of buckets the others read

    #No clock, no dice

    Nothing here reads a clock or draws a random number. now is whatever your clock read, in nanoseconds; where a decision is a roll, you pass a draw from [0, 1). That is what makes an hour of traffic a loop over now, and a retry sequence testable.

    let breaker = @moonward.Breaker::new()
    let now = 0L
    match breaker.allow(now, 0.42) {
      Some(call) => call.pass(now)   // or call.fail(now)
      None => println("refused")
    }

    #The breaker

    There is no open, half-open and closed. The breaker keeps ten seconds of outcomes and refuses each call with a probability that grows with the failures in it, so a struggling backend keeps receiving what it can still serve instead of being cut off and then flooded on recovery.

    w     = max( k − (k − floor) · failing / size , floor )
drop  = ( total − protection − w · accepts ) / ( total + 1 )
drop ×= ( size − working ) / size

    failing and working are runs at the head of the window, not totals. A success pays for k calls — at the preset's 1.5, a backend is left alone until more than a third of its calls fail.

    #The shedder

    A call is refused only when the server is both hot and full:

    • hot — the CPU is at or over the threshold, or was within the last second and the shedder has refused since;
    • full — both the moving average and the current count of calls in flight exceed the busiest bucket's passes per millisecond times the fastest bucket's mean response time, scaled down as the CPU nears 100% but never under a tenth.

    A CPU spike with nothing queued refuses nothing, and neither does a long queue on an idle CPU. CPU use is read by you, in thousandths, and passed to each allow.

    #A run

    moon run src/examples/limiters offers one burst of forty calls every two seconds to a bucket, a sliding window and a period, all with a quota of twenty a second. The burst straddles the second boundary:

    burst  bucket  sliding  period
    0      21       20      40
    1      21       20      40
    2      21       20      40
    3      21       20      40

five tokens in 200 ms

    The bucket spends a saved burst and then holds the rate. The slide refuses the twenty-first call in any one-second stretch. The period starts a fresh twenty when the second turns over, so the same burst is counted as twenty on each side and all forty pass.

    #Presets

    Each policy is a record with a preset of the same name. Copy it and change one field — { ..sre, k: 2.0 } — or build it with Sre::new(k=2.0). There are no per-call overrides.

    PresetValues
    srek 1.5 decaying to 1.1, protection 5, a probe after 1 s, 40 buckets over 10 s
    shed50 buckets over 5 s, CPU threshold 900‰, 1 s assumed response time, average weighted 0.9, 1 s to cool, never under 0.1 of capacity
    backoff3 attempts, 100 ms doubling to a 1 s cap, each wait spread by a fifth either way

    Durations are @moondate.Span, so the package brings moondate with it.

    #Examples

    ProgramWhat it shows
    src/examples/minimalone breaker, one failing backend, four lines of output — the smallest thing that works
    src/examples/limitersthe same bursty traffic through all three quotas
    src/examples/toura breaker under an outage, a shedder under rising load, one step through each limiter
    src/examples/guidethe guide as one program, in the order you read it

    The bay runs the same code compiled to JavaScript, with the knobs on the outside.

    #Licence

    Apache-2.0. See LICENSE.

    Backoff

    source
    pub(all) struct Backoff {
      base : 
    Span
      factor : Double
      cap : 
    Span
      attempts : Int
      jitter : Jitter
    } derive(Eq, 
    Debug    )

    When to try again, and how long to wait first.

    The preset is [backoff]. Build another with [Backoff::new], or update one in place with { ..backoff, attempts: 5 }.

    Backoff::ceiling

    source
    fn Backoff::ceiling(self : Backoff, retry : Int) -> 
    Span

    The undelayed ceiling before the retry-th retry, the first retry being 1: min(base × factor^(retry-1), cap).

    This is what a jitter-free client sleeps and what every other strategy draws within, so it is public: a caller comparing this library against a policy written elsewhere compares this.

    Parameters

    • retry : which retry, the first being one; zero or less returns nothing

    Backoff::delay

    source
    fn Backoff::delay(self : Backoff, retry : Int, random? : Double, previous? : 
    Span    ) -> 
    Span

    How long to wait before the retry-th retry.

    random is a value in [0, 1) that the caller draws; passing the same one twice gives the same delay twice, which is what makes a retry sequence testable. previous is what the last delay turned out to be, which only Decorrelated reads — it grows from where it landed rather than from the attempt number.

    Parameters

    • retry : which retry this is, the first being one
    • random : a draw from [0, 1) that the strategy scales; the same draw gives the same wait
    • previous : what the last wait turned out to be; only Decorrelated reads it

    Backoff::equal

    source
    fn Backoff::equal(Backoff, Backoff) -> Bool

    Backoff::more

    source
    fn Backoff::more(self : Backoff, made : Int) -> Bool

    Whether a call that has made made attempts may make another.

    The first attempt is 1, so a policy of three attempts allows two retries.

    Parameters

    • made : attempts made so far, the first attempt being one

    Backoff::new

    source
    fn Backoff::new(base? : 
    Span    , factor? : Double, cap? : 
    Span    , attempts? : Int, jitter? : Jitter) -> Backoff

    A backoff by name, every knob with the preset's value.

    Parameters

    • base : the wait before the first retry, 100 ms in the preset
    • factor : what each wait is multiplied by, 2.0 in the preset
    • cap : the ceiling that multiplication is held to, 1 s in the preset
    • attempts : calls allowed in total, the first one included; 3 in the preset
    • jitter : how much of that schedule is randomised, Spread(0.2) in the preset

    Backoff::not_equal

    source
    fn Backoff::not_equal(x : Backoff, y : Backoff) -> Bool

    Backoff::schedule

    source
    fn Backoff::schedule(self : Backoff, randoms? : ArrayView[Double]) -> Array[
    Span    ]

    Every delay a run of retries would wait, for a caller that wants the whole sequence rather than one step — a test, or a log line explaining a plan.

    randoms supplies the draws; when it runs short the remaining steps use zero, which is the low end of whatever strategy is in force.

    Parameters

    • randoms : the draws to use, one per retry; when it runs short the rest use zero

    Backoff::to_repr

    source

    Breaker

    source
    pub struct Breaker {
      sre : Sre
      win : Window[Tally]
      last : Int64?
    }

    A circuit breaker that sheds a fraction of calls rather than all of them.

    There is no open, half-open and closed: the breaker keeps a rolling window of outcomes and refuses each new call with a probability that grows with the failures in it. A struggling backend keeps receiving as much as it can still serve, instead of being cut off and then flooded again when it recovers.

    With accepts successes out of total calls in the window, failing the run of all-failure buckets at its head and working the run of all-success ones:

    w     = max(k - (k - floor) × failing / size, floor)
drop  = (total - protection - w × accepts) / (total + 1)
drop ×= (size - working) / size

    A drop at or below zero lets everything through, which is the whole healthy case.

    Breaker::allow

    source
    fn Breaker::allow(self : Breaker, now : Int64, roll : Double) -> Promise?

    Ask to make one call at now.

    roll is a draw from [0, 1); the call is refused when it falls under the drop fraction. Some means go ahead, and the caller settles the promise once the call has finished. A refusal is recorded as a Drop, so a breaker that is refusing never reads back as idle and throttles itself off.

    Parameters

    • now : the caller's clock reading, in nanoseconds
    • roll : a draw from [0, 1)

    Breaker::new

    source
    fn Breaker::new(sre? : Sre, now? : Int64) -> Breaker

    A breaker with an empty window opening at now.

    Parameters

    • sre : the policy; the sre preset by default
    • now : the clock reading the window opens at, 0L by default

    Breaker::ratio

    source
    fn Breaker::ratio(self : Breaker, now : Int64) -> Double

    The fraction of calls the breaker would refuse at now, zero while the window is healthy.

    The probe is left out: this is the standing throttle, which is what a dashboard or an alert wants.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Bucket

    source
    pub struct Bucket {
      rate : Double
      burst : Double
      tokens : Double
      at : Int64
    }

    A token bucket: rate tokens accrue per second up to burst, and each request takes one.

    It is the rate limiter that allows a burst after a quiet period, which is what a limit on a human-facing API wants — nobody should be refused for making two requests in the same second after an hour of making none.

    Time is nanoseconds from the caller's clock, as everywhere in this package.

    Bucket::new

    source
    fn Bucket::new(rate : Double, burst? : Double, now? : Int64) -> Bucket

    A bucket that starts full, which is what lets the first burst through.

    rate is tokens per second and burst is the most it will ever hold. A burst smaller than one means nothing ever passes, so it is raised to one.

    Parameters

    • rate : tokens accrued per second
    • burst : the most the bucket will ever hold; with none given, the rate, and never under one
    • now : the clock reading the bucket starts at, 0L by default

    Bucket::next

    source
    fn Bucket::next(self : Bucket, now : Int64, n? : Double) -> 
    Span

    How long until n tokens would be available, without taking them.

    Zero when they are available now. This is what a caller sleeps for rather than spinning, and what a Retry-After header is computed from.

    Parameters

    • now : the caller's clock reading, in nanoseconds
    • n : how many tokens are wanted, one by default

    Bucket::take

    source
    fn Bucket::take(self : Bucket, now : Int64, n? : Double) -> Bool

    Whether n tokens are available now, taking them if they are.

    Parameters

    • now : the caller's clock reading, in nanoseconds
    • n : how many tokens the call costs, one by default

    Bucket::tokens

    source
    fn Bucket::tokens(self : Bucket, now : Int64) -> Double

    How many tokens are in the bucket, for a caller that reports its own headroom.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Deadline

    source
    pub struct Deadline {
      start : Int64
      budget : Int64
    }

    A time budget measured from the instant it started.

    A budget of zero or less never expires, which is how most servers spell "no timeout": a configuration that leaves the timeout unset should not time everything out at once.

    Deadline::expired

    source
    fn Deadline::expired(self : Deadline, now : Int64) -> Bool

    Whether the budget is spent at now.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Deadline::remaining

    source
    fn Deadline::remaining(self : Deadline, now : Int64) -> 
    Span    ?

    What is left of the budget at now, never negative. None when there is no budget, which has no finite remainder.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Deadline::start

    source
    fn Deadline::start(budget : 
    Span    , now : Int64) -> Deadline

    A deadline of budget starting at now.

    Parameters

    • budget : how long the deadline lasts; zero or less never expires
    • now : the clock reading it starts at

    Gate

    source
    pub struct Gate {
      limit : Int
      held : Int
    }

    A count with a ceiling: the concurrency limit a server puts on itself.

    Unlike a rate, this does not depend on time at all — it is how many things are happening at once. A server past its limit answers rather than queues, which is what keeps a queue from becoming the outage.

    Gate::enter

    source
    fn Gate::enter(self : Gate) -> Bool

    Whether there is room, taking it if there is.

    Gate::held

    source
    fn Gate::held(self : Gate) -> Int

    How many are inside.

    Gate::leave

    source
    fn Gate::leave(self : Gate) -> Unit

    Give the room back.

    Gate::new

    source
    fn Gate::new(limit : Int) -> Gate

    A gate that nothing has entered.

    Parameters

    • limit : how many calls may be inside at once; zero or less means no ceiling

    Jitter

    source
    pub(all) enum Jitter {
      Rigid
      Full
      Equal
      Decorrelated
      Spread(Double)
    } derive(Eq, 
    Debug    )

    How much randomness goes into a delay.

    A retry with no jitter makes every client that failed together try again together, which is how one outage becomes a second one. Full, Equal and Decorrelated are the three shapes the literature names and measures; Spread keeps the wait within a fraction of the ceiling instead, which is the easier one to reason about when the schedule is fixed by something else:

    Rigidthe computed delay, unchanged. Reproducible, and it synchronises clients
    Fulluniform over [0, cap]. The one that spreads clients furthest
    Equalhalf the cap plus uniform over [0, cap/2]. Keeps a floor under the wait
    Decorrelateduniform over [base, previous × 3]. Grows from where it last landed rather than from the attempt number
    Spread(f)uniform over [cap × (1 − f), cap × (1 + f)], f held to [0, 1]. So the wait can pass the cap by f

    Jitter::equal

    source
    fn Jitter::equal(Jitter, Jitter) -> Bool

    Jitter::not_equal

    source
    fn Jitter::not_equal(x : Jitter, y : Jitter) -> Bool

    Jitter::to_repr

    source

    Outcome

    source
    pub(all) enum Outcome {
      Success
      Failure
      Drop
    } derive(Eq, 
    Debug    )

    How a call ended, as a breaker records it.

    A Drop is a call refused before it ran: traffic, but neither a success nor a failure of whatever is downstream.

    Outcome::equal

    source
    fn Outcome::equal(Outcome, Outcome) -> Bool

    Outcome::not_equal

    source
    fn Outcome::not_equal(x : Outcome, y : Outcome) -> Bool

    Outcome::to_repr

    source

    Period

    source
    pub struct Period {
      quota : Int
      period : Int64
      align : Bool
      start : Int64?
      count : Int
    }

    A fixed window counter: at most quota events per period, the count starting again when a period ends.

    Cheaper than [Sliding], which keeps every stamp, and coarser: a client can spend one quota at the end of a period and another at the start of the next, so forty events can pass in a second that allows twenty. That is the trade a fixed window makes, and the reason it is the cheap one.

    A period opens on the first event after the last one closed. With align it opens on a multiple of period on the caller's clock instead, so with a Unix clock a daily quota resets at midnight UTC rather than a day after first use.

    Period::left

    source
    fn Period::left(self : Period, now : Int64) -> Int

    How many events remain in the current period.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Period::new

    source
    fn Period::new(quota : Int, period : 
    Span    , align? : Bool) -> Period

    A counter no event has touched. A period shorter than a nanosecond is raised to one.

    Parameters

    • quota : events allowed per period
    • period : how long a period is; anything under a nanosecond is raised to one
    • align : open periods on multiples of period on the caller's clock instead of a period after first use

    Period::next

    source
    fn Period::next(self : Period, now : Int64) -> 
    Span

    How long until the current period ends. Zero when no period is open, because the next event would open one.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Period::take

    source
    fn Period::take(self : Period, now : Int64) -> Quota

    Count one event at now and say where the quota stands.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Promise

    source
    pub struct Promise {
      breaker : Breaker
    }

    A call the breaker let through. Exactly one of pass and fail settles it; until then the window holds no record of how the call went.

    Promise::fail

    source
    fn Promise::fail(self : Promise, now : Int64) -> Unit

    The call failed.

    Parameters

    • now : the clock reading the call finished at

    Promise::pass

    source
    fn Promise::pass(self : Promise, now : Int64) -> Unit

    The call succeeded.

    Parameters

    • now : the clock reading the call finished at

    Quota

    source
    pub(all) enum Quota {
      Under
      Hit
      Over
    } derive(Eq, 
    Debug    )

    Where a [Period] stands after an event: under the quota, exactly on it, or past it.

    The event that hits the quota is still admitted. It is told apart because it is the last one, which is when a caller would warn that the next will be refused.

    Quota::equal

    source
    fn Quota::equal(Quota, Quota) -> Bool

    Quota::not_equal

    source
    fn Quota::not_equal(x : Quota, y : Quota) -> Bool

    Quota::to_repr

    source

    Shed

    source
    pub(all) struct Shed {
      size : Int
      window : 
    Span
      cpu : Int
      latency : 
    Span
      beta : Double
      cool : 
    Span
      floor : Double
    } derive(Eq, 
    Debug    )

    The constants of adaptive load shedding.

    The preset is [shed]. Build another with [Shed::new], or update one in place with { ..shed, cpu: 800 }.

    Shed::equal

    source
    fn Shed::equal(Shed, Shed) -> Bool

    Shed::new

    source
    fn Shed::new(size? : Int, window? : 
    Span    , cpu? : Int, latency? : 
    Span    , beta? : Double, cool? : 
    Span    , floor? : Double) -> Shed

    A policy by name, every knob with the preset's value.

    Parameters

    • size : buckets in the window, 50 in the preset
    • window : how far back that window reaches, 5 s in the preset
    • cpu : CPU use in thousandths at or above which the server counts as hot, 900 in the preset
    • latency : response time assumed while the window has none, 1 s in the preset
    • beta : weight of the old value in the moving average of calls in flight, 0.9 in the preset
    • cool : how long the server stays hot after the CPU falls back, 1 s in the preset
    • floor : the least share of capacity admitted however high the CPU is, 0.1 in the preset

    Shed::not_equal

    source
    fn Shed::not_equal(x : Shed, y : Shed) -> Bool

    Shed::to_repr

    source

    Shedder

    source
    pub struct Shedder {
      shed : Shed
      passes : Window[Total]
      latencies : Window[Total]
      flying : Int64
      average : Double
      hot : Int64?
      dropped : Bool
    }

    Load shedding that refuses work only when the server is both hot and full.

    Hot means the CPU is at or over the threshold, or was within the cooling time and the shedder has refused since. Full means more calls are in flight than the server has shown it can finish: the busiest bucket's passes per millisecond times the fastest bucket's mean response time, scaled down as the CPU nears 100%. Both the moving average and the current count of calls in flight must be over that figure.

    A CPU spike with nothing queued sheds nothing, and neither does a queue on an idle CPU. CPU use is read by the caller, in thousandths, and passed to each [Shedder::allow].

    Shedder::allow

    source
    fn Shedder::allow(self : Shedder, now : Int64, cpu : Int) -> Ticket?

    Ask to start one call at now, with the CPU at cpu thousandths.

    Some means go ahead, and the caller settles the ticket once the call has finished.

    Parameters

    • now : the caller's clock reading, in nanoseconds
    • cpu : CPU use in thousandths, read by the caller

    Shedder::flying

    source
    fn Shedder::flying(self : Shedder) -> Int64

    How many calls are in flight.

    Shedder::new

    source
    fn Shedder::new(shed? : Shed, now? : Int64) -> Shedder

    A shedder with empty windows opening at now.

    Parameters

    • shed : the policy; the shed preset by default
    • now : the clock reading both windows open at, 0L by default

    Sliding

    source
    pub struct Sliding {
      quota : Int
      window : 
    Span
      stamps : Array[Int64]
    }

    A sliding window counter: at most quota events in any window long stretch.

    Where a bucket smooths, a window counts. A quota of "a thousand a day" is a window, not a rate, and answering it with a bucket would let a client spend the whole day's allowance in the first minute.

    The stamps of the events in the window are kept, so the count is exact rather than the approximation a two-bucket scheme gives.

    Sliding::left

    source
    fn Sliding::left(self : Sliding, now : Int64) -> Int

    How many events remain in the current window.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Sliding::new

    source
    fn Sliding::new(quota : Int, window : 
    Span    ) -> Sliding

    An empty window.

    Parameters

    • quota : events allowed in any window-long stretch
    • window : the length of that stretch

    Sliding::next

    source
    fn Sliding::next(self : Sliding, now : Int64) -> 
    Span

    When the window will next have room, which is when its oldest event falls out.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Sliding::take

    source
    fn Sliding::take(self : Sliding, now : Int64) -> Bool

    Whether an event is within the quota now, counting it if it is.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Sre

    source
    pub(all) struct Sre {
      k : Double
      floor : Double
      protection : Double
      probe : 
    Span
      size : Int
      window : 
    Span
    } derive(Eq, 
    Debug    )

    The constants of probability-based client-side throttling.

    The preset is [sre]. Build another with [Sre::new], or update one in place with { ..sre, k: 2.0 }.

    Sre::equal

    source
    fn Sre::equal(Sre, Sre) -> Bool

    Sre::new

    source
    fn Sre::new(k? : Double, floor? : Double, protection? : Double, probe? : 
    Span    , size? : Int, window? : 
    Span    ) -> Sre

    A policy by name, every knob with the preset's value.

    Parameters

    • k : how many calls each success pays for; the preset's 1.5 leaves a backend alone until more than a third of its calls fail
    • floor : what k decays to during an outage, 1.1 in the preset
    • protection : calls every backend gets regardless, 5 in the preset
    • probe : how long with nothing let through before one call is let through anyway
    • size : buckets in the window, 40 in the preset
    • window : how far back the window reaches, 10 s in the preset

    Sre::not_equal

    source
    fn Sre::not_equal(x : Sre, y : Sre) -> Bool

    Sre::to_repr

    source

    Tally

    source
    pub struct Tally {
      total : Int64
      success : Int64
      failure : Int64
      drop : Int64
    } derive(Default, Eq, 
    Debug    )

    Outcomes counted in one bucket.

    total counts every call including the dropped ones, so a bucket being throttled still reads as busy rather than idle.

    Tally::add

    source
    fn Tally::add(self : Tally, o : Outcome) -> Unit

    Count one call ending as o.

    Parameters

    • o : how the call ended

    Tally::default

    source
    fn Tally::default() -> Tally

    Tally::equal

    source
    fn Tally::equal(Tally, Tally) -> Bool

    Tally::not_equal

    source
    fn Tally::not_equal(x : Tally, y : Tally) -> Bool

    Tally::to_repr

    source

    Ticket

    source
    pub struct Ticket {
      shedder : Shedder
      start : Int64
    }

    A call the shedder let through. Exactly one of pass and fail settles it.

    Ticket::fail

    source
    fn Ticket::fail(self : Ticket) -> Unit

    The call failed. It leaves flight but says nothing about capacity.

    Ticket::pass

    source
    fn Ticket::pass(self : Ticket, now : Int64) -> Unit

    The call succeeded at now. Its response time counts towards what the server has shown it can do.

    Parameters

    • now : the clock reading the call finished at; its response time counts towards capacity

    Total

    source
    pub struct Total {
      sum : Int64
      count : Int64
    } derive(Default, Eq, 
    Debug    )

    A sum and how many values went into it, which is a mean once count is positive.

    Total::add

    source
    fn Total::add(self : Total, v : Int64) -> Unit

    Add one value.

    Parameters

    • v : the value to fold into the sum

    Total::default

    source
    fn Total::default() -> Total

    Total::equal

    source
    fn Total::equal(Total, Total) -> Bool

    Total::not_equal

    source
    fn Total::not_equal(x : Total, y : Total) -> Bool

    Total::to_repr

    source

    Window

    source
    pub struct Window[B] {
      width : Int64
      buckets : Array[B]
      head : Int
      start : Int64
    }

    A rolling window: size buckets that together cover the last window of time, oldest dropping off as time moves on.

    The bucket type is the caller's. A breaker keeps a [Tally] of outcomes per bucket, a shedder a [Total] of response times; anything with a Default can be a bucket, and the default is what a bucket is cleared back to.

    Time never moves on its own: every call carries the now it happens at. Writing at a later now clears the buckets the gap swept past; reading at a later now skips them without clearing, so a window nobody writes to still ages out, and two reads at the same now agree.

    Window::each

    source
    fn[B] Window::each(self : Window[B], now : Int64, f : (B) -> Unit, current? : Bool) -> Unit

    Visit the buckets still inside the window at now, oldest first.

    With current false the bucket now falls in is left out: it has seen only part of its time, and a reading that takes the busiest or the fastest bucket should not compare a partial one against whole ones.

    Parameters

    • now : the caller's clock reading, in nanoseconds
    • f : called once per live bucket, oldest first
    • current : whether to include the bucket now falls in, true by default

    Window::new

    source
    fn[B : Default] Window::new(size~ : Int, window~ : 
    Span    , now? : Int64) -> Window[B]

    An empty window of size buckets spanning window, its first bucket opening at now.

    A size below one is raised to one, and so is a bucket narrower than a nanosecond.

    Parameters

    • size : how many buckets; anything under one is raised to one
    • window : how much time the buckets together cover
    • now : the clock reading the first bucket opens at, 0L by default

    Window::size

    source
    fn[B] Window::size(self : Window[B]) -> Int

    How many buckets the window has.

    Window::slot

    source
    fn[B : Default] Window::slot(self : Window[B], now : Int64) -> B

    The bucket now falls in, for the caller to add to.

    The buckets the gap since the last write swept past are cleared first. The head is snapped back to a bucket boundary, so buckets keep their width rather than drifting with the arrival time of each write.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Window::width

    source
    fn[B] Window::width(self : Window[B]) -> 
    Span

    How much time one bucket covers.

    backoff

    source
    let backoff : Backoff

    The preset: three attempts, a hundred milliseconds doubling to a one-second ceiling, spread by a fifth either way.

    Each wait is the computed ceiling spread by a fifth below or above it, so a plan of 0.8 to 1.2 of the schedule is easy to explain to whoever asks why the second retry waited 168 ms and not 200.

    shed

    source
    let shed : Shed

    The preset: five seconds in fifty buckets, overloaded at 90% CPU, a one-second assumed response time, an average weighted 0.9, one second to cool and never less than a tenth of capacity.

    The memory is short on purpose: a five-second window follows the load it is in rather than averaging a burst away, and the floor keeps a server already at full CPU from being told it can serve nobody.

    sre

    source
    let sre : Sre

    The preset: k 1.5 decaying to 1.1, five calls of protection, a probe every second, and ten seconds of history in forty buckets.

    k and protection are the slope and the floor of the standard client-side throttling formula; the decay, the probe and the recovery scaling are what keep a long outage from hurting past successes, and a short one from being remembered for long.

    Source Files

    Powered by MoonBit

    Site sourceReport issueModulesSkillsCLIBuild queueStatistics

    © 2026 mooncakes.io