🥮 mooncakes.io

    moonward

    Resilience for MoonBit without I/O: circuit breaking, load shedding, rate limits and deadlines.

    resilience
    fault-tolerance
    circuit-breaker
    rate-limit
    rate-limiter
    token-bucket
    sliding-window
    load-shedding
    backoff
    retry
    deadline
    timeout
    throttling
    moonbit
    Download zip
    Version
    0.3.0
    License
    Apache-2.0
    Repository
    Last updated
    6 hours ago
    Downloads
    4K

    moonward

    #What it is

    Twelve types that answer one question each: should this call go ahead.

    AnswersRefuses when
    BreakerIs this backend likely to work?a draw falls under a refusal fraction built from the last ten seconds of outcomes
    ShedderShould this server take on another call?the CPU is over its threshold and more calls are in flight than the server has shown it can finish
    LimiterHow many calls may be in flight, judging by how fast they come back?in flight has reached a limit that AIMD, Vegas or Gradient moves with every call that finishes
    BucketIs this call within a rate?the token is spent; next says when the next one lands
    SlidingIs it within a quota over any window-long stretch?the count reaches the quota
    PeriodIs it within a quota this period?the count for the open period is spent
    GateIs there room right now?held has reached limit — it never asks what time it is
    ChainedDoes the call get past every limiter in a row?any of them refuses; what the others gave is given back
    PartitionedIs this caller within its own allowance?the limiter kept for its key refuses
    BackoffWhen to try again, and how long first?more(attempts) is false
    DeadlineHow much of the budget is left?expired is true
    WindowWhat does the rolling window hold?never; it is the array of buckets the others read

    #No clock, no dice

    Nothing here reads a clock or draws a random number. now is whatever your clock read, in nanoseconds; where a decision is a roll, you pass a draw from [0, 1). That is what makes an hour of traffic a loop over now, and a retry sequence testable.

    let breaker = @moonward.Breaker::new()
    let now = 0L
    match breaker.allow(now, 0.42) {
      Some(call) => call.pass(now)   // or call.fail(now)
      None => println("refused")
    }

    #The breaker

    There is no open, half-open and closed. The breaker keeps ten seconds of outcomes and refuses each call with a probability that grows with the failures in it, so a struggling backend keeps receiving what it can still serve instead of being cut off and then flooded on recovery.

    w     = max( k − (k − floor) · failing / size , floor )
drop  = ( total − protection − w · accepts ) / ( total + 1 )
drop ×= ( size − working ) / size

    failing and working are runs at the head of the window, not totals. A success pays for k calls — at the preset's 1.5, a backend is left alone until more than a third of its calls fail.

    call wraps a call in all of that. A refusal raises Unavailable, or answers with fallback; an error the call raises counts as a failure unless acceptable says it is the caller's own, a not found or a bad request:

    let breaker = @moonward.Breaker::new()
    let clock = () => 0L
    let answer = breaker.call(clock, 0.42, () => 7, fallback=_ => 0) catch { _ => -1 }
    assert_eq(answer, 7)

    #The shedder

    A call is refused only when the server is both hot and full:

    • hot — the CPU is at or over the threshold, or was within the last second and the shedder has refused since;
    • full — both the moving average and the current count of calls in flight exceed the busiest bucket's passes per millisecond times the fastest bucket's mean response time, scaled down as the CPU nears 100% but never under a tenth.

    A CPU spike with nothing queued refuses nothing, and neither does a long queue on an idle CPU. CPU use is read by you, in thousandths, and passed to each allow.

    #The adaptive limit

    Limiter is a concurrency limit that finds itself. Each call takes a lease; when the call finishes, its response time is a sample, and the rule moves the limit: up while calls come back as fast as ever, down once they queue.

    let limiter = @moonward.Limiter::new(rule=Gradient(@moonward.gradient))
    match limiter.acquire(0L) {
      Some(lease) => lease.pass(12_000_000L) // or lease.fail(now) on a timeout, lease.ignore() on a bad request
      None => println("over the limit")
    }
    assert_eq(limiter.limit(), 20)

    RuleMoves the limit by
    Aimdone up while at least half of it is used, a tenth down on a drop or a call over the timeout
    Vegashow many calls are queued: the response time over the shortest one seen, which it measures again now and then
    Gradienthow much slower the latest call is than the long-run average, plus a few slots to grow into
    MinRtthow much slower the latest call is than the fastest since the last probe; every probe calls or so it drops to the queue size and measures the fastest again
    Fixednever

    Vegas is the default. A rule of your own implements Adapt and goes in through Limiter::custom.

    #One allowance, several processes

    A bucket is two numbers — the tokens it holds and the moment they were brought up to date — and step answers a call against that pair instead of a bucket of your own:

    let bucket = @moonward.Bucket::new(10.0, burst=20.0)
    let stored = bucket.opening(0L)        // what a key that is not there starts as
    let grant = bucket.step(stored, 0L)
    if grant.granted {
      // write grant.state back, if the row still holds `stored`
    }
    assert_eq(grant.state.tokens, 19.0)
    assert_eq(grant.wait.nanos, 0L)

    grant.state goes into the store only when the row still holds what was read — a compare-and-set, a transaction, or the script a Redis client runs — and a caller that lost the race reads again and asks again. That is the whole of it: the count is fractional, so the store has to keep it as a number rather than an integer.

    A quota per period is shared the same way. Its state is when the open period started and how many events it has counted:

    let daily = @moonward.Period::new(1000, @moondate.Span::new(days=1L), align=true)
    let counted = daily.step(daily.opening(), 0L)
    assert_eq(counted.quota, Under)
    assert_eq(counted.state.used, 1)

    Two processes counting through one store spend one quota between them, and the count starts again for both when the period ends.

    #Limiters together

    Every limiter here can hand out one permit and take it back, which is the Permit trait. Chained asks several in a row and gives back what the earlier ones gave when a later one refuses, so a call that does not run spends nothing. Partitioned keeps one limiter per key, at most limit keys, dropping the one used least recently:

    let per_client = @moonward.Partitioned::new(_ => @moonward.Bucket::new(5.0, burst=10.0))
    let both = @moonward.Chained::new([
      @moonward.Bucket::new(100.0, burst=100.0),
      @moonward.Gate::new(64),
    ])
    assert_true(per_client.acquire("203.0.113.7", 0L))
    assert_true(both.acquire(0L))

    A limiter of your own implements Permit and goes into either.

    #A run

    moon run src/examples/limiters offers one burst of forty calls every two seconds to a bucket, a sliding window and a period, all with a quota of twenty a second. The burst straddles the second boundary:

    burst  bucket  sliding  period
    0      21       20      40
    1      21       20      40
    2      21       20      40
    3      21       20      40

five tokens in 200 ms

    The bucket spends a saved burst and then holds the rate. The slide refuses the twenty-first call in any one-second stretch. The period starts a fresh twenty when the second turns over, so the same burst is counted as twenty on each side and all forty pass.

    #Presets

    Each policy is a record with a preset of the same name. Copy it and change one field — { ..sre, k: 2.0 } — or build it with Sre::new(k=2.0). There are no per-call overrides.

    PresetValues
    srek 1.5 decaying to 1.1, protection 5, a probe after 1 s, 40 buckets over 10 s
    shed50 buckets over 5 s, CPU threshold 900‰, 1 s assumed response time, average weighted 0.9, 1 s to cool, never under 0.1 of capacity
    backoff3 attempts, 100 ms doubling to a 1 s cap, each wait spread by a fifth either way
    aimdstart at 20 within 20 and 200, keep 0.9 on a drop, a call over 5 s is a drop
    vegasstart at 20, at most 1000, no smoothing, alpha 3, beta 6, threshold and step 1, probe 30
    gradientstart at 20 within 20 and 200, smoothing 0.2, 4 slots on top, a 600-call average, tolerance 1.5

    Durations are @moondate.Span, so the package brings moondate with it.

    #Examples

    ProgramWhat it shows
    src/examples/minimalone breaker, one failing backend, four lines of output — the smallest thing that works
    src/examples/limitersthe same bursty traffic through all three quotas
    src/examples/toura breaker under an outage, a shedder under rising load, one step through each limiter
    src/examples/guidethe guide as one program, in the order you read it

    The bay runs the same code compiled to JavaScript, with the knobs on the outside.

    #Licence

    Apache-2.0. See LICENSE.

    Adapt

    source
    pub(open) trait Adapt {
      fn limit(Self) -> Int
      fn sample(Self, Sample) -> Unit
    }

    How a concurrency limit moves as calls finish.

    The four rules that ship are [Rule]'s variants. Implement this to bring your own, and hand it to [Limiter::custom].

    Permit

    source
    pub(open) trait Permit {
      fn acquire(Self, now : Int64) -> Bool
      fn release(Self, now : Int64) -> Unit
    }

    Something that admits one call at a time and can take an admission back.

    The four limiters here implement it. Implement it for your own, and it can be partitioned and chained like them.

    Unavailable

    source
    pub(all) suberror Unavailable

    A call the breaker would not let through.
    impl Show for Unavailable

    Unavailable::output

    source
    fn Unavailable::output(_self : Unavailable, logger : &Logger) -> Unit

    Unavailable::to_string

    source
    fn Unavailable::to_string(self : Unavailable) -> String

    Aimd

    source
    pub(all) struct Aimd {
      initial : Int
      min : Int
      max : Int
      backoff : Double
      timeout : 
    Span
    } derive(Eq, 
    Debug    )

    Additive increase, multiplicative decrease: one more while the limit is being used, a fixed share less on a drop or a call slower than the timeout.

    The preset is [aimd]. Build another with [Aimd::new], or update one in place with { ..aimd, max: 500 }.

    Aimd::equal

    source
    fn Aimd::equal(Aimd, Aimd) -> Bool

    Aimd::new

    source
    fn Aimd::new(initial? : Int, min? : Int, max? : Int, backoff? : Double, timeout? : 
    Span    ) -> Aimd

    A rule by name, every knob with the preset's value.

    Parameters

    • initial : the limit before any call has finished, 20 in the preset
    • min : the least the limit goes to, 20 in the preset
    • max : the most the limit goes to, 200 in the preset
    • backoff : the share kept after a drop, 0.9 in the preset
    • timeout : a call slower than this counts as a drop, 5 s in the preset

    Aimd::not_equal

    source
    fn Aimd::not_equal(x : Aimd, y : Aimd) -> Bool

    Aimd::to_repr

    source

    Backoff

    source
    pub(all) struct Backoff {
      base : 
    Span
      factor : Double
      cap : 
    Span
      attempts : Int
      jitter : Jitter
    } derive(Eq, 
    Debug    )

    When to try again, and how long to wait first.

    The preset is [backoff]. Build another with [Backoff::new], or update one in place with { ..backoff, attempts: 5 }.

    Backoff::ceiling

    source
    fn Backoff::ceiling(self : Backoff, retry : Int) -> 
    Span

    The undelayed ceiling before the retry-th retry, the first retry being 1: min(base × factor^(retry-1), cap).

    This is what a jitter-free client sleeps and what every other strategy draws within, so it is public: a caller comparing this library against a policy written elsewhere compares this.

    Parameters

    • retry : which retry, the first being one; zero or less returns nothing

    Backoff::delay

    source
    fn Backoff::delay(self : Backoff, retry : Int, random? : Double, previous? : 
    Span    ) -> 
    Span

    How long to wait before the retry-th retry.

    random is a value in [0, 1) that the caller draws; passing the same one twice gives the same delay twice, which is what makes a retry sequence testable. previous is what the last delay turned out to be, which only Decorrelated reads — it grows from where it landed rather than from the attempt number.

    Parameters

    • retry : which retry this is, the first being one
    • random : a draw from [0, 1) that the strategy scales; the same draw gives the same wait
    • previous : what the last wait turned out to be; only Decorrelated reads it

    Backoff::equal

    source
    fn Backoff::equal(Backoff, Backoff) -> Bool

    Backoff::more

    source
    fn Backoff::more(self : Backoff, made : Int) -> Bool

    Whether a call that has made made attempts may make another.

    The first attempt is 1, so a policy of three attempts allows two retries.

    Parameters

    • made : attempts made so far, the first attempt being one

    Backoff::new

    source
    fn Backoff::new(base? : 
    Span    , factor? : Double, cap? : 
    Span    , attempts? : Int, jitter? : Jitter) -> Backoff

    A backoff by name, every knob with the preset's value.

    Parameters

    • base : the wait before the first retry, 100 ms in the preset
    • factor : what each wait is multiplied by, 2.0 in the preset
    • cap : the ceiling that multiplication is held to, 1 s in the preset
    • attempts : calls allowed in total, the first one included; 3 in the preset
    • jitter : how much of that schedule is randomised, Spread(0.2) in the preset

    Backoff::not_equal

    source
    fn Backoff::not_equal(x : Backoff, y : Backoff) -> Bool

    Backoff::schedule

    source
    fn Backoff::schedule(self : Backoff, randoms? : ArrayView[Double]) -> Array[
    Span    ]

    Every delay a run of retries would wait, for a caller that wants the whole sequence rather than one step — a test, or a log line explaining a plan.

    randoms supplies the draws; when it runs short the remaining steps use zero, which is the low end of whatever strategy is in force.

    Parameters

    • randoms : the draws to use, one per retry; when it runs short the rest use zero

    Backoff::to_repr

    source

    Breaker

    source
    pub struct Breaker {
      sre : Sre
      win : Window[Tally]
      last : Int64?
    }

    A circuit breaker that sheds a fraction of calls rather than all of them.

    There is no open, half-open and closed: the breaker keeps a rolling window of outcomes and refuses each new call with a probability that grows with the failures in it. A struggling backend keeps receiving as much as it can still serve, instead of being cut off and then flooded again when it recovers.

    With accepts successes out of total calls in the window, failing the run of all-failure buckets at its head and working the run of all-success ones:

    w     = max(k - (k - floor) × failing / size, floor)
drop  = (total - protection - w × accepts) / (total + 1)
drop ×= (size - working) / size

    A drop at or below zero lets everything through, which is the whole healthy case.

    Breaker::allow

    source
    fn Breaker::allow(self : Breaker, now : Int64, roll : Double) -> Promise?

    Ask to make one call at now.

    roll is a draw from [0, 1); the call is refused when it falls under the drop fraction. Some means go ahead, and the caller settles the promise once the call has finished. A refusal is recorded as a Drop, so a breaker that is refusing never reads back as idle and throttles itself off.

    Parameters

    • now : the caller's clock reading, in nanoseconds
    • roll : a draw from [0, 1)

    Breaker::call

    source
    fn[T] Breaker::call(self : Breaker, now : () -> Int64, roll : Double, f : () -> T raise, acceptable? : (Error) -> Bool, fallback? : (Error) -> T raise) -> T raise

    f through the breaker, which is go-zero's Do and its siblings in one call.

    A refusal raises [Unavailable], or answers with fallback when one is given. An error from f is raised as it came, and counts against the breaker unless acceptable says it is the caller's business rather than the service's: a not found, a validation error.

    Parameters

    • now : the caller's clock, in nanoseconds; read before the call and again after it
    • roll : a draw from [0, 1)
    • f : the call
    • acceptable : which errors count as a success; none unless given
    • fallback : the answer when the breaker refuses; [Unavailable] is raised unless given

    Breaker::new

    source
    fn Breaker::new(sre? : Sre, now? : Int64) -> Breaker

    A breaker with an empty window opening at now.

    Parameters

    • sre : the policy; the sre preset by default
    • now : the clock reading the window opens at, 0L by default

    Breaker::ratio

    source
    fn Breaker::ratio(self : Breaker, now : Int64) -> Double

    The fraction of calls the breaker would refuse at now, zero while the window is healthy.

    The probe is left out: this is the standing throttle, which is what a dashboard or an alert wants.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Bucket

    source
    pub struct Bucket {
      rate : Double
      burst : Double
      state : Tokens
    }

    A token bucket: rate tokens accrue per second up to burst, and each request takes one.

    It is the rate limiter that allows a burst after a quiet period, which is what a limit on a human-facing API wants — nobody should be refused for making two requests in the same second after an hour of making none.

    Time is nanoseconds from the caller's clock, as everywhere in this package.
    impl Permit for Bucket

    Bucket::acquire

    source
    fn Bucket::acquire(self : Bucket, now : Int64) -> Bool

    Bucket::new

    source
    fn Bucket::new(rate : Double, burst? : Double, now? : Int64) -> Bucket

    A bucket that starts full, which is what lets the first burst through.

    rate is tokens per second and burst is the most it will ever hold. A burst smaller than one means nothing ever passes, so it is raised to one.

    Parameters

    • rate : tokens accrued per second
    • burst : the most the bucket will ever hold; with none given, the rate, and never under one
    • now : the clock reading the bucket starts at, 0L by default

    Bucket::next

    source
    fn Bucket::next(self : Bucket, now : Int64, n? : Double) -> 
    Span

    How long until n tokens would be available, without taking them.

    Zero when they are available now. This is what a caller sleeps for rather than spinning, and what a Retry-After header is computed from.

    Parameters

    • now : the caller's clock reading, in nanoseconds
    • n : how many tokens are wanted, one by default

    Bucket::opening

    source
    fn Bucket::opening(self : Bucket, now : Int64) -> Tokens

    The state a bucket that has seen no traffic is in at now: full, so the first burst goes through.

    This is what a shared store writes when the key is missing, which is the same state [Bucket::new] starts from.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Bucket::release

    source
    fn Bucket::release(self : Bucket, now : Int64) -> Unit

    Bucket::step

    source
    fn Bucket::step(self : Bucket, state : Tokens, now : Int64, n? : Double) -> Grant

    Answer one call against state at now, leaving the bucket untouched.

    This is the whole of the arithmetic, and the seam for a bucket shared by several processes: read the state out of the store — [Bucket::opening] when the key is missing — ask here, and write grant.state back only if the stored value has not changed in between; on a clash, read again and ask again. The compare-and-set is what stops two writers from spending the same token, since an answer is only true of the state it was computed from. Token counts are fractional, so a store that rounds them to an integer loses the point of the bucket.

    Parameters

    • state : the bucket's state as the store has it
    • now : the caller's clock reading, in nanoseconds
    • n : how many tokens the call costs, one by default; a negative cost is taken as zero

    Bucket::take

    source
    fn Bucket::take(self : Bucket, now : Int64, n? : Double) -> Bool

    Whether n tokens are available now, taking them if they are.

    Parameters

    • now : the caller's clock reading, in nanoseconds
    • n : how many tokens the call costs, one by default; a negative cost is taken as zero

    Bucket::tokens

    source
    fn Bucket::tokens(self : Bucket, now : Int64) -> Double

    How many tokens are in the bucket, for a caller that reports its own headroom.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Chained

    source
    pub struct Chained {
      links : Array[&Permit]
    }

    Several limiters a call has to get past, in order: a per-second rate and a daily quota, a quota and a concurrency gate. Any refusal refuses the call, and the permits already taken from the limiters before it are given back, so a call that does not run spends nothing.
    impl Permit for Chained

    Chained::acquire

    source
    fn Chained::acquire(self : Chained, now : Int64) -> Bool

    Chained::new

    source
    fn Chained::new(links : Array[&Permit]) -> Chained

    The limiters in the order a call meets them. With none, every call passes.

    Chained::release

    source
    fn Chained::release(self : Chained, now : Int64) -> Unit

    Counted

    source
    pub(all) struct Counted {
      quota : Quota
      state : Usage
    } derive(Eq, 
    Debug    )

    What one event gets from a [Period]: where the quota stands after it, and the state to keep.

    Counted::equal

    source
    fn Counted::equal(Counted, Counted) -> Bool

    Counted::not_equal

    source
    fn Counted::not_equal(x : Counted, y : Counted) -> Bool

    Counted::to_repr

    source

    Deadline

    source
    pub struct Deadline {
      start : Int64
      budget : Int64
    }

    A time budget measured from the instant it started.

    A budget of zero or less never expires, which is how most servers spell "no timeout": a configuration that leaves the timeout unset should not time everything out at once.

    Deadline::expired

    source
    fn Deadline::expired(self : Deadline, now : Int64) -> Bool

    Whether the budget is spent at now.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Deadline::remaining

    source
    fn Deadline::remaining(self : Deadline, now : Int64) -> 
    Span    ?

    What is left of the budget at now, never negative and never more than the budget itself, which a clock that moved backwards would otherwise report. None when there is no budget, which has no finite remainder.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Deadline::start

    source
    fn Deadline::start(budget : 
    Span    , now : Int64) -> Deadline

    A deadline of budget starting at now.

    Parameters

    • budget : how long the deadline lasts; zero or less never expires
    • now : the clock reading it starts at

    Gate

    source
    pub struct Gate {
      limit : Int
      held : Int
    }

    A count with a ceiling: the concurrency limit a server puts on itself.

    Unlike a rate, this does not depend on time at all — it is how many things are happening at once. A server past its limit answers rather than queues, which is what keeps a queue from becoming the outage.
    impl Permit for Gate

    Gate::acquire

    source
    fn Gate::acquire(self : Gate, _now : Int64) -> Bool

    Gate::enter

    source
    fn Gate::enter(self : Gate) -> Bool

    Whether there is room, taking it if there is.

    Gate::held

    source
    fn Gate::held(self : Gate) -> Int

    How many are inside.

    Gate::leave

    source
    fn Gate::leave(self : Gate) -> Unit

    Give the room back.

    Gate::new

    source
    fn Gate::new(limit : Int) -> Gate

    A gate that nothing has entered.

    Parameters

    • limit : how many calls may be inside at once; zero or less means no ceiling

    Gate::release

    source
    fn Gate::release(self : Gate, _now : Int64) -> Unit

    Gradient

    source
    pub(all) struct Gradient {
      initial : Int
      min : Int
      max : Int
      smoothing : Double
      queue : Int
      window : Int
      tolerance : Double
    } derive(Eq, 
    Debug    )

    The gradient between a long-run average response time and the latest one: when calls get slower than usual, the limit shrinks in proportion, and a few slots are always added on top so it can find room to grow.

    The preset is [gradient]. Build another with [Gradient::new], or update one in place with { ..gradient, queue: 8 }.

    Gradient::equal

    source
    fn Gradient::equal(Gradient, Gradient) -> Bool

    Gradient::new

    source
    fn Gradient::new(initial? : Int, min? : Int, max? : Int, smoothing? : Double, queue? : Int, window? : Int, tolerance? : Double) -> Gradient

    A rule by name, every knob with the preset's value.

    Parameters

    • initial : the limit before any call has finished, 20 in the preset
    • min : the least the limit goes to, 20 in the preset
    • max : the most the limit goes to, 200 in the preset
    • smoothing : how far towards each new estimate the limit moves, 0.2 in the preset
    • queue : slots added to every estimate, 4 in the preset
    • window : calls the long-run average spans, 600 in the preset
    • tolerance : how much slower than the average a call may be, 1.5 in the preset

    Gradient::not_equal

    source
    fn Gradient::not_equal(x : Gradient, y : Gradient) -> Bool

    Gradient::to_repr

    source

    Grant

    source
    pub(all) struct Grant {
      granted : Bool
      state : Tokens
      wait : 
    Span
    } derive(Eq, 
    Debug    )

    What one call gets: whether it may go ahead, what the bucket holds afterwards, and how long a refused call has to wait.

    Grant::equal

    source
    fn Grant::equal(Grant, Grant) -> Bool

    Grant::not_equal

    source
    fn Grant::not_equal(x : Grant, y : Grant) -> Bool

    Grant::to_repr

    source

    Jitter

    source
    pub(all) enum Jitter {
      Rigid
      Full
      Equal
      Decorrelated
      Spread(Double)
    } derive(Eq, 
    Debug    )

    How much randomness goes into a delay.

    A retry with no jitter makes every client that failed together try again together, which is how one outage becomes a second one. Full, Equal and Decorrelated are the three shapes the literature names and measures; Spread keeps the wait within a fraction of the ceiling instead, which is the easier one to reason about when the schedule is fixed by something else:

    Rigidthe computed delay, unchanged. Reproducible, and it synchronises clients
    Fulluniform over [0, cap]. The one that spreads clients furthest
    Equalhalf the cap plus uniform over [0, cap/2]. Keeps a floor under the wait
    Decorrelateduniform over [base, previous × 3]. Grows from where it last landed rather than from the attempt number
    Spread(f)uniform over [cap × (1 − f), cap × (1 + f)], f held to [0, 1]. So the wait can pass the cap by f

    Jitter::equal

    source
    fn Jitter::equal(Jitter, Jitter) -> Bool

    Jitter::not_equal

    source
    fn Jitter::not_equal(x : Jitter, y : Jitter) -> Bool

    Jitter::to_repr

    source

    Lease

    source
    pub struct Lease {
      limiter : Limiter
      start : Int64
      flying : Int
    }

    A call the limiter let through. Exactly one of pass, fail and ignore settles it.

    Lease::fail

    source
    fn Lease::fail(self : Lease, now : Int64) -> Unit

    The call timed out or was refused further down at now: the rule backs off.

    Lease::ignore

    source
    fn Lease::ignore(self : Lease) -> Unit

    The call ended in a way that says nothing about load, such as a bad request. It leaves flight and is not a sample.

    Lease::pass

    source
    fn Lease::pass(self : Lease, now : Int64) -> Unit

    The call finished at now, and its response time is a sample.

    Limiter

    source
    pub struct Limiter {
      adapt : &Adapt
      flying : Int
    }

    A concurrency limit that adapts to the response times of the calls it lets through.

    Ask with [Limiter::acquire] before each call. None means the limit is reached and the call should be refused or queued; Some is a lease to settle once the call is done, which is where the rule learns.

    Limiter::acquire

    source
    fn Limiter::acquire(self : Limiter, now : Int64) -> Lease?

    Ask to start one call at now, in nanoseconds from the caller's clock.

    Limiter::custom

    source
    fn Limiter::custom(adapt : &Adapt) -> Limiter

    A limiter on a rule of your own.

    Limiter::flying

    source
    fn Limiter::flying(self : Limiter) -> Int

    How many calls are in flight.

    Limiter::limit

    source
    fn Limiter::limit(self : Limiter) -> Int

    The most calls that may be in flight now.

    Limiter::new

    source
    fn Limiter::new(rule? : Rule, roll? : () -> Double) -> Limiter

    A limiter on one of the rules that ship.

    Parameters

    • rule : how the limit moves; Vegas(vegas) by default, as in Netflix concurrency-limits
    • roll : a draw from 0 to 1, used by Vegas and MinRtt to spread their measurements of the fastest call so a fleet does not take them in step; a constant 0.5 by default

    MinRtt

    source
    pub(all) struct MinRtt {
      initial : Int
      min : Int
      max : Int
      smoothing : Double
      queue : Queue
      tolerance : Double
      probe : Int
      backoff : Double
    } derive(Eq, 
    Debug    )

    The first gradient rule: the gradient between the fastest response time seen and the latest one. Calls slower than the fastest by more than tolerance shrink the limit in proportion; a drop cuts it by backoff. Every probe calls or so, the limit falls to the queue size and the fastest time is measured afresh, so that a baseline taken when the service was idle does not hold forever.

    [Gradient] replaced it as the default: a minimum never forgets a lucky call, and the probes cost a dip in throughput. It is here for the services tuned to it. The preset is [min_rtt]; build another with [MinRtt::new], or update one in place with { ..min_rtt, probe: 0 }.

    MinRtt::equal

    source
    fn MinRtt::equal(MinRtt, MinRtt) -> Bool

    MinRtt::new

    source
    fn MinRtt::new(initial? : Int, min? : Int, max? : Int, smoothing? : Double, queue? : Queue, tolerance? : Double, probe? : Int, backoff? : Double) -> MinRtt

    The first gradient rule by name, every knob with the preset's value.

    Parameters

    • initial : the limit before any call has finished, 50 in the preset
    • min : the least the limit goes to, 1 in the preset
    • max : the most the limit goes to, 1000 in the preset
    • smoothing : how far towards a lower estimate the limit moves, 0.2 in the preset
    • queue : slots added to every estimate, Root(4) in the preset
    • tolerance : how much slower than the fastest call a call may be, 2.0 in the preset
    • probe : calls between probes of the fastest time, 1000 in the preset; 0 never probes
    • backoff : the share of the limit kept after a drop, 0.9 in the preset

    MinRtt::not_equal

    source
    fn MinRtt::not_equal(x : MinRtt, y : MinRtt) -> Bool

    MinRtt::to_repr

    source

    Outcome

    source
    pub(all) enum Outcome {
      Success
      Failure
      Drop
    } derive(Eq, 
    Debug    )

    How a call ended, as a breaker records it.

    A Drop is a call refused before it ran: traffic, but neither a success nor a failure of whatever is downstream.

    Outcome::equal

    source
    fn Outcome::equal(Outcome, Outcome) -> Bool

    Outcome::not_equal

    source
    fn Outcome::not_equal(x : Outcome, y : Outcome) -> Bool

    Outcome::to_repr

    source

    Partitioned

    source
    pub struct Partitioned[L] {
      make : (String) -> L
      limit : Int
      entries : Map[String, L]
    }

    A limiter per key: per client, per tenant, per route. make builds the limiter for a key the first time it is seen. At most limit keys are kept; past that the one used least recently is dropped, and comes back fresh if it is seen again.

    Partitioned::acquire

    source
    fn[L : Permit] Partitioned::acquire(self : Partitioned[L], key : String, now : Int64) -> Bool

    Admit one call for key at now.

    Partitioned::get

    source
    fn[L] Partitioned::get(self : Partitioned[L], key : String) -> L

    The limiter for key, made if this is the first time, and marked as used now.

    Partitioned::length

    source
    fn[L] Partitioned::length(self : Partitioned[L]) -> Int

    How many keys have a limiter now.

    Partitioned::new

    source
    fn[L] Partitioned::new(make : (String) -> L, limit? : Int) -> Partitioned[L]

    No key seen yet.

    Parameters

    • make : the limiter for a key seen for the first time
    • limit : the most keys kept, [partitions] unless given; at least one

    Partitioned::release

    source
    fn[L : Permit] Partitioned::release(self : Partitioned[L], key : String, now : Int64) -> Unit

    Take back the call last admitted for key.

    Period

    source
    pub struct Period {
      quota : Int
      period : Int64
      align : Bool
      state : Usage
    }

    A fixed window counter: at most quota events per period, the count starting again when a period ends.

    Cheaper than [Sliding], which keeps every stamp, and coarser: a client can spend one quota at the end of a period and another at the start of the next, so forty events can pass in a second that allows twenty. That is the trade a fixed window makes, and the reason it is the cheap one.

    A period opens on the first event after the last one closed. With align it opens on a multiple of period on the caller's clock instead, so with a Unix clock a daily quota resets at midnight UTC rather than a day after first use.

    Several processes share one quota by keeping its [Usage] in a store and counting with [Period::step], as a [Bucket] shares its tokens.
    impl Permit for Period

    Period::acquire

    source
    fn Period::acquire(self : Period, now : Int64) -> Bool

    Period::left

    source
    fn Period::left(self : Period, now : Int64) -> Int

    How many events remain in the current period.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Period::new

    source
    fn Period::new(quota : Int, period : 
    Span    , align? : Bool) -> Period

    A counter no event has touched. A period shorter than a nanosecond is raised to one.

    Parameters

    • quota : events allowed per period
    • period : how long a period is; anything under a nanosecond is raised to one
    • align : open periods on multiples of period on the caller's clock instead of a period after first use

    Period::next

    source
    fn Period::next(self : Period, now : Int64) -> 
    Span

    How long until the current period ends. Zero when no period is open, because the next event would open one.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Period::opening

    source
    fn Period::opening(self : Period) -> Usage

    The state of a period no event has touched: what a store with no entry for the key stands for.

    Period::release

    source
    fn Period::release(self : Period, now : Int64) -> Unit

    Period::step

    source
    fn Period::step(self : Period, state : Usage, now : Int64) -> Counted

    Count one event at now against state, leaving the period untouched.

    This is the whole of the arithmetic, and the seam for a quota shared by several processes: read the state out of the store — [Period::opening] when the key is missing — count here, and write state back only if the stored value has not changed in between; on a clash, read again and count again. The compare-and-set is what stops two writers from both spending the last event of a quota, as with [Bucket::step].

    Parameters

    • state : the period's state as the store has it
    • now : the caller's clock reading, in nanoseconds

    Period::take

    source
    fn Period::take(self : Period, now : Int64) -> Quota

    Count one event at now and say where the quota stands.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Promise

    source
    pub struct Promise {
      breaker : Breaker
    }

    A call the breaker let through. Exactly one of pass and fail settles it; until then the window holds no record of how the call went.

    Promise::fail

    source
    fn Promise::fail(self : Promise, now : Int64) -> Unit

    The call failed.

    Parameters

    • now : the clock reading the call finished at

    Promise::pass

    source
    fn Promise::pass(self : Promise, now : Int64) -> Unit

    The call succeeded.

    Parameters

    • now : the clock reading the call finished at

    Queue

    source
    pub(all) enum Queue {
      Root(Int)
      Fixed(Int)
    } derive(Eq, 
    Debug    )

    How many slots a [MinRtt] rule adds on top of each estimate.

    Queue::equal

    source
    fn Queue::equal(Queue, Queue) -> Bool

    Queue::not_equal

    source
    fn Queue::not_equal(x : Queue, y : Queue) -> Bool

    Queue::to_repr

    source

    Quota

    source
    pub(all) enum Quota {
      Under
      Hit
      Over
    } derive(Eq, 
    Debug    )

    Where a [Period] stands after an event: under the quota, exactly on it, or past it.

    The event that hits the quota is still admitted. It is told apart because it is the last one, which is when a caller would warn that the next will be refused.

    Quota::equal

    source
    fn Quota::equal(Quota, Quota) -> Bool

    Quota::not_equal

    source
    fn Quota::not_equal(x : Quota, y : Quota) -> Bool

    Quota::to_repr

    source

    Rule

    source
    pub(all) enum Rule {
      Fixed(Int)
      Aimd(Aimd)
      Vegas(Vegas)
      Gradient(Gradient)
      MinRtt(MinRtt)
    } derive(Eq, 
    Debug    )

    The rules that ship.

    Rule::equal

    source
    fn Rule::equal(Rule, Rule) -> Bool

    Rule::not_equal

    source
    fn Rule::not_equal(x : Rule, y : Rule) -> Bool

    Rule::to_repr

    source

    Sample

    source
    pub(all) struct Sample {
      rtt : Int64
      flying : Int
      dropped : Bool
    } derive(Eq, 
    Debug    )

    One finished call, as a rule sees it.

    Sample::equal

    source
    fn Sample::equal(Sample, Sample) -> Bool

    Sample::not_equal

    source
    fn Sample::not_equal(x : Sample, y : Sample) -> Bool

    Sample::to_repr

    source

    Shed

    source
    pub(all) struct Shed {
      size : Int
      window : 
    Span
      cpu : Int
      latency : 
    Span
      beta : Double
      cool : 
    Span
      floor : Double
    } derive(Eq, 
    Debug    )

    The constants of adaptive load shedding.

    The preset is [shed]. Build another with [Shed::new], or update one in place with { ..shed, cpu: 800 }.

    Shed::equal

    source
    fn Shed::equal(Shed, Shed) -> Bool

    Shed::new

    source
    fn Shed::new(size? : Int, window? : 
    Span    , cpu? : Int, latency? : 
    Span    , beta? : Double, cool? : 
    Span    , floor? : Double) -> Shed

    A policy by name, every knob with the preset's value.

    Parameters

    • size : buckets in the window, 50 in the preset
    • window : how far back that window reaches, 5 s in the preset
    • cpu : CPU use in thousandths at or above which the server counts as hot, 900 in the preset
    • latency : response time assumed while the window has none, 1 s in the preset
    • beta : weight of the old value in the moving average of calls in flight, 0.9 in the preset
    • cool : how long the server stays hot after the CPU falls back, 1 s in the preset
    • floor : the least share of capacity admitted however high the CPU is, 0.1 in the preset

    Shed::not_equal

    source
    fn Shed::not_equal(x : Shed, y : Shed) -> Bool

    Shed::to_repr

    source

    Shedder

    source
    pub struct Shedder {
      shed : Shed
      passes : Window[Total]
      latencies : Window[Total]
      flying : Int64
      average : Double
      hot : Int64?
      dropped : Bool
    }

    Load shedding that refuses work only when the server is both hot and full.

    Hot means the CPU is at or over the threshold, or was within the cooling time and the shedder has refused since. Full means more calls are in flight than the server has shown it can finish: the busiest bucket's passes per millisecond times the fastest bucket's mean response time, scaled down as the CPU nears 100%. Both the moving average and the current count of calls in flight must be over that figure.

    A CPU spike with nothing queued sheds nothing, and neither does a queue on an idle CPU. CPU use is read by the caller, in thousandths, and passed to each [Shedder::allow].

    Shedder::allow

    source
    fn Shedder::allow(self : Shedder, now : Int64, cpu : Int) -> Ticket?

    Ask to start one call at now, with the CPU at cpu thousandths.

    Some means go ahead, and the caller settles the ticket once the call has finished.

    Parameters

    • now : the caller's clock reading, in nanoseconds
    • cpu : CPU use in thousandths, read by the caller

    Shedder::flying

    source
    fn Shedder::flying(self : Shedder) -> Int64

    How many calls are in flight.

    Shedder::new

    source
    fn Shedder::new(shed? : Shed, now? : Int64) -> Shedder

    A shedder with empty windows opening at now.

    Parameters

    • shed : the policy; the shed preset by default
    • now : the clock reading both windows open at, 0L by default

    Sliding

    source
    pub struct Sliding {
      quota : Int
      window : 
    Span
      stamps : Array[Int64]
    }

    A sliding window counter: at most quota events in any window long stretch.

    Where a bucket smooths, a window counts. A quota of "a thousand a day" is a window, not a rate, and answering it with a bucket would let a client spend the whole day's allowance in the first minute.

    The stamps of the events in the window are kept, so the count is exact rather than the approximation a two-bucket scheme gives.
    impl Permit for Sliding

    Sliding::acquire

    source
    fn Sliding::acquire(self : Sliding, now : Int64) -> Bool

    Sliding::left

    source
    fn Sliding::left(self : Sliding, now : Int64) -> Int

    How many events remain in the current window.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Sliding::new

    source
    fn Sliding::new(quota : Int, window : 
    Span    ) -> Sliding

    An empty window.

    Parameters

    • quota : events allowed in any window-long stretch
    • window : the length of that stretch

    Sliding::next

    source
    fn Sliding::next(self : Sliding, now : Int64) -> 
    Span

    When the window will next have room, which is when its oldest event falls out.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Sliding::release

    source
    fn Sliding::release(self : Sliding, _now : Int64) -> Unit

    Sliding::take

    source
    fn Sliding::take(self : Sliding, now : Int64) -> Bool

    Whether an event is within the quota now, counting it if it is.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Sre

    source
    pub(all) struct Sre {
      k : Double
      floor : Double
      protection : Double
      probe : 
    Span
      size : Int
      window : 
    Span
    } derive(Eq, 
    Debug    )

    The constants of probability-based client-side throttling.

    The preset is [sre]. Build another with [Sre::new], or update one in place with { ..sre, k: 2.0 }.

    Sre::equal

    source
    fn Sre::equal(Sre, Sre) -> Bool

    Sre::new

    source
    fn Sre::new(k? : Double, floor? : Double, protection? : Double, probe? : 
    Span    , size? : Int, window? : 
    Span    ) -> Sre

    A policy by name, every knob with the preset's value.

    Parameters

    • k : how many calls each success pays for; the preset's 1.5 leaves a backend alone until more than a third of its calls fail
    • floor : what k decays to during an outage, 1.1 in the preset
    • protection : calls every backend gets regardless, 5 in the preset
    • probe : how long with nothing let through before one call is let through anyway
    • size : buckets in the window, 40 in the preset
    • window : how far back the window reaches, 10 s in the preset

    Sre::not_equal

    source
    fn Sre::not_equal(x : Sre, y : Sre) -> Bool

    Sre::to_repr

    source

    Tally

    source
    pub struct Tally {
      total : Int64
      success : Int64
      failure : Int64
      drop : Int64
    } derive(Default, Eq, 
    Debug    )

    Outcomes counted in one bucket.

    total counts every call including the dropped ones, so a bucket being throttled still reads as busy rather than idle.

    Tally::add

    source
    fn Tally::add(self : Tally, o : Outcome) -> Unit

    Count one call ending as o.

    Parameters

    • o : how the call ended

    Tally::default

    source
    fn Tally::default() -> Tally

    Tally::equal

    source
    fn Tally::equal(Tally, Tally) -> Bool

    Tally::not_equal

    source
    fn Tally::not_equal(x : Tally, y : Tally) -> Bool

    Tally::to_repr

    source

    Ticket

    source
    pub struct Ticket {
      shedder : Shedder
      start : Int64
    }

    A call the shedder let through. Exactly one of pass and fail settles it.

    Ticket::fail

    source
    fn Ticket::fail(self : Ticket) -> Unit

    The call failed. It leaves flight but says nothing about capacity.

    Ticket::pass

    source
    fn Ticket::pass(self : Ticket, now : Int64) -> Unit

    The call succeeded at now. Its response time counts towards what the server has shown it can do.

    Parameters

    • now : the clock reading the call finished at; its response time counts towards capacity

    Tokens

    source
    pub(all) struct Tokens {
      tokens : Double
      at : Int64
    } derive(Eq, 
    Debug    )

    What a bucket holds at one instant: the tokens, and when they were last brought up to date.

    This is the bucket apart from its policy and its clock, which is what two processes sharing one allowance have to agree on. See [Bucket::step].

    Tokens::equal

    source
    fn Tokens::equal(Tokens, Tokens) -> Bool

    Tokens::not_equal

    source
    fn Tokens::not_equal(x : Tokens, y : Tokens) -> Bool

    Tokens::to_repr

    source

    Total

    source
    pub struct Total {
      sum : Int64
      count : Int64
    } derive(Default, Eq, 
    Debug    )

    A sum and how many values went into it, which is a mean once count is positive.

    Total::add

    source
    fn Total::add(self : Total, v : Int64) -> Unit

    Add one value.

    Parameters

    • v : the value to fold into the sum

    Total::default

    source
    fn Total::default() -> Total

    Total::equal

    source
    fn Total::equal(Total, Total) -> Bool

    Total::not_equal

    source
    fn Total::not_equal(x : Total, y : Total) -> Bool

    Total::to_repr

    source

    Usage

    source
    pub(all) struct Usage {
      start : Int64?
      used : Int
    } derive(Eq, 
    Debug    )

    Where a [Period] stands: when the open period started, if one is open, and how many events it has counted. This is the whole of a period's state, and what a store shared by several processes holds.

    Usage::equal

    source
    fn Usage::equal(Usage, Usage) -> Bool

    Usage::not_equal

    source
    fn Usage::not_equal(x : Usage, y : Usage) -> Bool

    Usage::to_repr

    source

    Vegas

    source
    pub(all) struct Vegas {
      initial : Int
      max : Int
      smoothing : Double
      alpha : Int
      beta : Int
      threshold : Int
      step : Int
      probe : Int
    } derive(Eq, 
    Debug    )

    TCP Vegas carried over to requests: the shortest response time seen is the time with no queue, and how far a call's time is over it says how many calls are queued. A short queue grows the limit, a long one shrinks it.

    The thresholds scale with the limit: each is a multiple of the limit's decimal logarithm, rounded down and at least one, so a limit of 10 and one of 10 000 react in proportion.

    The preset is [vegas]. Build another with [Vegas::new], or update one in place with { ..vegas, smoothing: 0.5 }.

    Vegas::equal

    source
    fn Vegas::equal(Vegas, Vegas) -> Bool

    Vegas::new

    source
    fn Vegas::new(initial? : Int, max? : Int, smoothing? : Double, alpha? : Int, beta? : Int, threshold? : Int, step? : Int, probe? : Int) -> Vegas

    A rule by name, every knob with the preset's value.

    Parameters

    • initial : the limit before any call has finished, 20 in the preset
    • max : the most the limit goes to, 1000 in the preset
    • smoothing : how far towards each new estimate the limit moves, 1.0 in the preset
    • alpha : queued calls under which the limit grows a step, times the logarithm, 3 in the preset
    • beta : queued calls over which the limit shrinks a step, times the logarithm, 6 in the preset
    • threshold : queued calls at or under which the limit grows by beta, times the logarithm, 1 in the preset
    • step : one step, times the logarithm, 1 in the preset
    • probe : calls per unit of limit between measurements of the no-queue time, 30 in the preset

    Vegas::not_equal

    source
    fn Vegas::not_equal(x : Vegas, y : Vegas) -> Bool

    Vegas::to_repr

    source

    Window

    source
    pub struct Window[B] {
      width : Int64
      buckets : Array[B]
      head : Int
      start : Int64
    }

    A rolling window: size buckets that together cover the last window of time, oldest dropping off as time moves on.

    The bucket type is the caller's. A breaker keeps a [Tally] of outcomes per bucket, a shedder a [Total] of response times; anything with a Default can be a bucket, and the default is what a bucket is cleared back to.

    Time never moves on its own: every call carries the now it happens at. Writing at a later now clears the buckets the gap swept past; reading at a later now skips them without clearing, so a window nobody writes to still ages out, and two reads at the same now agree.

    Window::each

    source
    fn[B] Window::each(self : Window[B], now : Int64, f : (B) -> Unit, current? : Bool) -> Unit

    Visit the buckets still inside the window at now, oldest first.

    With current false the bucket now falls in is left out: it has seen only part of its time, and a reading that takes the busiest or the fastest bucket should not compare a partial one against whole ones.

    Parameters

    • now : the caller's clock reading, in nanoseconds
    • f : called once per live bucket, oldest first
    • current : whether to include the bucket now falls in, true by default

    Window::new

    source
    fn[B : Default] Window::new(size~ : Int, window~ : 
    Span    , now? : Int64) -> Window[B]

    An empty window of size buckets spanning window, its first bucket opening at now.

    A size below one is raised to one, and so is a bucket narrower than a nanosecond.

    Parameters

    • size : how many buckets; anything under one is raised to one
    • window : how much time the buckets together cover
    • now : the clock reading the first bucket opens at, 0L by default

    Window::size

    source
    fn[B] Window::size(self : Window[B]) -> Int

    How many buckets the window has.

    Window::slot

    source
    fn[B : Default] Window::slot(self : Window[B], now : Int64) -> B

    The bucket now falls in, for the caller to add to.

    The buckets the gap since the last write swept past are cleared first. The head is snapped back to a bucket boundary, so buckets keep their width rather than drifting with the arrival time of each write.

    Parameters

    • now : the caller's clock reading, in nanoseconds

    Window::width

    source
    fn[B] Window::width(self : Window[B]) -> 
    Span

    How much time one bucket covers.

    aimd

    source
    let aimd : Aimd

    The preset, the same numbers as Netflix concurrency-limits: start at 20, stay within 20 and 200, keep 90% on a drop, and count a call over five seconds as one.

    backoff

    source
    let backoff : Backoff

    The preset: three attempts, a hundred milliseconds doubling to a one-second ceiling, spread by a fifth either way.

    Each wait is the computed ceiling spread by a fifth below or above it, so a plan of 0.8 to 1.2 of the schedule is easy to explain to whoever asks why the second retry waited 168 ms and not 200.

    gradient

    source
    let gradient : Gradient

    The preset, the same numbers as Netflix concurrency-limits' second gradient rule: start at 20, stay within 20 and 200, move a fifth of the way, four slots on top, a 600-call average and a tolerance of 1.5.

    min_rtt

    source
    let min_rtt : MinRtt

    The preset, the same numbers as Netflix concurrency-limits' first gradient rule (GradientLimit): start at 50, stay within 1 and 1000, move a fifth of the way down, the square root of the limit (at least 4) on top, a tolerance of 2, a probe every 1000 to 2000 calls, and nine tenths kept after a drop.

    partitions

    source
    let partitions : Int

    The most keys a [Partitioned] keeps a limiter for unless told otherwise. No common default exists for this; a bound is the safe side, since a key per client address would otherwise grow without end, and ten thousand limiters of a few words each fit in well under a megabyte.

    shed

    source
    let shed : Shed

    The preset: five seconds in fifty buckets, overloaded at 90% CPU, a one-second assumed response time, an average weighted 0.9, one second to cool and never less than a tenth of capacity.

    The memory is short on purpose: a five-second window follows the load it is in rather than averaging a burst away, and the floor keeps a server already at full CPU from being told it can serve nobody.

    sre

    source
    let sre : Sre

    The preset: k 1.5 decaying to 1.1, five calls of protection, a probe every second, and ten seconds of history in forty buckets.

    k and protection are the slope and the floor of the standard client-side throttling formula; the decay, the probe and the recovery scaling are what keep a long outage from hurting past successes, and a short one from being remembered for long.

    vegas

    source
    let vegas : Vegas

    The preset, the same numbers as Netflix concurrency-limits, whose default rule this is: start at 20, at most 1000, no smoothing, alpha 3, beta 6, threshold and step 1, and a fresh no-queue time every 15 to 30 calls per unit of limit.

    Source Files