🥮 mooncakes.io

    #gaato/mttp/client/retry

    Retry decisions as data. A RetryRule says which responses and transport failures may be retried, for which methods, how long to wait and when to give up; RetryRule::decide turns one failed attempt into Wait(ms) or Stop(reason). The package never sleeps, sends or reads a clock: the caller passes the jitter sample, the current time (for an HTTP-date Retry-After) and the time left in its budget, so one rule gives the same answer on every backend.

    • statuses lists retryable statuses (Code, Range) with the methods they apply to; Idempotent means the caller's idempotent fact, which gaato/mttp/client takes from the method or the replay_safe attribute, never from an Idempotency-Key header alone.
    • unsent_failures retries a TransportError that proves nothing was sent (is_unsent) whatever the method; sent_failures covers the rest.
    • retry_after reads the server's delay (Retry-After, or retry-after-ms first) and bounds it: stop above a limit, fall back to the backoff, or wait as asked. parse_retry_after and retry_after_ms are the parsers.
    • backoff doubles base_ms per attempt up to max_ms, spread by Jitter.

    gaato/mttp/client wraps a rule in RetryPolicy and applies it in its one send loop; vendor presets belong to the SDK that needs them.

    ///|
    test "a rate-limited POST waits as the server asks" {
      let rule : @retry.RetryRule = {
        ..@retry.RetryRule::never(),
        max_retries: 3,
        statuses: [{ statuses: [Code(429)], methods: Any, }],
        retry_after: {
          source: Header,
          cap: StopAbove(60000),
          zero_means_backoff: false,
        },
      }
      let post = @http.Request::post("https://api.example/messages").head()
      let limited = @retry.Response(
        status=429,
        headers=@http.Headers::of([("Retry-After", "2")]),
      )
      let decide = attempt => {
        rule.decide(
          limited,
          post,
          attempt~,
          random=0.5,
          idempotent=false,
          replayable=true,
        )
      }
      assert_eq(decide(0), Wait(2000))
      assert_eq(decide(3), Stop(Exhausted))
      assert_eq(@retry.parse_retry_after("1.5"), Some(1500))
    }

    AttemptOutcome

    source
    pub(all) enum AttemptOutcome {
      Response(status~ : Int, headers~ : 
    Headers    )
      Failure(
    TransportError    )
    } derive(
    Debug    )

    The result of an attempt that did not succeed.

    AttemptOutcome::to_repr

    source

    Debug representation of an outcome.

    Backoff

    source
    pub(all) struct Backoff {
      base_ms : Int
      max_ms : Int
      jitter : Jitter
    } derive(Eq, 
    Debug    )

    Exponential backoff: base_ms doubled per attempt up to max_ms, then jittered.

    Backoff::delay_ms

    source
    fn Backoff::delay_ms(self : Backoff, attempt : Int, random : Double) -> Int

    The delay before retry number attempt + 1, in milliseconds.

    The exponential delay is base_ms * 2^attempt up to max_ms; a negative attempt counts as 0, and a nonpositive base_ms or max_ms gives 0. random is clamped into [0, 1), NaN to 0. The jitter then applies: NoJitter keeps the delay, Full takes trunc(delay * random), and Proportional(lo~, hi~) multiplies by lo + trunc(random * (hi - lo)) per mille, in integers. The jittered delay is capped at max_ms again, so a hi above 1000 spreads delays below the cap but never past it.

    test {
      let backoff : @retry.Backoff = {
        base_ms: 500,
        max_ms: 30000,
        jitter: Proportional(lo=800, hi=1200),
      }
      assert_eq(backoff.delay_ms(2, 0.5), 2000)
      assert_eq(backoff.delay_ms(0, 0.0), 400)
    }

    Backoff::equal

    source
    fn Backoff::equal(Backoff, Backoff) -> Bool

    Equality for backoff settings.

    Backoff::not_equal

    source
    fn Backoff::not_equal(x : Backoff, y : Backoff) -> Bool

    Equality for backoff settings.

    Backoff::to_repr

    source

    Debug representation of backoff settings.

    Decision

    source
    pub(all) enum Decision {
      Wait(Int)
      Stop(StopReason)
    } derive(Eq, 
    Debug    )

    What to do after a failed attempt.

    Decision::equal

    source
    fn Decision::equal(Decision, Decision) -> Bool

    Equality for decisions.

    Decision::not_equal

    source
    fn Decision::not_equal(x : Decision, y : Decision) -> Bool

    Equality for decisions.

    Decision::to_repr

    source

    Debug representation of a decision.

    Jitter

    source
    pub(all) enum Jitter {
      NoJitter
      Full
      Proportional(lo~ : Int, hi~ : Int)
    } derive(Eq, 
    Debug    )

    How the exponential delay is spread, so that clients do not retry in lockstep. Multipliers are per mille.

    Jitter::equal

    source
    fn Jitter::equal(Jitter, Jitter) -> Bool

    Equality for jitter settings.

    Jitter::not_equal

    source
    fn Jitter::not_equal(x : Jitter, y : Jitter) -> Bool

    Equality for jitter settings.

    Jitter::to_repr

    source

    Debug representation of a jitter setting.

    Methods

    source
    pub(all) enum Methods {
      Never
      Idempotent
      Listed(Array[
    Method    ])
      Any
    } derive(Eq, 
    Debug    )

    Which request methods a retry is allowed for.

    Methods::equal

    source
    fn Methods::equal(Methods, Methods) -> Bool

    Equality for method sets.

    Methods::not_equal

    source
    fn Methods::not_equal(x : Methods, y : Methods) -> Bool

    Equality for method sets.

    Methods::to_repr

    source

    Debug representation of a method set.

    RetryAfter

    source
    pub(all) struct RetryAfter {
      source : RetryAfterSource
      cap : RetryAfterCap
      zero_means_backoff : Bool
    } derive(Eq, 
    Debug    )

    How a server-requested delay is read and bounded.

    RetryAfter::equal

    source
    fn RetryAfter::equal(RetryAfter, RetryAfter) -> Bool

    Equality for delay settings.

    RetryAfter::not_equal

    source
    fn RetryAfter::not_equal(x : RetryAfter, y : RetryAfter) -> Bool

    Equality for delay settings.

    RetryAfter::to_repr

    source

    Debug representation of delay settings.

    RetryAfterCap

    source
    pub(all) enum RetryAfterCap {
      StopAbove(Int)
      BackoffAbove(Int)
      NoCap
    } derive(Eq, 
    Debug    )

    What to do with a server-requested delay longer than a limit, in milliseconds.

    RetryAfterCap::equal

    source

    Equality for delay caps.

    RetryAfterCap::not_equal

    source
    fn RetryAfterCap::not_equal(x : RetryAfterCap, y : RetryAfterCap) -> Bool

    Equality for delay caps.

    RetryAfterCap::to_repr

    source

    Debug representation of a delay cap.

    RetryAfterSource

    source
    pub(all) enum RetryAfterSource {
      Ignore
      Header
      MsThenHeader
    } derive(Eq, 
    Debug    )

    Where a server-requested delay is read from.

    RetryAfterSource::equal

    source

    Equality for delay sources.

    RetryAfterSource::not_equal

    source
    fn RetryAfterSource::not_equal(x : RetryAfterSource, y : RetryAfterSource) -> Bool

    Equality for delay sources.

    RetryAfterSource::to_repr

    source

    Debug representation of a delay source.

    RetryRule

    source
    pub(all) struct RetryRule {
      max_retries : Int
      statuses : Array[StatusRule]
      sent_failures : Methods
      unsent_failures : Bool
      override_header : String?
      retry_after : RetryAfter
      backoff : Backoff
    } derive(Eq, 
    Debug    )

    A retry policy as data: what may be retried, how long to wait, and when to stop. decide applies it to one failed attempt.

    test {
      let rule : @retry.RetryRule = {
        ..@retry.RetryRule::never(),
        max_retries: 2,
        statuses: [{ statuses: [Code(429), Range(500, 599)], methods: Idempotent, }],
        backoff: { base_ms: 500, max_ms: 8000, jitter: NoJitter, },
      }
      let get = @http.Request::get("/items")
      let busy = @retry.Response(status=503, headers=@http.Headers::empty())
      assert_eq(
        rule.decide(
          busy,
          get.head(),
          attempt=1,
          random=0.0,
          idempotent=true,
          replayable=true,
        ),
        Wait(1000),
      )
      assert_eq(
        rule.decide(
          busy,
          get.head(),
          attempt=2,
          random=0.0,
          idempotent=true,
          replayable=true,
        ),
        Stop(Exhausted),
      )
    }

    RetryRule::decide

    source
    fn RetryRule::decide(self : RetryRule, outcome : AttemptOutcome, request : 
    RequestHead    , attempt~ : Int, random~ : Double, now_unix_ms? : Int64, remaining_ms? : Int64, server_delay_ms? : Int, idempotent~ : Bool, replayable~ : Bool) -> Decision

    Decides whether to retry after a failed attempt, and how long to wait.

    attempt counts the retries already made: 0 after the first attempt fails. The checks run in order, and the first that applies wins:

    1. attempt >= max_retries stops with Exhausted.
    2. A body that cannot be sent again (replayable false) stops with NotReplayable.
    3. A response carrying override_header with the value true (ignoring case and surrounding whitespace) skips step 4; false stops with ServerSaidNo; any other value is ignored.
    4. The outcome must be retryable for the request's method: a response through the first StatusRule containing its status; Protocol never; Proxy when unsent_failures is set and its status matches a StatusRule; another unsent failure when unsent_failures is set; any other failure through sent_failures. Idempotent methods are those of a request for which idempotent is true.
    5. The wait is server_delay_ms when given, otherwise the response's header per retry_after.source (an HTTP date needs now_unix_ms), bounded by retry_after.cap; without one it is backoff.delay_ms(attempt, random).
    6. With remaining_ms, a wait that is not shorter than it stops with Budget(wait).

    test {
      let rule : @retry.RetryRule = {
        ..@retry.RetryRule::never(),
        max_retries: 2,
        statuses: [{ statuses: [Code(429)], methods: Any, }],
        retry_after: {
          source: Header,
          cap: StopAbove(60000),
          zero_means_backoff: false,
        },
      }
      let post = @http.Request::post("/messages")
      let headers = @http.Headers::of([("Retry-After", "3")])
      let limited = @retry.Response(status=429, headers~)
      assert_eq(
        rule.decide(
          limited,
          post.head(),
          attempt=0,
          random=0.5,
          idempotent=false,
          replayable=true,
        ),
        Wait(3000),
      )
      assert_eq(
        rule.decide(
          limited,
          post.head(),
          attempt=0,
          random=0.5,
          remaining_ms=2000,
          idempotent=false,
          replayable=true,
        ),
        Stop(Budget(3000)),
      )
    }

    RetryRule::equal

    source
    fn RetryRule::equal(RetryRule, RetryRule) -> Bool

    Equality for retry rules.

    RetryRule::never

    source
    fn RetryRule::never() -> RetryRule

    A rule that never retries, and a base to build others on with { ..never(), ... }.

    RetryRule::not_equal

    source
    fn RetryRule::not_equal(x : RetryRule, y : RetryRule) -> Bool

    Equality for retry rules.

    RetryRule::to_repr

    source

    Debug representation of a retry rule.

    StatusMatch

    source
    pub(all) enum StatusMatch {
      Code(Int)
      Range(Int, Int)
    } derive(Eq, 
    Debug    )

    A status code, or an inclusive range of status codes.

    test {
      let rule : @retry.StatusRule = {
        statuses: [Code(429), Range(500, 599)],
        methods: Any,
      }
      assert_eq(rule.statuses.length(), 2)
    }

    StatusMatch::equal

    source
    fn StatusMatch::equal(StatusMatch, StatusMatch) -> Bool

    Equality for status matches.

    StatusMatch::not_equal

    source
    fn StatusMatch::not_equal(x : StatusMatch, y : StatusMatch) -> Bool

    Equality for status matches.

    StatusMatch::to_repr

    source

    Debug representation of a status match.

    StatusRule

    source
    pub(all) struct StatusRule {
      statuses : Array[StatusMatch]
      methods : Methods
    } derive(Eq, 
    Debug    )

    Response statuses that may be retried, and for which methods.

    StatusRule::equal

    source
    fn StatusRule::equal(StatusRule, StatusRule) -> Bool

    Equality for status rules.

    StatusRule::not_equal

    source
    fn StatusRule::not_equal(x : StatusRule, y : StatusRule) -> Bool

    Equality for status rules.

    StatusRule::to_repr

    source

    Debug representation of a status rule.

    StopReason

    source
    pub(all) enum StopReason {
      Exhausted
      NotRetryable
      MethodNotAllowed
      NotReplayable
      ServerSaidNo
      RetryAfterTooLong(Int)
      Budget(Int)
    } derive(Eq, 
    Debug    )

    Why decide stopped.

    StopReason::equal

    source
    fn StopReason::equal(StopReason, StopReason) -> Bool

    Equality for stop reasons.

    StopReason::not_equal

    source
    fn StopReason::not_equal(x : StopReason, y : StopReason) -> Bool

    Equality for stop reasons.

    StopReason::to_repr

    source

    Debug representation of a stop reason.

    parse_retry_after

    source
    fn parse_retry_after(value : String, now_unix_ms? : Int64) -> Int?

    Parses nonnegative seconds (including decimals) or, given a clock, an HTTP-date in any of RFC 9110's three forms. Results are truncated to milliseconds and saturated at Int max.

    test {
      assert_eq(@retry.parse_retry_after("1.2349"), Some(1234))
    }

    retry_after_ms

    source
    fn retry_after_ms(headers : 
    Headers    , now_unix_ms? : Int64) -> Int?

    Prefers the first retry-after-ms value; falls back to retry-after when that header is absent or unparseable, as the official OpenAI SDKs do.

    Source Files