🥮 mooncakes.io

    #gaato/mttp_core/h1

    A sans-IO HTTP/1.1 codec.

    It turns bytes into heads, framing and body payloads and back, and never performs I/O. The client and server connection engines of gaato/mttp_core read and write every message with it, so both refuse the same inputs. It follows the message syntax and framing of RFC 9112, stricter where lenient parsing lets two implementations read one message differently.

    #Reading a message

    Feed arriving bytes to a HeadDecoder until it returns the head; the bytes after consumed start the body. Parse the head, resolve its framing, then read the body by length or through a ChunkedDecoder. Every decoder scans each byte once, keeps only bounded state and is sticky: after an error, every later call raises the same error.

    ///|
    test "read a chunked request" {
      let wire = b"POST /upload HTTP/1.1\r\nHost: example.com\r\nTransfer-Encoding: chunked\r\n\r\n5\r\nhello\r\n0\r\n\r\n"
      let heads = @h1.HeadDecoder::new(skip_leading_crlf=true)
      let result = heads.feed(wire)
      guard result.head is Some(head) else { fail("head incomplete") }
      let request = @h1.parse_request_head(head)
      assert_eq(request.meth, Post)
      assert_eq(request.target, "/upload")
      guard @h1.request_framing(request.headers) is Chunked else {
        fail("expected a chunked body")
      }
      let body = @h1.ChunkedDecoder::new(@h1.Limits::default())
      let decoded = body.feed(wire[result.consumed:])
      assert_eq(decoded.chunks, [b"hello"])
      assert_true(decoded.done)
      body.finish()
    }

    A response's framing also depends on the request method and the status: a response to HEAD, a 1xx, 204 or 304 and a successful CONNECT have no body, and a response with neither Content-Length nor Transfer-Encoding ends when the connection closes.

    ///|
    test "resolve response framing" {
      let response = @h1.parse_response_head(
        b"HTTP/1.1 200\r\nContent-Length: 12\r\n\r\n",
      )
      assert_eq(
        @h1.response_framing(response.headers, status=response.status),
        Length(12),
      )
      assert_eq(
        @h1.response_framing(response.headers, status=response.status, meth=Head),
        NoBody,
      )
    }

    #Writing a message

    write_request_head and write_response_head replace Content-Length and Transfer-Encoding with the ones the given Framing implies and raise instead of emitting bytes the parser would refuse. What they write reads back with the same framing: NoBody becomes Content-Length: 0 on a response that would otherwise be read until close, and on a POST, PUT or PATCH request, while a response that never has a body (to HEAD or a successful CONNECT, a 1xx, 204 or 304) and a request whose method expects no content get no framing field. write_response_head takes the request method for this, as response_framing does. encode_chunk frames one chunk; an empty one ends the body.

    ///|
    test "write a chunked response" {
      let head = @h1.write_response_head(
        200,
        @mttp_core.Headers::of([("Content-Type", "text/plain")]),
        Chunked,
        Http11,
      )
      let parsed = @h1.parse_response_head(head)
      assert_eq(@h1.response_framing(parsed.headers, status=parsed.status), Chunked)
      assert_eq(@h1.encode_chunk(b"hi"), b"2\r\nhi\r\n")
    }

    #What is refused

    Every refusal is a ParseError whose kind says how to answer:

    KindInputsA server answers
    HeadTooLargea head, chunk-size line or trailer section past its Limits431 for a head; 400 inside a body
    Malformedbare CR or LF, an empty line before a status line, obs-fold, whitespace before a colon, controls in a field, a missing or invalid Host on HTTP/1.1, an invalid target form, any duplicate Content-Length (even identical), an invalid or overflowing length or chunk size400
    SmugglingContent-Length with Transfer-Encoding, Transfer-Encoding on HTTP/1.0, a framing or routing field in trailers400
    Unsupporteda transfer coding other than a single chunked, an HTTP version other than 1.x501 or 505

    Each case closes the connection. ParseKind may grow, so a match on it needs a wildcard arm.

    Accepted although some parsers refuse them: a status line without a reason phrase (HTTP/1.1 200), obs-text in field values (kept as Latin-1 code points), and an HTTP/1 minor version above 1, read as HTTP/1.1.

    A server skips empty lines (CRLF) before a request line, as RFC 9112 section 2.2 asks, because some clients send an extra CRLF after a request body: parse_request_head does, and so does a HeadDecoder or scan_head with skip_leading_crlf=true. The skipped bytes count against the head limit. A leading bare LF is still refused, and so is an empty line before a status line.

    #Limits

    BoundDefault
    Head, start line, final CRLF CRLF and skipped empty lines included16 KiB
    Fields per head, and per trailer section100
    Chunk-size line, extensions and CRLF included4 KiB
    Trailer section, final CRLF included16 KiB

    Chunk payloads are never buffered: a chunk declared as gigabytes long is returned in pieces as its bytes arrive.

    ParseError

    source
    pub(all) suberror ParseError {
      ParseError(kind~ : ParseKind, detail~ : String)
    } derive(
    Debug    )

    A codec refusal. detail is diagnostic text, not a stable contract; match on kind.

    ParseError::to_repr

    source

    Formats codec refusals.

    ChunkedDecoder

    source
    pub struct ChunkedDecoder {
      // private fields
    }

    An incremental decoder of the chunked transfer coding (RFC 9112, section 7.1). It keeps only the current size or trailer line and the trailer fields, never a chunk's payload, so its memory is bounded by Limits whatever chunk sizes the peer declares. Errors are sticky.

    ChunkedDecoder::feed

    source
    fn ChunkedDecoder::feed(self : ChunkedDecoder, bytes : BytesView) -> Decoded raise ParseError

    Consumes the bytes of bytes that belong to the body and returns the payload they carried. Report the end of input with finish: a zero-size chunk is not complete until the trailer section's final CRLF arrives. A size line or trailer section past its limit raises HeadTooLarge; a trailer that names a framing or routing field raises Smuggling.

    ChunkedDecoder::finish

    source
    fn ChunkedDecoder::finish(self : ChunkedDecoder) -> Unit raise ParseError

    Checks at end of input that the body ended; a truncated body raises, and the error is sticky.

    ChunkedDecoder::new

    source
    fn ChunkedDecoder::new(limits : Limits) -> ChunkedDecoder

    Creates a decoder bounded by limits.max_chunk_line_bytes per size line and by max_trailer_bytes and max_headers for the trailer section. A negative limit is refused by the first feed.

    test {
      let decoder = @h1.ChunkedDecoder::new(@h1.Limits::default())
      let result = decoder.feed(b"4\r\nWiki\r\n0\r\nX-Sum: 1\r\n\r\nnext")
      assert_eq(result.chunks, [b"Wiki"])
      assert_true(result.done)
      assert_eq(result.trailers.get("x-sum"), Some("1"))
      assert_eq(result.consumed, 24)
      decoder.finish()
    }

    Decoded

    source
    pub struct Decoded {
      chunks : Array[Bytes]
      done : Bool
      trailers : 
    Headers
      consumed : Int
    } derive(Eq, 
    Debug    )

    What one ChunkedDecoder::feed produced. chunks are nonempty payload pieces whose boundaries follow the input. trailers holds the trailer fields received so far and is complete once done. consumed counts the bytes of this feed that belong to the body; bytes after it belong to the next message.

    Decoded::equal

    source
    fn Decoded::equal(Decoded, Decoded) -> Bool

    Compares decoder results.

    Decoded::not_equal

    source
    fn Decoded::not_equal(x : Decoded, y : Decoded) -> Bool

    Compares decoder results.

    Decoded::to_repr

    source

    Formats decoder results.

    Framing

    source
    pub(all) enum Framing {
      NoBody
      Length(Int64)
      Chunked
      UntilClose
    } derive(Eq, 
    Debug    )

    How the bytes after a head belong to its message body.

    • NoBody: the message has no body; the next bytes start the next message (or, after a successful CONNECT, the tunnel).
    • Length(n): exactly n bytes follow.
    • Chunked: the body is chunked; decode it with ChunkedDecoder.
    • UntilClose: the body ends when the peer closes (responses only).

    Framing::equal

    source
    fn Framing::equal(Framing, Framing) -> Bool

    Compares body framing.

    Framing::not_equal

    source
    fn Framing::not_equal(x : Framing, y : Framing) -> Bool

    Compares body framing.

    Framing::to_repr

    source

    Formats body framing.

    HeadDecoder

    source
    pub struct HeadDecoder {
      // private fields
    }

    A bounded head accumulator that scans each byte once, so feeding a head in any number of pieces costs linear time. After completion, later feeds return the same head with zero bytes consumed. Errors are sticky: every later feed or finish raises the first error again.

    HeadDecoder::feed

    source
    fn HeadDecoder::feed(self : HeadDecoder, bytes : BytesView) -> HeadResult raise ParseError

    Takes the bytes of bytes that belong to the head, up to the decoder's limit, and returns the head once its final CRLF CRLF arrived.

    HeadDecoder::finish

    source
    fn HeadDecoder::finish(self : HeadDecoder) -> Bytes raise ParseError

    Returns the completed head at end of input, or refuses a truncated one.

    HeadDecoder::new

    source
    fn HeadDecoder::new(max? : Int, skip_leading_crlf? : Bool) -> HeadDecoder

    Starts a head accumulator that refuses heads longer than max bytes, final CRLF CRLF included (default 16 KiB, as Limits::default). With skip_leading_crlf, empty lines (CRLF) before the start line are consumed, left out of the returned head and counted against max, as a server reading requests should (RFC 9112, section 2.2); a client reading responses leaves it off, so a leading empty line is refused.

    test {
      let decoder = @h1.HeadDecoder::new()
      assert_eq(decoder.feed(b"HTTP/1.1 200 OK\r\n").consumed, 17)
      let result = decoder.feed(b"\r\nbody")
      assert_eq(result.consumed, 2)
      assert_eq(result.head, Some(b"HTTP/1.1 200 OK\r\n\r\n"))
      let requests = @h1.HeadDecoder::new(skip_leading_crlf=true)
      let result = requests.feed(b"\r\nGET / HTTP/1.1\r\nHost: h\r\n\r\n")
      assert_eq(result.consumed, 29)
      assert_eq(result.head, Some(b"GET / HTTP/1.1\r\nHost: h\r\n\r\n"))
    }

    HeadResult

    source
    pub struct HeadResult {
      head : Bytes?
      consumed : Int
    } derive(Eq, 
    Debug    )

    The result of feeding a HeadDecoder: the head once it is complete, and how many of the fed bytes it took. The bytes after consumed belong to the body or to a pipelined message; the decoder never keeps them.

    HeadResult::equal

    source
    fn HeadResult::equal(HeadResult, HeadResult) -> Bool

    Compares head decoder results.

    HeadResult::not_equal

    source
    fn HeadResult::not_equal(x : HeadResult, y : HeadResult) -> Bool

    Compares head decoder results.

    HeadResult::to_repr

    source

    Formats head decoder results.

    Limits

    source
    pub(all) struct Limits {
      max_head_bytes : Int
      max_chunk_line_bytes : Int
      max_trailer_bytes : Int
      max_headers : Int
    } derive(Eq, 
    Debug    )

    Input bounds of the codec. Byte limits include terminating CRLFs. The field count excludes the start line and applies to a head and, separately, to the trailer section of a chunked body. A negative limit is refused when it is first used.

    Limits::default

    source
    fn Limits::default() -> Limits

    Defaults: 16 KiB heads, 4 KiB chunk-size lines, 16 KiB trailer sections, 100 fields.

    test {
      let limits = @h1.Limits::default()
      assert_eq(limits.max_head_bytes, 16384)
      let strict = { ..limits, max_headers: 20, }
      assert_eq(strict.max_headers, 20)
    }

    Limits::equal

    source
    fn Limits::equal(Limits, Limits) -> Bool

    Compares codec limits.

    Limits::not_equal

    source
    fn Limits::not_equal(x : Limits, y : Limits) -> Bool

    Compares codec limits.

    Limits::to_repr

    source

    Formats codec limits.

    ParseKind

    source
    pub(all) enum ParseKind {
      HeadTooLarge
      Malformed
      Smuggling
      Unsupported
    } derive(Eq, 
    Debug    )

    Why the codec refused input, which decides the answer and that the connection closes. More kinds may be added; a match needs a wildcard.

    • HeadTooLarge: a head, a chunk-size line or a trailer section exceeded its byte or field limit (a server answers 431 for a head).
    • Malformed: the bytes are not valid RFC 9112 syntax (400).
    • Smuggling: the framing is ambiguous between implementations, such as Content-Length with Transfer-Encoding (400).
    • Unsupported: valid syntax the codec does not implement, such as a transfer coding other than chunked (501) or HTTP/2 (505).

    ParseKind::equal

    source
    fn ParseKind::equal(ParseKind, ParseKind) -> Bool

    Compares parse error kinds.

    ParseKind::not_equal

    source
    fn ParseKind::not_equal(x : ParseKind, y : ParseKind) -> Bool

    Compares parse error kinds.

    ParseKind::to_repr

    source

    Formats parse error kinds.

    ParsedRequest

    source
    pub struct ParsedRequest {
      meth : 
    Method
      target : String
      version : Version
      headers : 
    Headers
    } derive(Eq, 
    Debug    )

    A parsed request head. The fields are read-only, so later versions can add fields without breaking callers.

    ParsedRequest::equal

    source

    Compares parsed request heads.

    ParsedRequest::not_equal

    source
    fn ParsedRequest::not_equal(x : ParsedRequest, y : ParsedRequest) -> Bool

    Compares parsed request heads.

    ParsedRequest::to_repr

    source

    Formats parsed request heads.

    ParsedResponse

    source
    pub struct ParsedResponse {
      version : Version
      status : Int
      headers : 
    Headers
    } derive(Eq, 
    Debug    )

    A parsed response head. The reason phrase is validated and dropped: it carries no information (RFC 9112, section 4).

    ParsedResponse::equal

    source

    Compares parsed response heads.

    ParsedResponse::not_equal

    source
    fn ParsedResponse::not_equal(x : ParsedResponse, y : ParsedResponse) -> Bool

    Compares parsed response heads.

    ParsedResponse::to_repr

    source

    Formats parsed response heads.

    Version

    source
    pub(all) enum Version {
      Http10
      Http11
    } derive(Eq, 
    Debug    )

    The wire versions the codec reads and writes. A received HTTP/1.x with a minor version above 1 is read as Http11 (RFC 9112, section 2.3).

    Version::equal

    source
    fn Version::equal(Version, Version) -> Bool

    Compares wire versions.

    Version::not_equal

    source
    fn Version::not_equal(x : Version, y : Version) -> Bool

    Compares wire versions.

    Version::to_repr

    source

    Formats wire versions.

    encode_chunk

    source
    fn encode_chunk(bytes : BytesView) -> Bytes

    Encodes one chunk of the chunked coding; an empty bytes encodes the last chunk with an empty trailer section, which ends the body.

    test {
      assert_eq(@h1.encode_chunk(b"Wiki"), b"4\r\nWiki\r\n")
      assert_eq(@h1.encode_chunk(b""), b"0\r\n\r\n")
    }

    parse_request_head

    source
    fn parse_request_head(head : BytesView, limits? : Limits) -> ParsedRequest raise ParseError

    Parses a strict request head: one complete head within limits (empty lines before the request line are skipped and counted), a request line of three parts separated by single spaces, a target of the form its method requires, and at most one valid Host, which HTTP/1.1 requires. Fields keep their order and duplicates. Body framing is checked separately by request_framing, except that Transfer-Encoding on HTTP/1.0 is refused here as smuggling.

    test {
      let request = @h1.parse_request_head(
        b"GET /items?page=2 HTTP/1.1\r\nHost: example.com\r\nAccept: */*\r\n\r\n",
      )
      assert_eq(request.meth, Get)
      assert_eq(request.target, "/items?page=2")
      assert_eq(request.version, Http11)
      assert_eq(request.headers.get("host"), Some("example.com"))
    }

    parse_response_head

    source
    fn parse_response_head(head : BytesView, limits? : Limits) -> ParsedResponse raise ParseError

    Parses a strict response head: one complete head within limits, with no empty line before it, and a status line of the version, a single space, three digits from 100 to 599 and an optional reason phrase after another space. A status line without a reason phrase, with or without the space before it, is accepted (RFC 9112, section 4). Fields keep their order, duplicates and obs-text.

    test {
      let response = @h1.parse_response_head(
        b"HTTP/1.1 200\r\nSet-Cookie: a=1\r\nSet-Cookie: b=2\r\n\r\n",
      )
      assert_eq(response.status, 200)
      assert_eq(response.headers.get_all("set-cookie"), ["a=1", "b=2"])
    }

    request_framing

    source
    fn request_framing(headers : 
    Headers    ) -> Framing raise ParseError

    Resolves the framing of a request body: Length from one valid Content-Length, Chunked from a Transfer-Encoding of exactly chunked, and NoBody when neither field is present. Refuses Content-Length with Transfer-Encoding as smuggling, any duplicate or invalid Content-Length as malformed, and any other transfer coding as unsupported.

    test {
      let chunked = @mttp_core.Headers::of([("Transfer-Encoding", "chunked")])
      assert_eq(@h1.request_framing(chunked), Chunked)
      let both = chunked.add("Content-Length", "5")
      try @h1.request_framing(both) catch {
        @h1.ParseError(kind~, ..) => assert_eq(kind, Smuggling)
      } noraise {
        _ => fail("CL+TE must be refused")
      }
    }

    response_framing

    source
    fn response_framing(headers : 
    Headers    , status~ : Int, meth? : 
    Method    ) -> Framing raise ParseError

    Resolves the framing of a response body to meth. The fields are checked as for request_framing even when the status has no body; then a response to HEAD, a 1xx, 204 or 304, and a 2xx to CONNECT have NoBody, and a response with neither field reads UntilClose. After a successful CONNECT the caller hands the remaining bytes to the tunnel.

    scan_head

    source
    fn scan_head(buf : BytesView, max~ : Int, skip_leading_crlf? : Bool) -> Int? raise ParseError

    Finds the first complete head in buf within max bytes and returns its exclusive end, or None while the head is incomplete and within max. Refuses a bare CR or LF. With skip_leading_crlf, empty lines (CRLF) before the start line are part of the head instead of ending it, and count against max; a server reading requests sets it (RFC 9112, section 2.2). Each call scans buf from the start; feed arriving input to a HeadDecoder instead, which scans each byte once.

    test {
      let wire = b"HTTP/1.1 204\r\n\r\nnext"
      assert_eq(@h1.scan_head(wire, max=16384), Some(16))
      assert_eq(@h1.scan_head(wire[:10], max=16384), None)
      let request = b"\r\nGET / HTTP/1.1\r\nHost: h\r\n\r\n"
      assert_eq(
        @h1.scan_head(request, max=16384, skip_leading_crlf=true),
        Some(request.length()),
      )
    }

    write_request_head

    source
    fn write_request_head(meth : 
    Method    , target : String, headers : 
    Headers    , framing : Framing, v : Version) -> Bytes raise ParseError

    Serializes a request head, replacing Content-Length and Transfer-Encoding with the ones framing implies, and keeping duplicate fields. NoBody writes Content-Length: 0 for POST, PUT and PATCH, whose semantics anticipate content, and no framing field for other methods (RFC 9110, section 8.6); pass Length(0) to send it for another method. The result passes the same checks as parse_request_head (target form, Host), and a request cannot be UntilClose. No body bytes are written.

    test {
      let head = @h1.write_request_head(
        Post,
        "/items",
        @mttp_core.Headers::of([("Host", "example.com")]),
        Chunked,
        Http11,
      )
      assert_eq(
        head, b"POST /items HTTP/1.1\r\nhost: example.com\r\ntransfer-encoding: chunked\r\n\r\n",
      )
    }

    write_response_head

    source
    fn write_response_head(status : Int, headers : 
    Headers    , framing : Framing, v : Version, meth? : 
    Method    ) -> Bytes raise ParseError

    Serializes a response to meth with the standard reason phrase, replacing Content-Length and Transfer-Encoding with the ones framing implies, so that response_framing reads back the same body:

    • NoBody on a status that has a body writes Content-Length: 0, on HTTP/1.0 too, since a response without either field is read until close. A response that never has a body (to HEAD, a 1xx, 204 or 304, a 2xx to CONNECT) gets no framing field.
    • Length(n) and Chunked on a response to HEAD or a 304 describe the body a GET would have had (RFC 9110, sections 8.6 and 15.4.5); no body follows either way.
    • UntilClose adds Connection: close.

    Refuses a status outside 100 to 599, body framing on a 1xx, 204 or 2xx to CONNECT (RFC 9110, section 9.3.6), a negative length, chunked HTTP/1.0 and invalid fields. Values are written as Latin-1 octets, matching the parser; encode text beyond Latin-1 before putting it in a field. No body bytes are written.

    test {
      let head = @h1.write_response_head(
        200,
        @mttp_core.Headers::of([("Content-Type", "text/plain")]),
        Length(5),
        Http11,
      )
      assert_eq(
        head, b"HTTP/1.1 200 OK\r\ncontent-type: text/plain\r\ncontent-length: 5\r\n\r\n",
      )
      let empty = @h1.write_response_head(
        200,
        @mttp_core.Headers::empty(),
        NoBody,
        Http11,
      )
      assert_eq(empty, b"HTTP/1.1 200 OK\r\ncontent-length: 0\r\n\r\n")
      let none = @h1.write_response_head(
        204,
        @mttp_core.Headers::empty(),
        NoBody,
        Http11,
      )
      assert_eq(none, b"HTTP/1.1 204 No Content\r\n\r\n")
    }

    Source Files