Sign in

    moonurl

    URI references as RFC 3986 defines them: parsing, recomposing, resolving and percent coding by component, form encoding, and data URLs. No sockets.

    url
    uri
    rfc3986
    percent-encoding
    data-url
    urlencoded
    moonbit
    Download zip
    Version
    0.2.0
    License
    Apache-2.0
    Last updated
    3 hours ago
    Downloads
    50

    #moonurl

    URI references for MoonBit, as RFC 3986 defines them: taking one apart, putting it back, resolving a relative one against a base, normalising, and percent coding by component. Form encoding and data URLs ride on top. No sockets.

    let base = @moonurl.Url::decode("http://a/b/c/d;p?q")
    base.resolve(@moonurl.Url::decode("../g")).encode() // "http://a/b/g"
    @moonurl.escape_text("a b&c/d", kind=Query) // "a%20b&c/d"
    @moonurl.Query::decode(b"q=two+words&tag=a&tag=b").values("tag") // ["a", "b"]
    @dataurl.Url::decode("data:,A%20brief%20note").data // b"A brief note"

    moon run examples/tour takes a reference apart, resolves the RFC 3986 §5.4 examples, escapes one string for four components and builds a query; moon run examples/data reads the four examples RFC 2397 prints.

    #Packages

    PackageWhatSpecification
    moonurlURI references: parsing, recomposing, resolving, normalising, percent coding by componentRFC 3986
    moonurlQuery: the pairs of a query or a form body, read and written as application/x-www-form-urlencodedWHATWG URL Standard §5
    moonurl/dataurlA whole representation written inside the reference to itRFC 2397

    Not here: internationalised host names (RFC 3492), and what belongs to the WHATWG URL Standard rather than to RFC 3986 — special schemes, default ports, Windows drive letters.

    #Form encoding

    Query is a URL's query or a form body as ordered pairs, repeated names kept. + is a space there and nowhere else in a URL, and the serialiser keeps only letters, digits and *-._, so ~ goes out as %7E although RFC 3986 leaves it alone. Both directions pass the parser and serialiser cases of web-platform-tests.

    Query::decode refuses the whole input past its limits: a thousand pairs of up to 1 MiB each by default, which are Starlette's max_fields and max_part_size. Change one with Limits::new(pairs=100) or { ..@moonurl.limits, pairs: 100 }; the two are the same value.

    #Licence

    Apache-2.0. See LICENSE.

    Refused

    pub(all) suberror Refused {
    Port(at~ : Int)
    Host(at~ : Int)
    Pairs(limit~ : Int)
    Pair(limit~ : Int, got~ : Int)
    } derive(Eq,
    Debug
    )

    Why a string is not a URI reference. Only the authority can fail: §B splits any string at all into the five top-level components, and does not split that one.

    Refused::equal

    fn Refused::equal(Refused, Refused) -> Bool

    Refused::not_equal

    fn Refused::not_equal(x : Refused, y : Refused) -> Bool

    Refused::to_repr

    Kind

    pub(all) enum Kind {
    Bare
    Host
    Userinfo
    Segment
    Path
    Query
    Fragment
    Whole
    } derive(Eq,
    Debug
    )

    Which component an octet is going into, and so which octets may stand for themselves (RFC 3986 §2.2, §3).

    & is ordinary inside a path and a separator inside a query. The grammar answers that, not the caller.

    Kind::equal

    fn Kind::equal(Kind, Kind) -> Bool

    Kind::not_equal

    fn Kind::not_equal(x : Kind, y : Kind) -> Bool

    Kind::to_repr

    Limits

    pub(all) struct Limits {
    pairs : Int
    size : Int
    } derive(Eq,
    Debug
    )

    What a decode accepts. Past either bound the whole input is refused rather than cut short, so a handler never acts on part of a form.

    Limits::equal

    fn Limits::equal(Limits, Limits) -> Bool

    Limits::new

    fn Limits::new(pairs? : Int, size? : Int) -> Limits

    Limits by name, each defaulting to the preset's.

    Limits::not_equal

    fn Limits::not_equal(x : Limits, y : Limits) -> Bool

    Limits::to_repr

    Query

    pub struct Query {
    // private fields
    } derive(Eq,
    Debug
    )

    The name–value pairs of an application/x-www-form-urlencoded string: a URL's query, or a form body (WHATWG URL Standard §5). Order and repeated names are kept, as URLSearchParams and Go's url.Values keep them.

    Query::add

    fn Query::add(self : Query, name : String, value : String) -> Unit

    Add a pair at the end, keeping any already under name.

    Query::decode

    fn Query::decode(raw : BytesView, limits? : Limits) -> Query raise Refused

    Read an application/x-www-form-urlencoded string (WHATWG URL Standard §5.1).

    & separates pairs and the first = separates a name from its value; + is a space here and nowhere else in a URL. Escapes are undone to octets and the octets read as UTF-8, a broken sequence becoming U+FFFD. An empty piece between two & is no pair at all.

    Query::encode

    fn Query::encode(self : Query) -> String

    Write the pairs as application/x-www-form-urlencoded (WHATWG URL Standard §5.2): letters, digits and *-._ stand for themselves, a space is +, and every other octet of the UTF-8 is %XX.

    Query::entries

    fn Query::entries(self : Query) -> Array[(String, String)]

    Every pair, in order.

    Query::equal

    fn Query::equal(Query, Query) -> Bool

    Query::get

    fn Query::get(self : Query, name : StringView) -> String?

    The first value under name.

    Query::has

    fn Query::has(self : Query, name : StringView) -> Bool

    Whether any pair is named name.

    Query::new

    fn Query::new(pairs? : Array[(String, String)]) -> Query

    A query holding pairs, in order.

    Query::not_equal

    fn Query::not_equal(x : Query, y : Query) -> Bool

    Query::remove

    fn Query::remove(self : Query, name : StringView) -> Unit

    Remove every pair under name.

    Query::set

    fn Query::set(self : Query, name : String, value : String) -> Unit

    Leave exactly one pair under name, holding value: the first one's place if there was one, the end if not.

    Query::to_repr

    Query::values

    fn Query::values(self : Query, name : StringView) -> Array[String]

    Every value under name, in order.

    Url

    pub(all) struct Url {
    scheme : String?
    userinfo : String?
    host : String?
    port : Int?
    path : String
    query : String?
    fragment : String?
    } derive(Eq,
    Debug
    )

    A URI reference taken apart (RFC 3986 §3).

    No scheme makes it relative (§4.2). No host means no authority, which is not an empty host — file:///tmp has one, mailto:x@y has none.

    Fields stay percent-encoded. Decoding on the way in would lose the difference between a / that separates two segments and a %2F inside one.

    Url::authority

    fn Url::authority(self : Url) -> String?

    The authority as one string, brackets restored around an IP-literal (§3.2).

    Url::decode

    fn Url::decode(text : StringView) -> Url raise Refused

    Take a URI reference apart (RFC 3986 §3, by the split §B states).

    Any string splits, so what comes back is a reference even when it is one nobody would write. Only the authority can be malformed.

    Url::encode

    fn Url::encode(self : Url) -> String

    Put the reference back together (RFC 3986 §5.3).

    Url::equal

    fn Url::equal(Url, Url) -> Bool

    Url::is_absolute

    fn Url::is_absolute(self : Url) -> Bool

    A reference with a scheme and no fragment is a URI (§3); anything else is a relative reference or a same-document one.

    Url::new

    fn Url::new(scheme? : String, userinfo? : String, host? : String, port? : Int, path? : String, query? : String, fragment? : String) -> Url

    A reference this shape, for building one rather than reading one.

    Url::normalize

    fn Url::normalize(self : Url) -> Url

    Syntax-based normalisation (RFC 3986 §6.2.2): scheme and host to lower case, escapes normalised, dot segments out.

    Not §6.2.3: dropping :80 is true of http and false in general, and this package holds no register of schemes.

    Url::not_equal

    fn Url::not_equal(x : Url, y : Url) -> Bool

    Url::resolve

    fn Url::resolve(self : Url, reference : Url) -> Url

    Resolve reference against this base (RFC 3986 §5.2.2, strict).

    Strict: a reference that repeats the base's scheme stays absolute. The non-strict reading is for pre-1998 parsers and turns http:g into something it does not say.

    Url::to_repr

    escape

    fn escape(raw : BytesView, kind? : Kind, spare? : StringView) -> String

    Percent-encode octets for kind (RFC 3986 §2.1).

    Octets, not text: what travels inside a URI is whatever the caller encoded it as. [escape_text] is the one that assumes UTF-8.

    spare is for percent coding that is not going into a URI at all — gRPC's grpc-message keeps all of printable ASCII but %, which is no component of RFC 3986's. A URI component names its kind and leaves spare alone.

    escape_text

    fn escape_text(text : StringView, kind? : Kind, spare? : StringView) -> String

    Percent-encode text as UTF-8 (RFC 3986 §2.5).

    escaped

    fn escaped(text : StringView, kind? : Kind, spare? : StringView) -> Bool

    Whether every % is a complete escape and nothing outside kind's set is there unescaped — the check [unescape] does not make.

    kind

    let kind : Kind

    The strictest component, for when nothing says. Escaping an octet that did not need it changes nothing; leaving one that did changes where the component ends.

    limits

    let limits : Limits

    A thousand pairs of up to 1 MiB each: Starlette's max_fields and max_part_size, and the parameterLimit of Node's qs.

    normalize_escapes

    fn normalize_escapes(text : StringView) -> String

    Percent-encoding normalised (RFC 3986 §6.2.2.2): hex digits in upper case, and escapes of unreserved octets written out plainly.

    remove_dots

    fn remove_dots(path : String) -> String

    Take out the . and .. segments (RFC 3986 §5.2.4).

    The section's loop rather than a split-and-rejoin: the two differ at exactly the inputs that matter — a trailing .., a path that climbs above its own root, /..//.

    unescape

    fn unescape(raw : BytesView) -> Bytes

    Undo percent-encoding (RFC 3986 §2.1).

    A % without two hex digits after it stands for itself. This runs on request targets and form bodies, where refusing the whole message over one stray % helps nobody; [escaped] is for the caller who would rather be told.

    unescape_text

    fn unescape_text(text : StringView) -> String

    Undo percent-encoding and read the result as UTF-8, replacing what is not.

    Source Files