moonp8s

    Prometheus metrics for MoonBit.

    prometheus
    p8s
    metrics
    openmetrics
    exposition
    monitoring
    observability
    telemetry
    counter
    gauge
    histogram
    summary
    exemplar
    pushgateway
    client
    asgi
    moonbit
    Download zip
    Version
    0.1.1
    License
    Apache-2.0
    Last updated
    13 hours ago
    Downloads
    166

    #moonp8s

    Prometheus metrics for MoonBit: the metric types, a registry, both text exposition formats, a parser for them, push gateway requests, and an ASGI handler. It is a data model and a format implementation, not a server: it never opens a socket and never reads a clock, so a scrape and a push stay something the caller schedules.

    The site is at https://moonbitstack.github.io/moonp8s/ — a guide, the generated reference, a parser to try, and five runnable examples.

    #Install

    moon add moonbitstack/moonp8s

    #Annotations

    Count and time a function by saying so on it. The function stays as it is, and a generator adds one beside it:

    #p8s.counter("orders_total", help="Orders asked for.")
    #p8s.counter("orders_refused_total", help="Orders refused.", on=raised)
    #p8s.timed("order_seconds", help="Seconds to place an order.", buckets="0.5, 1, 5")
    fn place(item : String) -> Int raise Sold {
    ...
    }

    The generator writes one metric per annotation as a top-level let, place_metered, and register_metrics, which puts them all in a registry:

    let registry = @moonp8s.Registry::new()
    register_metrics(registry)
    let receipt = place_metered("tea", now=clock)

    on says when a counter moves: call (the default), done for a call that returned, or raised. buckets defaults to the same list as Histogram::new. now is a clock in seconds, asked for only when something is timed. The wrapper records a timing whichever way the call ends, a cancelled one included.

    The generator wires in as a dev_build rule, in moon.mod:

    rule(name: "metrics", command: "moonx moonbitstack/moonp8s/cmd/metrics -C \"$mod_dir\" \"$input\" -o \"$output\"")

    and in the moon.pkg of the package, which imports moonbitstack/moonp8s and moonbitstack/moonp8s/metric:

    dev_build(rule: "metrics", input: "orders.mbt", output: "orders.g.mbt") formatter(ignore: [ "orders.g.mbt" ])

    moon run src/examples/08-annotated shows it.

    #Quick start

    import {
    "moonbitstack/moonp8s",
    "moonbitstack/moonp8s/metric",
    }

    try {
    let registry = @moonp8s.Registry::new()
    let requests = registry.counter("http_requests", "HTTP requests.")
    requests.inc()
    let latency = registry.histogram("request_latency_seconds", "Latency.")
    latency.observe(0.25)
    println(@moonp8s.text(registry.collect()))
    } catch {
    @metric.Refused(message) => println("refused: " + message)
    }

    text writes the compatibility format, version 0.0.4. openmetrics writes OpenMetrics 1.0 and ends with # EOF:

    let body = @moonp8s.openmetrics(registry.collect())
    let content_type = @moonp8s.Format::OpenMetrics100.content_type()

    #Metric types

    TypeMethodsNotes
    Counterinc(by~, exemplar?), reset(created?), get()Only increases; a negative step or NaN is refused. A reset dates the series anew, or forgets when it started.
    Gaugeset(v, timestamp?), inc(by~), dec(by~), set_function(f), get()Pass the timestamp to date a sample; a function replaces the stored value.
    Histogramobserve(v, exemplar?), snapshot()Buckets from buckets~; linear and exponential build them.
    GaugeHistogramobserve, snapshot()A gauge's sum and count over histogram buckets.
    Summaryobserve(v), snapshot()Count and sum by default; quantiles~ adds estimates.
    InfoInfo::new(labels)A fixed label set whose value is 1.
    StateSetset, unset, set_many, getStateSet::exclusive keeps exactly one state true.

    A hot counter can accumulate through a shard and flush when it is scraped:

    let shard = requests.shard()
    shard.inc()
    shard.flush()

    ProcessCollector exports the seven process_* series from a ProcessStats value; the caller reads the numbers, so the collector stays portable and does no I/O of its own.

    The default histogram buckets are the shared client defaults, 0.005 through 10:

    let h = @moonp8s.Histogram::new(buckets=@moonp8s.linear(1.0, 1.0, 5))

    A summary with quantiles uses a Greenwald-Khanna sketch, so memory grows with the log of the observation count rather than with the count:

    let s = @moonp8s.Summary::new(quantiles=[(0.5, 0.01), (0.9, 0.01), (0.99, 0.001)])

    #Labels and families

    Family fixes the label names at construction and creates one child per combination of values:

    let requests = @moonp8s.Family::new(["method", "code"], fn() {
    @moonp8s.Counter::new()
    })
    requests.labels(["GET", "200"]).inc()
    requests.labels(["POST", "500"]).inc(by=2.0)

    When the label combination is better expressed as a record, implement LabelSet and use TypedFamily:

    struct Route { method : String path : String } derive(Eq, Hash) impl @moonp8s.LabelSet for Route with pairs(self) { [("method", self.method), ("path", self.path)] }

    #Registry

    A registry applies an optional name prefix and constant labels, refuses an invalid name, a duplicate name, and an invalid unit, and collects everything in registration order. A counter's name may end in _total or not: the family is named without it, and its samples are written with it, so it comes out once:

    let registry = @moonp8s.Registry::new(prefix="app", labels=[("region", "eu")])
    let requests = registry.counter("http_requests", "HTTP requests.")
    let sub = registry.sub(prefix="http", labels=[("server", "a")])

    A custom source implements Collector and is added with add; the registry applies its prefix and constant labels to the families the collector returns.

    A name is refused when it would collide with a sample another family already writes: a gauge called orders_total next to a counter called orders would put two different series under one name, so the second registration is refused with the name it would have taken.

    #Several processes

    A worker pool counts the same metric once per worker. Each process writes what it has counted to its own file in one directory; the process that answers /metrics reads the directory and adds the files up. Nothing is shared, so a scrape never waits for a worker that is busy.

    // moon.pkg: import { "moonbitstack/moonp8s/multiproc/directory" }
    let written = @directory.write(
    dir,
    7,
    registry.collect(),
    modes=[("queue_depth", @moonp8s.Mode::Sum)],
    )

    write replaces this process's files. They hold the totals since the process started, not a delta, so flushing twice is flushing once; each file is written beside its final name and renamed into place, so a scrape reads a whole file or the previous one.

    The scrape side is one call:

    let families = @directory.collect(dir, at=1_700_000_015.0)
    let body = @moonp8s.openmetrics(families)

    Counters, histograms, and summaries add up across processes. A gauge does not, so its mode says what to do with the values from the other processes:

    ModeThe series becomes
    Mode::All (default)one series per process, each labelled pid
    Mode::Min, Mode::Maxthe smallest or the largest value
    Mode::Sumthe sum of the values
    Mode::MostRecentthe value written last; at equal timestamps the earlier file wins

    Every mode also has a Live spelling — LiveAll, LiveMin, and so on — which aggregates the same way and changes what happens when the process exits:

    @directory.mark_dead(dir, 7)

    A live gauge describes a process that is running now, so its file goes when the supervisor says the process is gone. A counter's file stays: what the process counted is still true after it exits.

    A summary's quantiles are estimated from one process's samples and cannot be added up, so only its count and sum travel. An info metric and a state set are process-local and have no cross-process meaning; write names them in the result it returns rather than dropping them quietly.

    The files use the layout the official clients' multiprocess mode uses, which is what makes a directory portable: a worker written here can be read by them, and a directory they wrote can be read here.

    #Exposition

    FunctionResult
    text(families)Prometheus text format, version 0.0.4
    openmetrics(families)OpenMetrics text format, version 1.0.0, ending in # EOF
    negotiate(accept)Format chosen from an Accept header
    parse(text, format~)the families an exposition describes

    parse reads both formats. The compatibility format is lossy for info and state sets, which it spells as gauges; OpenMetrics keeps them.

    #Push gateway

    push, push_add, and delete build the request; the caller sends it:

    let request = @moonp8s.push(
    "pushgateway.local",
    "batch",
    registry.collect(),
    grouping_key=[("instance", "worker-1")],
    )
    println(request.http_method + " " + request.path)
    println(request.body)

    Grouping key values that contain a slash or a space, and empty values, use the gateway's @base64 form.

    #ASGI

    app serves /metrics; middleware counts requests and observes their duration. The clock is a parameter so a test can drive time:

    let handler = @moonp8s.app(registry)
    let wrapped = @moonp8s.middleware(registry)

    #Packages

    PackageContents
    moonbitstack/moonp8sRe-exports the common types and functions.
    moonbitstack/moonp8s/metricMetric types, families, the registry, and the snapshot model.
    moonbitstack/moonp8s/expoBoth text formats, their negotiation, and the parser.
    moonbitstack/moonp8s/pushPush gateway request construction.
    moonbitstack/moonp8s/asgiThe /metrics handler and the counting middleware.
    moonbitstack/moonp8s/multiprocGauge modes, a worker's file, and the merge. No I/O.
    moonbitstack/moonp8s/multiproc/directoryWriting and reading those files. Native only.

    #Conformance

    The OpenMetrics 1.0 specification was read MUST by MUST; the verdict for each of the hundred is in the delivery notes for this version. Three are not met, and here is what each would cost:

    • Negative observations. A histogram or summary that records a negative value carries a negative sum, which the specification forbids because a sum is a counter. Both official clients accept negative observations and document the consequence, and a temperature delta is a real thing to record; refusing them would move the problem from the exposition to the caller's arithmetic.
    • Reserved names. Label names beginning with an underscore belong to the ingestor. A registry here accepts them: __name__ is the ingestor's to set, but a metric named _internal_total is a naming mistake, not an unreadable exposition.
    • Counts rendered as floats. Counts and bucket values are integers in the model, and written 17.0 rather than 17, as every official client writes them. The format's own grammar accepts both, and a reader must not reject either.

    The exposition formats themselves are checked by tooling rather than by reading: promtool check metrics lints the 0.0.4 output clean, a real Prometheus server scrapes both formats and the query results are compared with the values the workers recorded, and the multiprocess directory is exchanged with the official Python client in both directions, with the two renderings compared line for line — the two writers order the series of one family differently, and neither format gives series an order.

    #Migration

    moonbitstack/moonp8s/compat rebuilds the legacy counter vector, server metric set, and remote report writer on the registry model, so an existing caller can move over one import at a time.

    #License

    Apache-2.0.

    Counter

    The metric model re-exported for callers that want one import.

    Entry

    How several processes' values become one series, and the model of the files they leave behind. The directory these files live in is a native package of its own: moonbitstack/moonp8s/multiproc/directory.

    Exemplar

    The metric model re-exported for callers that want one import.

    Family

    The metric model re-exported for callers that want one import.

    Format

    The exposition formats and their encoders and parser.

    Gauge

    The metric model re-exported for callers that want one import.

    GaugeHistogram

    The metric model re-exported for callers that want one import.

    Histogram

    The metric model re-exported for callers that want one import.

    Info

    The metric model re-exported for callers that want one import.

    Kind

    The metric model re-exported for callers that want one import.

    LocalCounter

    The metric model re-exported for callers that want one import.

    MetricFamily

    The metric model re-exported for callers that want one import.

    Mode

    How several processes' values become one series, and the model of the files they leave behind. The directory these files live in is a native package of its own: moonbitstack/moonp8s/multiproc/directory.

    Part

    How several processes' values become one series, and the model of the files they leave behind. The directory these files live in is a native package of its own: moonbitstack/moonp8s/multiproc/directory.

    Point

    The metric model re-exported for callers that want one import.

    ProcessCollector

    The metric model re-exported for callers that want one import.

    ProcessStats

    The metric model re-exported for callers that want one import.

    PushRequest

    Push gateway request construction.

    Registry

    The metric model re-exported for callers that want one import.

    StateSet

    The metric model re-exported for callers that want one import.

    Summary

    The metric model re-exported for callers that want one import.

    TypedFamily

    The metric model re-exported for callers that want one import.

    Value

    The metric model re-exported for callers that want one import.

    app

    A /metrics handler for a registry. GET negotiates the format from the Accept header, OPTIONS advertises the allowed methods, anything else is refused with 405, and /favicon.ico is answered empty so a browser probing the endpoint stays quiet.

    buckets

    let buckets : Array[Double]

    The default bucket bounds the official Prometheus clients share: 0.005 through 10, fourteen finite bounds plus the implicit +Inf bucket every histogram has.

    delete

    fn delete(gateway : String, job : String, grouping_key? : Array[(String, String)]) ->
    PushRequest

    Build a DELETE that removes every series under job and grouping_key.

    exemplar_label_limit

    let exemplar_label_limit : Int

    OpenMetrics caps an exemplar's combined label names and values at 128 characters; a longer set makes the exemplar unrepresentable.

    exponential

    fn exponential(start : Double, factor : Double, count : Int) -> Array[Double] raise
    Refused

    count bounds starting at start and multiplying by factor.

    format_float

    fn format_float(d : Double) -> String

    Format a float the way the official clients do: +Inf, -Inf, and NaN by name; otherwise the shortest decimal that round-trips, with a .0 on integers and Go's switch to a two-digit exponent once the integer part is longer than six digits (so 1000000.0 is 1e+06, matching what a Prometheus scrape expects to read).

    is_valid_label_name

    fn is_valid_label_name(name : String) -> Bool

    Whether name is a label name under the OpenMetrics character rules: [a-zA-Z_][a-zA-Z0-9_]*.

    is_valid_metric_name

    fn is_valid_metric_name(name : String) -> Bool

    Whether name is a metric name under the OpenMetrics character rules: [a-zA-Z_:][a-zA-Z0-9_:]*.

    is_valid_unit

    fn is_valid_unit(unit : String) -> Bool

    Whether unit may be appended to a metric name. A unit starts with a letter and continues with letters, digits, or underscores.

    linear

    fn linear(start : Double, width : Double, count : Int) -> Array[Double] raise
    Refused

    count bounds starting at start and growing by width.

    merge

    Fold the parts into one set of families.

    Counters and summaries add up across processes; histograms add up per bucket and are made cumulative again; a gauge follows its mode. Families come out in the order their first part mentions them, and series in the order they were first seen, so two runs over the same directory write the same bytes.

    A family that two files disagree about — a name that is a counter in one process and a histogram in another, or a gauge written under two modes — is refused. The numbers would be meaningless, and a scrape that says so is better than one that quietly picks a side.

    middleware

    A middleware that counts handled requests (http_requests, labelled by method, route, and status) and observes their duration in seconds (http_request_duration_seconds, labelled by method and route). The clock is a parameter so a test can drive time; it defaults to wall-clock seconds.

    negotiate

    fn negotiate(accept : String) ->
    Format

    Pick a format from an HTTP Accept header. application/openmetrics-text wins when it names a version this library speaks (or names no version); otherwise a scrape gets the compatibility format, which is also the default when the header is absent or unrecognised.

    openmetrics

    fn openmetrics(families : Array[
    MetricFamily
    ]) -> String

    Write families in the OpenMetrics text format, version 1.0.0, ending with the # EOF marker the format requires.

    push

    fn push(gateway : String, job : String, families : Array[
    MetricFamily
    ], grouping_key? : Array[(String, String)], format? :
    Format
    ) ->
    PushRequest

    Build a PUT that replaces every series under job and grouping_key with the exposition of families.

    push_add

    fn push_add(gateway : String, job : String, families : Array[
    MetricFamily
    ], grouping_key? : Array[(String, String)], format? :
    Format
    ) ->
    PushRequest

    Build a POST that replaces only the series named in families, leaving other series under the same job and grouping key alone.

    text

    Write families in the Prometheus text exposition format, version 0.0.4. Counters gain a _total suffix, info becomes a gauge ending in _info, state sets become gauges, and _created, _gcount, and _gsum series are emitted as separate gauge families, as the compatibility format requires.

    Source Files