Sign in

    moon-lander-logging-hosting-async

    Console and rolling file delivery for Moon-Lander logging, with bounded async sessions

    Download zip
    Author
    Version
    0.7.0
    License
    Apache-2.0
    Last updated
    8 minutes ago
    Downloads
    200

    #moonrockz/moon-lander-logging-hosting-async

    Console and rolling file delivery for Moon-Lander logging. A session owns one logger, one bounded queue and one writer task. The logger keeps the synchronous Logger trait: a log call captures the record and returns at once, and the writer task writes it later to a Literate console, readable text files and JSON Lines files.

    The module is optional. A library that only writes events depends on moonrockz/moon-lander-logging and on nothing else. This module imports core, logging and moonbitlang/async, and never configuration, mediator or hosting, so an application can use sinks without the application host.

    PackageWhat it holdsTargets
    moonrockz/moon-lander-logging-hosting-asyncTyped settings and report values, the color decisionwasm, wasm-gc, js, native
    moonrockz/moon-lander-logging-hosting-async/runtimeSessions, the writer task, console streams and files (guide)native, js (Node.js and Bun)
    moonrockz/moon-lander-hosting/loggingThe opt-in host bridge that drains before the process ends (guide)native, js

    The pure layouts and rolling rules are in logging/formatting and logging/rolling. The runtime has no sink on wasm, wasm-gc or a browser: an import of the runtime there fails at build time, and it never drops records silently. Bring your own Logger on those targets.

    #Settings

    Each value with a rule has a smart constructor and a typed SettingsError. Parse a relative directory against an explicit base at your boundary first; LogDirectory takes an already parsed AbsolutePath.

    ///|
    test "settings for a console and two file formats" {
    let logs = "/var/log/acme"
    |> @lander_core.AbsolutePath::parse
    |> @sinks.LogDirectory::from_absolute
    let stem = @rolling.LogFileStem::parse("acme")
    let settings = @sinks.SessionSettings::new(
    console=@sinks.ConsoleSink::new(minimum=Debug),
    files=[
    @sinks.FileSink::new(logs, stem),
    @sinks.FileSink::new(logs, stem, format=JsonLines, retention=KeepAll),
    ],
    event_limit=@sinks.QueueEventLimit::count(4096),
    )
    inspect(settings.files()[1].marker_path(), content="/var/log/acme/acme.jsonl.lock")
    }

    SettingDefaultRule
    ConsoleSink streamStandardErrorStandardOutput by explicit selection
    ConsoleSink minimum, color, themeInformation, Auto, LiterateTyped cases
    FileSink format, policy, retentionText, DailyAndSize(10 MiB), KeepNewest(31)Typed cases; the count includes the active file
    FileSink json_propertiesOrderedArrayNamedObject selects schema version 2; applies only to JsonLines
    QueueEventLimit1024 records1 or more
    QueueByteLimit8 MiB1 or more
    RecordByteLimit1 MiB1 or more, and not more than QueueByteLimit
    Drain limit5 secondsA TimeLimit
    File setsTwo FileSink values cannot share a directory, stem and format

    The queue limits count each record that is accepted and not yet written, including the record that the writer writes now, and the payload bytes of all its destinations. A single large input can still need temporary memory to encode before the session rejects it.

    #Named JSON file properties

    Select json_properties=NamedObject on a JsonLines sink for schema version 2. The last occurrence wins under properties. duplicateProperties holds all occurrences of repeated names in original per-name order, including the winner, and is absent without repeats. Nulls, arrays and objects keep their JSON types. Event names cannot overwrite record metadata. See the formatter contract.

    ///|
    test "select named properties for a rolling JSON file" {
    let logs = "/var/log/acme"
    |> @lander_core.AbsolutePath::parse
    |> @sinks.LogDirectory::from_absolute
    let file = @sinks.FileSink::new(
    logs, @rolling.LogFileStem::parse("diagnostics"),
    format=JsonLines, json_properties=NamedObject,
    )
    assert_true(file.json_properties() is NamedObject)
    assert_true(file.files().format() is JsonLines)
    }

    The default remains schema version 1 with an ordered array. The option does not change readable text output, file names, ownership, rolling or retention. Use distinct stems for simultaneous sinks with different JSON policies. A change of policy for an existing file set can mix schema versions in that file, so select a new stem when readers require one schema per file.

    #Colors

    ColorMode::Auto writes styles only when the runtime reports a color terminal, the stream is not redirected, TERM is not dumb, and NO_COLOR is empty or not set. Always forces the Literate theme; Never writes plain text.

    RuntimeWhat Auto knows
    Node.js, BunisTTY and hasColors() of the selected stream
    nativeNothing: the pinned moonbitlang/async has no public terminal check, so Auto stays plain. Use Always to force colors

    ///|
    test "Auto colors are off for an unknown terminal and with NO_COLOR" {
    let console = @sinks.ConsoleSink::new()
    let hints = @sinks.ColorHints::from_env(no_color=Some("1"), term=None)
    let none = @sinks.ColorHints::none()
    assert_true(console.render_theme(ColorTerminal, none) is Literate)
    assert_true(console.render_theme(Unknown, none) is Plain)
    assert_true(console.render_theme(ColorTerminal, hints) is Plain)
    }

    #Reports

    Session::status() and Session::close() give an immutable SessionReport: the accepted records, the drops by reason (queue_full, oversized, closed, reentrant), and for each destination the written and undelivered counts, the failure that stopped it and the latest retention failure. delivery_failed() is true when a destination failed, a record stayed undelivered or a drain used its full limit. A drop at admission is visible in the counts, but it is not a delivery failure.

    #Limits

    • This is not an audit sink. A full queue drops the newest record, and a log call never waits.
    • Completion means that the OS or the Node.js stream callback accepted the bytes. There is no fsync, and a crash or a forced end can lose records.
    • One file set has one writer. An ownership marker <stem>.<extension>.lock in the log directory guards it. A marker that a crash left stays, and the next session reports AcquireOwnership and writes nothing to that set. Make sure that no writer is active, then delete the marker by hand.
    • A failed write can leave a partial line at the end of a file. The writer reports it, stops that destination and does not write the record again.
    • One session runs in one async runtime. Do not share it across threads.

    ReportError

    pub suberror ReportError {
    NegativeCount(count~ : Int64)
    TimeoutBeforeClose
    } derive(
    Debug
    )

    A report value that breaks a rule.

    ReportError::message

    fn ReportError::message(self : ReportError) -> String

    Text for people that names the input and the broken rule.

    SettingsError

    pub suberror SettingsError {
    InvalidQueueEventLimit(count~ : Int)
    InvalidQueueByteLimit(bytes~ : Int64)
    InvalidRecordByteLimit(bytes~ : Int64)
    RecordLimitAboveQueueLimit(record~ : Int64, queue~ : Int64)
    InvalidLogDirectory(path~ : String, rule~ : LogDirectoryRule)
    DuplicateFileSet(directory~ : String, name~ : String)
    } derive(
    Debug
    )

    A session setting that breaks a rule. Each case names the input and the rule.

    SettingsError::message

    fn SettingsError::message(self : SettingsError) -> String

    Text for people that names the input and the broken rule.

    ColorHints

    pub struct ColorHints {
    // private fields
    } derive(Eq,
    Debug
    )

    The environment hints for Auto, read once at the boundary.

    ColorHints::equal

    fn ColorHints::equal(ColorHints, ColorHints) -> Bool

    ColorHints::from_env

    fn ColorHints::from_env(no_color~ : String?, term~ : String?) -> ColorHints

    The hints from the values of NO_COLOR and TERM. A nonempty NO_COLOR turns off Auto colors. TERM=dumb turns them off too.

    ColorHints::none

    fn ColorHints::none() -> ColorHints

    No hint against colors.

    ColorHints::not_equal

    fn ColorHints::not_equal(x : ColorHints, y : ColorHints) -> Bool

    ColorMode

    pub(all) enum ColorMode {
    Auto
    Always
    Never
    } derive(Eq,
    Debug
    )

    When the console writes ANSI styles.

    ColorMode::equal

    fn ColorMode::equal(ColorMode, ColorMode) -> Bool

    ColorMode::not_equal

    fn ColorMode::not_equal(x : ColorMode, y : ColorMode) -> Bool

    ColorMode::styles

    fn ColorMode::styles(self : ColorMode, capability : TerminalCapability, hints : ColorHints) -> Bool

    Whether the console writes styles for this mode, terminal and hints.

    ConsoleSink

    pub struct ConsoleSink {
    // private fields
    } derive(Eq,
    Debug
    )

    A console destination: a stream, a minimum level, a color mode and a theme.

    ConsoleSink::color

    fn ConsoleSink::color(self : ConsoleSink) -> ColorMode

    The color mode of the destination.

    ConsoleSink::equal

    fn ConsoleSink::equal(ConsoleSink, ConsoleSink) -> Bool

    ConsoleSink::minimum

    The lowest level that the destination writes.

    ConsoleSink::new

    A console destination. The defaults are standard error, Information, Auto colors and the Literate theme.

    ConsoleSink::not_equal

    fn ConsoleSink::not_equal(x : ConsoleSink, y : ConsoleSink) -> Bool

    ConsoleSink::render_theme

    The theme to render with: the destination theme when the mode, terminal and hints permit styles, else Plain.

    ConsoleSink::stream

    fn ConsoleSink::stream(self : ConsoleSink) -> ConsoleStream

    The stream of the destination.

    ConsoleSink::theme

    The theme that the destination uses when it writes styles.

    ConsoleStream

    pub(all) enum ConsoleStream {
    StandardError
    StandardOutput
    } derive(Eq,
    Debug
    )

    The process stream of a console destination.

    ConsoleStream::equal

    ConsoleStream::not_equal

    fn ConsoleStream::not_equal(x : ConsoleStream, y : ConsoleStream) -> Bool

    ConsoleStream::output

    fn ConsoleStream::output(self : ConsoleStream, logger : &Logger) -> Unit

    ConsoleStream::to_string

    fn ConsoleStream::to_string(self : ConsoleStream) -> String

    DestinationReport

    pub struct DestinationReport {
    // private fields
    } derive(Eq,
    Debug
    )

    The counts and the latest failures of one destination.

    DestinationReport::equal

    DestinationReport::failure

    The failure that stopped the destination, if any. The destination writes no more records after it.

    DestinationReport::new

    fn DestinationReport::new(target : SinkTarget, written~ : Int64, undelivered~ : Int64, failure? : SinkFailure, retention_failure? : SinkFailure) -> DestinationReport raise ReportError

    Make a destination report. The runtime and test adapters make these. It raises NegativeCount for a count less than 0.

    DestinationReport::not_equal

    fn DestinationReport::not_equal(x : DestinationReport, y : DestinationReport) -> Bool

    DestinationReport::retention_failure

    fn DestinationReport::retention_failure(self : DestinationReport) -> SinkFailure?

    The latest retention or ownership-release failure, if any. It does not stop the destination.

    DestinationReport::target

    The destination.

    DestinationReport::undelivered

    fn DestinationReport::undelivered(self : DestinationReport) -> Int64

    The count of accepted records for this destination that it did not write, because the destination failed or the final drain ended first.

    DestinationReport::written

    fn DestinationReport::written(self : DestinationReport) -> Int64

    The count of records that the destination wrote.

    DropCounts

    pub struct DropCounts {
    // private fields
    } derive(Eq,
    Debug
    )

    The counts of records that the session did not accept, one for each reason.

    DropCounts::closed

    fn DropCounts::closed(self : DropCounts) -> Int64

    Events received after the session closed.

    DropCounts::equal

    fn DropCounts::equal(DropCounts, DropCounts) -> Bool

    DropCounts::new

    fn DropCounts::new(queue_full~ : Int64, oversized~ : Int64, closed~ : Int64, reentrant~ : Int64) -> DropCounts raise ReportError

    Make drop counts. The runtime and test adapters make these. It raises NegativeCount for a count less than 0.

    DropCounts::not_equal

    fn DropCounts::not_equal(x : DropCounts, y : DropCounts) -> Bool

    DropCounts::oversized

    fn DropCounts::oversized(self : DropCounts) -> Int64

    Records discarded because they were larger than the record limit.

    DropCounts::queue_full

    fn DropCounts::queue_full(self : DropCounts) -> Int64

    Records discarded because a queue limit was full.

    DropCounts::reentrant

    fn DropCounts::reentrant(self : DropCounts) -> Int64

    Events received while the session logger or its failure observer ran, for example a log call from the Show of an error or from the observer.

    DropCounts::total

    fn DropCounts::total(self : DropCounts) -> Int64

    The sum of all drop counts.

    FileSink

    pub struct FileSink {
    // private fields
    }

    A file destination: one rolling file set in one directory.

    FileSink::directory

    fn FileSink::directory(self : FileSink) -> LogDirectory

    The directory of the file set.

    FileSink::files

    The file set: stem, format and rolling policy.

    FileSink::json_properties

    The JSON property policy. It affects JsonLines only.

    FileSink::marker_path

    fn FileSink::marker_path(self : FileSink) -> String

    The path of the ownership marker of the file set.

    FileSink::minimum

    The lowest level that the destination writes.

    FileSink::new

    A file destination. The defaults are the Text format, daily and 10 MiB rolling, the newest 31 files including the active file, and Information. JsonLines uses schema version 1 OrderedArray properties by default. Select NamedObject for schema version 2. The policy has no effect on Text.

    FileSink::retention

    The retention of old files.

    FileSink::target

    fn FileSink::target(self : FileSink) -> SinkTarget

    The target of the file set in failures and reports.

    LogDirectory

    pub struct LogDirectory {
    // private fields
    } derive(Eq,
    Debug
    )

    An absolute directory for log files. It holds an already parsed AbsolutePath, so it does not parse the path again; it adds only the rules of file I/O.

    LogDirectory::equal

    LogDirectory::file

    The path of the owned log file name in the directory, with a / separator.

    LogDirectory::from_absolute

    Make a log directory from an absolute path. It raises InvalidLogDirectory when the path has a NUL character. Parse relative text against an explicit base at the application boundary first.

    LogDirectory::marker

    The path of the ownership marker of files in the directory.

    LogDirectory::not_equal

    fn LogDirectory::not_equal(x : LogDirectory, y : LogDirectory) -> Bool

    LogDirectory::output

    fn LogDirectory::output(self : LogDirectory, logger : &Logger) -> Unit

    LogDirectory::path

    The absolute path of the directory.

    LogDirectory::to_string

    fn LogDirectory::to_string(self : LogDirectory) -> String

    LogDirectoryRule

    pub enum LogDirectoryRule {
    NulCharacter
    } derive(Eq,
    Debug
    )

    A rule of a log directory.

    LogDirectoryRule::equal

    LogDirectoryRule::not_equal

    fn LogDirectoryRule::not_equal(x : LogDirectoryRule, y : LogDirectoryRule) -> Bool

    QueueByteLimit

    pub struct QueueByteLimit {
    // private fields
    } derive(Eq,
    Debug
    )

    The largest count of encoded payload bytes that are accepted but not yet written. The count includes the payload of each destination and the record that the writer writes now.

    QueueByteLimit::bytes

    fn QueueByteLimit::bytes(bytes : Int64) -> QueueByteLimit raise SettingsError

    Make a queue byte limit. It raises InvalidQueueByteLimit when bytes is less than 1.

    QueueByteLimit::default

    The default limit: 8 MiB.

    QueueByteLimit::equal

    QueueByteLimit::not_equal

    fn QueueByteLimit::not_equal(x : QueueByteLimit, y : QueueByteLimit) -> Bool

    QueueByteLimit::to_int64

    fn QueueByteLimit::to_int64(self : QueueByteLimit) -> Int64

    The limit in bytes.

    QueueEventLimit

    pub struct QueueEventLimit {
    // private fields
    } derive(Eq,
    Debug
    )

    The largest count of records that are accepted but not yet written, including the record that the writer writes now.

    QueueEventLimit::count

    fn QueueEventLimit::count(count : Int) -> QueueEventLimit raise SettingsError

    Make a queue event limit. It raises InvalidQueueEventLimit when count is less than 1.

    QueueEventLimit::default

    The default limit: 1024 records.

    QueueEventLimit::equal

    QueueEventLimit::not_equal

    fn QueueEventLimit::not_equal(x : QueueEventLimit, y : QueueEventLimit) -> Bool

    QueueEventLimit::to_int

    fn QueueEventLimit::to_int(self : QueueEventLimit) -> Int

    The limit as a count of records.

    RecordByteLimit

    pub struct RecordByteLimit {
    // private fields
    } derive(Eq,
    Debug
    )

    The largest count of encoded bytes of one record, for all its destinations together.

    RecordByteLimit::bytes

    fn RecordByteLimit::bytes(bytes : Int64) -> RecordByteLimit raise SettingsError

    Make a record byte limit. It raises InvalidRecordByteLimit when bytes is less than 1.

    RecordByteLimit::default

    The default limit: 1 MiB.

    RecordByteLimit::equal

    RecordByteLimit::not_equal

    fn RecordByteLimit::not_equal(x : RecordByteLimit, y : RecordByteLimit) -> Bool

    RecordByteLimit::to_int64

    fn RecordByteLimit::to_int64(self : RecordByteLimit) -> Int64

    The limit in bytes.

    SessionReport

    pub struct SessionReport {
    // private fields
    } derive(Eq,
    Debug
    )

    An immutable snapshot of a session: accepted records, drops, the result of each destination, and whether a drain used its full time limit.

    SessionReport::accepted

    fn SessionReport::accepted(self : SessionReport) -> Int64

    The count of accepted records.

    SessionReport::closed

    fn SessionReport::closed(self : SessionReport) -> Bool

    Whether the session is closed.

    SessionReport::delivery_failed

    fn SessionReport::delivery_failed(self : SessionReport) -> Bool

    Whether delivery failed: a destination failed, a record stayed undelivered, or a drain timed out. Drops at admission and retention failures do not count.

    SessionReport::destinations

    fn SessionReport::destinations(self : SessionReport) -> Array[DestinationReport]

    The report of each destination, console first, then files in settings order. The array is a copy.

    SessionReport::drain_timed_out

    fn SessionReport::drain_timed_out(self : SessionReport) -> Bool

    Whether a drain ended at its time limit before all accepted records were written.

    SessionReport::dropped

    fn SessionReport::dropped(self : SessionReport) -> DropCounts

    The counts of records that the session did not accept.

    SessionReport::equal

    SessionReport::failure_messages

    fn SessionReport::failure_messages(self : SessionReport) -> Array[String]

    Each failure in the report, for people, one text for each failure: the destination failures, then the retention failures, then a drain timeout.

    SessionReport::new

    fn SessionReport::new(accepted~ : Int64, dropped~ : DropCounts, destinations~ : Array[DestinationReport], drain_timed_out~ : Bool, closed~ : Bool) -> SessionReport raise ReportError

    Make a session report. The runtime and test adapters make these. The report keeps a copy of destinations. It raises NegativeCount for a count less than 0, and TimeoutBeforeClose for a drain timeout of an open session.

    SessionReport::not_equal

    fn SessionReport::not_equal(x : SessionReport, y : SessionReport) -> Bool

    SessionSettings

    pub struct SessionSettings {
    // private fields
    }

    The settings of one logging session: its destinations and its limits.

    SessionSettings::byte_limit

    The outstanding payload limit.

    SessionSettings::console

    The console destination, if any.

    SessionSettings::drain_limit

    The time limit of a final drain.

    SessionSettings::event_limit

    The outstanding record limit.

    SessionSettings::files

    The file destinations, in order. The array is a copy.

    SessionSettings::new

    fn SessionSettings::new(console? : ConsoleSink, files? : Array[FileSink], event_limit? : QueueEventLimit, byte_limit? : QueueByteLimit, record_limit? : RecordByteLimit, drain_limit? :
    TimeLimit
    ) -> SessionSettings raise SettingsError

    Make session settings. The defaults are no console, no files, 1024 outstanding records, 8 MiB outstanding payload, 1 MiB per record and a 5-second drain. It raises RecordLimitAboveQueueLimit when one record could never enter the queue, and DuplicateFileSet when two file destinations use one file set.

    SessionSettings::record_limit

    fn SessionSettings::record_limit(self : SessionSettings) -> RecordByteLimit

    The limit of one record.

    SessionSettings::standard

    The standard settings: a console on standard error with the defaults of ConsoleSink::new, no files and the default limits.

    SinkFailure

    pub struct SinkFailure {
    // private fields
    } derive(Eq,
    Debug
    )

    One failure of a destination: the operation, the target, the path or name that the operation used, if any, and the text of the error.

    SinkFailure::detail

    fn SinkFailure::detail(self : SinkFailure) -> String

    The text of the error.

    SinkFailure::equal

    fn SinkFailure::equal(SinkFailure, SinkFailure) -> Bool

    SinkFailure::message

    fn SinkFailure::message(self : SinkFailure) -> String

    Text for people, for example logging: write failed for files /logs/acme.log.lock (acme-20261027.log): disk full.

    SinkFailure::new

    fn SinkFailure::new(operation : SinkOperation, target : SinkTarget, subject? : String, detail : String) -> SinkFailure

    Make a failure report. The runtime and test adapters make these.

    SinkFailure::not_equal

    fn SinkFailure::not_equal(x : SinkFailure, y : SinkFailure) -> Bool

    SinkFailure::operation

    fn SinkFailure::operation(self : SinkFailure) -> SinkOperation

    The operation that failed.

    SinkFailure::subject

    fn SinkFailure::subject(self : SinkFailure) -> String?

    The path or file name that the operation used, if any.

    SinkFailure::target

    fn SinkFailure::target(self : SinkFailure) -> SinkTarget

    The destination of the operation.

    SinkOperation

    pub(all) enum SinkOperation {
    CreateDirectory
    AcquireOwnership
    ListDirectory
    PlanFile
    OpenFile
    Write
    CloseFile
    ReleaseOwnership
    DeleteOldFile
    Drain
    } derive(Eq,
    Debug
    )

    The operation of a destination that failed.

    SinkOperation::equal

    SinkOperation::not_equal

    fn SinkOperation::not_equal(x : SinkOperation, y : SinkOperation) -> Bool

    SinkOperation::output

    fn SinkOperation::output(self : SinkOperation, logger : &Logger) -> Unit

    SinkOperation::to_string

    fn SinkOperation::to_string(self : SinkOperation) -> String

    SinkTarget

    The destination of a failure or a count.
    impl Show for SinkTarget

    SinkTarget::equal

    fn SinkTarget::equal(SinkTarget, SinkTarget) -> Bool

    SinkTarget::not_equal

    fn SinkTarget::not_equal(x : SinkTarget, y : SinkTarget) -> Bool

    SinkTarget::output

    fn SinkTarget::output(self : SinkTarget, logger : &Logger) -> Unit

    SinkTarget::to_string

    fn SinkTarget::to_string(self : SinkTarget) -> String

    TerminalCapability

    pub(all) enum TerminalCapability {
    ColorTerminal
    PlainTerminal
    Redirected
    Unknown
    } derive(Eq,
    Debug
    )

    What the runtime knows about the terminal of a console stream.

    TerminalCapability::equal

    TerminalCapability::not_equal

    default_drain_limit

    The default final drain limit: 5 seconds.