🥮 mooncakes.io

myshell

A small shell-free process EDSL for MoonBit

process
pipeline
async
sandbox
agent
moon add bobzhang/myshell@0.3.0
Download zip
Author
Version
0.3.0
License
Apache-2.0
Repository
Last updated
2 hours ago
Downloads
10

Dependencies

README

#myshell

bobzhang/myshell is a small, shell-free process EDSL built on moonbitlang/async. It is meant for sandboxed programs and agents that need a familiar process API without receiving a shell as ambient authority.

#The invariant

Cmd never invokes a shell. The executable and argument vector stay separate, and every argument is passed literally. |, >, &&, $(), and * have no special meaning. Use Pipeline for pipes and ordinary MoonBit for control flow. Embedded NUL characters in process metadata are rejected before spawning rather than being truncated by an operating-system argv boundary.

The library supports the wasm and native targets. Its preferred target is wasm, where process creation is delegated through the host operations used by moonbitlang/async/process.

To exercise the included smoke program under MoonBit's deny-by-default Wasm policy, with only process spawning enabled:

moon runwasm --experimental-policy examples/moonrun-policy.json cmd/smoke

#Install

moon add bobzhang/myshell

Add the async runtime and this library to the executable package:

///|
import {
  "bobzhang/myshell",
  "moonbitlang/async",
}

Use an async fn main; MoonBit async code does not use an await keyword.

#The whole API

A process is described by one constructor and executed by one method. There are no builder chains: a Cmd is written the way it is read.

///|
Cmd(
  program,                 // executable name or path
  arguments,               // Array[String], each passed literally
  cwd? : String,           // default: inherited working directory
  env? : Map[String, String], // default: {}
  inherit_env? : Bool,     // default: true
  stdin? : Stdin,          // default: closed
  stdout? : Redirect,      // default: Capture
  stderr? : Redirect,      // default: Capture
  cancel? : Cancel,        // default: Kill
  no_console_window? : Bool, // default: false, Windows only
) -> Cmd

Pipeline(commands : Array[Cmd]) -> Pipeline

///|
enum Stdin {
  Text(String)
  Binary(Bytes)
  FromFile(String)
} // `< path`

///|
enum Redirect {
  Capture
  Inherit
  ToFile(String)
  AppendToFile(String)
} // `> path`, `>> path`

Execution: output collects the streams, status returns only the exit code, and each_line follows standard output as it is produced. All three exist on Cmd; Pipeline has output and each_line.

Cmd and Pipeline are abstract and immutable. They are read back through program(), arguments(), cwd(), env(), inherit_env(), stdin(), stdout(), stderr(), cancel(), no_console_window(), and commands() — so a plan that has been inspected is the same plan that runs. Output, Stdin, and Redirect are transparent, because a result is data and a stream setting is a choice.

#1. Run one command

///|
#cfg(not(platform="windows"))
async test {
  let output = @myshell.Cmd("printf", ["hello"]).output()
  assert_eq(output.exit_code, 0)
  assert_eq(output.stdout, "hello")
}

#2. Pass shell characters literally

This prints the characters; it does not execute echo and does not create a pipe.

///|
#cfg(not(platform="windows"))
async test {
  let output = @myshell.Cmd("printf", ["%s", "$(echo no) | *"]).output()
  assert_eq(output.stdout, "$(echo no) | *")
}

#3. Set cwd and environment

Options are labelled arguments on the constructor, so the whole plan is one expression.

///|
test {
  let command = @myshell.Cmd("moon", ["check"], cwd="workspace", env={
    "NO_COLOR": "1",
  })
  debug_inspect(command.cwd(), content="Some(\"workspace\")")
  assert_eq(command.env()["NO_COLOR"], "1")
}

Pass inherit_env=false to start from an empty environment instead of adding to the parent's.

#4. Inspect a plan before running it

A plan can be logged, diffed, or checked against a policy before anything is spawned. Because Cmd is immutable, nothing can change it between the check and the run.

///|
test {
  let command = @myshell.Cmd("rm", ["-rf", "/"])
  assert_eq(command.program(), "rm")
  assert_eq(command.arguments().length(), 2)
}

Building a command list is ordinary MoonBit; there is no builder to learn.

///|
test {
  let arguments = ["check"]
  if true {
    arguments.push("--target")
    arguments.push("wasm")
  }
  let command = @myshell.Cmd("moon", arguments)
  assert_eq(command.arguments(), ["check", "--target", "wasm"])
}

#5. Pipe commands

Every stage is separately visible to the process host. The stages use real operating-system pipes and run concurrently in one structured task group.

///|
#cfg(not(platform="windows"))
async test {
  let output = @myshell.Pipeline([
    Cmd("printf", ["alpha\nbeta\n"]),
    Cmd("grep", ["beta"]),
    Cmd("tr", ["a-z", "A-Z"]),
  ]).output()
  assert_eq(output.stdout, "BETA\n")
  assert_eq(output.stage_exit_codes, [0, 0, 0])
}

#6. Supply standard input

Standard input is closed by default, so non-interactive runs cannot accidentally wait on ambient input. stdin on the first stage also feeds a pipeline; giving it to a later stage is rejected.

///|
#cfg(not(platform="windows"))
async test {
  let output = @myshell.Pipeline([
    Cmd("cat", [], stdin=Text("hello")),
    Cmd("tr", ["a-z", "A-Z"]),
  ]).output()
  assert_eq(output.stdout, "HELLO")
}

Use Binary when the input is not text:

///|
#cfg(not(platform="windows"))
async test {
  let output = @myshell.Cmd("cat", [], stdin=Binary(b"\x00\xff")).output()
  assert_eq(output.stdout_bytes, b"\x00\xff")
}

#7. Redirect to and from files

ToFile and AppendToFile replace a shell's > path and >> path, and FromFile replaces < path. Without them the only shell-free option would be to capture in memory and write the file yourself.

///|
#cfg(not(platform="windows"))
async test {
  let path = "/tmp/myshell-readme.log"
  @myshell.Cmd("printf", ["one\n"], stdout=ToFile(path)).status() |> ignore
  @myshell.Cmd("printf", ["two\n"], stdout=AppendToFile(path)).status()
  |> ignore
  let back = @myshell.Cmd("cat", [], stdin=FromFile(path)).output()
  assert_eq(back.stdout, "one\ntwo\n")
  @myshell.Cmd("rm", ["-f", path]).status() |> ignore
}

A redirected stream is not captured, so it arrives empty in Output and does not count against max_output_bytes. Use Inherit to hand a stream to the parent's own descriptor. Only the last stage of a pipeline may redirect stdout, since every earlier stage's stdout is the pipe.

#8. Follow output as it is produced

output returns nothing until the command finishes. each_line delivers standard output line by line while the command runs, which is what a long-running build or test needs in order to report progress. Completed lines are not retained, so total output is unbounded; max_line_bytes (8 MiB by default) caps one line's content exactly, so a child that never emits a newline cannot exhaust memory. Both \n and \r\n are recognised as terminators, and a CRLF's CR does not count against the limit.

///|
#cfg(not(platform="windows"))
async test {
  let seen = []
  let code = @myshell.Cmd("printf", ["alpha\nbeta\n"]).each_line(line => {
    seen.push(line)
  })
  assert_eq(code, 0)
  assert_eq(seen, ["alpha", "beta"])
}

Pipeline::each_line follows the last stage the same way.

#9. Run without capturing output

Use status when stdout and stderr should be inherited by the current process. It returns only the exit code and has no capture limit.

///|
#cfg(not(platform="windows"))
async test {
  assert_eq(@myshell.Cmd("false", []).status(), 1)
}

#10. Inspect exit status

Cmd::output does not turn a non-zero exit status into an exception. Use ordinary MoonBit control flow or call check() when failure should raise.

///|
#cfg(not(platform="windows"))
async test {
  let output = @myshell.Cmd("false", []).output()
  if !output.success() {
    assert_eq(output.exit_code, 1)
  }
}

#11. Pipeline status uses pipefail

exit_code is the rightmost non-zero stage status, or zero when all stages succeed. Every individual status remains available in stage_exit_codes.

///|
#cfg(not(platform="windows"))
async test {
  let output = @myshell.Pipeline([Cmd("false", []), Cmd("cat", [])]).output()
  assert_eq(output.exit_code, 1)
  assert_eq(output.stage_exit_codes, [1, 0])
}

#12. Capture stderr

For pipelines, stage_stderr preserves one string per stage and stderr concatenates them in stage order. Exact bytes remain available as stdout_bytes and stage_stderr_bytes; the text fields use lossy UTF-8 decoding so arbitrary process output cannot cancel sibling stages.

///|
#cfg(not(platform="windows"))
async test {
  let output = @myshell.Cmd("/usr/bin/grep", [
    "needle", "/definitely/not/a/real/myshell-file",
  ]).output()
  assert_true(!output.stderr.is_empty())
  assert_eq(output.stage_stderr.length(), 1)
}

#13. Add a timeout

A timeout cancels the structured task and stops each direct child according to its cancel policy, which kills immediately by default — see section 14 to ask a child to stop first instead. A timeout is not a process-tree deadline: descendants must be contained and reaped by the host's native process sandbox.

///|
#cfg(not(platform="windows"))
async test {
  try @myshell.Cmd("sleep", ["1"]).output(timeout_ms=20) catch {
    @async.TimeoutError => ()
    _ => fail("expected TimeoutError")
  } noraise {
    _ => fail("expected TimeoutError")
  }
}

Captured stdout plus stderr is limited to 8 MiB by default. This prevents an untrusted child from growing the Wasm instance without bound. Override it when needed:

///|
let output = command.output(max_output_bytes=32 * 1024 * 1024)

If all captured streams together exceed the limit, execution raises ProcessError::OutputLimitExceeded and cancels the structured process group.

#14. Choose how a cancelled child is stopped

Cancellation — from a timeout, a capture limit, or a failing pipeline stage — kills the child outright by default, so a sandboxed runtime never waits on an untrusted process. Graceful instead asks the child to stop first, with SIGTERM (or SIGBREAK on Windows), and kills it only if it is still running after grace_ms. That is what a child needs in order to flush a file or release a lock.

///|
test {
  let command = @myshell.Cmd("moon", ["check"], cancel=Graceful(grace_ms=2000))
  debug_inspect(command.cancel(), content="Graceful(grace_ms=2000)")
}

A graceful teardown delays the cancelling call by up to grace_ms, so a timeout_ms of 20 with a grace_ms of 1000 raises TimeoutError after about a second rather than after 20 milliseconds. As with any timeout, this reaches only the direct child; descendants remain the host sandbox's responsibility.

#One-file use

After the package is published, a complete sandboxed probe can be run without creating a project:

moon run -e 'import {
  "bobzhang/myshell",
  "moonbitlang/async",
}

async fn main {
  let output = @myshell.Pipeline([
    @myshell.Cmd("/usr/bin/printf", ["hello\nworld\n"], inherit_env=false),
    @myshell.Cmd("/usr/bin/grep", ["world"], inherit_env=false),
  ]).output()
  print(output.stdout)
}'

#Known limitation: captured output can truncate

Captured output can be cut short, silently, when a child writes faster than this library drains the pipe. moonbitlang/async creates its pipes with O_NONBLOCK, and that flag survives into the child, so once the pipe fills the child's own write fails with EAGAIN rather than blocking. Most programs report a write error and exit non-zero at that point, so Output holds a prefix of what the command meant to say.

This is moonbitlang/async#553 and cannot be fixed from here; the fix is to clear O_NONBLOCK on the descriptors handed to a child.

It is a race, not a fixed ceiling. Small outputs are unaffected in practice. Larger ones usually survive on the native target and usually do not on wasm, but a loaded machine or a slower consumer can lose either. Until it is fixed upstream:

  • status is unaffected, because nothing is captured.
  • Redirect::ToFile and AppendToFile are unaffected, because a regular file is never made non-blocking.
  • output and each_line are affected whenever a command produces more than a pipe buffer, roughly 64 KiB.

#Security boundary

This package removes shell parsing and keeps each process invocation structured; it does not decide which executables or effects are safe. Because a Cmd is immutable and fully readable, a caller can apply its own policy to a plan before calling output or status — the description and the execution are separate steps.

On wasm, the host still owns executable authorization and the native process sandbox. A strong deployment should grant each child only the filesystem, network, environment, and secret capabilities required for that invocation.

MoonBit's current moonrun policy gates process spawning as a boolean. Enabling process.spawn does not apply the policy's filesystem or network restrictions to the native child; the child receives host-user ambient access. Production agent runtimes therefore need the separate native exec-sandbox described in the architecture, with child capabilities no greater than the parent grant.

#
ProcessError

source
pub(all) suberror ProcessError {
  EmptyProgram
  EmptyPipeline
  InvalidEnvironmentName(String)
  InvalidOutputLimit(Int)
  InvalidGracePeriod(Int)
  OutputLimitExceeded(stream~ : String, limit~ : Int)
  NulByte(String)
  StdinOnNonFirstStage(Int)
  RedirectOnNonFinalStage(Int)
  StdoutNotCaptured
  CommandFailed(Output)
} derive(
Debug)

Errors reported by the process EDSL itself.

These are raised before or instead of spawning; a non-zero child status is reported through Output rather than as an error.

#
ProcessError::to_repr

source

#
Cancel

source
pub(all) enum Cancel {
  Kill
  Graceful(grace_ms~ : Int)
} derive(
Debug)

How a child is stopped when a run is cancelled.

Cancellation happens when a timeout expires, when a capture limit is exceeded, or when any stage of a pipeline fails.

Kill ends the child immediately and is the default, because a sandboxed runtime should not have to wait on an untrusted child. Graceful first asks the child to stop — SIGTERM, or SIGBREAK on Windows — and kills it only if it has not exited within grace_ms, which is what a child that must flush a file or release a lock needs.

A graceful teardown delays the cancelling call by up to grace_ms: a timeout_ms of 20 with grace_ms of 1000 raises TimeoutError after roughly a second, not after 20 milliseconds.

#
Cancel::to_repr

source

#
Cmd

source

A shell-free process description.

program is an executable name or path. Every element of arguments is passed as one literal argument. Shell operators such as |, >, &&, $(), and * have no special meaning.

The representation is abstract and a Cmd is immutable once built, so a plan that has been inspected or approved is the same plan that runs.

#
Cmd::Cmd

source
fn Cmd::Cmd(program : String, arguments : Array[String], cwd? : String, env? : Map[String, String], inherit_env? : Bool, stdin? : Stdin, stdout? : Redirect, stderr? : Redirect, cancel? : Cancel, no_console_window? : Bool) -> Cmd

Describe one process.

Standard input is closed unless stdin is given, so a non-interactive run cannot accidentally wait on ambient input. env adds to the inherited environment; pass inherit_env=false to start from an empty one. no_console_window suppresses a console window on Windows and is ignored elsewhere.

stdout and stderr default to Capture, so output collects them.

arguments and env are copied, so later changes to the caller's collections do not reach the command.

Example

test {
  let cmd = @myshell.Cmd("git", ["status", "--short"], cwd="workspace")
  inspect(cmd.program(), content="git")
  debug_inspect(cmd.cwd(), content="Some(\"workspace\")")
}

#
Cmd::arguments

source
fn Cmd::arguments(self : Cmd) -> ArrayView[String]

The literal argument vector, as a read-only view.

#
Cmd::cancel

source
fn Cmd::cancel(self : Cmd) -> Cancel

How the child is stopped when the run is cancelled.

#
Cmd::cwd

source
fn Cmd::cwd(self : Cmd) -> String?

The configured working directory.

#
Cmd::each_line

source
async fn Cmd::each_line(self : Cmd, on_line : async (String) -> Unit, timeout_ms? : Int, max_line_bytes? : Int) -> Int

Execute this command, delivering standard output one line at a time.

on_line receives each line without its terminator — \n or \r\n — as soon as the child flushes it, so a long-running command can report progress instead of arriving as one block at the end. A trailing fragment with no newline is delivered as a final line. Standard error follows the same rule as in status.

Completed lines are not retained, so total output is unbounded. One line is held while it is assembled, and max_line_bytes caps its content exactly: any single line longer than that raises OutputLimitExceeded, whether or not a newline ever arrives, so a child that emits no newline cannot exhaust memory. A CRLF terminator's CR is punctuation rather than content and does not consume the allowance.

Raises StdoutNotCaptured when stdout sends the stream elsewhere, since there would be nothing to read.

Example

#cfg(not(platform="windows"))
async test {
  let seen = []
  let code = @myshell.Cmd("printf", ["a\nb\n"]).each_line(line => {
    seen.push(line)
  })
  assert_eq(code, 0)
  assert_eq(seen, ["a", "b"])
}

#
Cmd::env

source
fn Cmd::env(self : Cmd) -> Map[String, String]

A copy of the environment entries added for the child.

#
Cmd::inherit_env

source
fn Cmd::inherit_env(self : Cmd) -> Bool

Whether the child also receives the parent environment.

#
Cmd::no_console_window

source
fn Cmd::no_console_window(self : Cmd) -> Bool

Whether creation of a console window is suppressed on Windows.

#
Cmd::output

source
async fn Cmd::output(self : Cmd, timeout_ms? : Int, max_output_bytes? : Int) -> Output

Execute this command and capture stdout and stderr.

Constructing a Cmd does not execute it. Captured stdout and stderr share max_output_bytes; streams sent elsewhere by stdout or stderr do not count against it and arrive empty in Output. When timeout_ms expires, each direct child is stopped according to its cancel policy — immediately by default, or after a grace period under Graceful; descendants require enforcement by the host's native process sandbox.

Example

#cfg(not(platform="windows"))
async test {
  let output = @myshell.Cmd("printf", ["hello"]).output()
  assert_eq(output.stdout, "hello")
}

#
Cmd::program

source
fn Cmd::program(self : Cmd) -> String

The executable name or path.

#
Cmd::status

source
async fn Cmd::status(self : Cmd, timeout_ms? : Int) -> Int

Execute this command without capturing stdout or stderr.

Streams left as Capture are inherited by the current process, because status has no channel to return them on; explicit ToFile, AppendToFile, and Inherit settings still apply. This method returns only the exit status and has no capture limit.

Example

#cfg(not(platform="windows"))
async test {
  assert_eq(@myshell.Cmd("false", []).status(), 1)
}

#
Cmd::stderr

source
fn Cmd::stderr(self : Cmd) -> Redirect

Where standard error goes.

#
Cmd::stdin

source
fn Cmd::stdin(self : Cmd) -> Stdin?

The configured standard input, if any.

#
Cmd::stdout

source
fn Cmd::stdout(self : Cmd) -> Redirect

Where standard output goes.

#
Cmd::to_repr

source

#
Output

source
pub struct Output {
  exit_code : Int
  stage_exit_codes : Array[Int]
  stdout : String
  stdout_bytes : Bytes
  stderr : String
  stage_stderr : Array[String]
  stage_stderr_bytes : Array[Bytes]
} derive(
Debug)

Captured output and exit status from a command or pipeline.

A single command reports one element in stage_exit_codes and one in stage_stderr. For a pipeline, exit_code uses pipefail semantics: it is the rightmost non-zero stage status, or zero when every stage succeeds, and stderr is the concatenation of stage_stderr in stage order.

Text fields use lossy UTF-8 decoding so arbitrary process output cannot cancel a run. stdout_bytes and stage_stderr_bytes keep the exact bytes.

#
Output::check

source
fn Output::check(self : Output) -> Output raise ProcessError

Raise CommandFailed when the output is not successful.

Example

#cfg(not(platform="windows"))
async test {
  try @myshell.Cmd("false", []).output().check() catch {
    @myshell.ProcessError::CommandFailed(failed) =>
      inspect(failed.exit_code, content="1")
    _ => fail("unexpected error")
  } noraise {
    _ => fail("expected CommandFailed")
  }
}

#
Output::success

source
fn Output::success(self : Output) -> Bool

Whether the command, or every stage of the pipeline, exited with zero.

#
Output::to_repr

source

#
Pipeline

source

A sequence of commands connected by operating-system pipes.

Every stage is separately visible to the process host, and an empty pipeline is rejected when execution is requested.

#
Pipeline::Pipeline

source
fn Pipeline::Pipeline(commands : Array[Cmd]) -> Pipeline

Describe a pipeline whose stages are connected by real pipes.

Only the first stage may carry stdin; the rest read from the previous stage. commands is copied, so later changes to the caller's array do not reach the pipeline.

Example

test {
  let plan = @myshell.Pipeline([
    Cmd("printf", ["alpha\nbeta\n"]),
    Cmd("grep", ["beta"]),
  ])
  inspect(plan.commands().length(), content="2")
}

#
Pipeline::commands

source
fn Pipeline::commands(self : Pipeline) -> ArrayView[Cmd]

The stages in execution order, as a read-only view.

#
Pipeline::each_line

source
async fn Pipeline::each_line(self : Pipeline, on_line : async (String) -> Unit, timeout_ms? : Int, max_line_bytes? : Int) -> Int

Execute this pipeline, delivering the last stage's output one line at a time.

The returned status uses the same pipefail rule as output. See Cmd::each_line for how lines are delivered and bounded.

Example

#cfg(not(platform="windows"))
async test {
  let seen = []
  @myshell.Pipeline([Cmd("printf", ["alpha\nbeta\n"]), Cmd("grep", ["beta"])]).each_line(line => {
      seen.push(line)
    },
  )
  |> ignore
  assert_eq(seen, ["beta"])
}

#
Pipeline::output

source
async fn Pipeline::output(self : Pipeline, timeout_ms? : Int, max_output_bytes? : Int) -> Output

Execute this pipeline and capture final stdout plus every stage's stderr.

Stages use real operating-system pipes and run concurrently in one structured task group. timeout_ms, when present, applies to the whole pipeline.

Example

#cfg(not(platform="windows"))
async test {
  let output = @myshell.Pipeline([
    Cmd("printf", ["alpha\nbeta\n"]),
    Cmd("grep", ["beta"]),
  ]).output()
  assert_eq(output.stdout, "beta\n")
}

#
Pipeline::to_repr

source

#
Redirect

source
pub(all) enum Redirect {
  Capture
  Inherit
  ToFile(String)
  AppendToFile(String)
} derive(
Debug)

Where a child process's standard output or standard error goes.

Capture returns the stream to the caller: output collects it into Output, each_line delivers stdout line by line, and status has no return channel so it inherits instead. Inherit hands the stream to the parent's own descriptor.

ToFile and AppendToFile are the structured forms of a shell's > path and >> path. Both create the file if it is missing; ToFile truncates an existing one. A redirected stream is absent from Output, where it appears as an empty string.

#
Redirect::to_repr

source

#
Stdin

source
pub(all) enum Stdin {
  Text(String)
  Binary(Bytes)
  FromFile(String)
} derive(
Debug)

Where a child process's standard input comes from.

Text is encoded as UTF-8. Binary is written verbatim, which is what a child expecting a compressed or otherwise non-textual stream needs. FromFile is the structured form of a shell's < path; the file must already exist.

#
Stdin::to_repr

source

#
default_max_output_bytes

source
let default_max_output_bytes : Int

Default aggregate limit for captured stdout and stderr: 8 MiB.

Source Files

Powered by MoonBit

Site sourceReport issuePackagesBuild queueSkillsStatistics

© 2026 mooncakes.io