dtbscope

    Device tree compiler and inspector in pure MoonBit: DTS to DTB, DTB back to DTS, and overlay application, with no dtc binary present.

    devicetree
    dtb
    dts
    fdt
    embedded
    Download zip
    Version
    0.1.0
    License
    Apache-2.0
    Last updated
    2 hours ago
    Downloads
    2

    #dtbscope

    Device tree compiler and inspector in pure MoonBit.

    Reads and writes the flattened device tree blob format that firmware hands to the kernel at boot. No dtc binary, no C toolchain, no host operating system assumptions: the same code compiles to Wasm and to native, so a device tree can be inspected in a browser as easily as in a build script.

    #Why

    Every ARM and RISC-V board in production describes its hardware in a device tree, and the only tool that compiles one is dtc, a C program that has to be built and installed before it can run. That is fine on a Linux build host and awkward everywhere else: a Mac developer cannot read a .dtb pulled off a board, a CI runner has to apt-get a native package to check that an overlay applies cleanly, and a browser cannot do either.

    MoonBit compiles to both Wasm and native from one source, which is the property this project is built around.

    #Status

    Early. The blob header parses and is validated; the struct and strings blocks are next.

    CapabilityStatus
    40-byte blob header, version 17done
    Block tiling and alignment validationdone
    Bounds-checked primitive readsdone
    Struct block walk (nodes, properties)done
    Strings block lookupdone
    Nesting limit and cycle guarddone
    DTS lexer: identifiers, strings, numbers, commentsdone
    DTS parser: nodes, properties, typed valuesdone
    Labels, &label and &{/path} referencesdone
    /bits/ and /memreserve/done
    DTB writerdone
    Byte-identical to dtc on all three fixturesdone
    CLI: compile and dump modesdone
    Reference resolution, phandle assignmentdone
    DTS emitternot started
    Overlay applicationnot started

    #Quick start

    moon test

    Encode a source file and compare it with dtc:

    moon run cmd/main -- internal/fixtures/nested.dts -o /tmp/out.dtb dtc -q -I dts -O dtb -o /tmp/dtc.dtb internal/fixtures/nested.dts cmp /tmp/out.dtb /tmp/dtc.dtb

    Three fixtures. minimal.dtb is a flat board tree; nested.dtb has a CPU subtree, an interrupt controller with a phandle and a ranges property that encodes to nothing; realview-pb1176 is the ARM RealView PB1176 from the Linux tree, 68 nodes with aliases, dma-ranges and 14 phandles. All three are compared byte for byte against dtc, and node by node against dtc -I dtb -O dts.

    The RealView source is stored in its preprocessed form. The file in the kernel tree begins with #include, and cpp has to run before either tool sees it; keeping the post-cpp text means the comparison does not depend on which header revisions the machine has.

    Fixtures live in internal/fixtures as DTS sources plus the DTB that dtc 1.8.1 produced from them. The blob is checked in so the tests do not need dtc; tools/gen-fixtures.sh regenerates it, and tools/gen-fixtures.sh --check verifies it still matches, which is what CI runs.

    #Layout

    FileContents
    dtbscope.mbtError type, cursor, hex formatting
    reader.mbtBounds-checked reads of the primitive types
    header.mbtBlob header parse and validation
    struct_block.mbtToken walk, node tree, structural warnings
    dts_token.mbtToken kinds, spans, scanner state
    dts_lex.mbtCharacter rules and token scanning
    dts_parse.mbtSource grammar, value parsing, source tree
    phandle.mbtLabel table, phandle numbering, reference rewrite
    dtb_write.mbtBlob encoder
    cmd/main/Command line front end
    tools/gen-fixtures.shFixture regeneration
    internal/fixtures/DTS sources and their compiled DTBs

    #Scope

    The C preprocessor is out of scope. Kernel device tree sources use #include and macros, and the kernel's own build runs cpp before dtc; this library expects the same, so the input it takes is what cpp produced rather than the file as it sits in the tree. /incbin/ is likewise refused: it would make the parse depend on files outside the source, and the resulting bytes depend on data the source does not contain.

    #Acknowledgements

    The format is specified in the Devicetree Specification, v0.4. Behaviour is compared against dtc from dgibson/dtc, which is used only to produce fixtures and to check output; no dtc code is part of this module.

    Licensed under Apache-2.0.

    DtbError

    pub(all) suberror DtbError {
    UnexpectedEnd(offset~ : Int64, needed~ : Int64, available~ : Int64)
    BadMagic(magic~ : UInt)
    BadValue(field~ : String, value~ : Int64)
    OutOfBounds(what~ : String, offset~ : Int64, size~ : Int64, limit~ : Int64)
    BadName(offset~ : Int64)
    Inconsistent(what~ : String, detail~ : String)
    LimitExceeded(what~ : String, limit~ : Int64)
    Unsupported(what~ : String)
    }

    Every way a device tree blob can be rejected.

    Parsing never returns a half-built tree. A damaged blob surfaces as one of these, and the payload carries the offset of the bytes that failed so a caller can point at them instead of guessing.
    impl Show for DtbError

    DtbError::output

    fn DtbError::output(self : DtbError, logger : &Logger) -> Unit

    Exposes the message as a plain method so callers can format it themselves.

    DtbError::to_string

    fn DtbError::to_string(self : DtbError) -> String

    Exposes the message as a plain method so callers can format it themselves.

    LexerError

    pub(all) suberror LexerError {
    UnexpectedChar(char~ : Char, span~ : Span)
    UnterminatedComment(start~ : Span)
    UnterminatedString(start~ : Span)
    BadEscape(char~ : Char, span~ : Span)
    BadNumber(text~ : String, span~ : Span)
    }

    Every way the scanner can reject a source file.

    These are syntax defects, distinct from DtbError which describes a damaged binary blob. Keeping them separate means a caller can tell "the file I was given is not valid DTS" from "the tree I was given is inconsistent".
    impl Show for LexerError

    LexerError::output

    fn LexerError::output(self : LexerError, logger : &Logger) -> Unit

    Exposes the scanner error message as a plain method.

    LexerError::to_string

    fn LexerError::to_string(self : LexerError) -> String

    Exposes the scanner error message as a plain method.

    OverlayError

    pub(all) suberror OverlayError {
    UnknownSymbol(label~ : String)
    MissingTarget(path~ : String)
    BadFixup(detail~ : String)
    NoSymbols
    BadFragment(name~ : String)
    Resolve(inner~ : ResolveError)
    }

    Every way building or applying an overlay can fail.

    OverlayError::output

    fn OverlayError::output(self : OverlayError, logger : &Logger) -> Unit

    Exposes the message as a plain method.

    OverlayError::to_string

    fn OverlayError::to_string(self : OverlayError) -> String

    Exposes the message as a plain method.

    ParseError

    pub(all) suberror ParseError {
    Unexpected(token~ : TokenKind, expected~ : String, span~ : Span)
    Lexical(inner~ : LexerError)
    BadValue(detail~ : String, span~ : Span)
    LimitExceeded(what~ : String, limit~ : Int)
    Unsupported(what~ : String, span~ : Span)
    }

    Every way a source file can fail to parse.

    Distinct from LexerError, which is about characters, and from DtbError, which is about a binary blob. A caller can therefore report the three failure modes differently, which matters when the input came from a user pasting text.
    impl Show for ParseError

    ParseError::output

    fn ParseError::output(self : ParseError, logger : &Logger) -> Unit

    Exposes the parse error message as a plain method.

    ParseError::to_string

    fn ParseError::to_string(self : ParseError) -> String

    Exposes the parse error message as a plain method.

    ResolveError

    pub(all) suberror ResolveError {
    UnknownLabel(label~ : String)
    UnknownPath(path~ : String)
    DuplicateLabel(label~ : String, first~ : String, second~ : String)
    DuplicatePhandle(value~ : UInt, first~ : String, second~ : String)
    BadPhandle(value~ : UInt, node~ : String)
    }

    Every way resolution can fail.

    These are separate from ParseError because a source file can be perfectly well-formed and still refer to a node that does not exist. That is a linking problem, not a syntax one, and dtc reports it the same way.

    ResolveError::output

    fn ResolveError::output(self : ResolveError, logger : &Logger) -> Unit

    Exposes the resolution error message as a plain method.

    ResolveError::to_string

    fn ResolveError::to_string(self : ResolveError) -> String

    Exposes the resolution error message as a plain method.

    Bounds

    pub(all) struct Bounds {
    bytes : Bytes
    start : Int64
    limit : Int64
    }

    A buffer together with the range of it that may be read.

    start and limit are absolute offsets into bytes. Keeping them absolute means a sub-blob (the struct block inside a larger blob, say) can be described without copying, while error messages still report a position the caller can find in the original file.

    Bounds::cstr

    fn Bounds::cstr(self : Bounds, offset : Int64, limit : Int64) -> (String, Int64) raise DtbError

    Reads a NUL-terminated string that must end inside [offset, limit).

    Returns the string and the offset just past its terminator. The DTB strings block holds all property names back to back, so the tail version is what the caller actually needs.

    Bounds::length

    fn Bounds::length(self : Bounds) -> Int64

    Number of readable bytes.

    Bounds::of_bytes

    fn Bounds::of_bytes(bytes : Bytes) -> Bounds

    Wraps a whole buffer as readable from 0 to its length.

    Bounds::of_range

    fn Bounds::of_range(bytes : Bytes, start : Int64, limit : Int64) -> Bounds raise DtbError

    Restricts a buffer to [start, limit), rejecting a range that runs off the end of the underlying bytes.

    Bounds::read_u16

    fn Bounds::read_u16(self : Bounds, cur : Cursor) -> (UInt, Cursor) raise DtbError

    Decodes an unsigned 16-bit integer in the cursor's byte order.

    Bounds::read_u32

    fn Bounds::read_u32(self : Bounds, cur : Cursor) -> (UInt, Cursor) raise DtbError

    Decodes an unsigned 32-bit integer in the cursor's byte order.

    Accumulated in Int64 so that a value with bit 31 set does not go negative before it is reinterpreted.

    Bounds::read_u64

    fn Bounds::read_u64(self : Bounds, cur : Cursor) -> (Int64, Cursor) raise DtbError

    Decodes an unsigned 64-bit integer in the cursor's byte order.

    Returned as Int64 so all 64 bits survive; callers that need the unsigned reading mask at the use site, which is exactly where they have to think about it anyway.

    Bounds::read_u8

    fn Bounds::read_u8(self : Bounds, cur : Cursor) -> (UInt, Cursor) raise DtbError

    Decodes an unsigned 8-bit integer.

    Bounds::require

    fn Bounds::require(self : Bounds, offset : Int64, size : Int64, what : String) -> Unit raise DtbError

    Fails unless [offset, offset + size) lies inside the readable range.

    Parsers call this before trusting any length field that came out of the file, so a corrupt length is reported with the field name that carried it.

    Bounds::slice

    fn Bounds::slice(self : Bounds, offset : Int64, n : Int64, what : String) -> Bytes raise DtbError

    Copies n bytes starting at offset into a fresh buffer.

    Cursor

    pub(all) struct Cursor {
    pos : Int64
    endian : Endian
    }

    A position in a buffer together with the byte order to decode it in.

    Cursor::big

    fn Cursor::big(pos : Int64) -> Cursor

    Produces a cursor at pos reading big-endian fields, which is what every field inside a DTB uses.

    Cursor::little

    fn Cursor::little(pos : Int64) -> Cursor

    Produces a cursor at pos reading little-endian fields.

    Endian

    pub(all) enum Endian {
    Little
    Big
    }

    Byte order of a field.

    The DTB format is big-endian throughout, but the DTS front end has to quote /bits/ 16 cell lists in the host's own order when it dumps them back out, so the reader keeps the axis explicit rather than hard-coding it.

    Fragment

    pub(all) struct Fragment {
    target : ValuePart
    span : Span
    node : SourceNode
    }

    One top-level patch in a plugin source, written &label { ... }.

    The compiler turns each of these into a fragment@N node holding a target reference and an __overlay__ subtree, which is what the binary form stores. They are kept separate here rather than folded at parse time so that a file with no fragments still parses as a plain tree.
    pub(all) struct Header {
    totalsize : Int64
    off_dt_struct : Int64
    off_dt_strings : Int64
    off_mem_rsvmap : Int64
    version : UInt
    last_comp_version : UInt
    boot_cpuid_phys : UInt
    size_dt_strings : Int64
    size_dt_struct : Int64
    }

    The four blocks a blob is made of, with their offsets resolved against the start of the file.

    Offsets here are relative to the start of the blob, which is what every tool that walks the tree needs. Absolute offsets would break the moment a blob is embedded inside another container.

    Header::is_modern

    fn Header::is_modern(self : Header) -> Bool

    Whether the blob uses the version 17 alignment rules.

    Version 16 and earlier packed the strings block without padding, so callers that care about byte-identical round-tripping have to know which rules were in force when the input was produced.

    Header::size

    fn Header::size(self : Header) -> Int64

    Total size the header claims, which is also the length a verbatim copy has to preserve.

    LabelTable

    pub(all) struct LabelTable {
    labels : Map[String, String]
    explicit : Map[String, UInt]
    }

    Where every label points, and where every explicit phandle already is.

    Built in one pass over the tree before any reference is rewritten, so that a forward reference — a node referred to by a node that appears before it — works the same as a backward one. dtc allows both; a streaming resolver would not.

    Lexer

    pub(all) struct Lexer {
    source : String
    pos : Int
    line : Int
    column : Int
    in_bytes : Bool
    }

    The scanner state: the source text plus a position in it.

    The position is a byte offset into the string together with the line and column that offset sits at. Both are advanced together so that Span never has to be recomputed by rescanning from the start of the file.

    Lexer::at_end

    fn Lexer::at_end(self : Lexer) -> Bool

    Whether the scanner has reached the end.

    Lexer::clone

    fn Lexer::clone(self : Lexer) -> Lexer

    Copies the scanner so a caller can look at the next token without consuming it.

    The alternative is a token buffer in the parser, which would mean every directive check allocating. A clone is two integers and a string reference, and the string is immutable, so this is the cheaper of the two.

    Lexer::new

    fn Lexer::new(source : String) -> Lexer

    Creates a scanner at the first character of source.

    Lexer::next

    fn Lexer::next(self : Lexer) -> Token raise LexerError

    Scans the next token, skipping whitespace and comments first.

    Lexer::scan_hex_byte

    fn Lexer::scan_hex_byte(self : Lexer, span : Span) -> TokenKind raise LexerError

    Scans one byte inside a [...] string.

    Bytes are exactly two hex digits. Older dtc accepted a single digit and padded it, which made [1 2] mean [01 02]; current dtc rejects it, and so does this scanner, because silently changing the byte count of a value is the kind of difference that shows up much later as a wrong MAC address.

    Node

    pub(all) struct Node {
    name : String
    properties : Array[Property]
    children : Array[Node]
    }

    One node of the tree.

    The root node's name is the empty string, matching the / of the source form. Property and child order is preserved as the blob lists them, because dtc writes properties before children and byte-identical round-tripping depends on keeping that order rather than sorting it.

    Parser

    pub(all) struct Parser {
    lexer : Lexer
    current : Token
    }

    Parser state: the scanner plus one token of lookahead.

    One token is enough for this grammar. The only construct that needs to choose between alternatives by looking further is a node body, where an identifier may be followed by {, = or ; — and the token right after the identifier settles which. Directives need two tokens of lookahead (/, word, /), which directive_word gets by cloning the scanner instead of buffering.

    Parser::new

    fn Parser::new(source : String) -> Parser raise ParseError

    Creates a parser positioned at the first token.

    Property

    pub(all) struct Property {
    name : String
    value : Bytes
    offset : Int64
    }

    A property as it appears in the block: a name resolved through the strings block and the value bytes exactly as stored.

    The value is left as raw bytes rather than decoded into cells. A property can be a string, a cell list, a byte string, or an interleaved list of those, and which one it is depends on a binding rather than on the bytes. Decoding here would mean guessing, so the caller decodes with the schema it has.

    SourceFile

    pub(all) struct SourceFile {
    root : SourceNode
    memreserves : Array[(UInt64, UInt64)]
    is_plugin : Bool
    fragments : Array[Fragment]
    }

    A parsed source file: the root node plus the memory reservations declared before it.

    The two are kept apart because the blob stores them in different blocks and the reservation map is not part of the node tree.

    SourceNode

    pub(all) struct SourceNode {
    name : String
    labels : Array[String]
    properties : Array[SourceProperty]
    children : Array[SourceNode]
    }

    A node in the source tree, before references are resolved.

    This is deliberately a different type from the binary Node: a source property holds unresolved references and typed cell lists, neither of which has a byte representation until the whole file has been read and phandles assigned. The conversion happens in one pass afterwards, so the encoder never has to look ahead.

    SourceProperty

    pub(all) struct SourceProperty {
    name : String
    value : Value
    span : Span
    }

    A property in the source tree, before its value is encoded.

    Span

    pub(all) struct Span {
    line : Int
    column : Int
    }

    A position in the source file, 1-based so that it matches what an editor shows.
    impl Show for Span

    Span::output

    fn Span::output(self : Span, logger : &Logger) -> Unit

    Exposes the span text as a plain method.

    Span::to_string

    fn Span::to_string(self : Span) -> String

    Exposes the span text as a plain method.

    StringTable

    pub(all) struct StringTable {
    bytes :
    Buffer

    offsets : Map[String, Int]
    }

    A name table for the strings block.

    Insertion order matters: dtc writes a name the first time it appears and never moves it, so a tree encoded in source order reproduces the same block layout. A hash map alone would not, which is why the offsets are also kept in a list.

    StringTable::new

    Creates an empty table.

    StructTag

    pub(all) enum StructTag {
    BeginNode
    EndNode
    Prop
    Nop
    End
    }

    Token tags that appear in the struct block.

    0x5 through 0x8 are unspecified and must be skipped by a reader, which is why parse_struct treats an unknown tag as an error rather than trying to guess a skip length: the format gives no way to know how long an unknown token is.

    Token

    pub(all) struct Token {
    kind : TokenKind
    span : Span
    }

    A token together with where it came from.

    Declared after TokenKind because MoonBit resolves a struct's field types at the point the struct is declared, and the scanner returns these by value, so the order matters.

    TokenKind

    pub(all) enum TokenKind {
    Ident(String)
    String(String)
    Number(UInt64)
    LBrace
    RBrace
    LAngle
    RAngle
    LBracket
    RBracket
    LParen
    RParen
    Comma
    Semicolon
    Equals
    Slash
    Amp
    Colon
    Minus
    HexByte(Byte)
    Eof
    }

    The kinds of token the source format needs.

    A label appears before a node rather than inside the value grammar, so it needs no separate token kind: the parser sees Ident, Colon and decides from what follows.
    impl Show for TokenKind

    TokenKind::output

    fn TokenKind::output(self : TokenKind, logger : &Logger) -> Unit

    Exposes the token text as a plain method.

    TokenKind::to_string

    fn TokenKind::to_string(self : TokenKind) -> String

    Exposes the token text as a plain method.

    Tree

    pub(all) struct Tree {
    root : Node
    warnings : Array[Warning]
    }

    The result of walking the struct block: the root node plus anything noteworthy that did not justify failing the parse.

    Value

    pub(all) struct Value {
    parts : Array[ValuePart]
    }

    A property value: one or more parts concatenated.

    reg = <0 1>, <2 3> and compatible = "a", "b" are both a list of one kind, while mixed = "x", <1> is legal too. Keeping the parts separate means the encoder does not have to re-parse the bytes to find the seams.

    ValuePart

    pub(all) enum ValuePart {
    Text(String)
    Cells(Array[UInt64])
    Bytes(Array[Byte])
    Ref(String)
    PathRef(String)
    Refs(Array[ValuePart])
    TypedCells(UInt64, Array[UInt64])
    }

    The kinds of thing a property value can hold once parsed.

    This is the intermediate form between the source text and the bytes the binary format stores. Strings, cells and byte strings all end up in one value buffer, but they go in differently, and a reference cannot be encoded at all until phandles are assigned, so the distinction has to survive until the encoding pass.

    Warning

    pub(all) enum Warning {
    MissingEnd(offset~ : Int64)
    TrailingData(offset~ : Int64, remaining~ : Int64)
    }

    A structural fact about the blob that a strict reader rejects but that appears in firmware produced by older generators.

    These are reported rather than raised: the tree is still readable, and a tool that refused to show it would be less useful than one that shows it with the defect named.
    impl Show for Warning

    Warning::output

    fn Warning::output(self : Warning, logger : &Logger) -> Unit

    Exposes the warning message as a plain method.

    Warning::to_string

    fn Warning::to_string(self : Warning) -> String

    Exposes the warning message as a plain method.

    build_overlay

    fn build_overlay(file : SourceFile) -> SourceFile raise OverlayError

    Builds an overlay from a plugin source.

    The result is a normal blob whose __fixups__ node lists where each unresolved reference landed. dtc produces the same bytes for the same source with -@.

    Only references that resolve outside the overlay remain unresolved. A plugin that refers to a label it defines itself gets a real phandle, which is what dtc does too: the fixup list is for what the base tree has to supply.

    encode_blob

    fn encode_blob(file : SourceFile) -> Bytes raise DtbError

    Encodes a source tree as a blob.

    The four blocks are laid out in the order the format fixes: header, memory reservation map, struct block, strings block. Sizes are written last, once the blocks exist.

    References must already be resolved. Encoding refuses a tree that still contains one, because a reference has no byte form: the whole point of phandles is that the encoder decided the number.

    parse_header

    fn parse_header(bytes : Bytes) -> Header raise DtbError

    Reads the header at the start of a blob and validates that the four blocks tile the file.

    The alignment requirements from the specification are checked here rather than at each use site: the struct block is 4-byte aligned, and in version 17 and later so is the strings block and the memory reservation map. A blob that violates them is rejected before any block is decoded, so no later parser has to defend against a misaligned read.

    parse_source

    fn parse_source(source : String) -> SourceFile raise ParseError

    Parses a whole source file.

    The leading /dts-v1/; is required: without it the file is not a device tree source, and treating it as one would accept a fragment as if it were complete. /plugin/; may appear after it and is recorded rather than acted on, because its only effect is on how unresolved references are reported.

    parse_struct

    fn parse_struct(struct_bounds : Bounds, strings_bounds : Bounds) -> Tree raise DtbError

    Walks the struct block and builds the node tree.

    Every child node is appended to its parent as soon as its own END_NODE is seen, so a partially built tree can never be observed by the caller: the only way out is the completed Tree or an error.

    resolve_references

    fn resolve_references(file : SourceFile) -> SourceFile raise ResolveError

    Assigns numbers to every node that needs one, and rewrites references into cells.

    The numbering follows the order in which the source first refers to each node, not the order the nodes appear in the tree. That distinction is not cosmetic: dtc numbers references as it resolves them, so a board that refers to its clock before it refers to its interrupt controller gets 1 for the clock even when the controller appears first in the file. A tree walk produces a different assignment, every reference then points at the wrong node, and the blob is still structurally valid — which is why the byte comparison against dtc is the only test that catches it.

    References under /aliases do not count. They become path strings rather than phandles, so a node reached only through an alias gets no number.