cmark

    A port of pulldown-cmark's parser to MoonBit.

    Download zip
    Author
    Version
    0.1.1
    License
    AGPL-3.0
    Last updated
    26 days ago
    Downloads
    18

    #kokic/cmark

    A behavior-complete MoonBit port of the pulldown-cmark 0.13.1 parser (block + inline structure and the resulting event stream).

    The parser's behavior is validated against the full spec corpus — CommonMark, GitHub-flavored, and all pulldown-cmark extension specs — by comparing the event dump produced by this package against the reference Rust implementation, case by case.

    #Scope

    Implemented:

    • Block parsing (headings, lists, blockquotes, code blocks, HTML blocks, tables, footnotes, definition lists, math, container extensions, metadata blocks, ...).
    • Inline parsing (emphasis, links, images, autolinks, code spans, math spans, smart punctuation, wikilinks, strikethrough, highlight, superscript/subscript, ...).
    • Event stream (Event/Tag/TagEnd) mirroring pulldown-cmark::Event.
    • All extension flags via Options, bit-compatible with pulldown-cmark's Options.

    Not implemented (out of scope, per the porting task):

    • HTML rendering.
    • Serde support.
    • Broken-link / parser callbacks.

    #Usage

    let events = @cmark.parse(source, options)

    Parse options are combined with Options::insert:

    let options = @cmark.Options::empty()
    options.insert(@cmark.enable_tables())
    options.insert(@cmark.enable_footnotes())
    let events = @cmark.parse(markdown, options)

    #License

    AGPL-3.0

    Alignment

    pub(all) enum Alignment {
    None
    Left
    Center
    Right
    } derive(
    Debug
    )

    Table column text alignment.

    Allocations

    type Allocations

    Allocations of owned strings, links, alignments and headings.

    BlockQuoteKind

    pub(all) enum BlockQuoteKind {
    Note
    Tip
    Important
    Warning
    Caution
    } derive(
    Debug
    )

    Block quote kind (Note, Tip, Important, Warning, Caution).

    CodeBlockKind

    pub(all) enum CodeBlockKind {
    Indented
    Fenced(String)
    } derive(
    Debug
    )

    Code block kind.

    CodeBlockKind::is_fenced

    fn CodeBlockKind::is_fenced(self : CodeBlockKind) -> Bool

    CodeBlockKind::is_indented

    fn CodeBlockKind::is_indented(self : CodeBlockKind) -> Bool

    CodeDelims

    type CodeDelims

    Tracks tree indices of code span delimiters of each length.

    ContainerKind

    pub(all) enum ContainerKind {
    Default
    Spoiler
    } derive(
    Debug
    )

    Container block kind (Spoiler only).

    Event

    pub(all) enum Event {
    Start(Tag)
    End(TagEnd)
    Text(String)
    Code(String)
    InlineMath(String)
    DisplayMath(String)
    Html(String)
    InlineHtml(String)
    FootnoteReference(String)
    SoftBreak
    HardBreak
    Rule
    TaskListMarker(Bool)
    } derive(
    Debug
    )

    Markdown events generated in a preorder traversal of the document tree.

    Event::to_repr

    HeadingLevel

    pub(all) enum HeadingLevel {
    H1
    H2
    H3
    H4
    H5
    H6
    } derive(
    Debug
    )

    Heading levels.

    HtmlScanGuard

    type HtmlScanGuard

    InlineStack

    type InlineStack

    Item

    type Item

    LinkStack

    type LinkStack

    LinkType

    pub(all) enum LinkType {
    Inline
    Reference
    ReferenceUnknown
    Collapsed
    CollapsedUnknown
    Shortcut
    ShortcutUnknown
    Autolink
    Email
    WikiLink(Bool)
    } derive(
    Debug
    )

    LinkType::to_repr

    MathDelims

    type MathDelims

    Tracks brace contexts and delimiter length for math delimiters.

    MetadataBlockKind

    pub(all) enum MetadataBlockKind {
    YamlStyle
    PlusesStyle
    } derive(
    Debug
    )

    Metadata block kind.

    Options

    pub struct Options {
    flags : UInt
    }

    Option flags for the parser.

    Bit values are a stable ABI, so option sets are interchangeable.

    Options::contains

    fn Options::contains(self : Options, other : Options) -> Bool

    Options::empty

    fn Options::empty() -> Options

    Options::insert

    fn Options::insert(self : Options, other : Options) -> Unit

    Options::is_empty

    fn Options::is_empty(self : Options) -> Bool

    Parser

    pub struct Parser {
    text : Bytes
    options : Options
    tree : Tree[Item]
    allocs : Allocations
    html_scan_guard : HtmlScanGuard
    link_ref_expansion_limit : Int
    inline_stack : InlineStack
    link_stack : LinkStack
    wikilink_stack : LinkStack
    code_delims : CodeDelims
    math_delims : MathDelims
    }

    Markdown event and source range iterator.

    Parser::next_event

    fn Parser::next_event(self : Parser) -> Event?

    Returns the next event, or None when the document is exhausted.

    Parser::next_event_range

    fn Parser::next_event_range(self : Parser) -> (Event, Int, Int)?

    Returns the next event and its source range (start, end), or None.

    Tag

    pub(all) enum Tag {
    Paragraph
    Heading(level~ : HeadingLevel, id~ : String?, classes~ : Array[String], attrs~ : Array[(String, String?)])
    BlockQuote(BlockQuoteKind?)
    CodeBlock(CodeBlockKind)
    ContainerBlock(ContainerKind, String)
    HtmlBlock
    List(Int64?)
    Item
    FootnoteDefinition(String)
    DefinitionList
    DefinitionListTitle
    DefinitionListDefinition
    Table(Array[Alignment])
    TableHead
    TableRow
    TableCell
    Emphasis
    Strong
    Strikethrough
    Highlight
    Superscript
    Subscript
    Link(link_type~ : LinkType, dest_url~ : String, title~ : String, id~ : String)
    Image(link_type~ : LinkType, dest_url~ : String, title~ : String, id~ : String)
    MetadataBlock(MetadataBlockKind)
    } derive(
    Debug
    )

    Tags for elements that can contain other elements.

    Tag::to_repr

    TagEnd

    pub(all) enum TagEnd {
    Paragraph
    Heading(HeadingLevel)
    BlockQuote(BlockQuoteKind?)
    CodeBlock
    ContainerBlock(ContainerKind)
    HtmlBlock
    List(Bool)
    Item
    FootnoteDefinition
    DefinitionList
    DefinitionListTitle
    DefinitionListDefinition
    Table
    TableHead
    TableRow
    TableCell
    Emphasis
    Strong
    Strikethrough
    Highlight
    Superscript
    Subscript
    Link
    Image
    MetadataBlock(MetadataBlockKind)
    } derive(
    Debug
    )

    The end of a Tag.

    TagEnd::to_repr

    Tree

    type Tree[T]

    A tree abstraction, intended for fast building as a preorder traversal.

    enable_cjk_friendly_emphasis

    fn enable_cjk_friendly_emphasis() -> Options

    Enables CJK-friendly emphasis.

    enable_container_extensions

    fn enable_container_extensions() -> Options

    Enables colon-delimited container extension blocks.

    enable_definition_list

    fn enable_definition_list() -> Options

    Enables definition lists.

    enable_footnotes

    fn enable_footnotes() -> Options

    Enables GitHub-compatible footnote syntax.

    enable_gfm

    fn enable_gfm() -> Options

    Enables miscellaneous GFM features (blockquote tags).

    enable_heading_attributes

    fn enable_heading_attributes() -> Options

    Enables heading attributes (# text {#id .class}).

    enable_highlight

    fn enable_highlight() -> Options

    Enables ==highlight==.

    enable_math

    fn enable_math() -> Options

    Enables inline and display math.

    enable_math_everywhere

    fn enable_math_everywhere() -> Options

    Enables treating every $ as a math delimiter, regardless of surrounding whitespace or digits. A literal $ must be escaped as \$. Implies enable_math.

    enable_old_footnotes

    fn enable_old_footnotes() -> Options

    Enables the old (hoedown-style) footnote syntax. Implies enable_footnotes.

    enable_pluses_delimited_metadata_blocks

    fn enable_pluses_delimited_metadata_blocks() -> Options

    Enables +++-delimited metadata blocks.

    enable_smart_punctuation

    fn enable_smart_punctuation() -> Options

    Enables smart punctuation.

    enable_strikethrough

    fn enable_strikethrough() -> Options

    Enables strikethrough (GFM ~~x~~).

    enable_subscript

    fn enable_subscript() -> Options

    Enables subscript.

    enable_superscript

    fn enable_superscript() -> Options

    Enables superscript.

    enable_tables

    fn enable_tables() -> Options

    Enables GitHub-style tables.

    enable_tasklists

    fn enable_tasklists() -> Options

    Enables GitHub-style task lists.
    fn enable_wikilinks() -> Options

    Enables Obsidian-style wikilinks.

    enable_yaml_style_metadata_blocks

    fn enable_yaml_style_metadata_blocks() -> Options

    Enables YAML-style metadata blocks.

    event_dump

    fn event_dump(source : String, options : Options) -> String

    Parses source and returns a line-per-event canonical dump.

    new_parser

    fn new_parser(text : Bytes, options : Options) -> Parser

    Creates a new event parser for a markdown string with given options.

    parse

    fn parse(source : String, options : Options) -> Array[Event]