cmark

A port of pulldown-cmark's parser to MoonBit.

moon add kokic/cmark@0.1.1
Download zip
Author
Version
0.1.1
License
AGPL-3.0
Last updated
1 hour ago
Downloads
7
README

#kokic/cmark

A behavior-complete MoonBit port of the pulldown-cmark 0.13.1 parser (block + inline structure and the resulting event stream).

The parser's behavior is validated against the full spec corpus — CommonMark, GitHub-flavored, and all pulldown-cmark extension specs — by comparing the event dump produced by this package against the reference Rust implementation, case by case.

#Scope

Implemented:

  • Block parsing (headings, lists, blockquotes, code blocks, HTML blocks, tables, footnotes, definition lists, math, container extensions, metadata blocks, ...).
  • Inline parsing (emphasis, links, images, autolinks, code spans, math spans, smart punctuation, wikilinks, strikethrough, highlight, superscript/subscript, ...).
  • Event stream (Event/Tag/TagEnd) mirroring pulldown-cmark::Event.
  • All extension flags via Options, bit-compatible with pulldown-cmark's Options.

Not implemented (out of scope, per the porting task):

  • HTML rendering.
  • Serde support.
  • Broken-link / parser callbacks.

#Usage

let events = @cmark.parse(source, options)

Parse options are combined with Options::insert:

let options = @cmark.Options::empty()
options.insert(@cmark.enable_tables())
options.insert(@cmark.enable_footnotes())
let events = @cmark.parse(markdown, options)

#License

AGPL-3.0

#
Alignment

pub(all) enum Alignment {
None
Left
Center
Right
} derive(
Debug
)

Table column text alignment.

#
Allocations

type Allocations

Allocations of owned strings, links, alignments and headings.

#
BlockQuoteKind

pub(all) enum BlockQuoteKind {
Note
Tip
Important
Warning
Caution
} derive(
Debug
)

Block quote kind (Note, Tip, Important, Warning, Caution).

#
CodeBlockKind

pub(all) enum CodeBlockKind {
Indented
Fenced(String)
} derive(
Debug
)

Code block kind.

#
CodeBlockKind::is_fenced

fn CodeBlockKind::is_fenced(self : CodeBlockKind) -> Bool

#
CodeBlockKind::is_indented

fn CodeBlockKind::is_indented(self : CodeBlockKind) -> Bool

#
CodeDelims

type CodeDelims

Tracks tree indices of code span delimiters of each length.

#
ContainerKind

pub(all) enum ContainerKind {
Default
Spoiler
} derive(
Debug
)

Container block kind (Spoiler only).

#
Event

pub(all) enum Event {
Start(Tag)
End(TagEnd)
Text(String)
Code(String)
InlineMath(String)
DisplayMath(String)
Html(String)
InlineHtml(String)
FootnoteReference(String)
SoftBreak
HardBreak
Rule
TaskListMarker(Bool)
} derive(
Debug
)

Markdown events generated in a preorder traversal of the document tree.

#
Event::to_repr

#
HeadingLevel

pub(all) enum HeadingLevel {
H1
H2
H3
H4
H5
H6
} derive(
Debug
)

Heading levels.

#
HtmlScanGuard

type HtmlScanGuard

#
InlineStack

type InlineStack

#
Item

type Item

#
LinkStack

type LinkStack

#
LinkType

pub(all) enum LinkType {
Inline
Reference
ReferenceUnknown
Collapsed
CollapsedUnknown
Shortcut
ShortcutUnknown
Autolink
Email
WikiLink(Bool)
} derive(
Debug
)

#
LinkType::to_repr

#
MathDelims

type MathDelims

Tracks brace contexts and delimiter length for math delimiters.

#
MetadataBlockKind

pub(all) enum MetadataBlockKind {
YamlStyle
PlusesStyle
} derive(
Debug
)

Metadata block kind.

#
Options

pub struct Options {
flags : UInt
}

Option flags for the parser.

Bit values are a stable ABI, so option sets are interchangeable.

#
Options::contains

fn Options::contains(self : Options, other : Options) -> Bool

#
Options::empty

fn Options::empty() -> Options

#
Options::insert

fn Options::insert(self : Options, other : Options) -> Unit

#
Options::is_empty

fn Options::is_empty(self : Options) -> Bool

#
Parser

pub struct Parser {
text : Bytes
options : Options
tree : Tree[Item]
allocs : Allocations
html_scan_guard : HtmlScanGuard
link_ref_expansion_limit : Int
inline_stack : InlineStack
link_stack : LinkStack
wikilink_stack : LinkStack
code_delims : CodeDelims
math_delims : MathDelims
}

Markdown event and source range iterator.

#
Parser::next_event

fn Parser::next_event(self : Parser) -> Event?

Returns the next event, or None when the document is exhausted.

#
Parser::next_event_range

fn Parser::next_event_range(self : Parser) -> (Event, Int, Int)?

Returns the next event and its source range (start, end), or None.

#
Tag

pub(all) enum Tag {
Paragraph
Heading(level~ : HeadingLevel, id~ : String?, classes~ : Array[String], attrs~ : Array[(String, String?)])
BlockQuote(BlockQuoteKind?)
CodeBlock(CodeBlockKind)
ContainerBlock(ContainerKind, String)
HtmlBlock
List(Int64?)
Item
FootnoteDefinition(String)
DefinitionList
DefinitionListTitle
DefinitionListDefinition
Table(Array[Alignment])
TableHead
TableRow
TableCell
Emphasis
Strong
Strikethrough
Highlight
Superscript
Subscript
Link(link_type~ : LinkType, dest_url~ : String, title~ : String, id~ : String)
Image(link_type~ : LinkType, dest_url~ : String, title~ : String, id~ : String)
MetadataBlock(MetadataBlockKind)
} derive(
Debug
)

Tags for elements that can contain other elements.

#
Tag::to_repr

#
TagEnd

pub(all) enum TagEnd {
Paragraph
Heading(HeadingLevel)
BlockQuote(BlockQuoteKind?)
CodeBlock
ContainerBlock(ContainerKind)
HtmlBlock
List(Bool)
Item
FootnoteDefinition
DefinitionList
DefinitionListTitle
DefinitionListDefinition
Table
TableHead
TableRow
TableCell
Emphasis
Strong
Strikethrough
Highlight
Superscript
Subscript
Link
Image
MetadataBlock(MetadataBlockKind)
} derive(
Debug
)

The end of a Tag.

#
TagEnd::to_repr

#
Tree

type Tree[T]

A tree abstraction, intended for fast building as a preorder traversal.

#
enable_cjk_friendly_emphasis

fn enable_cjk_friendly_emphasis() -> Options

Enables CJK-friendly emphasis.

#
enable_container_extensions

fn enable_container_extensions() -> Options

Enables colon-delimited container extension blocks.

#
enable_definition_list

fn enable_definition_list() -> Options

Enables definition lists.

#
enable_footnotes

fn enable_footnotes() -> Options

Enables GitHub-compatible footnote syntax.

#
enable_gfm

fn enable_gfm() -> Options

Enables miscellaneous GFM features (blockquote tags).

#
enable_heading_attributes

fn enable_heading_attributes() -> Options

Enables heading attributes (# text {#id .class}).

#
enable_highlight

fn enable_highlight() -> Options

Enables ==highlight==.

#
enable_math

fn enable_math() -> Options

Enables inline and display math.

#
enable_math_everywhere

fn enable_math_everywhere() -> Options

Enables treating every $ as a math delimiter, regardless of surrounding whitespace or digits. A literal $ must be escaped as \$. Implies enable_math.

#
enable_old_footnotes

fn enable_old_footnotes() -> Options

Enables the old (hoedown-style) footnote syntax. Implies enable_footnotes.

#
enable_pluses_delimited_metadata_blocks

fn enable_pluses_delimited_metadata_blocks() -> Options

Enables +++-delimited metadata blocks.

#
enable_smart_punctuation

fn enable_smart_punctuation() -> Options

Enables smart punctuation.

#
enable_strikethrough

fn enable_strikethrough() -> Options

Enables strikethrough (GFM ~~x~~).

#
enable_subscript

fn enable_subscript() -> Options

Enables subscript.

#
enable_superscript

fn enable_superscript() -> Options

Enables superscript.

#
enable_tables

fn enable_tables() -> Options

Enables GitHub-style tables.

#
enable_tasklists

fn enable_tasklists() -> Options

Enables GitHub-style task lists.
fn enable_wikilinks() -> Options

Enables Obsidian-style wikilinks.

#
enable_yaml_style_metadata_blocks

fn enable_yaml_style_metadata_blocks() -> Options

Enables YAML-style metadata blocks.

#
event_dump

fn event_dump(source : String, options : Options) -> String

Parses source and returns a line-per-event canonical dump.

#
new_parser

fn new_parser(text : Bytes, options : Options) -> Parser

Creates a new event parser for a markdown string with given options.

#
parse

fn parse(source : String, options : Options) -> Array[Event]