Convert HTML to Markdown (CommonMark), a MoonBit port of JohannesKaufmann/html-to-markdown v2.

html

markdown

converter

commonmark

moon add hustcer/html2md@0.1.1

Download zip

Author

hustcer

Version

0.1.1

License

Apache-2.0

Repository

github.com/hustcer/html2md

Last updated

last month

Downloads

Dependencies

bobzhang/html_parser0.1.7

README

#html2md

Convert HTML to Markdown (CommonMark) in MoonBit.

This is a MoonBit port of the Go library JohannesKaufmann/html-to-markdown v2.

#Features

HTML-string entry via convert, plus convert_dom for already-parsed DOM nodes, with the same CommonMark option set on both paths.
Markdown image rendering for <img> and GFM pipe-table rendering for <table>.
GFM strikethrough for s, del and strike tags.
Table-specific behavior for escaped pipes, header alignment markers, simple colspan/rowspan placeholder cells and role="presentation" fallback.
Smart escaping that protects fake Markdown syntax in text while preserving real formatting produced from HTML tags.
Configurable CommonMark output for heading style, emphasis delimiters, horizontal rules, bullet markers, code fences, empty-link behavior and list separation comments.
Relative-link resolution against domain, plus RFC 3986 query-component normalization for spaces, Unicode, valid percent octets and literal +.

#Install

moon add hustcer/html2md

Add the import to the package that needs it (moon.pkg):

import { "hustcer/html2md" }

#Usage

///|
test "basic" {
  let html =
    #|<h1>Hello</h1>
    #|<p>Some <b>bold</b> and <i>italic</i> text with a
    #|<a href="https://example.com">link</a>.</p>
  inspect(
    @html2md.convert(html),
    content=(
      #|# Hello
      #|
      #|Some **bold** and *italic* text with a [link](https://example.com).
    ),
  )
}

#Lists, code and blockquotes

///|
test "blocks" {
  let html =
    #|<ul><li>first</li><li>second</li></ul>
    #|<pre><code class="language-go">x := 1</code></pre>
    #|<blockquote>a quote</blockquote>
  inspect(
    @html2md.convert(html),
    content=(
      #|- first
      #|- second
      #|
      #|```go
      #|x := 1
      #|```
      #|
      #|> a quote
    ),
  )
}

#GFM tables and strikethrough

Pipes inside table cells are escaped, and s/del/strike tags render as GFM strikethrough:

///|
test "gfm" {
  let html =
    #|<p><del>old</del> value</p>
    #|<table>
    #|  <tr><th>Name</th><th>Value</th></tr>
    #|  <tr><td>A</td><td>1 | 2</td></tr>
    #|</table>
  inspect(
    @html2md.convert(html),
    content=(
      #|~~old~~ value
      #|
      #|| Name | Value |
      #|| --- | --- |
      #|| A | 1 \| 2 |
    ),
  )
}

#Smart escaping

Markdown-special characters in text are escaped only when they would otherwise be parsed as markdown, while real formatting from tags is preserved:

///|
test "escaping" {
  inspect(
    @html2md.convert("<p>fake **bold** and real <strong>bold</strong></p>"),
    content=(
      #|fake \*\*bold\** and real **bold**
    ),
  )
}

Escaping can be turned off entirely:

///|
test "escaping disabled" {
  inspect(
    @html2md.convert("<p>1. not a list</p>", escape_mode=Disabled),
    content="1. not a list",
  )
}

#Relative links

Provide a domain to turn relative URLs into absolute ones. Query components are percent-encoded with RFC 3986 URL semantics: spaces become %20, Unicode is UTF-8 percent-encoded, valid %HH octets are preserved, and + stays +.

///|
test "domain" {
  inspect(
    @html2md.convert("<a href=\"/page\">link</a>", domain="https://example.com"),
    content="[link](https://example.com/page)",
  )
  inspect(
    @html2md.convert(
      "<a href=\"?q=hello world&next=/a?b=1&plus=a+b\">search</a>",
      domain="https://example.com/path?old=1",
    ),
    content="[search](https://example.com/path?q=hello%20world&next=/a?b=1&plus=a+b)",
  )
}

#Options

convert (and convert_dom, which takes an already-parsed @html_parser/dom.Node) accept the following labeled options:

Option	Type	Default	Notes
domain	String	""	Base domain for relative URLs
heading_style	HeadingStyle	Atx	Atx (# H) or Setext (underlined h1/h2)
em_delimiter	String	"*"	"*" or "_"
strong_delimiter	String	"**"	"**" or "__"
horizontal_rule	String	"* * *"	any thematic break (≥3 of */_/-)
bullet_list_marker	String	"-"	"-", "+" or "*"
code_block_fence	String	"```"	"```" or "~~~"
escape_mode	EscapeMode	Smart	Smart or Disabled
link_empty_href_behavior	LinkBehavior	Render	Render keeps [text](), Skip drops the link
link_empty_content_behavior	LinkBehavior	Render	Render keeps [](href), Skip drops the link
list_end_comment	Bool	true	insert <!--THE END--> between adjacent lists

Invalid option values raise ConvertError::InvalidConfig.

#Development

Useful validation commands before committing:

moon fmt
moon check --target all
moon test --target all
moon info

#License

Apache-2.0

#
ConvertError

source

pub(all) suberror ConvertError {
  InvalidConfig(String)
  ParseFailed(String)
} derive(@moonbitlang/core/debug.Debug
Debug)

Error raised by convert / convert_dom.

#
EscapeMode

source

pub(all) enum EscapeMode {
  Smart
  Disabled
} derive(Eq, @moonbitlang/core/debug.Debug
Debug)

Controls the escaping of characters that have a special meaning in markdown.

Smart (default): only escape a character when it would otherwise be parsed as markdown syntax in its surrounding context.
Disabled: never escape.

#
HeadingStyle

source

pub(all) enum HeadingStyle {
  Atx
  Setext
} derive(Eq, @moonbitlang/core/debug.Debug
Debug)

Heading rendering style.

Atx: ## Heading (default)
Setext: underline style for h1/h2, e.g. Heading\n-------

#
LinkBehavior

source

pub(all) enum LinkBehavior {
  Render
  Skip
} derive(Eq, @moonbitlang/core/debug.Debug
Debug)

Controls how links with an empty href / empty content are rendered.

Render: still render as a link, e.g. [content]() (default)
Skip: fall back to rendering the content without link syntax

#
convert

source

fn convert(html : String, domain? : String, heading_style? : HeadingStyle, em_delimiter? : String, strong_delimiter? : String, horizontal_rule? : String, bullet_list_marker? : String, code_block_fence? : String, escape_mode? : EscapeMode, link_empty_href_behavior? : LinkBehavior, link_empty_content_behavior? : LinkBehavior, list_end_comment? : Bool) -> String raise ConvertError

Convert an HTML string to CommonMark Markdown.

Mirrors htmltomarkdown.ConvertString of the Go library JohannesKaufmann/html-to-markdown v2 (base + commonmark plugins).

Example

test {
  inspect(
    @html2md.convert("<h1>Title</h1><p>Some <b>bold</b> text.</p>"),
    content=(
      #|# Title
      #|
      #|Some **bold** text.
    ),
  )
}

#
convert_dom

source

fn convert_dom(doc : @bobzhang/html_parser/dom.Node
Node, domain? : String, heading_style? : HeadingStyle, em_delimiter? : String, strong_delimiter? : String, horizontal_rule? : String, bullet_list_marker? : String, code_block_fence? : String, escape_mode? : EscapeMode, link_empty_href_behavior? : LinkBehavior, link_empty_content_behavior? : LinkBehavior, list_end_comment? : Bool) -> String raise ConvertError

Convert an already parsed DOM tree (from bobzhang/html_parser) to CommonMark Markdown.

Note: the tree is normalized in place during conversion (nodes may be removed, merged or renamed). Pass a clone_node(deep=true) copy if the tree must stay untouched.