conllu

    CoNLL-U / Universal Dependencies toolchain: lossless parse and serialize, spec validation, tree queries, UAS/LAS evaluation, corpus statistics, document grouping, coverage analysis.

    conllu
    universal-dependencies
    nlp
    dependency-parsing
    treebank
    Download zip
    Author
    Version
    0.2.0
    License
    Apache-2.0
    Last updated
    15 hours ago
    Downloads
    25

    #PaperY0/conllu

    CI mooncakes license

    MoonBit 里的 CoNLL-U / Universal Dependencies 工具链:无损解析与序列化、规范校验、依存树查询、文档/段落分组、语料覆盖分析、UAS/LAS 评测、语料统计,外加一个可直接跑的 CLI(含 gzip 读与写)。

    CoNLL-U 是 UD 树库(Universal Dependencies 项目,200+ 语言树库)的唯一交换格式:每句 10 列制表符分隔,句子之间空行分隔,# 开头是注释,ID 可以是 12(普通词)、5-6(多词 token)、5.1(空节点)。生态里做分词、词性的包已经不少,但句法层(HEAD/DEPREL 树、投射性、UAS/LAS)此前是空白,这个包补的就是这一层。

    #特性

    • 无损:原始 10 列全部保留在 Token 里,to_string 能逐字节还原(含多词 token、空节点、注释),不认识的列不会被"顺手规范化"。
    • 严格报错:parse 对列数、ID 形态、HEAD 数字性做硬校验,出错抛 ConlluError(line, message),行号直接给到源文件。
    • 规范校验:17 个 UPOS、37 个 UD v2 依存关系(含增强 ref)、ID 连续性、单根、自环、成环、DEPS 语法与范围,都按 CoNLL-U 规范检查。
    • 评测:evaluate(gold, pred) 给出 UAS / LAS(百分制),按 1-based 词序对齐,长度不等的句子计入 skipped_sentences 而不是静默丢掉。
    • 树查询:children / descendants / depth / is_ancestor / is_projective / roots / path / is_single_rooted。
    • 文档分组:按 # newdoc / # newpar 还原 Document → Paragraph → Sentence 三层结构,没有边界声明的文件仍算一个匿名文档。
    • 覆盖分析:coverage 一次遍历统计 UPOS 与 DEPREL 命中率、规范外标签、投射性与树深、长句与空节点,CLI 里可直接当语料门槛用。
    • 统计:句数、词数、多词 token、空节点、注释、最长句、平均句长、涉及的树库。
    • 对拍过:与 Python conllu 参考解析器在 UD_English-EWT、UD_Chinese-GSD、UD_Japanese-GSD 的 dev/test 六个切分(6,128 句、107,091 行)上逐字段零差异,见 DIFF-REPORT.md。

    #安装

    moon add PaperY0/conllu

    #解析与序列化

    ///|
    test {
    let text = "1\tI\tI\tPRON\tPRP\t_\t2\tnsubj\t_\t_\n2\tsaw\tsee\tVERB\tVBD\t_\t0\troot\t_\t_\n3\tthe\tthe\tDET\tDT\t_\t4\tdet\t_\t_\n4\tcat\tcat\tNOUN\tNN\t_\t2\tobj\tSpaceAfter=No\t_\n5\t.\t.\tPUNCT\t.\t_\t2\tpunct\t_\t_\n"
    let sentences = @conllu.parse(text) catch { _ => [] }
    assert_eq(sentences.length(), 1)
    let sentence = sentences[0]
    assert_eq(sentence.word_count(), 5)
    assert_eq(sentence.words()[1].lemma, "see")
    // HEAD 是解析过的整数,可直接用于树操作
    assert_eq(sentence.words()[1].head(), Some(0))
    // 序列化补回规范要求的句末空行
    assert_eq(@conllu.to_string(sentences), text + "\n")
    }

    Sentence 同时保留注释和全部行:sentence.rows() 拿到 Word / Multiword / Empty 三类行,sentence.comment("text") 直接取 # text = ... 的值。

    #校验

    ///|
    test {
    let broken = "1\ta\ta\tNOUN\tNN\t_\t0\troot\t_\t_\n2\tb\tb\tNOUN\tNN\t_\t0\troot\t_\t_\n"
    let sentences = @conllu.parse(broken) catch { _ => [] }
    let diagnostics = @conllu.validate(sentences)
    assert_eq(@conllu.count_errors(diagnostics), 1)
    assert_eq(diagnostics[0].code, "ROOT_COUNT")
    }

    校验规则按 code 分类,方便 CI 里按类型放行或拦截:

    code含义
    ID_SEQUENCE词 ID 不是从 1 开始连续递增
    MULTIWORD_SPAN多词 token 的 5-6 越界
    HEAD_MISSING / HEAD_RANGE / HEAD_SELFHEAD 非数字、越界、指向自己
    ROOT_COUNT根节点不是恰好 1 个
    CYCLE沿 HEAD 走不到根
    DEPS_FORMAT / DEPS_RANGE / DEPS_DEPREL / DEPS_MISSINGDEPS 语法、范围、关系名、空节点缺关系
    UPOS_UNKNOWN / DEPREL_UNKNOWN不在 UD v2 标签集内
    FORM_EMPTY / FORM_SPACEFORM 为空或含空白

    #文档与段落

    树库不是扁平的句子列表:# newdoc 与 # newpar 行标出每句属于哪个文档、哪一段。documents() 把这层关系还原成三层结构,id 直接取自注释;没有边界声明的文件返回一个 id 为空的匿名文档,而不是空数组。

    ///|
    test {
    let text = "# newdoc id = d1\n# newpar id = p1\n# sent_id = 1\n1\ta\ta\tNOUN\tNN\t_\t0\troot\t_\t_\n\n# newpar id = p2\n# sent_id = 2\n1\tb\tb\tNOUN\tNN\t_\t0\troot\t_\t_\n"
    let docs = @conllu.documents(@conllu.parse(text) catch { _ => [] })
    assert_eq(docs.length(), 1)
    assert_eq(docs[0].id(), "d1")
    assert_eq(docs[0].paragraph_count(), 2)
    assert_eq(docs[0].paragraphs()[0].id(), "p1")
    assert_eq(docs[0].paragraphs()[1].id(), "p2")
    assert_eq(docs[0].sentence_count(), 2)
    }

    # newdoc 一行同时开启新段落,因此文档不会继承上一段的尾巴;分组信息本身就是普通注释,Document::to_string 写回时这些行原样保留。

    #语料覆盖分析

    句数词数只能说明体量,说明不了质量。coverage 一次遍历回答四个问题:规范里的 17 个 UPOS、37 个 DEPREL 各用到了几个;有没有规范外的标签;树有多深、多少句不投射;长句和空节点有多少。

    ///|
    test {
    let text = "1\ta\ta\tNOUN\tNN\t_\t0\troot\t_\t_\n2\tb\tb\tUNKNOWN\tNN\t_\t1\tweird\t_\t_\n"
    let cov = @conllu.coverage(@conllu.parse(text) catch { _ => [] })
    assert_eq(cov.upos_used.length(), 1)
    assert_eq(cov.unknown_upos[0], "UNKNOWN")
    assert_eq(cov.unknown_deprel[0], "weird")
    assert_false(cov.full_upos())
    assert_eq(cov.projectivity(), 100.0)
    }

    #评测 UAS / LAS

    ///|
    test {
    let gold = "1\ta\ta\tNOUN\tNN\t_\t2\tnsubj\t_\t_\n2\tb\tb\tVERB\tVB\t_\t0\troot\t_\t_\n"
    let pred = "1\ta\ta\tNOUN\tNN\t_\t0\tnsubj\t_\t_\n2\tb\tb\tVERB\tVB\t_\t0\troot\t_\t_\n"
    let g = @conllu.parse(gold) catch { _ => [] }
    let p = @conllu.parse(pred) catch { _ => [] }
    let result = @conllu.evaluate(g, p)
    assert_eq(result.uas(), 50.0)
    assert_eq(result.las(), 50.0)
    }

    #命令行

    包自带 cmd/main,子命令都直接吃文件,纯文本和 .conllu.gz 都支持,输出去 .gz 时自动压缩:

    moon run cmd/main -- stats treebank.conllu # 句数/词数/多词/空节点/最长句/树库 moon run cmd/main -- validate treebank.conllu # 诊断列表,有 error 时退出码 1 moon run cmd/main -- eval gold.conllu pred.conllu moon run cmd/main -- labels gold.conllu pred.conllu # 按关系拆开的 LAS moon run cmd/main -- labels gold.conllu pred.conllu --format csv moon run cmd/main -- docs treebank.conllu # 文档/段落结构与句数 moon run cmd/main -- analyze treebank.conllu # 覆盖报告,规范外标签时退出码 1 moon run cmd/main -- dump treebank.conllu # 归一化 dump,用于对拍 moon run cmd/main -- dump treebank.conllu --format json moon run cmd/main -- roundtrip treebank.conllu # 解析再序列化,校验无损 moon run cmd/main -- compress treebank.conllu out.conllu.gz # 规范序列化 + gzip 写回

    compress 会把自己写出的文件再读回来比对,写坏了不会静默产出一个"看着像"的文件。UD_English-EWT dev 实测 1,899,904 → 357,823 字节(18.83%)。

    labels --format csv 输出 label,gold,correct,las 四列,按 RFC 4180 处理引号,可直接喂给表格或 pandas;默认仍是终端对齐表。

    dump 的格式是稳定的行式契约(D/S/w/m/e 前缀),差分脚本就靠它跟别的实现比对:

    D 2001 S 7 0 0 4 w 1 From ADP 3 case

    需要结构化输出时用 --format json:一份文件一个 JSON 对象,句子带行号、注释、词/多词/空节点计数,token 保留原始 10 列值,方便别的语言写包装器。

    #差分对拍

    tools/diff_test.py 把本包的 dump 与 Python conllu 的输出逐行逐字段比对,另外检查 validate 零 error、roundtrip 无损、自评 UAS=LAS=100%。

    python3 tools/fetch_corpus.py # 下载 UD 官方语料到 data/ud/(语料不随包分发) python3 tools/diff_test.py --report

    当前结果(详见 DIFF-REPORT.md):

    文件句数dump 行数字段不一致
    en_ewt-ud-dev2001275130
    en_ewt-ud-test2077275280
    zh_gsd-ud-dev500131660
    zh_gsd-ud-test500125110
    ja_gsd-ud-dev507127950
    ja_gsd-ud-test543135780

    对拍过程里发现并修掉的两个真实 bug:8.1:obj 这类指向空节点的 DEPS 被当成数字 81(越界误报),以及增强 DEPS 里合法的 ref 关系被判成未知标签。

    #API 参考

    完整文档注释可用 moon doc --serve 在本地预览(默认 http://127.0.0.1:3000),发布后也会出现在 mooncakes 的文档页。

    解析与序列化

    函数说明
    parse(text) -> Array[Sentence] raise ConlluError按空行分句,校验 10 列与 ID 形态,出错带行号
    to_string(sentences) -> String序列化整份文档,每句后补规范空行
    Sentence::to_string()单句序列化
    Sentence::words() / rows() / comments()词行 / 全部行 / 注释
    Sentence::word_count()词行数量(不含多词与空节点)
    Sentence::word(i) -> Token?按 1-based 词序取词
    Sentence::comment(key)取 # key = value 的 value
    Sentence::comment_value(key)同上的通用版本,Some(value) / None
    Token::head() -> Int32?HEAD 解析为整数,非数字返回 None
    Token::is_word() / is_multiword() / is_empty_node()行类型判定
    Token::to_line()还原一行 10 列

    校验

    函数说明
    validate(sentences) -> Array[Diagnostic]全文档校验
    validate_sentence(sentence)单句校验
    count_errors(diagnostics) -> Int32只数 Error 级别
    Diagnostic::to_string()line 12: error [ROOT_COUNT] ...

    依存树

    函数说明
    children(s, head) / descendants(s, node)直接孩子 / 全部后代
    path(s, from, to) -> Array[Int32]?从 from 沿 HEAD 到祖先 to(to = 0 走到根)
    is_ancestor(s, a, b)a 是否为 b 的祖先
    depth(s, i) / max_depth(s)深度 / 最大深度
    is_projective(s)是否无交叉弧
    roots(s) / is_single_rooted(s)根节点 / 是否单根

    文档与段落

    函数说明
    documents(sentences) -> Array[Document]按 # newdoc / # newpar 分组
    Document::id() / paragraph_count() / sentence_count()文档 id 与两级计数
    Document::paragraphs() / sentences()段落列表 / 展开成句子列表
    Paragraph::id() / sentences() / sentence_count()段落 id 与句数
    Sentence::has_newdoc() / has_newpar()是否开启新文档 / 新段落
    Sentence::newdoc_id() / newpar_id()对应注释的 id 值

    覆盖分析

    函数说明
    coverage(sentences) -> Coverage标签命中、规范外标签、树形与句长统计
    Coverage::upos_used() / deprel_used()实际出现的标签(字母序)
    Coverage::upos_missing() / deprel_missing()规范里没出现的标签
    Coverage::projectivity()投射句占比(百分制)
    Coverage::full_upos() / full_deprel()是否覆盖全部规范标签
    Coverage::to_string()多行报告,analyze 直接打印

    评测与统计

    函数说明
    evaluate(gold, pred) -> EvalResult对齐后计算 UAS / UUAS / LAS
    EvalResult::uas() / uuas() / las()百分制分数;uuas 不计方向
    evaluate_by_label(gold, pred)按 gold 关系拆开的 LAS,最差的排最前
    LabelScore::las() / to_string()单关系准确率
    stats(sentences) -> Stats句数、词数、多词、空节点、注释、最长句、树库
    Stats::average_length() / to_string()平均句长 / 一行摘要

    #许可与语料

    代码 Apache-2.0。UD 树库是 CC BY-SA 4.0,只作为测试夹具下载,不进仓库、不进发布包;引用请遵循 Universal Dependencies 项目的署名要求。

    ConlluError

    pub(all) suberror ConlluError {
    ConlluError(Int, String)
    }

    Raised when the input is not well-formed at the lexical level (wrong column count, unparsable ID, ...). Structural problems are reported by validate so that a broken tree can still be inspected.

    ConlluError::line

    fn ConlluError::line(self : ConlluError) -> Int

    The 1-based source line a ConlluError points at.

    ConlluError::message

    fn ConlluError::message(self : ConlluError) -> String

    Coverage

    pub struct Coverage {
    sentences : Int
    words : Int
    upos_used : Array[String]
    upos_missing : Array[String]
    deprel_used : Array[String]
    deprel_missing : Array[String]
    unknown_upos : Array[String]
    unknown_deprel : Array[String]
    untagged : Int
    projective : Int
    non_projective : Int
    single_rooted : Int
    multi_rooted : Int
    max_depth : Int
    avg_depth : Double
    long_sentences : Int
    empty_nodes : Int
    multiword_tokens : Int
    documents : Int
    paragraphs : Int
    }

    What a corpus actually covers: which universal tags and relations appear, which tags it uses that the UD spec does not define, how well-formed its trees are, and the shape of its sentences. Answers the questions a corpus maintainer asks before trusting a treebank.

    Coverage::full_deprel

    fn Coverage::full_deprel(self : Coverage) -> Bool

    Same for the 37 universal relations.

    Coverage::full_upos

    fn Coverage::full_upos(self : Coverage) -> Bool

    True when every universal tag and relation of the spec appears at least once. A false here is not a defect, but it does tell you the corpus will not exercise those labels in a parser evaluation.

    Coverage::projectivity

    fn Coverage::projectivity(self : Coverage) -> Double

    Share of sentences whose tree is projective, in percent.

    Coverage::to_string

    fn Coverage::to_string(self : Coverage) -> String

    Human-readable report, one section per dimension. Printed by conllu analyze.

    Diagnostic

    pub struct Diagnostic {
    line : Int
    severity : Severity
    code : String
    message : String
    }

    One validation finding, tied to a physical line of the source file.

    Diagnostic::to_string

    fn Diagnostic::to_string(self : Diagnostic) -> String

    Document

    pub struct Document {
    id : String
    paragraphs : Array[Paragraph]
    }

    A document (# newdoc) split into the paragraphs it contains. Sentences that appear before the first # newdoc line form one document with an empty id, which is what a file without document boundaries looks like.

    Document::id

    fn Document::id(self : Document) -> String

    Document::paragraph_count

    fn Document::paragraph_count(self : Document) -> Int

    Document::paragraphs

    fn Document::paragraphs(self : Document) -> Array[Paragraph]

    Document::sentence_count

    fn Document::sentence_count(self : Document) -> Int

    Document::sentences

    fn Document::sentences(self : Document) -> Array[Sentence]

    All sentences of the document, in file order.

    Document::to_string

    fn Document::to_string(self : Document) -> String

    Serialises the document with the canonical writer, one blank line between sentences. The # newdoc / # newpar lines travel as ordinary comments, so the grouping survives a write.

    EvalResult

    pub struct EvalResult {
    sentence_pairs : Int
    skipped_sentences : Int
    gold_tokens : Int
    aligned_tokens : Int
    uas_correct : Int
    uuas_correct : Int
    las_correct : Int
    }

    Aggregated attachment scores of a predicted parse against a gold parse.

    EvalResult::las

    fn EvalResult::las(self : EvalResult) -> Double

    Labelled attachment score in percent.

    EvalResult::to_string

    fn EvalResult::to_string(self : EvalResult) -> String

    One-line summary of the evaluation, as printed by the CLI.

    EvalResult::uas

    fn EvalResult::uas(self : EvalResult) -> Double

    Unlabelled attachment score in percent.

    EvalResult::uuas

    fn EvalResult::uuas(self : EvalResult) -> Double

    Undirected attachment score in percent: a token counts as correct when it points at the same head as the gold parse, ignoring the direction of the dependency. Useful when a parser gets the attachment right but the direction or the label wrong.

    LabelScore

    pub struct LabelScore {
    label : String
    gold : Int
    correct : Int
    }

    Labelled accuracy of one dependency relation.

    LabelScore::las

    fn LabelScore::las(self : LabelScore) -> Double

    Labelled accuracy of this relation in percent.

    LabelScore::to_string

    fn LabelScore::to_string(self : LabelScore) -> String

    One-line summary: label=nsubj gold=210 correct=198 las=94.29%.

    Paragraph

    pub struct Paragraph {
    id : String
    sentences : Array[Sentence]
    }

    A run of sentences inside one paragraph. id is the # newpar id value, or the empty string when the corpus does not name paragraphs.

    Paragraph::id

    fn Paragraph::id(self : Paragraph) -> String

    Paragraph::sentence_count

    fn Paragraph::sentence_count(self : Paragraph) -> Int

    Paragraph::sentences

    fn Paragraph::sentences(self : Paragraph) -> Array[Sentence]

    RowKind

    pub(all) enum RowKind {
    Word
    Multiword
    Empty
    }

    CoNLL-U row kind. A sentence mixes word rows, multiword-token rows (1-2) and empty-node rows (5.1).

    Sentence

    pub struct Sentence {
    comments : Array[String]
    rows : Array[Token]
    line : Int
    }

    One CoNLL-U sentence: the comment block that precedes it and its rows in file order.

    Sentence::comment

    fn Sentence::comment(self : Sentence, key : StringView) -> String?

    Sentence::comment_value

    fn Sentence::comment_value(self : Sentence, key : StringView) -> String?

    Value of a sentence comment written as # <key> = <value>, the form used by # newdoc id = ... and # newpar id = .... Returns None when the sentence carries no such comment.

    Sentence::comments

    fn Sentence::comments(self : Sentence) -> Array[String]

    Sentence::has_newdoc

    fn Sentence::has_newdoc(self : Sentence) -> Bool

    True when the sentence opens a new document (# newdoc).

    Sentence::has_newpar

    fn Sentence::has_newpar(self : Sentence) -> Bool

    True when the sentence opens a new paragraph (# newpar).

    Sentence::newdoc_id

    fn Sentence::newdoc_id(self : Sentence) -> String?

    The document id of a # newdoc id = ... line, if any.

    Sentence::newpar_id

    fn Sentence::newpar_id(self : Sentence) -> String?

    The paragraph id of a # newpar id = ... line, if any.

    Sentence::rows

    fn Sentence::rows(self : Sentence) -> Array[Token]

    Sentence::to_string

    fn Sentence::to_string(self : Sentence) -> String

    Serialises one sentence back to CoNLL-U, without the trailing blank line.

    Sentence::word

    fn Sentence::word(self : Sentence, index : Int) -> Token?

    1-based lookup of a word row; None when out of range or not a word row.

    Sentence::word_count

    fn Sentence::word_count(self : Sentence) -> Int

    Sentence::words

    fn Sentence::words(self : Sentence) -> Array[Token]

    The word rows of a sentence, in ID order.

    Severity

    pub(all) enum Severity {
    Error
    Warning
    }

    Severity of a validation diagnostic.

    Stats

    pub struct Stats {
    sentences : Int
    words : Int
    multiwords : Int
    empty_nodes : Int
    comments : Int
    max_length : Int
    treebanks : Array[String]
    }

    Corpus-level statistics of a CoNLL-U document.

    Stats::average_length

    fn Stats::average_length(self : Stats) -> Double

    Stats::to_string

    fn Stats::to_string(self : Stats) -> String

    Token

    pub struct Token {
    id_raw : String
    form : String
    lemma : String
    upos : String
    xpos : String
    feats : String
    head_raw : String
    deprel : String
    deps : String
    misc : String
    kind : RowKind
    start : Int
    end : Int
    line : Int
    }

    One physical row of a CoNLL-U sentence: the ten tab-separated columns plus positional information derived by the parser.

    Token::head

    fn Token::head(self : Token) -> Int?

    The HEAD column as an integer, or None when the column is _ (legal only for empty nodes).

    Token::is_empty_node

    fn Token::is_empty_node(self : Token) -> Bool

    Token::is_multiword

    fn Token::is_multiword(self : Token) -> Bool

    Token::is_word

    fn Token::is_word(self : Token) -> Bool

    Token::to_line

    fn Token::to_line(self : Token) -> String

    Formats a single row back to one line.

    children

    fn children(sentence : Sentence, head : Int) -> Array[Int]

    Direct children of a word (by 1-based index); the root's children are returned for head == 0.

    count_errors

    fn count_errors(diagnostics : Array[Diagnostic]) -> Int

    coverage

    fn coverage(sentences : Array[Sentence]) -> Coverage

    Measures one corpus.

    depth

    fn depth(sentence : Sentence, index : Int) -> Int

    Depth of a word in its tree: root children have depth 1.

    descendants

    fn descendants(sentence : Sentence, ancestor : Int) -> Array[Int]

    All words whose HEAD chain passes through ancestor.

    documents

    fn documents(sentences : Array[Sentence]) -> Array[Document]

    Groups sentences by # newdoc and then by # newpar, in file order. A # newdoc line always starts a new paragraph as well, so a document never inherits the tail of the previous one.

    evaluate

    fn evaluate(gold : Array[Sentence], pred : Array[Sentence]) -> EvalResult

    Compares two parses sentence by sentence. Sentences whose word counts differ cannot be aligned token by token and are counted as skipped instead of silently scored.

    evaluate_by_label

    fn evaluate_by_label(gold : Array[Sentence], pred : Array[Sentence]) -> Array[LabelScore]

    Labelled accuracy broken down by the gold relation, worst first, so a parser can see which relations it confuses instead of a single aggregate number.

    is_ancestor

    fn is_ancestor(sentence : Sentence, ancestor : Int, node : Int) -> Bool

    True when following HEAD links from node reaches ancestor.

    is_projective

    fn is_projective(sentence : Sentence) -> Bool

    A sentence is projective when no two dependency arcs cross. An arc head -> dep is projective iff every word strictly between them is a descendant of head.

    is_single_rooted

    fn is_single_rooted(sentence : Sentence) -> Bool

    True when the sentence has exactly one word attached to the root. A well-formed CoNLL-U sentence is single-rooted by definition; this is the cheap check for tooling that only needs that one property.

    max_depth

    fn max_depth(sentence : Sentence) -> Int

    Longest root-to-leaf distance in the sentence.

    parse

    fn parse(text : String) -> Array[Sentence] raise ConlluError

    Parses a complete CoNLL-U document. Blank lines separate sentences; a leading comment block belongs to the sentence that follows it.

    path

    fn path(sentence : Sentence, from : Int, to : Int) -> Array[Int]?

    Dependency path from from up to to, both inclusive, in the order from, ..., to. Returns None when to is not an ancestor of from or an index is out of range. to = 0 walks all the way up to the artificial root.

    roots

    fn roots(sentence : Sentence) -> Array[Int]

    Root words of a sentence (HEAD == 0).

    stats

    fn stats(sentences : Array[Sentence]) -> Stats

    to_string

    fn to_string(sentences : Array[Sentence]) -> String

    Serialises a document with one blank line between sentences.

    validate

    fn validate(sentences : Array[Sentence]) -> Array[Diagnostic]

    Validates every sentence of a document.

    validate_sentence

    fn validate_sentence(sentence : Sentence) -> Array[Diagnostic]

    Checks the structure of one sentence: ID sequence, HEAD targets, single root, acyclicity, DEPS syntax, tag sets and token surface constraints. Lexical errors are already rejected by parse, so everything reported here keeps a sentence inspectable.