🥮 mooncakes.io
    Sign in

    jonesmelton/sqlbrook/lexer does not have a README file

    LexError

    source
    pub suberror LexError {
      UnexpectedByte(Byte, offset~ : Int)
      PostgresCast(offset~ : Int)
      PostgresDollarQuote(offset~ : Int)
    }

    Input that is not lexable SQLite SQL.

    offset is a UTF-8 byte offset into the source, so it points at the same place a byte-oriented editor or tool would.
    impl Show for LexError

    LexError::output

    source
    fn LexError::output(self : LexError, logger : &Logger) -> Unit

    LexError::to_string

    source
    fn LexError::to_string(self : LexError) -> String

    Spanned

    source
    pub struct Spanned {
      token : Token
      start : Int
      end : Int
    } derive(Eq, 
    Debug    )

    A token and the source range [start, end) it was lexed from.

    Offsets are String indices (UTF-16 code units). A multi-word keyword spans from its first word to its last, whitespace between them included.

    Spanned::equal

    source
    fn Spanned::equal(Spanned, Spanned) -> Bool

    Spanned::not_equal

    source
    fn Spanned::not_equal(x : Spanned, y : Spanned) -> Bool

    Spanned::to_repr

    source

    Token

    source
    pub(all) enum Token {
      Keyword(String)
      Ident(String)
      Quoted(String)
      Number(String)
      Operator(String)
      Placeholder(String)
      LParen
      RParen
      Comma
      Semicolon
      Comment(String)
    } derive(Eq, 
    Debug    )

    A SQLite token. Whitespace is not a token.

    Keyword payloads are lowercased and may be multi-word phrases such as order by, with the words joined by one space. Every other payload is the exact source text, so to_string reproduces the token as written.
    impl Show for Token

    Token::equal

    source
    fn Token::equal(Token, Token) -> Bool

    Token::not_equal

    source
    fn Token::not_equal(x : Token, y : Token) -> Bool

    Token::output

    source
    fn Token::output(self : Token, logger : &Logger) -> Unit

    Token::to_repr

    source

    Token::to_string

    source
    fn Token::to_string(self : Token) -> String

    lex

    source
    fn lex(source : String) -> Array[Spanned] raise LexError

    Splits SQLite source into tokens with their source ranges, dropping whitespace.

    test {
      let spans = @lexer.lex("SELECT a  FROM t")
      inspect(
        spans.map(sp => "\{sp.token}@\{sp.start}-\{sp.end}").join(" "),
        content="select@0-6 a@7-8 from@10-14 t@15-16",
      )
    }

    Errors

    Raises LexError on a character that starts no token, on an unterminated quote, or on PostgreSQL-only syntax (::, $$).

    tokenize

    source
    fn tokenize(source : String) -> Array[Token] raise LexError

    Splits SQLite source into tokens, dropping whitespace. Keywords come back lowercased; everything else keeps its source spelling.

    test {
      debug_inspect(
        @lexer.tokenize("Order  By x"),
        content=(
          #|[Keyword("order by"), Ident("x")]
        ),
      )
    }

    Errors

    Raises LexError under the same conditions as lex.

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io