Package dev.omnist.oml

Class OmlLexer

java.lang.Object

dev.omnist.oml.OmlLexer

public class OmlLexer
extends Object

Normative OML Tokenizer (omnist-spec §4.2). Scans with maximal munch under a fixed 9-rule priority order: 1. STRING family (dquote or raw/multiline) 2. Punctuation: { } [ ] : , 3. DATETIME 4. DATE (with T+TIME lookahead rule) 5. TIME 6. NUMBER 7. Reserved float spellings nan, inf, -inf (emitted as NUMBER) 8. INTEGER (maxIntegerDigits limit enforced here) 9. IDENT

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final record	OmlLexer.Token	An individual token produced by OmlLexer.
static enum	OmlLexer.TokenType	Token types produced by the OML lexer (omnist-spec §4.2).

Constructor Summary

Constructors

Constructor	Description
OmlLexer(String source, Limits limits)	Constructs a lexer for the given OML source text.

Method Summary

Modifier and Type	Method	Description
OmlLexer.Token	nextToken()	Scans and returns the next token from the source, advancing the lexer position.
List<OmlLexer.Token>	tokenizeAll()	Tokenizes the entire source, returning a list ending with a single OmlLexer.TokenType.EOF token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

OmlLexer

public OmlLexer(String source,
 Limits limits)

Constructs a lexer for the given OML source text.

Parameters:

source - the OML text to tokenize; null is treated as an empty string

limits - safety limits applied during tokenization (e.g. Limits.maxIntegerDigits()); null defaults to Limits.DEFAULT

Method Details

tokenizeAll

public List<OmlLexer.Token> tokenizeAll()

Tokenizes the entire source, returning a list ending with a single OmlLexer.TokenType.EOF token. Calls nextToken() repeatedly until EOF is reached.

Returns:

an unmodifiable snapshot of all tokens, never empty (always contains at least EOF)

Throws:

OmlParseException - if any lexical error is encountered

nextToken

public OmlLexer.Token nextToken()

Scans and returns the next token from the source, advancing the lexer position. Applies the 9-rule priority order described in the class Javadoc (omnist-spec §4.2). Horizontal whitespace and #-comments are silently skipped before each token.

Returns:

the next OmlLexer.Token; returns OmlLexer.TokenType.EOF at end of input

Throws:

OmlParseException - if the character sequence does not match any rule, or if an integer literal exceeds Limits.maxIntegerDigits(), or if a string literal is malformed or unterminated