Lexer

The lexer is responsible for splitting your code into meaningful tokens. This framework offers an extensible lexer that can be easily configured. For simplicity the lexer emits two special tokens to determine the start and the end of the input. The SOF token represents the start of input and EOF the end of the input.

You can initialize the lexer by overriding the InitLexer method:

protected override void InitLexer(LexerConfig lexer)
{
       
}