Package jsyntaxpane.lexers
Class SimpleRegexLexer
java.lang.Object
jsyntaxpane.lexers.SimpleRegexLexer
- All Implemented Interfaces:
Lexer
This is a "dynamic" Lexer that will use Regex patterns to parse any document,
It is NOT as fast as other JFLex generated lexers.
The current implementation is about 20x slower than a JFLex lexer
(5000 lines in 100ms, vs 5ms for JFlex lexer)
This is still usable for a few 100 lines. 500 lines parse in about 10ms.
It also depends on how complex the Regexp and how many of them will actually
provide a match.
Since KEYWORD TokenType is by order less than IDENTIFIER, the higher
precedence of KEYWORD token will be used, even if the same regex matches
an IDENTIFIER. This is a neat side-effect of the ordering of the TokenTypes.
We now just need to add any non-overlapping matches. And since longer matches
are found first, we will properly match the longer identifiers which start with
a keyword.
This behaviour can easily be modified by overriding the
compareTo
method-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionvoid
This is the only method a Lexer needs to implement.putPattern
(TokenType type, String regex) putPatterns
(Map props)
-
Constructor Details
-
SimpleRegexLexer
-
SimpleRegexLexer
- Throws:
IOException
-
-
Method Details
-
parse
Description copied from interface:Lexer
This is the only method a Lexer needs to implement. It will be passed a Reader, and it should return non-overlapping Tokens for each recognized token in the stream.- Specified by:
parse
in interfaceLexer
- Parameters:
segment
- Text to parse.ofst
- offset to add to start of each token (useful for nesting)tokens
- List of Tokens to be added. This is done so that the caller creates the appropriate List implementation and size. The parse method just adds to the list
-
putPattern
-
putPatterns
-