NLT is complete

After an ugly start — a bug which was introduced with bugfix for another bug — I managed to go straight to the last missing piece in NLT, table driven lexer. It does not mean NLT is finished, but it is complete — all structural features are there and the table for lexer was the last one.

User can specify to compile all regex and string patterns while building a lexer into one big table. When the lexer runs there are no longer repetitions, classes, ranges, just a table with transitions. I was afraid of handling Unicode ranges, but after reading “Regular Expression Matching in the Wild” by Russ Cox I knew what to do, especially I was not building general purpose regex engine, but simplified multi regex one (no back references for example).

Without any optimization at all it is twice as fast as previous approach — checking patterns one by one during runtime. And I didn’t notice any difference in time when building the lexer, which is rather good news.

From now on, NLT is more for maintenance — fixing bugs, optimization, better error reporting, and so on.

Tagged

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: