TokenMakerMaker Updates

It’s been awhile since I’ve written about TokenMakerMaker, but it’s a great tool and it’s been getting some love lately, so I’ll talk about some of the recent change here.

TokenMakerMaker is a tool for folks who want to add support for custom languages to RSyntaxTextArea, but who don’t want to spend time learning JFlex (the most common way to add support for a new language).  TMM provides a simple GUI interface that lets you specify the makeup of your language (keywords, comment styles, etc.) and generates a TokenMaker (the class that defines syntax highlighting for a language) for you.  The best part is that it is using JFlex behind the scnes to generate your TokenMaker, so it should be every bit as fast as if you wrote a JFlex specification “by hand.”  Please see my original post on TokenMakerMaker for more information on running it.

So what’s new?  First off, number literals can now be specified in the “Numbers” tab.  You can specify whether, and in what format, integer, float, and hex literals are specified in your language:

Numbers Options

Numbers Options

Second, the TokenMaker class generated by TMM now properly registers your line comment style with the “toggle line comment” action (Ctrl+/, Cmd+/ on Macs).

TMM has also been updated for recent changes in RSyntaxTextArea’s token types.  For example, the General tab contains a check box allowing you to specify whether your language allows the boolean literals “true” and “false”, and has a new tab allowing you to specify keywords that should be highlighted as “secondary” keywords (commonly used for control flow keywords such as “return”).

Boolean Literal Highlighting

Boolean Literal Highlighting

A new option has been added to allow the customizing of the theme installed on test RSyntaxTextAreas when testing out your TokenMaker:

RSTA Theming Support

RSTA Theming Support

Finally, and perhaps most importantly, a new “Output” tab shows you a log of what’s going on when you click “Generate” to create your TokenMaker.  This will allow you to easily spot any errors that occur (errors are highlighted in red).  No more having to watch the console for this information.

Output Panel

Output Panel

Give it a shot and see what you think!  I plan to keep evolving this tool.  Likely updates will include:

  1. Verbatim strings support
  2. A way to specify tokens to highlight as variables (languages such as Perl, with easily-identifiable variables would benefit from this)
  3. Specification of keywords & functions via regex as well as plain text

I’m also actively working on the Help system in the application, to hopefully help out first-time users.

 

Comments are closed.