I’ve gotten several requests asking for the ability to add syntax highlighting for custom languages to RSyntaxTextArea. Because of the way it is designed, RSTA doesn’t lend itself well to this type of thing. Creating a highlighter that just read through a properties file (or XML) and highlighted stuff via regular expressions would probably not perform well enough for most people. But the learning curve for how to hack at JFlex-generated output so that RSTA is happy with it is also rather high.
So I’m working on a new tool, called, for lack of a better name, “TokenMakerMaker” (name suggestions are welcome!). RSTA has uses a TokenMaker class to know how to highlight for a given language, such as Perl or Java. TokenMakerMaker is a GUI program that allows you to enter the specifications for your language – keywords, comment delimiters, data types, etc. It can take that input and generate the JFlex input file for your TokenMaker, as well as the actual Java source for your TokenMaker (it’s hooked into JFlex). Further, if you’re running TokenMakerMaker with a JDK, as opposed to a JRE, it can compile the Java source generated and allow you to test it out in an RSyntaxTextArea instance, at design time!
This tool is nice because the generated scanners will be extremely fast (optimized by JFlex). In fact, they should be every bit as fast as the scanners that come built into RSTA. The key will be making the UI give as many options as possible so that what a generated scanner can do isn’t too rigid.
To test it out, I’m using it to generate a (simplified) TokenMaker for Java. How is it looking so far? Well…
Here’s a screenshot of the panel allowing you to define the comment delimiters:
Defining the comment delimiters for Java in TMM
And the panel allowing you to define the data types:
Defining the data types for Java
And here’s the testing window you get after clicking “Generate”:
Testing our generated Java TokenMaker scanner
It’s still early on but I hope to have a few people test it out soon. Here’s what’s left to be done:
- Flesh out the UI
- Add panel for defining standard library functions
- Add more configurability for Strings (escapes allowed or not, etc.)
- Allow saving TokenMaker definition to XML. Create batch tool to generate .flex/.java directly from XML to facilitate automation of the process
If you’d like to try this tool out early and provide feedback, let me know!