Archive for August, 2009

“TokenMakerMaker” – Easy creation of custom syntax highlighters for RSTA

Saturday, August 22nd, 2009

I’ve gotten several requests asking for the ability to add syntax highlighting for custom languages to RSyntaxTextArea.  Because of the way it is designed, RSTA doesn’t lend itself well to this type of thing.  Creating a highlighter that just read through a properties file (or XML) and highlighted stuff via regular expressions would probably not perform well enough for most people.  But the learning curve for how to hack at JFlex-generated output so that RSTA is happy with it is also rather high.

So I’m working on a new tool, called, for lack of a better name, “TokenMakerMaker” (name suggestions are welcome!).  RSTA has uses a TokenMaker class to know how to highlight for a given language, such as Perl or Java.  TokenMakerMaker is a GUI program that allows you to enter the specifications for your language – keywords, comment delimiters, data types, etc.  It can take that input and generate the JFlex input file for your TokenMaker, as well as the actual Java source for your TokenMaker (it’s hooked into JFlex).  Further, if you’re running TokenMakerMaker with a JDK, as opposed to a JRE, it can compile the Java source generated and allow you to test it out in an RSyntaxTextArea instance, at design time!

This tool is nice because the generated scanners will be extremely fast (optimized by JFlex).  In fact, they should be every bit as fast as the scanners that come built into RSTA.  The key will be making the UI give as many options as possible so that what a generated scanner can do isn’t too rigid.

To test it out, I’m using it to generate a (simplified) TokenMaker for Java.  How is it looking so far?  Well…

Here’s a screenshot of the panel allowing you to define the comment delimiters:

Defining the comment delimiters for Java in TMM

Defining the comment delimiters for Java in TMM

And the panel allowing you to define the data types:

Defining the data types for Java

Defining the data types for Java

And here’s the testing window you get after clicking “Generate”:

Testing our generated Java TokenMaker scanner

Testing our generated Java TokenMaker scanner

It’s still early on but I hope to have a few people test it out soon.  Here’s what’s left to be done:

  1. Flesh out the UI
  2. Add panel for defining standard library functions
  3. Add more configurability for Strings (escapes allowed or not, etc.)
  4. Allow saving TokenMaker definition to XML.  Create batch tool to generate .flex/.java directly from XML to facilitate automation of the process

If you’d like to try this tool out early and provide feedback, let me know!

Error Strip Component

Tuesday, August 11th, 2009

An “error strip” is the vertical component on the right-hand side of an editor.  It contains colorful little rectangles that indicate the relative location of parser errors and other places of interest in the editor’s content.  Clicking on one of those colorful rectangles takes you to that location in the document.

When I first saw this feature in IDE’s, I didn’t like it.  It seemed like a waste of space.  Eventually however, I grew to like it.  It was nice, for example, to quickly get an idea of how much a variable was used by clicking on it and seeing the “marked occurrences” show up in the error strip.

So I’ve decided to add it as yet-another-new-feature to RSTA.  It nicely compliments the Parser work currently being done, so it’s fun to do them in parallel.

The component itself is very simple and was easy enough to create.  The real work comes in efficiently synchronizing it with changes in the editor’s state.  I’ve got something working but the code is very messy and I’ll have to work on it to get it presentable.

Anyway, here’s a screenshot.  I’m using the spelling Parser again.  The error strip shows little rectangles for each misspelled word in the editor.  Clicking on one of them takes you to that misspelled word:

Error Strip, take 1

Error Strip, take 1

As always, more to come soon!

FocusableTips and SpellChecker

Saturday, August 1st, 2009

The latest push to SVN added FocusableTips and the SpellChecker add-on makes full use of them.  I’ve prettied up the SpellChecker tips.  They now make use of a few icons from the excellent Silk icon set, and here’s how they currently look:

Latest SpellChecker changes

Latest SpellChecker changes

Note that the “Add to dictionary” functionality doesn’t work yet, but clicking on a suggested fix does correct the word in the editor.

FocusableTips are now the default tooltips used by RSyntaxTextArea.  I still need to work on the placement code to better mimic placement of “real” tooltips.  You can toggle whether “real” JToolTips are used or FocusableTips by these new methods in the RSyntaxTextArea class:

public void setUseFocusableTips(boolean use);
public boolean getUseFocusableTips();

Still more to come!