Fifesoft Blog

The Java code completion I’m working on has hit a new milestone.  Code completion for fields and local variables is starting to work:

Field Completion

Both RSyntaxTextArea and SpellChecker 1.4.0 were just released on SourceForge!  Go grab them and try them out!  A lot of new features and bug fixes have gone into the library with this release.  Spell checking alone has been an oft-requested feature, but there’s lots more than that:

• “FocusableTips” were added, which are tool tips that become focusable when the
  user clicks on them or presses F2, like the tool tips in Eclipse.  These
  are the default tool tips for RSyntaxTextArea, and can be used by parsers
  (such as the spell checker) for user-friendly assistance.
• Added an “Error Strip” component, designed to sit on the right-hand side of an RSyntaxTextArea.  Also seen in Eclipse and NetBeans, this component is capable of displaying colored markers for various stuff in the editor – parser errors, marked occurrences of the currently selected identifier, etc.
• Delphi, Scala and BBCode syntax highlighting were added.
• RSTA will now automatically insert a closing curly brace when appropriate for C, C++, C#, Java and Perl.  This behavior is toggle-able in the API if you don’t like or want it.
• RSTA will also now automatically complete closing XML tags when “</” is typed (again, you can turn this off if you don’t want it, but it’s really nice!).
• Various bug fixes and improvements.
• The spell checker library is now officially released.  It’s version is “1.4.0″ to indicate that it should only be used with RSyntaxTextArea 1.4.0, not earlier releases.  It comes with a simple demo app showing how easy it is to use it with RSTA.
• RSTA comes with a “Task Parser” as well.  It scans source code for the tokens “TODO” and “FIXME” in comments.  Applications can use this parser to provide the user with a list of “tasks” they’ve reminded themselves of in code.

Now that the release is done, I plan on updating the RSyntaxTextArea site with documentation on how to use these cool new features.

Also coming up shortly, a new RText version should be released that will take full advantage of all of these new features!

To ensure the robustness of the AutoComplete library, I decided to see what it would take to implement some basic Java code completion.  This is of course much more complex than completion for markup languages, and presents a different set of challenges, but it’s something I’ve always been interested in.

The first thing that I needed was a Java parser, one that could break a source file down into an AST suitable for code completion.  This means you don’t have to go as far as parsing actual method bodies, you just have to grab imports, class/interface/enum names, method and field names, and local variables.  Of course, this isn’t a trivial task by any means; especially for local variables, you have to parse method bodies “well enough” to be able to get the variable names and types, without getting caught up in validating the method body’s structure.

Luckily, I had already been working on such a beast off and on (yes, I’m a geek) so I had something to go on.  It can currently parse all source files in the 1.6u16 JDK (well, classes and interfaces, I’ve put off enums for now to work on other areas), so I think it’s in pretty good shape.  Its error recovery (ability to keep parsing after seeing invalid Java source) is pretty terrible, mostly due to my lack of real education in parsing/compiler design.

So with the Java parser at least functional, the next step was to integrate it into RSyntaxTextArea’s Parser API.  That way the parser runs whenever the text component’s document is modified, allowing the user to get up-to-date completion choices automatically.  This part was easy, as RSTA’s Parser API was already up to the job.

Here’s what’s included in completion suggestions so far:

• Field and method names for members of the class being edited.
• Field and method names for all super classes and implemented interfaces.  Access flags are honored – e.g. you will not see suggestions for private members of extended classes.
• Local variable names for variables accessible at the current caret position.
• Names of imported classes (e.g. “import java.io.*;” allows everything in the java.io package to be suggested).  Again, access flags are honored – package private classes, etc. are not suggested (unless the source being parsed is part of that package).
• Anything fully-qualified is suggested as well, so typing “com.sun.java.swing.<ctrl+space>” will suggest everything in that package, whether you’ve imported it or not.

Local Variable Completion

Completion of a local (marked deprecated) method

Handling Fully-Qualified Class Names

Of course, this is no Eclipse (yet!), so there’s plenty of issues to work out:

• It can be slow when loading classes from imports of large packages the first time(e.g. importing java.util.*, java.io.*, javax.swing.* all together).  Once the classes are loaded, code completion is fast, it’s just the initial caching that can take some time.  This is because this package actually parses the completion info out of the jar (e.g. scans the .class files in a tools.jar), which can take some time.  We should be smart and load this stuff off the EDT when possible (e.g. when the user adds the import, but before they hit ctrl+space).
• It does not yet offer available methods for local variables or members.  For example, for the local variable “String str”, typing “str.get<ctr+space>” will not return “getBytes()”.  The parser does know that “str” is a String however, so this functionality is on the to-do list.  This could get very complicated however, with situations such as chained methods.
• Javadoc is not served up with the completion choices.  This can also be added, but probably won’t be seriously looked at until other issues are sorted out first.  Expect to see Javadoc returned for members in the being-parsed source before general purpose, src.zip-style documentation support.

I decided to give the markup languages (XML, HTML, JSP and PHP) a little love  in RSyntaxTextArea.  Their syntax is so structured it should be easier to support more advanced features with them than with more abstract languages.

The first thing I did was “soften” the default color scheme used for markup languages.  Here’s how an HTML file looks out-of-the-box now:

Softer color scheme

Note that element names are now differentiated from attributes.  I may end up softening the string color as well, as that’s still a little obnoxious.

With things looking nicer, I decided to implement a new time-saving feature.  Now, typing “</” in an XML file will automatically complete the closing tag with the correct tag name.  e.g. typing:

<foo></

will type the following into the editor:

<foo></foo>

A welcome time-saver.  This feature can also be toggled on with HTML, JSP and PHP, but by default it is off, due to HTML often accepting unclosed tags such as <p>, <hr>, etc., which would mess up close-tag completions.

Going one step further from completing closing tags, I also used the AutoComplete library to add auto-completion for HTML, JSP and PHP.  The completion is very smart:

• It only offers tag names as suggestions when you’re at the start of a tag (e.g. “<” or “<ht”).
• If you’ve already typed a tag name, only the attributes valid for that tag will be offered as completion choices until the tag is closed.
• No completion choices are offered if it doesn’t make sense (you’re in a comment, a string literal, in embedded JavaScript, etc.).

Auto-completing an HTML tag

Intelligent attribute completion

HTML/JSP/PHP code completion will definitely be integrated into the next RText release, but I’d like to figure out a way to release it alongside the standard AutoComplete library.  I don’t think it should be a part of the main library itself (why bloat the library with completion support for a language that might not be needed by all users?), but perhaps some kind of additional jar, that includes pre-built support for certain languages such as this?

If anybody would like to help filling in the gaps in the description for HTML tags and their attributes for this HTML code completion, get in contact with me.  I scraped some basic info off the web, but it’s really lacking.  You would of course receive credit for your input.

Though development has unfortunately come to a crawl, a first beta of TokenMakerMaker is available for testing.  Contact me if you’re interested in testing it out.  It will be a feedback-driven project – features that are asked for are what will be implemented.  I plan on using it to generate a few new scanners for the next version of RSTA – first will be ActionScript.