Archive for the ‘RSTALanguageSupport’ Category

Perl Language Support Enhancement

Friday, October 22nd, 2010

Newly added to the Perl Language Support in RSTALanguageSupport is the ability to modify the value of the PERL5LIB environment variable used when syntax-checking source code.  This feature has already been integrated into the RText Subversion trunk:

Missing a Perl library

Missing a Perl library

Adding it via PERL5LIB

Adding it via PERL5LIB

Problem solved!

Problem solved!

As the source code in the screenshots say, you can also solve this (a more proper solution in many cases, as well) with “use lib”, but I’m trying to give as much freedom with the language support library as possible.

Auto-Activation added to AutoComplete

Sunday, June 6th, 2010

Sorry for the overloading of the “auto” prefix here… The “AutoComplete” library should probably have been named “CodeComplete”, but the point was to emphasize that it was usable for more scenarios than just code completion…

Anyway, after a few requests, I’m adding what Eclipse refers to as Auto-Activation.  This means that you can have the completion popup appear automatically after certain characters are typed.  For example, typing a ‘.’ character in Java could cause the completion popup to appear after a small delay, removing the need to keep pressing Ctrl+Space.

The new methods are added to the AutoCompletion class, and look like this:

public void setAutoActivationEnabled(boolean enabled)
public boolean isAutoActivationEnabled()
public void setAutoActivationDelay(int millis)
public int getAutoActivationDelay()


These methods allow you to toggle not only whether auto-activation is enabled, but also how long the delay should be between when they stop typing and when the popup appears.  It can be set to 0, meaning to always appear when possible, but often users want a small delay (maybe 200 milliseconds), so that it only shows up if they genuinely need it and stop typing.

Note that auto-activation depends on auto-completion itself being enabled.  If you’ve called setAutoCompleteEnabled(false) on an AutoCompletion, it will not honor the auto-activation property.

As to what characters trigger auto-activation, that is done on a per-CompletionProvider basis, since this can (and should) vary depending on what programming language is being edited.  The CompletionProvider interface now has a method:

public boolean isAutoActivateOkay(JTextComponent)

that should return true or false, depending on whether the text at the current caret position is something that auto-activation should occur at.  The concrete base class, CompletionProviderBase (what all CompletionProviders actually extend from, allow you to set exactly what characters this method checks for:

public void setAutoActivationRules(boolean letters, String others)

The first parameter allows you to have auto-activation occur after any standard letter (e.g. Ascii).  I personally think this is annoying, but I have seen editors do it in the past (Visual Studio?).  The second parameter is a string, each char of which is treated as a char to auto-activate after.  So for example, with Java you could call “setAutoActivationRules(false, “.”)” to auto-activate after the user types a period.  For markup languages you could pass “<” as the “others” String to auto-activate for tag names after ‘<” is typed.

The RSTALanguageSupport library is already taking advantage of this new feature.  The Java, HTML, and PHP supports all now auto-activate after appropriate characters by default.  They’re extra smart, and won’t auto-activate when not appropriate (e.g. typing “foo.” while in Javadoc won’t start auto-completion in Java, or typing ‘<’ while in a comment or attribute in HTML – while invalid in and of itself – will not cause code completion either).  If you’re an early adopter, be sure to check out out!  It was first added in revision 209.

Integrating Language Support into RText

Saturday, May 22nd, 2010

I’ve spent the last week or so doing tedious, but important, stuff – making sure the RSTALangaugeSupport API is robust enough to be used in other applications.  The hard work is starting to pay off: Perl support is almost completely integrated already!

Perl options in RText

Perl options in RText

Here’s RText’s Options dialog, showing all the features related to Perl support that you can currently toggle.  As you can see, you can currently fiddle with both the code completion as well as the on-the-fly syntax checking I recently blogged about.  For the latter, you have to have a Perl install somewhere on your machine.  RText will scan your PATH for a Perl install, and default to using that, but you can change it to another installation if you want.  Still pending is the ability to add modules to the @INC path.

Language support will be fully integrated into all aspects of RText.  For example, compile errors and warnings will show up in the Error Strip component:

Compiler errors in Error Strip

Compiler errors in Error Strip

Also, function descriptions are displayed simply by hovering the mouse over them.  For example, below, my mouse is over the “print” function (sorry screen captures in Windows don’t capture the mouse pointer):

Function descriptions on mouse hover

Function descriptions on mouse hover

Perl is one of the harder languages to integrate, simply because its language support has so many features (runtime compilation, etc.) that others don’t have.  Hopefully other language supports will be integrated into RText shortly.  Stay tuned!

Perl Syntax Checking

Wednesday, May 12th, 2010

Perl syntax checking has been added to RSTALanguageSupport.  You can configure it with the location of a Perl install on your system (by default it scans your PATH for a Perl binary), and then PerlLanguageSupport will automatically begin scanning your code for errors as you type:

Perl Syntax Checking

Perl Syntax Checking

Next will be the ability to muck with the @INC path to add other libraries.  Currently it only uses the default @INC of your Perl install.

RSTALanguageSupport Progress

Sunday, May 2nd, 2010

The RSTALangaugeSupport library I’ve blogged about for so long has finally been added to the Subversion repository, but do note that it isn’t ready for prime time yet.  Use it at your own risk!  And send feedback when you do!  :)

If you check out the project, please read the readme file first.  It’s fairly up to date and explains how to compile and use the library.

The library currently includes code completion for the standard C library, and despite a couple of rough edges, should be mostly complete:

C code completion

C code completion

C support takes advantage of the parameter assistance feature of the AutoComplete library as well:

Parameter assistance for C functions

Parameter assistance for C functions

There’s also the start of decent support for Java, which is much more robust and dynamic than the support for other languages such as C.  It actually evaluates your code when you hit ctrl+space, and gives you accurate completion choices based on your current location in the code, based on classes you’ve imported (referencing jars on your “classpath”), local variables if you’re in a method, etc.  This has been discussed in several previous blog posts.

Java code completion

Java code completion

Yes, that tree widget on the left is actually included in RSTALanguageSupport as well!  However, it is currently Java-specific, and will not display an outline of source code in any other language.  It’s not high priority at the moment to generalize it, but it should probably happen sooner or later.  Java also supports parameter assistance when completing methods, like C does with its standard library.

There’s also code completion for Perl, supporting all built-in functions in Perl 5.10.x.  Rudimentary support for completions of variable names will appear for Perl in the next couple of days.

Perl code completion

Perl code completion

There’s also excellent support for HTML 5 – completion for all valid tag names, and attributes are suggested as well (only those attributes valid for the tag they are describing).  The description information displayed in the tool tip-style side window is rather lacking, however (as you can tell from the screenshot), so help improving this documentation is more than welcome!

HTML Completion

HTML Completion

PHP code completion is also included.  It uses the built-in HTML support when editing HTML, but when the caret is within PHP tags, PHP functions are suggested instead.  There is no documentation for PHP functions in the description window like there is for other languages just yet.

PHP code completion

PHP code completion

Similar to the C and Java support, parameter assistance is included for PHP functions:

Parameter assistance for PHP

Parameter assistance for PHP

Hopefully that is enough to whet some appetites!  Please discuss and ask questions over in the forums.

Javadoc assistance for Java Code Completion

Sunday, April 11th, 2010

I’ve added the ability to add “source attachments” to jars on the “class path” or “build path” (not sure what to call that, Eclipse uses “build path,” but RSTA isn’t compiling anything, since it’s just an editor…).  You can now get Javadoc in the description window alongside code completion choices.  In the example below, I’m editing java/lang/String.java, and I’ve attached the src.zip that comes with the JDK.  You’ll see that the Javadoc for the relevant String method is shown:

Source Attachments working

Source Attachments working

Beyond that, I’ve been working on the API, using RText as the test application.

Also, not the source outline tree on the left.  I’m working on getting this component simple and usable as well.  I’m tempted to shove it into the RSTALanguageSupport library, since it would only be used with RSTA instances using the library, but since it isn’t specifically related to the editor, it may just end up in RText itself.  It’s really nice – it displays the outline of the source as parsed by the code completion parser, and clicking on an item in the tree jumps to that item in the source code.  However, it is Java-specific and is not currently meant to be used for outlining just any language.

Java Code Completion Status

Thursday, April 1st, 2010

The code completion library for standard programming languages for RSyntaxTextArea is almost ready for a public repository.  I plan on calling this library “RSTALanguageSupport” (nice name, huh?), as it contains support for editing common programming languages with RSyntaxTextArea.

In getting this library tidied up, I’ve picked up the Java code completion yet again, since it’s by far the most complex.  I thought I’d give a status update now so people don’t have unrealistic expectations about the library when it’s first released.

Assuming you’ve added any jars you want on your “build path” (to use Eclipse terminology), the Java code completion support can handle:

  1. Class fields and methods:

    Completion of fields and methods

    Completion of fields and methods

  2. Local variables in scope in the current method:

    Completion for local variables

    Completion for local variables

  3. Methods and static fields of non-primitive fields and local variables:

    More local variable completion

    More local variable completion

  4. Common Javadoc completions that I personally find extremely useful in Eclipse:

    Javadoc completions

    Javadoc completions

It does not currently handle completions for return types of methods:

Can't do this just yet

Can't do this just yet

This is the single issue that will take the longest to resolve, as it’ll take a re-hashing of how the code completion is parsing the code.  Other features and bug fixes will appear before this one is fixed.

Also, please note that the Java code completion should not be considered “good design.”  I am (poorly) self-taught in lexing/parsing, and so what I have now is certainly not optimized, easy to read, or using the latest design techniques.  It is what it is.  :)  I consider this to be a learning experience, and that the library will improve over time.

That said, it seems reasonably performant (only a pause when it has to initially load completions for many new classes, such as multiple imports of large packages, such as “javax.swing.*”), and is more than usable on the machines I’ve tested on.  Memory usage could certainly be improved (stuff isn’t shared between multiple editors implementing Java completion that should be) but I plan to fix that soon.

As a side note – as you can see from the first screenshot, I’m also working on tying Javadoc into the description window.  Notice that the Javadoc for the “str” field is nicely formatted in the tool tip-style side window?  That happens for all fields and methods in the current source file.  I’m also working on the ability to add “source attachments” for jars on the “classpath,” just like Eclipse, which will be used for assistance with libraries.  So for example, you can point to the src.zip included with your JDK, and get a nice description of each class, field and method in the JDK, just like in your favorite real IDE!

Integration of this library into RText as a plugin is already underway.  As always, RText will be a showcase of all the cool stuff you can do with RSyntaxTextArea.

Java Field and Local Variable Completion

Thursday, December 10th, 2009

The Java code completion I’m working on has hit a new milestone.  Code completion for fields and local variables is starting to work:

Field Completion

Field Completion

Java Code Completion

Sunday, November 15th, 2009

To ensure the robustness of the AutoComplete library, I decided to see what it would take to implement some basic Java code completion.  This is of course much more complex than completion for markup languages, and presents a different set of challenges, but it’s something I’ve always been interested in.

The first thing that I needed was a Java parser, one that could break a source file down into an AST suitable for code completion.  This means you don’t have to go as far as parsing actual method bodies, you just have to grab imports, class/interface/enum names, method and field names, and local variables.  Of course, this isn’t a trivial task by any means; especially for local variables, you have to parse method bodies “well enough” to be able to get the variable names and types, without getting caught up in validating the method body’s structure.

Luckily, I had already been working on such a beast off and on (yes, I’m a geek) so I had something to go on.  It can currently parse all source files in the 1.6u16 JDK (well, classes and interfaces, I’ve put off enums for now to work on other areas), so I think it’s in pretty good shape.  Its error recovery (ability to keep parsing after seeing invalid Java source) is pretty terrible, mostly due to my lack of real education in parsing/compiler design.

So with the Java parser at least functional, the next step was to integrate it into RSyntaxTextArea’s Parser API.  That way the parser runs whenever the text component’s document is modified, allowing the user to get up-to-date completion choices automatically.  This part was easy, as RSTA’s Parser API was already up to the job.

Here’s what’s included in completion suggestions so far:

  • Field and method names for members of the class being edited.
  • Field and method names for all super classes and implemented interfaces.  Access flags are honored – e.g. you will not see suggestions for private members of extended classes.
  • Local variable names for variables accessible at the current caret position.
  • Names of imported classes (e.g. “import java.io.*;” allows everything in the java.io package to be suggested).  Again, access flags are honored – package private classes, etc. are not suggested (unless the source being parsed is part of that package).
  • Anything fully-qualified is suggested as well, so typing “com.sun.java.swing.<ctrl+space>” will suggest everything in that package, whether you’ve imported it or not.
Java Code Completion

Local Variable Completion

Completion of a local (marked deprecated) method

Completion of a local (marked deprecated) method

Handling Fully-Qualified Class Names

Handling Fully-Qualified Class Names

Of course, this is no Eclipse (yet!), so there’s plenty of issues to work out:

  • It can be slow when loading classes from imports of large packages the first time(e.g. importing java.util.*, java.io.*, javax.swing.* all together).  Once the classes are loaded, code completion is fast, it’s just the initial caching that can take some time.  This is because this package actually parses the completion info out of the jar (e.g. scans the .class files in a tools.jar), which can take some time.  We should be smart and load this stuff off the EDT when possible (e.g. when the user adds the import, but before they hit ctrl+space).
  • It does not yet offer available methods for local variables or members.  For example, for the local variable “String str”, typing “str.get<ctr+space>” will not return “getBytes()”.  The parser does know that “str” is a String however, so this functionality is on the to-do list.  This could get very complicated however, with situations such as chained methods.
  • Javadoc is not served up with the completion choices.  This can also be added, but probably won’t be seriously looked at until other issues are sorted out first.  Expect to see Javadoc returned for members in the being-parsed source before general purpose, src.zip-style documentation support.