Javascript autocomplete

Questions on using RSyntaxTextArea should go here.

Moderator: robert

Javascript autocomplete

Postby steven_upton » Tue Feb 07, 2012 9:10 am

Hi,

I have been doing some work with the new Javascript language support. I have written a post before about needing local variables.

I have made a few changes to org.fife.rsta.ac.js.SourceCompletionProvider that parses the Rhino AstRoot and split the code into CodeBlock objects (similar to the other autocomplete implementations). Then I can check for local variables and functions from the caret position.

I have had to add additional Tokens for Token.FOR, Token.WHILE, Token.TRY etc..... within addFunctionCompletions().

I am not sure if you are interested in the changes? Or what the procedure is for you to check/accept any changes. It is very much work in progress at the moment. Perhaps I can email my changes for you to look at?
steven_upton
 
Posts: 4
Joined: Mon Jan 30, 2012 11:06 am

Re: Javascript autocomplete

Postby robert » Tue Feb 07, 2012 2:08 pm

Thanks Steven. I haven't gotten around to improving the JS code completion yet myself, so I'm glad you're working on it. Emailing me a patch file is probably the easiest way to send your changes.
User avatar
robert
 
Posts: 805
Joined: Sat May 10, 2008 5:16 pm

Re: Javascript autocomplete

Postby steven_upton » Tue Feb 07, 2012 3:21 pm

Ok,

I have noticed that the recent JavaScriptTokenMaker changes now break one of my functions. I am using the JavaScriptTokenMaker to parse some javascript and convert a template into themed HTML... e.g

Input Text:
for (var i=0; i<10; i++)\n{\n\n}

Converted Text:
<span style='color:#7f0055; font-family:Monospaced; font-weight:bold; '>for</span><span style='color:#000000; font-family:Monospaced; '> (</span><span style='color:#7f0055; font-family:Monospaced; font-weight:bold; '>var</span><span style='color:#000000; font-family:Monospaced; '> i=0; i&lt;10; i++)</span>

The Input text is converted to Eclipse themed summary for the description popup. I am adding some predefined Javascript templates to my Javascript completion provider (for, while, try/catch etc..) and I want the HTML to be themed based on the Theme I am using for the editor. In this case Eclipse.

But note that my conversion is now missing the open { and the closing }

I reverted the JavaScriptTokenMaker to a version on 22/2/09 and I see the { and } tokens when I iterate through them.

Simple test with no styling:

Code: Select all
public static String convertStringToThemedHTML(String text, SyntaxScheme scheme, TokenMaker marker)
    {
        Segment s = new Segment(text.toCharArray(), 0, text.length());
        Token token = marker.getTokenList(s, Token.NULL, 0);
        StringBuffer sb = new StringBuffer();

        while (token != null)
        {
           if (token.text != null)
            {
                sb.append(token.getLexeme());
            }
            token = token.getNextToken();
        }
        return sb.toString();
    }
steven_upton
 
Posts: 4
Joined: Mon Jan 30, 2012 11:06 am

Re: Javascript autocomplete

Postby robert » Wed Feb 08, 2012 5:46 am

You're right, this was caused by the big changes to the JavaScriptTokenMaker pushed on 2/5. It was changed from a hand-made parser to one based off of JFlex, like most of the built-in TokenMakers. This allowed for not only easier maintenance, but overall better syntax highlighting (identifications of regular expressions, etc., that would be simply too difficult to identify with a hand-made parser).

Like the other JFlex-based TokenMakers, it makes an (undocumented) assumption that getTokenList() is fed one line's worth of code at a time. Thus, when it sees a newline or EOS, it stops parsing. That explains why you're only seeing the first line of tokens in your output.

One fix would be to change JavaScriptTokenMaker so that instead of stopping on newline characters, it treats it as regular whitespace (?) and keeps parsing. This is probably the most correct thing to do. However, this wouldn't match the behavior of the other TokenMakers, and it would be a fairly large change to modify all of them to behave in this way. I'm unsure if it would have ramifications elsewhere. Instead, I think you can work around it on your end by feeding it one line at a time, something like this (untested):

Code: Select all
Segment s = new Segment();
StringBuffer sb = new StringBuffer();
String[] lines = text.split("\n");
for (int i=0; i<lines.length; i++) {
   s.array = lines[i].toCharArray();
   s.offset = 0;
   s.count = lines[i].length();
   Token token = marker.getTokenList(s, Token.NULL, 0);
   while (token!=null) {
      // Same old ...
      if (i<lines.length-1) {
         sb.append("<br>");
      }
   }
   return sb.toString();
}


Will that work for you?
User avatar
robert
 
Posts: 805
Joined: Sat May 10, 2008 5:16 pm

Re: Javascript autocomplete

Postby steven_upton » Wed Feb 08, 2012 9:48 am

Thanks Robert,

That works great.

Steve
steven_upton
 
Posts: 4
Joined: Mon Jan 30, 2012 11:06 am


Return to Help

Who is online

Users browsing this forum: No registered users and 2 guests

cron