Generic function name highlighting

Questions on using RSyntaxTextArea should go here.

Moderator: robert

Generic function name highlighting

Postby kriro » Wed Dec 16, 2009 7:41 pm


I created a new TokenMaker for my custom language (which is very similar to Java). Using the JavaTokenMaker.jflex as the starting point, I was surprised how easy it was to get a basic version going. Documentation is quite good (compared to what I've seen elsewhere...).

Here's a note for using JFlex: In the docs it is stated that the generated files are hand-massaged and trimmed. I used JFlex's skeleton feature to apply the changes in the skeleton file, so now I get the mentioned changes automatically for each build and do not have to manually change the generated file (unless there are more edits than documented). This I find very convenient and it lends itself for automatic builds.

The one thing I did not yet manage to achieve is to create a function token for functions whose name is not known in advance, i.e. user-defined functions. The virtual rule would be

Code: Select all

i.e. an identifier followed by an opening round bracket, where the token should only comprise the identifier part, not the bracket, and parsing after detecting this should continue again at the opening bracket. Essentially, this is some kind of lookahead.

From what I see in the examples, functions are always explicitly listed, and not generically recognized. Is there some recipe on how to tackle this problem?

Posts: 3
Joined: Wed Dec 16, 2009 7:13 pm

Re: Generic function name highlighting

Postby Guest » Wed Dec 16, 2009 8:12 pm

Well, it looks like "lookahead" was the key, and the solution is simply

Code: Select all
{ident}/"("   { addToken(Token.FUNCTION); }

i.e. using the lookahead function indicated by the forward slash. :oops:


Re: Generic function name highlighting

Postby robert » Wed Dec 16, 2009 9:25 pm

Hey Christian,

Wow, you think making our own TokenMaker is "well documented"? That makes one of us. :D Seriously though, if you're comfortable using JFlex and find the doc comment at the top of the existing TokenMaker classes, you're right that it's pretty easy to get going.

As for the skeleton file, you're right that that's really the way to go. I should push a skeleton file somewhere in the project and start using that as well. The "TokenMakerMaker" utility I've mentioned elsewhere on the forums does exactly that. Unfortunately I'll probably be too lazy to regenerate all of the existing TokenMakers, so their class comments won't mention using this skeleton file for awhile. :D Perhaps a README file with a section on setting up JFlex and the skeleton file to use would be a good compromise.
User avatar
Posts: 852
Joined: Sat May 10, 2008 5:16 pm

Return to Help

Who is online

Users browsing this forum: No registered users and 2 guests