Asynchronous AbstractParser?

[preditcon]

Is there a way to move AbstractParser implementations off EDT (event dispatch thread aka gui thread) and then feed only ParseResult implementations back to ParserManager or to whatever processes them? I would ensure that my ParseResults are immutable.

As far as I can see org.fife.ui.rsyntaxtextarea.parser.Parser.parse(RSyntaxDocument doc, String style) is always called from EDT. Passing execution to another thread and not having to return immediately would be quite useful for my use case - as it is now minor gui freezes occur when I'm checking the document since multiple non-trivial checks must be done on it as a whole. Not sure if RSyntaxDocument is thread safe though.

btw: my custom XML editor with schema aware quasi-autocomplete is comming along quite nicely thanks to your component.

[robert]

This would be tough to do, because we'd have to clone the Document to have a snapshot of what to parse for the parsing Thread. Then, what do we do if the user edits the original Document before the Parser is done, and his ParserNotices don't match up with what the user is looking at?

I suppose we could put a read-lock on the Document and do all parsing on another Thread. This would allow the user to scroll around, and keep the UI responsive, but they could not edit the Document until the Parser completed. This might not be too difficult to implement, but I'm not sure if there are any other implications to doing things this way. This would also incur a good amount of overhead for simpler parsers (though even simple Parsers often do a fair amount of work themselves, such as spawn an external process to run a compiler).

I'll think about it, maybe there's something I haven't thought of yet. Any suggestions?

[preditcon]

Actually these are common threading problems which are solvable.

You could force implementors to mark each ParserNotice or at least it's containement ParseResult somehow - perhaps by putting a hash of the parsed document's contents into it and then comparing it with the current document's content hash - if a ParserNotice/ParseResult fails to pass such an equality check simply throw it away since new ones are sure to come when the new parser task is done. Cleaning outdated notices could also be achieved in a similar way or would simply be left to the user. You could even avoid re-parsing by using a cache based on this solution in case an Undo is issued. I am assuming that the implementation would provide some sort of a queue into which the parsing results would be entered - the same way EDT itself does it if I remember correctly.

There's also no need to lock the document for reading in all cases (by making a snapshot of some sort before each parser run like you suggested). I think that the majority of parsers are only interested in the string content of a document, so why not simply pass only that to parsers and not the whole RSyntaxDocument. Strings are inherently immutable and therefore thread safe. Perhaps I'm over simplifying things and there's a reason why the whole object is needed (some sort of localized parsing which requires the additional information contained in it but that's efficient enough for it to not be processed off EDT anyways). If it's for line:column to document offset conversion then that can be done internally after a valid notice is received. You could also clone the whole document though. I sure know that I abuse it violently during each parser run by re-parsing your XMLTokens into a tree structure which would be impossible without the whole document object.

I agree about the overhead problem in any case - multithreading always implies it. Perhaps all this could be optional and processing of queue entries could either be done the way it is now or in an async way.

Note that I'm just thinking out loud. I have almost zero experience implementing custom gui components.

[robert]

Oh I knew they were solvable, the question was more whether I had the time/motivation to implement them (and test them)!

I'm actually surprised the current parsing technique (all on the EDT, multiple parsers will parse sequentially so their run times are stacked) actually scales as well as it does. I'll admit the only times I've tried it with "huge" documents (hundreds of thousands of lines) has been simple test cases with the RSTALanguageSupport Java parser just to get a feel for performance. Things seemed more than fast enough for the common cases, but I know that's highly machine/environment dependent.

In any case, I'll be happy to explore better parsing handling moving forward. Your idea of using hashes to determine out-of-date content is a good one. I also like the idea of being able to toggle the current, synchronous method and the asynchronous one, but that would also mean maintaining two different implementations.