RSyntaxTextArea and large XML messages

[RogerLacroix]

Hi Robert,

I tried out RSyntaxTextArea a few months ago and I should have posted my issues back then. Anyway, I was going to use it to highlight XML messages. It works great for small XML messages (i.e. 1KB) but most of the XML messages passing through the messaging system are large, very large (i.e. 1MB or larger) and they typically do not have CRLF (or LF).

Hence, when I try to display a very large XML message with RSyntaxTextArea, the CPU spikes to 100% and stays there for a long while. When the CPU finally goes back to zero, if I try to scroll the pane then the CPU spikes again. (I have a Intel Quad CPU PC, so my PC normally flies!)

Is there anyway for your code to highlight "just what is in the viewport"? If I try to load an XML message that is 50MB then CPU spikes forever.

Regards,
Roger Lacroix

[robert]

RSyntaxTextArea unfortunately isn't very good at very long lines. It already only paints what's in the viewport, but has other issues with multi-MB-lines, due to its design. I'll try to see if there's some way around this.

[robert]

I'm not sure if I have an answer for you. I created a 50MB, single line text file and tried some stuff out. Loading it in a simple Java app, with just a line-wrapped JTextArea, even the JTextArea cannot handle it. It seems to burn CPU indefinitely (at least, > 5 minutes on a 3GHz CPU). I imagine it's having trouble calculating all of the line break positions. Since RSyntaxTextArea is built on top of JTextArea, if JTextArea can't do it, then RSTA can't either. We naturally do more work than JTextArea does (parsing the text, trying to break at token boundaries and not just word boundaries).

Interestingly, even Windows Notepad can't handle a file like that.

With word wrap enabled, Notepad becomes unresponsive. Never even paints the canvas. I stopped waiting after 2 minutes.

Even with word wrap disabled (should be an easier case), it appears that Notepad inserts line breaks for you, even though there aren't any in the file. These may be "logical" line breaks that aren't saved when you modify the file, but a workaround for a limitation in standard Windows text widgets and line length. But that's just a guess.

Notepad++ seems to have better luck. With word wrap enabled, it can load the file and is generally responsive. Editing is slow, but if you're just viewing the file it should be fine. Strangely enough, if the last logical line is in the viewport, the editor becomes considerably slower, even when just moving around with the arrow keys. Must be some implementation detail.

So in short, I think you're going to be out of luck finding a Java text component that can handle such large files, unless you use one that's not built on top of the standard Swing text package. Most editors in general (not just Java) aren't capable of handling very long lines, since this is a tricky scenario to handle; when looking into using one, you should check whether this is a handled case.