Exception in thread "File loading thread" java.lang.OutOfMem

Questions on using RSyntaxTextArea should go here.

Moderator: robert

Exception in thread "File loading thread" java.lang.OutOfMem

Postby Taneeda » Fri Jun 19, 2009 12:45 pm

Hi guys,

I use your RSyntaxTextArea component in my application as a text editor for all files. I have a log file, that is very big (round 95k lines). I open the file to a TextEditorPane in a new tab inside a jtabbedpane.
I open the file the the new tab, and close it after loading, several time, an OutOfMemoryException occurs.

Code: Select all
Exception in thread "File loading thread" java.lang.OutOfMemoryError: Java heap space
   at javax.swing.text.GapContent.allocateArray(Unknown Source)
   at javax.swing.text.GapVector.resize(Unknown Source)
   at javax.swing.text.GapVector.shiftEnd(Unknown Source)
   at javax.swing.text.GapContent.shiftEnd(Unknown Source)
   at javax.swing.text.GapVector.open(Unknown Source)
   at javax.swing.text.GapVector.replace(Unknown Source)
   at javax.swing.text.GapContent.insertString(Unknown Source)
   at javax.swing.text.AbstractDocument.handleInsertString(Unknown Source)
   at javax.swing.text.AbstractDocument.insertString(Unknown Source)
   at javax.swing.text.PlainDocument.insertString(Unknown Source)
   at org.fife.ui.rtextarea.RTextAreaEditorKit.read(RTextAreaEditorKit.java:341)
   at org.fife.ui.rtextarea.RTextArea.read(RTextArea.java:970)
   at org.fife.ui.rsyntaxtextarea.TextEditorPane.load(TextEditorPane.java:452)
   at client.view.guicomponents.texteditor.GuiTextEditor$4.run(GuiTextEditor.java:234)


Code: Select all
public void loadFile()
   {
      Thread t = new Thread()
      {
         @Override
         public void run()
         {
            SwingUtilities.invokeLater(new Runnable()
            {
               @Override
               public void run()
               {
                  getTEStatusBar().setMessage("loading...");
               }
            });
            try
            {
>>>>>>>>>>>>>         getRTextArea().load(FileLocation.create(file), null);
                 
                  /**
                   * Set cursor to the first line.
                   */
                  getRTextArea().setSelectionStart(1);
                  getRTextArea().setSelectionEnd(1);
              }
            catch (IOException ioe)
            {
               FailureMessage.printFailureMessage(MainClass.getController(),
                  "IOException;" + ioe.getMessage() + ";" + ioe.getCause());
              }
            finally
            {
               SwingUtilities.invokeLater(new Runnable()
               {
                  @Override
                  public void run()
                  {
                     getTEStatusBar().setMessage("");
                  }
               });
            }
         }
      };
      t.setName("File loading thread");
      t.setDaemon(true);
      t.start();
   }


Note:
- After startup, the application allocates round 34MB. After files was opened, the application allocates about 95MB!!!!!
- The file has round 95k lines, but in the TextEditoPane are only round90k lines displayed.
- When i use the JTextArea, there is no error and everything works fine

Does anybody knows where the failure is??? I don't know :(
Need help, please...
Image
User avatar
Taneeda
 
Posts: 10
Joined: Fri Jun 19, 2009 12:35 pm
Location: Oldenburg, Germany

Re: Exception in thread "File loading thread" java.lang.OutOfMem

Postby robert » Fri Jun 19, 2009 3:16 pm

Strange, as RSTA is very lightweight, and doesn't require much more memory than a standard JTextArea does (due to the way it doesn't store style information for the entire document like most other programming editors do).

From your stack trace, it appears the OOME occurs in JRE code (document loading), not RSTA code. We make a simiple call to EditorKit.read(), which loads the text into the Document, before doing anything else, and it looks like we never get to the "anything else." So I wonder why you wouldn't get this same problem with a JTextArea.

What type of memory usage do you see when you use a JTextArea?

I'll try to reproduce the problem, but it would be best if you sent me a simple test case (shouldn't be your real application since I'm assuming that's too big to be considered "simple"!) in case I cannot reproduce the problem. In the meantime, there's always the standby of bumping your max heap size via -Xmx192M or something similar.

You can send it to: robert -at- fifesoft dot com

I'll post back
User avatar
robert
 
Posts: 774
Joined: Sat May 10, 2008 5:16 pm

Re: Exception in thread "File loading thread" java.lang.OutOfMem

Postby Guest » Fri Jun 19, 2009 3:38 pm

Hi robert, thanx for your help, I wonder too, about this problem.

I tried a little to find the problem and i can see, that if i overwrite the read(Reader in, Object desc) method of TextEditorPane with the read(...) method of JTextArea, the problem doesn't occurs. I copied the code of read method from JTextComponent and created an anonymous inner type of TextEditorPane where is replaced the read(...) method. in the method, i just replaced the line
Code: Select all
EditorKit kit = getUI().getEditorKit(this);
(from JTextComponent) with
Code: Select all
RTextAreaEditorKit kit = (RTextAreaEditorKit)getUI().getEditorKit(this);


If i use this code from inner type, everything works fine, no problems or errors.

Code: Select all
textArea = new TextEditorPane()
            {
               private static final long serialVersionUID = 1629709814119814765L;

               @Override
               /**
                 * Initializes from a stream.  This creates a
                 * model of the type appropriate for the component
                 * and initializes the model from the stream.
                 * By default this will load the model as plain
                 * text.  Previous contents of the model are discarded.
                 *
                 * @param in the stream to read from
                 * @param desc an object describing the stream; this
                 *   might be a string, a File, a URL, etc.  Some kinds
                 *   of documents (such as html for example) might be
                 *   able to make use of this information; if non-<code>null</code>,
                 *   it is added as a property of the document
                 * @exception IOException as thrown by the stream being
                 *  used to initialize
                 * @see EditorKit#createDefaultDocument
                 * @see #setDocument
                 * @see PlainDocument
                 */
                public void read(Reader in, Object desc) throws IOException
                {
                  RTextAreaEditorKit kit = (RTextAreaEditorKit)getUI().getEditorKit(this);
                    Document doc = kit.createDefaultDocument();
                    if (desc != null) {
                        doc.putProperty(Document.StreamDescriptionProperty, desc);
                    }
                    try {
                        kit.read(in, doc, 0);
                        setDocument(doc);
                    } catch (BadLocationException e) {
                        throw new IOException(e.getMessage());
                    }
                }
            };


I hope this helps ^^
Guest
 

Re: Exception in thread "File loading thread" java.lang.OutOfMem

Postby robert » Fri Jun 19, 2009 5:08 pm

I think it's strange that your solution of overriding TextEditorPane's read() method works for you, because that really shouldn't change anything. If you don't override the method, the call boils down to RTextArea's read() method, which looks very similar to your new version:

Code: Select all
RTextAreaEditorKit kit = (RTextAreaEditorKit)getUI().getEditorKit(this);
setText(null);
Document doc = getDocument();
if (desc != null)
   doc.putProperty(Document.StreamDescriptionProperty, desc);
try {
   // NOTE:  Resets the "line separator" property.
   kit.read(in, doc, 0);
} catch (BadLocationException e) {
   throw new IOException(e.getMessage());
}


The only thing we do different is re-use the same Document, which is done to preserve our Undo manager.

Tell me: are you re-loading new content into the editor over and over (large amounts such as 95k of lines over and over), or just loading it up one time? If the latter, perhaps that has something to do with it.

I wrote a test application and couldn't reproduce this issue to the degree you are seeing. The app simply creates either a JTextArea or TextEditorPane, and loads it with a file (highlighted Java source, to be sure the syntax highlighting comes into play). I could load a 200,000 line file into the TextEditorPane and not run into an OOME. A 300,000 line file required a bump-up of -Xmx. I didn't try anything in between, because I figured 200k was sufficiently larger than your 95k. :D (Besides, 200,000 should be more than sufficient for 99.9% of source code editing out there!). There must be something I'm not doing that you are, to cause the problem. Here's the application:

Code: Select all
import java.awt.*;
import java.io.*;
import javax.swing.*;
import javax.swing.event.CaretEvent;
import javax.swing.event.CaretListener;
import javax.swing.text.BadLocationException;

import org.fife.ui.rsyntaxtextarea.FileLocation;
import org.fife.ui.rsyntaxtextarea.TextEditorPane;


public class TestCase extends JFrame {

   private static final File file = new File("Small.java");


   public TestCase() throws IOException {

      JPanel cp = new JPanel(new BorderLayout());
      cp.setBorder(BorderFactory.createEmptyBorder(5,5,5,5));

      final JTextArea textArea;

      TextEditorPane tep = new TextEditorPane();
      tep.setSyntaxEditingStyle("text/java");
      tep.load(FileLocation.create(file), null);
      textArea = tep;

//      textArea = new JTextArea();
//      textArea.read(new BufferedReader(new FileReader(file)), null);

      textArea.setRows(25);
      textArea.setColumns(40);
      cp.add(new JScrollPane(textArea));
      textArea.addCaretListener(new CaretListener() {
         public void caretUpdate(CaretEvent e) {
            try {
               System.out.println(textArea.getLineOfOffset(e.getDot())+1);
            } catch (BadLocationException ble) {
               ble.printStackTrace();
            }
         }
      });

      setContentPane(cp);
      setTitle("Memory usage test");
      setDefaultCloseOperation(EXIT_ON_CLOSE);
      pack();
      setLocationRelativeTo(null);

   }


   public static void main(String[] args) {
      SwingUtilities.invokeLater(new Runnable() {
         public void run() {
            try {
               new TestCase().setVisible(true);
            } catch (IOException ioe) {
               ioe.printStackTrace();
            }
         }
      });
   }
}


One pain point that springs to mind is the growth function for a data structure RSTA uses. For each line in the Document we store exactly 1 int, and for this we use a data structure similar to ArrayList, but for primitive ints. Since that class is basically a copy-and-paste of ArrayList, it uses its same growth function, e.g. (oldSize*3)/2+1. For "normal" sized files (a few hundred or a few thousand lines), this is fine, but when your files start getting 100,000+ lines, this growth is probably excessive. We could put an arbitrary cap on the size increase to keep memory from ballooning so much, say 50,000. 99.9% of users would see no difference, and the other 0.1% would be able to get a few thousand more lines displayed before they ran into OOME's. :D

Anyway, I'm glad you have found a solution that works for you. Let me know your usage pattern for this log viewer and perhaps I can look into optimizing memory in that scenario.
User avatar
robert
 
Posts: 774
Joined: Sat May 10, 2008 5:16 pm

Re: Exception in thread "File loading thread" java.lang.OutOfMem

Postby Guest » Sun Jun 21, 2009 12:09 pm

Hi Robert,

I run your test application for a file with round 96k lines and also get no error, that's is strange. The file has just the string "line" in each line.
Have you looken in the test application i sent you at friday? With this application, i get an OOME for the file in test case!?

Greetz Tanneda
Guest
 

Re: Exception in thread "File loading thread" java.lang.OutOfMem

Postby Taneeda » Sun Jun 21, 2009 12:16 pm

The last posts are from me, i forgot to login.

what did i wrong in my test application, i sent you? Can sou see the failure, i can't :(

I retried it with another file, and with that, i got the error

Code: Select all
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
   at javax.swing.text.GapContent.allocateArray(Unknown Source)
   at javax.swing.text.GapVector.resize(Unknown Source)
   at javax.swing.text.GapVector.shiftEnd(Unknown Source)
   at javax.swing.text.GapContent.shiftEnd(Unknown Source)
   at javax.swing.text.GapVector.open(Unknown Source)
   at javax.swing.text.GapVector.replace(Unknown Source)
   at javax.swing.text.GapContent.insertString(Unknown Source)
   at javax.swing.text.AbstractDocument.handleInsertString(Unknown Source)
   at javax.swing.text.AbstractDocument.insertString(Unknown Source)
   at javax.swing.text.PlainDocument.insertString(Unknown Source)
   at org.fife.ui.rtextarea.RTextAreaEditorKit.read(RTextAreaEditorKit.java:341)
   at org.fife.ui.rtextarea.RTextArea.read(RTextArea.java:970)
   at org.fife.ui.rsyntaxtextarea.TextEditorPane.load(TextEditorPane.java:452)
   at mypackage.TestCase.<init>(TestCase.java:29)
   at mypackage.TestCase$2.run(TestCase.java:61)
   at java.awt.event.InvocationEvent.dispatch(Unknown Source)
   at java.awt.EventQueue.dispatchEvent(Unknown Source)
   at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
   at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
   at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
   at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
   at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
   at java.awt.EventDispatchThread.run(Unknown Source)


File Info:
- 94807 lines
- line content like "[Wed May 20 22:06:38 CEST 2009|FINE]: (client.model.threads.RequestThread::runTestCase | [18] ): "Consuming entity content [RequestThread >-=|[whole test case [3 Steps]|=-<]"" (large lines with large strings)
- 12.500 KB file size

The error occurs just for TextEditorPane, not for JTextArea, but if add more lines to file, to increase line number to roun 126k, i also get the error for JTextArea.

What dou think robert?
Image
User avatar
Taneeda
 
Posts: 10
Joined: Fri Jun 19, 2009 12:35 pm
Location: Oldenburg, Germany


Return to Help

Who is online

Users browsing this forum: No registered users and 1 guest