How to stop RSyntaxTextArea from changing line breaks

General Discussion on RSyntaxTextArea.

Moderator: robert

How to stop RSyntaxTextArea from changing line breaks

Postby narupley » Fri Jan 31, 2014 5:03 pm

Say I have a string containing carriage returns (0x0D). I paste it into RSyntaxTextArea, then copy it from the text area and back into somewhere else. All my original carriage returns have been replaced with LFs (0x0A). This is the same thing Sublime Text does by default, and it's horrible.

I saw that the TextEditorPane has a setLineSeparator method, but it appears to do nothing. I set it to "\r" to test it out, but the incorrect behavior stayed exactly the same. In any case, that setLineSeparator method isn't what I need. The text area needs to be able to accept multiple types of line breaks (CR/LF/CRLF) and preserve them when copy/pasting between the pane and something else.

Is there a way for me to tell RSyntaxTextArea "No, contrary to popular belief you actually don't know what's best for me, leave my strings alone"?


Part 2: Is there a simple way to show "meta" characters in the text area? Like whenever a literal line feed is in the string, in the text pane it should show "\n" instead. Obviously when a user copies the string out from the text area it should not copy out a literal backslash and "n", but instead copy out the actual LF. We do this in our current application using JEdit. Here's what it looks like:

Image
Image
Image

Obviously JEdit doesn't do all the fancy things that RSyntaxTextArea does like code folding, but at least it got these basic things right.
User avatar
narupley
 
Posts: 7
Joined: Fri Jan 31, 2014 4:35 pm

Re: How to stop RSyntaxTextArea from changing line breaks

Postby robert » Sat Feb 01, 2014 12:35 am

Unfortunately all Swing JTextComponents (including RSyntaxTextArea) do not provide support for remembering individual, differing line terminators for different lines in a single document. They always normalize line endings to a single separator.

By default, I *think* JTextComponents choose the OS-specific default line terminator, but it might always just choose LF, I'm not sure about that. When reading a file via textArea.read(), it uses some heuristic to determine what the line terminator should be for that particular content when/if you write it back out. In any case, you can override this and tell the Document (not the text component) to use a specific line ending when writing its content:

java code:

Document doc = textArea.getDocument();
doc.putProperty(DefaultEditorKit.EndOfLineStringProperty, "\r\n"); // For example


This property is definitely used when writing file contents via textArea.write(Writer), but unfortunately I'm not 100% sure whether it is also used when copy/pasting text. And again, it won't preserve line endings read when the file was loaded either (if they are homogeneous in a file).

Preserving individual line endings would require some legwork on RSTA's part. Please enter a feature request on GitHub so this can be tracked.
User avatar
robert
 
Posts: 829
Joined: Sat May 10, 2008 5:16 pm

Re: How to stop RSyntaxTextArea from changing line breaks

Postby narupley » Fri Jan 09, 2015 12:49 am

Finally, I've come back to this, and implemented a solution.

First, I've gotten rid of the code in RTATextTransferHandler that normalized line breaks (the handleReaderImport method). Then to handle the line breaks myself, I created a custom document class:

java code:

package org.fife.ui.rsyntaxtextarea;

import java.util.TreeMap;

import javax.swing.text.AttributeSet;
import javax.swing.text.BadLocationException;

import org.apache.commons.lang3.ArrayUtils;

public class EOLPreservingRSyntaxDocument extends RSyntaxDocument {

private TreeMap<Integer, char[]> eolMap = new TreeMap<Integer, char[]>();

public EOLPreservingRSyntaxDocument(String syntaxStyle) {
super(syntaxStyle);
}

public EOLPreservingRSyntaxDocument(TokenMakerFactory tmf, String syntaxStyle) {
super(tmf, syntaxStyle);
}

@Override
public void insertString(int offset, String str, AttributeSet a) throws BadLocationException {
boolean wasCR = false;
char[] charArray = str.toCharArray();

for (int i = 0; i < charArray.length; i++) {
char c = charArray[i];

if (c == '\r') {
boolean updated = false;
if (i == charArray.length - 1) {
char[] eol = eolMap.get(offset);
if (eol != null && eol.length == 1 && eol[0] == '\n') {
eolMap.put(offset, "\r\n".toCharArray());
updated = true;
}
}

if (!updated) {
if (wasCR || i == charArray.length - 1) {
// Insert CR
insertEOL(offset, a, "\r");
offset++;
}
wasCR = true;
}
} else if (c == '\n') {
boolean updated = false;
if (i == 0) {
char[] eol = eolMap.get(offset - 1);
if (eol != null && eol.length == 1 && eol[0] == '\r') {
eolMap.put(offset - 1, "\r\n".toCharArray());
updated = true;
}
}

if (!updated) {
if (wasCR) {
// Insert CRLF
insertEOL(offset, a, "\r\n");
offset++;
} else {
// Insert LF
insertEOL(offset, a, "\n");
offset++;
}
}

wasCR = false;
} else {
if (wasCR) {
// Insert previous CR
insertEOL(offset, a, "\r");
offset++;
}

// Insert character
adjustEOLOffsets(offset);
super.insertString(offset, new String(new char[] { c }), a);
offset++;

wasCR = false;
}
}
}

private void insertEOL(int offs, AttributeSet a, String eol) throws BadLocationException {
adjustEOLOffsets(offs);
super.insertString(offs, "\n", a);
eolMap.put(offs, eol.toCharArray());
}

private void adjustEOLOffsets(int offs) {
for (Integer offset : eolMap.descendingKeySet().toArray(new Integer[eolMap.size()])) {
if (offset >= offs) {
eolMap.put(offset + 1, eolMap.remove(offset));
}
}
}

@Override
public void remove(int offs, int len) throws BadLocationException {
super.remove(offs, len);

for (Integer offset : eolMap.keySet().toArray(new Integer[eolMap.size()])) {
if (offset >= offs) {
char[] eol = eolMap.remove(offset);
if (offset >= offs + len) {
eolMap.put(offset - len, eol);
}
}
}
}

@Override
public void replace(int offset, int length, String text, AttributeSet a) throws BadLocationException {
remove(offset, length);
insertString(offset, text, a);
}

public char[] getEOL(int line) {
int i = 0;
for (char[] eol : eolMap.values()) {
if (i == line) {
return eol;
}
i++;
}
return null;
}

public String getEOLFixedText(int offset, int length) throws BadLocationException {
String text = getText(offset, length);
StringBuilder builder = new StringBuilder();

for (int i = 0; i < text.length(); i++) {
char[] eol = eolMap.get(offset + i);
if (ArrayUtils.isNotEmpty(eol)) {
builder.append(eol);
} else {
builder.append(text.charAt(i));
}
}

return builder.toString();
}
}


Now, I need to tap into the custom document to determine what to draw in the View. I opted to create a small utility method:

java code:

package org.fife.ui.rsyntaxtextarea;

import java.awt.Graphics2D;
import java.awt.Point;

import javax.swing.text.BadLocationException;

import org.apache.commons.lang3.ArrayUtils;

public class SyntaxViewUtil {

/**
* Draws custom end-of-line markers based on whether a line ends with a CR, LF, or CRLF.
*/
public static void drawEOL(RSyntaxTextArea textArea, Graphics2D g, float x, float y) {
if (textArea.getEOLMarkersVisible()) {
g.setColor(textArea.getForegroundForTokenType(Token.WHITESPACE));
g.setFont(textArea.getFontForTokenType(Token.WHITESPACE));

if (textArea.getDocument() instanceof EOLPreservingRSyntaxDocument) {
try {
int line = textArea.getLineOfOffset(textArea.viewToModel(new Point((int) x, (int) y)));
char[] eol = ((EOLPreservingRSyntaxDocument) textArea.getDocument()).getEOL(line);

if (ArrayUtils.isNotEmpty(eol)) {
String display = "";
for (char c : eol) {
switch (c) {
case '\r':
display += "\\r";
break;
case '\n':
display += "\\n";
break;
}
}

g.drawString(display, x, y);
}
} catch (BadLocationException e) {
}
} else {
g.drawString("\\n", x, y);
}
}
}
}


Then in SyntaxView and WrappedSyntaxView, I call out to the utility method wherever an EOL needs to be drawn (4 places total):

java code:

private float drawLine(TokenPainter painter, Token token, Graphics2D g,
float x, float y) {

float nextX = x; // The x-value at the end of our text.

while (token!=null && token.isPaintable() && nextX<clipEnd) {
nextX = painter.paint(token, g, nextX,y, host, this, clipStart);
token = token.getNextToken();
}

SyntaxViewUtil.drawEOL(host, g, nextX, y);

// Return the x-coordinate at the end of the painted text.
return nextX;
}


Finally, I set my custom document on my custom subclass of RSyntaxTextArea, which does a little something to grab the text with the preserved line breaks:

java code:

@Override
public String getText() {
if (getDocument() instanceof EOLPreservingRSyntaxDocument) {
try {
return ((EOLPreservingRSyntaxDocument) getDocument()).getEOLFixedText(0, getDocument().getLength());
} catch (BadLocationException e) {
}
}
return super.getText();
}


For good measure, I added a custom action to the input map of my text area, so that pressing Shift+Enter will insert a CR. Here's the result:

Image
User avatar
narupley
 
Posts: 7
Joined: Fri Jan 31, 2014 4:35 pm

Re: How to stop RSyntaxTextArea from changing line breaks

Postby robert » Fri Jan 09, 2015 3:13 am

That's really slick. I'll check it out, thanks! I have added a Feature Request on GitHub to track this getting added to RSTA proper. Pull requests are always welcome. :)

My only consideration is whether there is a performance impact. I'd bet good money it is negligible, but that factor would determine whether to take the subclass approach as you did (likely just for minimizing the intrusiveness of your fix), or building it into e.g. RSyntaxDocument directly. This would be a good place for some unit tests as well (I plan to make a push to increase code coverage in RSTA in the next release. RSTA's simply too mature to have so few unit tests).

Anyway, thanks for the really cool enhancement!
User avatar
robert
 
Posts: 829
Joined: Sat May 10, 2008 5:16 pm

Re: How to stop RSyntaxTextArea from changing line breaks

Postby narupley » Fri Jan 09, 2015 3:19 am

No prob! Sorry it took so long... was pulled in other directions at work for a while, and I'm just now working again on overhauling our editor panes.
User avatar
narupley
 
Posts: 7
Joined: Fri Jan 31, 2014 4:35 pm

Re: How to stop RSyntaxTextArea from changing line breaks

Postby narupley » Mon Jan 12, 2015 6:02 pm

I have an update for this... basically a few changes for performance reasons, but also I added the ability to show control characters (0x00-0x1F, 0x7F) as control pictures (␀-␟, ␡). The changes are simple, I just pass a boolean into the constructor, and when true I do the replacement either way during insertion/extraction of text. Here's the updated code:

java code:

package org.fife.ui.rsyntaxtextarea;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.Reader;
import java.io.StringReader;
import java.util.Arrays;
import java.util.TreeMap;

import javax.swing.text.AttributeSet;
import javax.swing.text.BadLocationException;

import org.apache.commons.lang3.ArrayUtils;
import org.apache.commons.lang3.StringUtils;

public class EOLPreservingRSyntaxDocument extends RSyntaxDocument {

private static final char[] CR = new char[] { '\r' };
private static final char[] LF = new char[] { '\n' };
private static final char[] CRLF = new char[] { '\r', '\n' };

private boolean replaceControlCharacters;
private TreeMap<Integer, char[]> eolMap = new TreeMap<Integer, char[]>();

public EOLPreservingRSyntaxDocument(String syntaxStyle) {
this(syntaxStyle, true);
}

public EOLPreservingRSyntaxDocument(String syntaxStyle, boolean replaceControlCharacters) {
super(syntaxStyle);
this.replaceControlCharacters = replaceControlCharacters;
}

public EOLPreservingRSyntaxDocument(TokenMakerFactory tmf, String syntaxStyle, boolean replaceControlCharacters) {
super(tmf, syntaxStyle);
this.replaceControlCharacters = replaceControlCharacters;
}

@Override
public void insertString(int offset, String str, AttributeSet a) throws BadLocationException {
if (StringUtils.isEmpty(str)) {
return;
}
PeekReader reader = null;

try {
reader = new PeekReader(new StringReader(str));
StringBuilder builder = new StringBuilder();
TreeMap<Integer, char[]> tempMap = new TreeMap<Integer, char[]>();
char[] buff = new char[1024];
int nch;
int cOffset = offset;
boolean wasCR = false;

while ((nch = reader.read(buff, 0, buff.length)) != -1) {
for (int i = 0; i < nch; i++) {
char c = buff[i];

if (c == '\r') {
boolean updated = false;
if (i == nch - 1 && !reader.peek() && Arrays.equals(eolMap.get(offset), LF)) {
eolMap.put(offset, CRLF);
updated = true;
}

if (!updated && (wasCR || i == nch - 1 && !reader.peek())) {
// Insert CR
tempMap.put(cOffset++, CR);
builder.append(LF);
}

wasCR = true;
} else if (c == '\n') {
boolean updated = false;
if (cOffset == offset) {
if (Arrays.equals(eolMap.get(offset - 1), CR)) {
eolMap.put(offset - 1, CRLF);
updated = true;
}
}

if (!updated) {
if (wasCR) {
// Insert CRLF
tempMap.put(cOffset++, CRLF);
builder.append(LF);
} else {
// Insert LF
tempMap.put(cOffset++, LF);
builder.append(LF);
}
}

wasCR = false;
} else if (replaceControlCharacters && c != '\t' && (c < ' ' || c == 0x7F)) {
// Insert control character
cOffset++;
builder.append((char) (c == 0x7F ? '\u2421' : '\u2400' + c));
wasCR = false;
} else {
if (wasCR) {
// Insert previous CR
tempMap.put(cOffset++, CR);
builder.append(LF);
}

// Insert regular character
cOffset++;
builder.append(c);
wasCR = false;
}
}
}

str = builder.toString();

Integer key = eolMap.isEmpty() ? null : eolMap.lastKey();
while (key != null && key >= offset) {
eolMap.put(key + str.length(), eolMap.remove(key));
key = eolMap.lowerKey(key);
}
eolMap.putAll(tempMap);
} catch (IOException e) {
// Only using a StringReader, so should not happen
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException e) {
}
}
}

super.insertString(offset, str, a);
}

@Override
public void remove(int offs, int len) throws BadLocationException {
// Combine edge CRs and LFs if necessary
if (Arrays.equals(eolMap.get(offs - 1), CR) && Arrays.equals(eolMap.get(offs + len), LF)) {
eolMap.put(offs - 1, CRLF);
len++;
}

super.remove(offs, len);

for (Integer offset : eolMap.keySet().toArray(new Integer[eolMap.size()])) {
if (offset >= offs) {
char[] eol = eolMap.remove(offset);
if (offset >= offs + len) {
eolMap.put(offset - len, eol);
}
}
}
}

@Override
public void replace(int offset, int length, String text, AttributeSet a) throws BadLocationException {
remove(offset, length);
insertString(offset, text, a);
}

public char[] getEOL(int line) {
int i = 0;
for (char[] eol : eolMap.values()) {
if (i == line) {
return eol;
}
i++;
}
return null;
}

@Override
public String getText(int offset, int length) throws BadLocationException {
if (replaceControlCharacters) {
try {
StringBuilder builder = new StringBuilder();
Reader reader = new StringReader(super.getText(offset, length));
char[] buff = new char[1024];
int nch;

while ((nch = reader.read(buff, 0, buff.length)) != -1) {
for (int i = 0; i < nch; i++) {
if (buff[i] >= '\u2400' && buff[i] <= '\u241F') {
buff[i] = (char) (buff[i] - '\u2400');
} else if (buff[i] == '\u2421') {
buff[i] = 0x7F;
}
}

builder.append(buff, 0, nch);
}

return builder.toString();
} catch (IOException e) {
// Only using a StringReader, so should not happen
}
}

return super.getText(offset, length);
}

public String getEOLFixedText(int offset, int length) throws BadLocationException {
String text = getText(offset, length);
StringBuilder builder = new StringBuilder();

for (int i = 0; i < text.length(); i++) {
char[] eol = eolMap.get(offset + i);
if (ArrayUtils.isNotEmpty(eol)) {
builder.append(eol);
} else {
builder.append(text.charAt(i));
}
}

return builder.toString();
}

private class PeekReader extends BufferedReader {

public PeekReader(Reader in) {
super(in);
}

public boolean peek() throws IOException {
mark(1);
try {
return read() != -1;
} finally {
reset();
}
}
}
}


Here's an example of how it looks with a sample ASTM E1381 / E1394 transmission capture:

Image

It would be more ideal to show a larger graphic for each control characters, like how other editors do (Sublime Text, Notepad++, PSPad, etc.). Here's how Sublime Text looks:

Image

However that would be much harder to do with RSTA... a bunch of custom token painting and keeping track of the correct x-positions (especially while still allowing for tab stops, ugh).
User avatar
narupley
 
Posts: 7
Joined: Fri Jan 31, 2014 4:35 pm


Return to Open Discussion

Who is online

Users browsing this forum: No registered users and 1 guest

cron