You may want to check out the
HTML support in RSTALanguageSupport, in particular the HtmlCompletionProvider class. It uses the RSyntaxDocument of the RSyntaxTextArea to get tokens to help with its completions. It uses it to determine wheher or not it's in a tag, and if it is, if the user is typing a tag name or attribute.
In theory the HtmlTokenMaker isn't specifically tied to HTML, but rather gets its completion information from
this XML file at runtime. I think you could check out the RSTALanguageSupport project, and modify HtmlCompletionProvider.java to load a different XML file with the same structure, and it would do completions for your special markup language. Keep in mind though that it's not smart enough to handle nested elements properly, since in HTML, all standard tags are always uniquely named, and you don't have to know the parent tag hierarchy to determine what attributes are appropriate for a given tag. If your markup doesn't meet this requirement, then HtmlCompletionProvider will not be sufficient.
Also note that HtmlCompletionProvider, and all of RSTALanguageSupport, is LGPL, so if you'll have to be okay with that license in order to use it.