Balisage Paper: Ragnarok: An Experimental XML environment

Slicing

The most obvious change is that DOM Elements are a true subclass of Python lists — consisting of their child nodes. This enables Python’s normal subscript or slicing notation: myNode[1], not just myNode.childNodes[1].

In minidom, the first raises TypeError: 'Element' object is not subscriptable. The second works as a reference, but assigning to it in obvious Python fashion only appears to work in minidom. For example, this reports no error:

myNode.childNodes[1] = myNode.childNodes[3]

If instead a new Node is spliced into position [1], minidom again corrupts its own data: iterating via nextSibling would never see the new Node, though it would be reachable as myNode.childNodes[1]). Yggdrasil instead does the right thing. In Yggdrasil Pythonically-obvious things just work, both on left and right sides.

Python [] notation supports more varied arguments than just a positive integer, and Yggdrasil also leverages that to support shorthands reminiscent of XPath:

x[-1]

x.childNodes[-1]  # right-side only

x[0:-3]

nl = NodeList()
for ch in x.childNodes[0:-3]: nl.append(ch)

x["para"]

nl = NodeList([ ch for ch in x.childNodes
    if ch.nodeName == "para" ])

x["@id"]

x.getAttribute("id")

x["para":1:3]

temp = NodeList[ ch for ch in x.childNodes
    if ch.nodeName == "para" ])
NodeList(temp[1:3])

x["#text"]

nl = NodeList([ ch for ch in x.childNodes
    if ch.nodeName == "#text" ])

x["*"]

nl = NodeList([ ch for ch in x.childNodes
    if ch.nodeType == Node.ELEMENT_NODE ])

x[".."]

x.parentNode

x["/"]

NodeList(x.childNodes)

As with Python’s own extended slices there are some limitations on the left side (they raise errors if tried). Yggdrasil is even faithful to a feature/quirk of Python slicing: An out-of-range index raises IndexError for a singleton like x[999] but quietly returns an empty list for a range like x[99:999] or x[99:999:2]. One difference, however, is that elements with no children do not cast to Boolean False like empty Python lists do. This is because empty elements have distinct data (such as attributes, not to mention their tree context), unlike other falsish things in Python: None, 0, "", [], {}, etc.

Finally, slicing supports scheme prefixes analogous to those in XPointers and URLs. For example, n["css:#mainText"]. Many CSS selector types are available (a few don’t make sense in non-browser contexts). Additional scheme prefixes can be registered along with functions to handle them.

Other list methods

Regular Python list operations such as insert and del (not to mention sort and many others) just work, as do regular DOM ones:

myNode.insert(-1, newNode)
myNode.extend(someNodeList)
del myNode[-1]
myNode.pop(5)
x[0:2] = myNodeList
howMany = myNode.count("p")
myNode.appendChild(newNode)
myNode.insertBefore(oldChild, newChild)

Other Yggdrasil features

Some other features of Yggdrasil are inspired by the HTML DOM, such as (configurable) case-ignoring^[3] and innerXml and outerXml (both setters and getters). WhatWG inspired adding insertAdjacentXml() and methods for tokenized attributes (such as, but not only, HTML CLASS).

The Yggdrasil API uses modern Python constructs. The much shorter nodeType test properties were shown earlier (n.isElement, n.isText, etc.); or of course Python isinstance() works fine. There are also methods to get the first node along any XPath axis (not just parent, child, and siblings), and generators to iterate along any XPath axis.^[4] There is also a generator for the SAX events that a parser would return were it parsing a given subtree. The generators can take a callable to use as a filter:

for i, n in enumerate(someNode.descendants(test=myCallback)):
     print(f"Descendant {i} is of type {n.nodeName}")

Yggdrasil accepts synonyms such as DOM previous/next vs. XPath preceding/following, and fills symmetry gaps such as prependChild().

Reserved string arguments are defined as enums, so in cases like

myNode.insertAdjacentXML("beforebegin", "<p>As seen below:</p>")

a reified type and value can be used: RelPosition.beforebegin. This lets pylint provide earlier detection of errors, and compilers like mypc optimize better. The enums also accept the strings as equivalent so existing code works without change.

DOM methods that normally require a child node as a location specifier are extended in Yggdrasil to also accept (signed) numbers. Whether a caller knows the position of a node (perhaps because it’s iterating by number) or the actual node (perhaps because it’s in a NodeList), it can just use what it has rather than having to convert to the other — the following are all interchangeable if oldNode is theParent[4]:

theParent.insertBefore(newNode, oldNode)
theParent.insertBefore(newNode, 4)
theParent.insert(4, newNode)

That last case is just the normal Python list insertion method, and therefore uses that order of arguments. Nodes can even delete themselves, using:

myNode.removeNode()

myNode.parentNode.removeChild(myNode)

Yggdrasil can generate and interpret basic XPointers [Clark and DeRose 2002] with or without IDs. As it turns out, generating XPointer child sequences for two nodes and comparing them is a very fast and intuitive way to compare document positions. Upgrades to support DOM/XPointer ranges and discontiguous (virtual) elements are underway but incomplete (to be named Gungnir after Odin’s spear which always hits its target). Those of course will need additional position, containment, traversal, and inheritance semantics.

Since Elements and NodeLists are actually Python lists, operations like sort, reverse, and even multiply are available. However, because no Node can appear in more than one place in a document, operations like multiplication on Elements return a new NodeList and not the same Element lengthened.

Relation to lxml and ElementTree

lxml is a popular parser, with an associated ElementTree package comparable to DOM. It has the advantages of being much more Pythonic than minidom, for example simply constructing nodes via Element(), Comment(), etc.; treating elements much like lists, leveraging Pythonisms such as is, in, len; treating attributes as a dictionary; etc. These conventions are also used in Yggdrasil. ElementTree also has its own names for various things: tag vs. nodeName, getParent() vs. parentNode, etc. These also are supported in the interest of users not having to care.^[6]

ElementTree itself, however, has what I consider a serious drawback: It has no concept of text nodes. Text is treated as a property of the preceding sibling (almost like an attribute), named tail. But of course there may not be a preceding sibling. In that case the text instead goes onto the parent as a property called text (tail on the parent would refer to text in a different place). For Comments and PIs, text means their data content and they have no tail. This works, but has consequences:

I find ElementTree’s approach to text less than intuitive. Nevertheless, along with the rest of the ElementTree API for nodes, Yggdrasil supplies setters and getters for text and tail.

Yggdrasil / Dominµs sibling representation

DOM implementations provide direct access to the previous and next sibling from any node. There are three main ways to implement this, which have significant space/time tradeoffs:

Perhaps surprisingly, profiling revealed that going to the parentNode and counting is very much faster than maintaining and using direct pointers, even for quite wide trees. This is a good example of why a pure Python experimental framework is useful: it’s very easy to change specific details and do head-to-head comparisons.

In keeping with Ragnarok’s overall philosophy Yggdrasil provides for choosing any of the three methods, and even for changing in mid-stream. For example, method 2 might be best during loading (it is very fast for additions at the end), then method 3 once loaded (it is very fast for modifications), or method 1 if there are very bushy nodes. It would not be very hard to have individual nodes switch methods when they get wide (perhaps with hundreds of child nodes). However, method 3 is so much faster in my profiling that I have not seen a need for that. Direct changes to nodes via list operations just work, regardless of the sibling implementation in effect at the time.

A node’s child number is available, counting from either end. Options enable counting only nodes with the same nodeName, ignoring white-space-only text nodes, and/or coalescing adjacent (non-normalized) text nodes.

Security options

Several options have little relation to XML syntax but aim to improve the security of entity usage. Python documentation [Python] gives fairly dire warnings about attack vectors like entities with excessive expansion or risky system identifiers:

<!ENTITY a "Boom">
<!ENTITY b
    "&a;&a;&a;&a;&a;&a;&a;&a;&a;&a;"></para>
<!ENTITY c
    "&b;&b;&b;&b;&b;&b;&b;&b;&b;&b;"></para>
<!ENTITY d
    "&c;&c;&c;&c;&c;&c;&c;&c;&c;&c;"></para>
<!ENTITY e
    "&d;&d;&d;&d;&d;&d;&d;&d;&d;&d;"></para>
<doc>&e;</doc>

<!ENTITY pw SYSTEM "../../../etc/passwd">
&pw;

These are indeed potential attacks, although they affect most any data format that provides a macro or include capability (a very long list). There is even a Python library called defusedxml to mitigate such risks [Heimes 2023]. Thor and Loki both provide mitigation options, because such options do not affect syntax per se:

Basic/practical parser options

Loki, like its namesake, is very good at shape-changing. It’s built as a subclass of Thor. A few of its basic options have already been mentioned. For example, case-folding is not XML but is fairly simple and ubiquitous — SGML and HTML both use it (SGML via separate options for element vs. entity names). XML does not offer it for users, but uses it internally (xmlns and xml prefixes, xml:stylesheet, etc.). XSD has it as a facet for datatypes. This all suggests that case-folding is pretty useful. It is also nearly trivial to implement in a parser, and not disruptive to much else. So Loki offers it as an option (actually several options: elementFold, attributeFold, entityFold, idFold, …).

However, case is not entirely trivial. I learned that some HTML versions, SGML, and Unicode all define it very slightly differently. I bit the bullet and Loki (via Runeheim) can use upper(), lower(), or full Unicode case_fold() (or of course none, as in Thor). More choices can be added. Unicode also defines several normalization forms [Unicode], and those are also available to deal with ligatures, diacritics, halfwidth vs. fullwidth, etc. Likewise where specs differ in what counts as whitespace.

A somewhat related option (htmlNames) enables all the usual SGML/HTML named character entities in one step. This otherwise requires 252 explicit ENTITY declarations (over 2,000 for HTML 5) – and a parser that handles them, and the time to parse them. Even if you’re using a schema language other than DTD your tools have to include DTD support to get this (or perhaps you can code a callback for unknown entities), which seems a bit silly. Like case-folding, a one-step way to enable these isn’t part of XML but it’s awfully handy.

Setting entEncoding enables an extra parameter for ENTITY declarations (thus available in Loki only), to declare the encoding for an external entity. This is especially useful because files oftern show up in unexpected encodings. Since Python provides an encoding parameter when opening streams, this is straightforward:

<!ENTITY chap2 SYSTEM "chap2.xml" ENCODING cp1252>

As noted earlier all options are off by default (except a few for avoiding risky entity fetches), in which case Loki has normal XML behavior just like Thor. But for those inclined to rush in where parsers fear to read, there are two ways to turn options on. First, construct a Loki parser and call its setOption() method. Second, enable options from within a document. At the moment, the settings go in the XML declaration (or perhaps I should call it the Loki declaration?). For example:

<?xml version="1.1" encoding="utf-8" elementFold="UPPER" htmlNames="yes"?>

There are many other ways this could be done. An advantage of the present approach is that it makes a document that uses extensions no longer be WF XML, while remaining extremely close — which is precisely the point. With extensions in use a document is only XML-ish, so XML parsers should reject the document rather than interpreting any extended usage incorrectly. This syntax makes them do so. It also makes the document reveal up front exactly which extensions it uses (if one is used but not enabled, Loki raises a WF error). Extended documents can be converted to regular XML: just load, then export them back out with Gleipnir. That works for most options, though not for those supporting non-hierarchical structures — those must wait for support in Yggdrasil.

Weightier extensions

Being interested in more complex document structures (and well-versed in Wall’s 3 Virtues of a Programmer [Wall, Christiansen, and Orwant 2000]), I also wanted a tool to help me examine related ideas, especially incremental support for non-hierarchical and virtual markup structures. In contrast, the security options discussed above have essentially no effect on parsing (so are in Thor), and case-folding options have only a very narrow effect (so are in Loki only). Enabling special syntax for non-hierarchical structures is a bigger deal (milestones are not a special syntax per se, but involve a special semantic of which an XML parser can be blissfully but unhelpfully unaware).

Nevertheless, non-hierarchy is not always as big a deal as it may seem at first glance. A relatively simple example is olist semantics (see Sperberg-McQueen and Huitfeldt 2000), where an element can be closed even if it isn’t current. The following is neither WF XML nor MECS syntax, but in an olist/overlap world it’s pretty clear what it must mean:

<sec><para>Quoth the raven,</para>
     <para><q>Nevermore.</para>
     <para>Except maybe on Tuesdays.</q></para>
</sec>

Because XML (and therefore XML parsers) can’t do this it is typical to either:

Option 1 requires building separate checking even for many constraints XML parsers already have most of the code for, such as ensuring that milestones are empty but paired, that starts come before ends and are of corresponding type, that end milestones lack attributes, and so on. The markup also has to be more complex: in the case above nothing is required to indicate what the q end tag pairs with, but with milestones they have to be co-indexed or have an interesting algorithm to match them up (not to mention that ID/IDREF attributes for co-indexing does not involve quite the same concept as their use elsewhere).

Option 2 requires constructing a new parser (or modding an existing one, which is harder if it is not in Python when the rest of your code is or if the new syntax is not very close to XML). At that point people commonly create, implement, and debug entirely new syntaxes and tools.

A key insight is that there can be syntax for things like olist that is only a tiny step from XML. By choosing such a syntax and integrating it as an option within a parser that also supports and is tested with regular XML, most of the constraints just mentioned become trivial to add. For olist semantics, in the parser itself the difference involves little but whether the open-element data is a stack or a list. Since Loki is designed for extensibility it is very easy to add an option to remove such an item from the stack of open elements, return a SAX event for it, and continue rather than issuing a WF error. Just add an elif exactly where XML end tag processing would instead issue a WF error:

if name == tagStack[-1]:
    tagStack.pop()
    yield SAXEvent.End, name
elif options.olist and name in tagStack:
    del tagStack[tagStack.rindex(name)]
    yield SAXEvent.End, name
else: raise SyntaxError(
    f"Unexpected end tag for ’{name}’ (context: {tagStack}).")

The parser change is surgical, the syntax and semantics are predictable given olist structural semantics, and everything else works as it did. When the olist option is not enabled the behavior is exactly that of XML. Put another way, Loki supports olist with a 3-line addition beyond Thor.

Yggdrasil cannot yet represent such structures directly, but a model that can (such as LMNL [Piez 2008, Piez 2012] or TagML [Bleeker, Buitendijk, and Haentjens Dekker 2020] or MECS [Sperberg-McQueen and Huitfeldt 2008]) can source the necessary information via Loki if desired.^[12]

Suspend and resume markup work similarly when the suspend option is enabled in Loki. It generates distinct SAX events, and a suspended element stays in the open-element stack (flagged as suspended when it is). The markup takes the form below. As with olist this requires very little code in Loki, and seems (at least to me) a fairly intuitive increment beyond XML syntax (an error is raised in cases like suspending an already suspended or closed element):

  <q>....<-q>...<+q>...</q>

In this and the prior case there could in principle be multiple q elements open and one might wish to operate on one other than the innermost. For this reason an endTagId option enables ID attributes to be repeated on suspend, resume, and/or end tags to co-identify the correct element if more than one of the same type is open (if not specified, the most recently encountered is closed):

 <p><q id="G">Say unto my people,
    <q id="J">....<-q id="G">...<+q id="G">...</q id="G">...</q id="J">...</p>

This feature can also be used with end tags in general, to co-identify the boundaries of very large elements for easier readability and more specific error reporting. If an end tag’s name matches the current element but it has an ID attribute that doesn’t, an error is raised that can specifically say where the intended (that is, matching) start tag was (or that it isn’t there at all). No other attributes are allowed in suspend, resume, or end tags. The implementation is a near-trivial re-use of the attribute-parsing code already there for starttags.

Also related to helping keep track of large element scopes, I am considering experimenting with Loki options reminiscent of SGML RANK. Many people have experienced trying to tag recursive structures right when the tags at all levels are identical (say, div/div/div instead of div1/div2/div3 or chap/sec/subsec), especially when converting from applications or data for which there are no real sections, but only headings (say, h1/h2/h3). I find it much easier to debug when something explicit relates the starts to their corresponding ends. Many schemas do this by providing different element types for each level; but it is easy to support the general notion at the syntactic rather than lexical level. Some obvious syntax possibilities are shown below; another is simply to permit a numeric suffix, which if present must match the depth:

<div@3>
<div x:level=3>
<div><?level 3?>

In semantic (not syntactic) homage to some LMNL concerns, simultaneous opens and closes are supported via the simultaneous option, using markup as shown below. This is one possible way to represent more than one element with the identical scope:^[13]

<b|i>.....</b|/i>

For the b/i case it is of course easier to create a portmanteau bi element; but the problem is far more general (e.g., <foreign+term>).

Milestone markup needs no extensions for parsing per se, though the DOM and validation implications are more complex. In the short term Yggdrasil will likely add virtual element support as mentioned earlier, leveraging Schemera syntax for declaring the relevant elements and attributes.

Other options

Loki makes options easy to experiment with. Several are available that recall SGML features, but never break the XML principle that the parser can produce the correct result with or without a schema. They also remain close to XML syntax for reasons already discussed:

</> — omit the name when closing the innermost element (emptyEnd).

Omit an end tag(s) immediately before another end tag (omitEnd, about which I am ambivalent) or at EOF (omitAtEOF).

<|> is not from SGML, but potentially useful. This closes the innermost element and reopens a new instance of it (the name is optional, but checked if present). Attributes (other than an ID) get the same values, but can be changed by specifying them after the |. This restart option is reminiscent of MediaWiki tables, though it’s not quite as short.

<div id=Intro> — omit quotes when an attribute value is a name or number token (unQuotedAttribute).

A booleanAttribute option enables Boolean-valued attributes (or any, really) to be set to 1 or 0 as shown below. This addresses the don’t repeat yourself awkwardness of things like border="border" (not to mention SGML’s rule that different attributes cannot share enum values):

<td +border -underline...>

There are also minor options to work around autocorrect problems sometimes introduced by uncooperative environments. For example, the parser can be instructed to recognize comments with different delimiters: em dash as equivalent to 2 hyphens (emComments), or <#...#> (poundComments) to avoid the issue altogether (nestable comments are on the possible list). For similar reasons curlyQuote permits various fancy quotes around attributes.

Such options can be helpful to XML’s paradigmatic desperate Perl hacker who has to make some documents work right now (see the prescient and still valuable Bray 2010). Note that none of these cases change existing XML syntax to have a new meaning. They intentionally use syntax that is unused in XML (that is, syntax that is not WF). A list of parser options is provided in an appendix.

Schemera and Loki

Some Loki options affect the schema directly or indirectly. One helps with the slight inconvenience of declaring defaults and types for attributes in the subset. People seem rarely to do this to get defaulting (unless they are providing a full DTD, which some parsers do not even support):

<!DOCTYPE mySchema SYSTEM "" [
<!ATTLIST p id ID #IMPLIED
    class NMTOKENS "regular">
]>

Loki can be configured to allow setting a default within the document, on the first use on a given element type (bangAttribute):^[14]

<p foo!=bar>

With bangAttrType a datatype can also be given (this would use colon in homage to Python type-hints, but that would conflict with namespacing):

<p foo!NMTOKEN=bar>

Assuming this is the first use of attribute foo on a p, it has the same effect as:

<!ATTLIST p foo NMTOKEN "bar">

There are other Loki/Schemera options to slightly extend markup declarations. For example, the SGML feature of declaring multiple element types in a single declaration (groupDcl).

Parser APIs

Both Thor and Loki have APIs almost identical to expat’s (as viewed from Python). Handlers are assigned in the same way, with the same names and arguments. However, there are a few options here too.

For example, each attribute can be returned as a separate SAX event immediately following its start tag event (saxAttribute). This has the advantage of making every event have a fixed number of arguments rather than start tag events having a potentially unbounded list of alternating names and values as some parsers produce (though not expat for Python). It also gives room to return additional information such as whether the attribute was explicit or defaulted, or the pre-normalized and normalized forms. The default is to just do what expat does for Python, which is to return a Python dictionary of the attributes.

A slight difference from expat is that neither Loki nor Thor breaks text into separate SAX events at every newline and character entity. However, that behavior can be achieved in either by setting option expatBreaks.

Although expat always hands attribute values to handlers as strings, with attrCast Loki (with Schemera’s help) can coerce them to their declared types — so a program can have them ready to go as Python ints, floats, datetimes; or as provided custom types that map directly to XSD’s builtin types.

Special character handling

The inconvenience that all special-character entities must be declared has been mentioned briefly. Sets such a those defined in Annex D of SGML and in HTML are useful and widely known, but activating them in the standard ways is tedious. Such sets can be activated in Loki merely by setting an option, either via the API:

myParser.setOption("htmlNames", 1)

<?xml version="1.1" encoding="utf-8" htmlNames="1" ?>

•
&GREEK.SMALL.LETTER.OMICRON.WITH.TONOS;

Case is ignored because Unicode character names do not mix case (this will likely be made subject to the entityFold option). Any of hyphen, underscore, or dot may be used instead of spaces (so may space itself when looking characters up via the API).^[15]

Some names are long. However, it turns out that abbreviating all but the last token down to the first 4 characters leads to only 11 collisions in the Unicode BMP. So abbreviations down as far as that also work, as do intermediate abbreviations:

&GREEK.SMAL.LETT.OMIC.WITH.TONOS;

In the rare case of a collision (which Loki will report), lengthen anywhere to disambiguate. For example, EQUIVALENT TO (U+0224d) and EQUIANGULAR TO (U+0225a) collide when fully abbreviated, but giving at least the first 5 characters of the first token succeeds: EQUIV.TO vs. EQUIA.TO. Further abbreviations are feasible, such as SMALL LETTER OMICRON to LC O or special-cases a few frequent/long substrings like CJK UNIFIED IDEOGRAPH to CJK and simply omitting LETTER; but you’ve already memorized the current rule.

I sometimes find it inconvenient that XML defines no special-character mechanism that applies inside PIs or comments. The piEscapes option makes Loki recognize and replace character references inside PIs.^[16]

Somewhat related, a piAttribute option makes Loki parse PI data as if it were an attribute list and return SAX events for PIs with the corresponding arguments (similar to start tag events). This is an obviously useful convention (XML employs it for the XML declaration, thoough it doesn’t offer the same thing to users). With this and the prior option users can avoid re-implementing some wheels. And finally, piAttrDcl enables ATTLIST declarations to constrain those PI quasi-attributes, merely by giving ? plus a PI target name where an element name would normally go (again applying Loki’s idea of XML-adjacent syntax, and trivially implemented by small adjustments to already-existing code):

<!ATTLIST ?ah hyphenate boolean #IMPLIED
              kern      float   "1.0">
...
<?ah hyphenate="1" kern="0.8"?>

Those accustomed to backslashed hexadecimal character codes can choose to enable a backslash option to recognize \n, \\, \xFF, \uFFFF, \U000FFFFF, etc. (though not \u{FFF} or \u{BULLET}, yet). As one might expect, \< and \& are then ok.

ID handling

Yggdrasil indexes IDs, with or without case and similar normalization. They do not (yet) update individually if the tree is modified (in fairness, minidom’s don’t either). The index can, however, be discarded and rebuilt without rebuilding the whole DOM.

There is a common problem of specifying just which attributes are IDs. XML thankfully reserves xml:id but many schemas assign their own. Such cases cannot be detected reliably without a schema. Ragnarok also provides for configuring what is recognized (and indexed) as IDs via the API, based on the name and namespace of the attribute and of the element it’s on. Multiple definitions and wildcards are possible.

Other ID-like semantics will likely be added. For example:

The types enable checking associated structural semantics such as that starts come before ends, milestone elements are empty, etc. The declarations must differ slightly to distinguish when the role is flagged by Loki syntax, element type, attribute name, or potentially other mechanisms.

Element declarations will have ways to declare whether they are suspendable, milestonable, olist-endable, etc. (though some of these may merely be inferred from what attribute types they declare and use).

Like attribute defaults and named special characters, IDs introduce semantics that require a schema to understand (though of course not to parse). They thus bump up against XML’s general principle that documents are correctly parsable without a DTD. This can be avoided by always using xml:id, but that’s slightly unwieldy.

XML inherits SGML’s restriction that there can be only one ID attribute for any given element type.^[17] Given that uniqueness it would be feasible to support a special syntax instead of a special attribute type. One example could be suffixing IDs to the element type: <p#p31> vs. <p xml:id="p31">. Such an IDSuffix option would save space and perhaps improve readability while avoiding the need for explicit declarations.

A longer view

I am working on support for XPointer ranges [Clark and DeRose 2002] and on DOM-integrated support for overlap features and potentially XCONCUR [Schonefeld 2008]. I and others have long lamented the lack of annotation, versioning, and collaborative editing capabilities both on the Web and in XML generally (see DeRose 2014b), for which overlap support is prerequisite.^[18] By promoting such structures into a separate but coordinated structure from the DOM itself, some of these capabilities might be brought within closer reach (see also DeRose and van Dam 1999):

I have also done some work to support mapping language and/or orthography codes to the Unicode ranges they use (an option to be called langChecking. This will check that text content comports with the xml:lang value in effect in its context. Short of that, there are already options to prohibit C0 control characters (other than cr/lf/tab), C1 control characters (which commonly arise via incorrect character-set handling), and/or private use characters.

Why try Ragnarok?

Why not try it?

Availability

The entire suite including the test suite will shortly be available on my github page at https://github.com/sderose/Ragnarok/tree/master/. Error reports, suggestions, etc. are welcome. Pythonistai (or perhaps Níðhǫggsfólk?) are also welcome to contribute.