Balisage Paper: Adventures in Single-Sourcing XQuery and XSLT

Overview

I write programs to make art using XQuery and XSLT. The XQuery generates a domain-specific XML output, which the XSLT transforms to SVG for rendering. To support this work I have created extensive function libraries for geometric operations, tiling, curve generation, random number generation, image and colour manipulations and much more. These libraries are written in XQuery.

However, there are several reasons to want the same functions available in XSLT:

Keeping the domain specific XML more abstract and pushing the details of rendering to the XSLT makes it easier to experiment with alternative presentations and designs. For example, a stroke can be rendered with a different "brush", turning a simple line into a more complex set of SVG objects. These brushes can be quite complex, involving extensive geometric calculations and randomizations. The requirement here is simply to be able to import the XQuery functions: translation to XSLT is not required. There are non-standard and rather trivial mechanisms to do this, although with a large collection of function libraries that is still a fair amount of repetitive work. The standard way to make XQuery functions available in XSLT involves extensive and tedious binding definitions.

Part of the goal of the function libraries is to share a variety of useful capabilities with the XML community. The fact is that there are many more people using XSLT in the world than XQuery. To the extent the collection of function libraries is useful, and worth sharing, it is more useful and shared more broadly as XSLT rather than XQuery. Importing into XSLT might be enough, except many processors do not support that, or support it only as a paid option. Making the libraries available to XSLT users who are unable or unwilling to pay the extra requires translation.

Finally, I wanted to experiment with interactive art in the browser without having to recode everything in a language I'd prefer not to use, and Saxon-JS looked to provide a vehicle for this. However, Saxon-JS only supports pure XSLT applications: packages that import XQuery modules are not supported. Translation of the XQuery libraries to XSLT packages is therefore a prerequisite.

There is also a fundamental requirement here: I am lazy and impatient. I don't want to have to type the same thing over and over again. I have a lot of existing code and I don't want to stop making art for months just to convert it all over. I want a system or tool that is easy to implement and simplifies the process enough that I will stick with it going forwards while meeting my other goals. There is ongoing development of the libraries as well as the creation of new libraries, and that development happens in XQuery. By the same token, I am not too concerned about perfecting the tool for every case: I am happy to have the tool do most of the work and finish the job in Emacs.

What follows is a description of the various approaches I took to making my XQuery functions available in XSLT: first using bindings to import them as-is, and then performing conversions into XSLT. I am not claiming all these approaches were good approaches. Indeed: simple string parsing is not a great approach at all, even if it proved a surprisingly effective one.

Calling XQuery Functions from XSLT

The truly easy way to make XQuery functions available in XSLT is decidedly non-standard: saxon:import-query instruction (which requires Saxon-PE or Saxon-EE) or its equivalent (e.g. MarkLogic's xdmp:import-module).

To generate a binding for an XQuery module is a fairly simple bit of scripting.

declare variable $loc as xs:string external;
declare variable $prefix as xs:string external;

let $query := unparsed-text($loc)=>normalize-space()
let $namespace :=
  $query=>
    substring-after("module namespace ")=>
    substring-after("=")=>
    substring-before(";")=>
    replace("['""]", "")
return 
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
                xmlns:saxon="http://saxon.sf.net/"
                exclude-result-prefixes="saxon {$prefix}"
                version="3.0">{
  namespace {$prefix} {$namespace},
  <saxon:import-query namespace="{$namespace}" href="{$loc}"/>
}</xsl:stylesheet>
      

This wrapper transform is given the location of the XQuery module and the preferred XSLT prefix to use^[1], and generates a small stub stylesheet by plucking out the module namespace. That stub stylesheet that can then be imported, providing access to the functions and variables in the XQuery library.

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
                xmlns:edge="http://mathling.com/geometric/edge"
                xmlns:saxon="http://saxon.sf.net/"
                exclude-result-prefixes="saxon edge"
                version="3.0">
  <saxon:import-query namespace="http://mathling.com/geometric/edge"
                      href="../geo/edge.xqy"/>
</xsl:stylesheet>
      

To create the binding transform in a standard way is a little more involved. The stub transform binds the results of the function load-xquery-module() to a variable. This is a map with information about the public functions and variables. The script needs to then generate XSL functions that use this information to call the appropriate function or return the appropriate variable value. Figure 3 shows what the bindings look like, once generated.

<xsl:stylesheet xmlns:map="http://www.w3.org/2005/xpath-functions/map"
                xmlns:svg="http://www.w3.org/2000/svg"
                xmlns:util="http://mathling.com/core/utilities"
                xmlns:xs="http://www.w3.org/2001/XMLSchema"
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                exclude-result-prefixes="util xs map"
                version="3.0">

   <xsl:variable name="mod.util" as="map(*)"
     select="load-xquery-module('http://mathling.com/core/utilities',
           map {'location-hints':'../core/utilities.xqy'})"
   />

   <xsl:variable name="util:PRIMES100" as="xs:integer*"
     select="$mod.util('variables')(
       QName('http://mathling.com/core/utilities','PRIMES100')
     )"
   />
   ...
   <xsl:variable name="util:is-prime" as="function(*)">
      <xsl:sequence select="
        $mod.util('functions')(
          QName('http://mathling.com/core/utilities','is-prime')
        )(1)"
      />
   </xsl:variable>
   <xsl:function name="util:is-prime" as="xs:boolean">
      <xsl:param name="i" as="xs:integer"/>
      <xsl:sequence select="$util:is-prime($i)"/>
   </xsl:function>

   <xsl:variable name="util:logK" as="function(*)">
      <xsl:sequence select="
        $mod.util('functions')(
          QName('http://mathling.com/core/utilities','logK')
        )(2)"
      />
   </xsl:variable>
   <xsl:function name="util:logK" as="xs:double">
      <xsl:param name="x" as="xs:double"/>
      <xsl:param name="k" as="xs:double"/>
      <xsl:sequence select="$util:logK($x,$k)"/>
   </xsl:function>
   ...
</xsl:stylesheet>
      

The variables are not actually needed here and could be collapsed into the body of the function. I chose to keep the mechanics of function lookup out of the function bodies themselves.

This is a substantially more involved bit of scripting. To accomplish it, there are several problems we need to address:

I have taken two distinct approaches to solving these problems, which I'll get to in section “Approaches”.

Translating XQuery Function Modules to XSLT

There are a number of papers on translating XSLT to XQuery. Bettentrupp06 and Fokoue05 direct most of their energy to determining the way to translate the template selection mechanism of XSLT, which is obviously of no concern when translating function libraries in the other direction. Laga06 gives a nice side-by-side summary of how XSLT 2.0 constructs can be mapped into XQuery 1.0. It uses templates applied using some kind of ad hoc implementation.

When mapping from XSLT to XQuery there are issues about replicating the template matching paradigm, issues about grouping constructs (for XQuery 1.0 as the target, at least), and issues around contextual features of XSLT like tunnel parameters and rules around precedence. Managing the syntax of XSLT is not problematic: it is XML and therefore easy to parse. The XPath expressions are subsets of XQuery expressions.

Going the other way is more syntactically challenging: XQuery has a fairly large and complex grammar, with a few subtleties around the use of non-reserved keywords and white space. My problem is more constrained than a general translation problem as well: I only wish to translate function libraries. Papers here are thin on the ground. Lechner02 is based on working drafts of the XQuery 1.0 specification and of XSLT 2.0, although the same principles hold. The approach is to do a complete translation of XQuery into XSLT using a mapping from an intermediate abstract syntax tree. A great deal of the effort centers around the problems of assigning nodes to variables, which is not an issue in XSLT 3.0. I also found some very incomplete references to papers but not, in fact, the papers themselves.

The target here is XSLT 3.0 packages. For the most part, therefore, we can wrap the body of the XQuery function inside the select attribute of xsl:sequence instruction to define the XSLT implementation. There are some XQuery constructs in the body that need to be translated, as we'll see in section “Difficulties”, but it is a good first step.

<xsl:package xmlns='http://www.w3.org/1999/xhtml' 
  xmlns:array='http://www.w3.org/2005/xpath-functions/array'
  xmlns:art='http://mathling.com/art'
  xmlns:callable='http://mathling.com/core/callable'
  xmlns:config='http://mathling.com/core/config'
  xmlns:errors='http://mathling.com/core/errors'
  xmlns:map='http://www.w3.org/2005/xpath-functions/map'
  xmlns:math='http://www.w3.org/2005/xpath-functions/math'
  xmlns:this='http://mathling.com/core/utilities'
  xmlns:xsl='http://www.w3.org/1999/XSL/Transform'
  name='http://mathling.com/core/utilities'
  version='3.0'
  exclude-result-prefixes='this array math map callable config errors'>

   <xsl:expose component='variable' names='this:*' visibility='public'/>
   <xsl:expose component='function' names='this:*' visibility='public'/>

   <xsl:use-package name='http://mathling.com/core/callable' package-version='*'/>
   <xsl:use-package name='http://mathling.com/core/config' package-version='*'/>
   <xsl:use-package name='http://mathling.com/core/errors' package-version='*'/>

   <!--
    : The first 100 primes
    -->
   <xsl:variable name='this:PRIMES100' as='xs:integer*'>
      <xsl:sequence select='
      (2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47,
      53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107,
      109, 113, 127, 131, 137, 139, 149, 151, 157, 163, 167,
      173, 179, 181, 191, 193, 197, 199, 211, 223, 227, 229,
      233, 239, 241, 251, 257, 263, 269, 271, 277, 281, 283,
      293, 307, 311, 313, 317, 331, 337, 347, 349, 353, 359,
      367, 373, 379, 383, 389, 397, 401, 409, 419, 421, 431,
      433, 439, 443, 449, 457, 461, 463, 467, 479, 487, 491,
      499, 503, 509, 521, 523, 541)'/>
   </xsl:variable>
   ...
   <!--
    : is-prime()
    : Is the number a prime?
    :
    : @param: $i: positive integer
    : @return: whether it is prime 
    -->
   <xsl:function name='this:is-prime' as='xs:boolean'>
      <xsl:param name='i' as='xs:integer'/>
      <xsl:sequence select='
      if ($i &lt; 6000) then $i = $this:CANVAS-PRIMES
      else (
        every $x in 2 to this:round(math:sqrt($i))
        satisfies $i mod $x != 0
      )
      '/>
   </xsl:function>

   <!--
    : logK()
    : Log base k of x: logk(x) = log(x)/log(k)
    :
    : @param: $x: number
    : @param: $k: base
    : @return: log base k 
    -->
   <xsl:function name='this:logK' as='xs:double'>
      <xsl:param name='x' as='xs:double'/>
      <xsl:param name='k' as='xs:double'/>
      <xsl:sequence select='math:log($x) div math:log($k)'/>
   </xsl:function>
   ...   
</xsl:package>
      

This kind of conversion needs everything we needed for the generated function bindings, plus a few additional items:

Approaches

declare function local:import-list($module-text as xs:string) as array(*)*
{
  let $head :=
    $module-text=>
      substring-after("module namespace")=>
      substring-after(";")=>
      substring-after("import module namespace ")=>
      substring-before("declare function")
  for $block in tokenize($head, "import module namespace ", "s")[. ne ""]
  let $prefix := substring-before($block,"=")
  let $ns := $block=>substring-after('"')=>substring-before('"')
  where $prefix ne ""
  return array {$prefix, $ns}
};
        

declare function local:function-list($module-text as xs:string) as array(*)*
{
  for $block in tokenize($module-text, "declare (%[^\s]+ )?function ", "s")
  let $name := 
    $block=>
      substring-before("(")=>
      substring-after(":")
  let $body := 
    $block=>
      substring-after("{")=>
      substring-before("};")
  let $parms :=
    tokenize(
      $block=>
        substring-after($block,"(")=>
        substring-before("{"), 
      "[$]", "s"
    )[. ne ""]
  let $parameter-names := 
    for $p in $parms return (
      tokenize($p, "[\s,]+", "s")[. ne "" and . ne "("][1]
    )
  let $arity := count($names)
  where $name ne ""
  return array{$name, $body, $arity, array{$parameter-names}}
};
      

public ExtensionFunctionCall makeCallExpression() {
  return new ExtensionFunctionCall() {
    @Override
    public Sequence call(XPathContext context, Sequence[] arguments) 
      throws XPathException
    {
      FunctionItemType fn =
        ((UserFunctionReference.BoundUserFunction)arguments[0]).
          getFunctionItemType();
      SequenceType seqType = fn.getResultType();
      return StringValue.makeStringValue(seqType.toExportString());
    }
  };
}
        

declare function local:guess-type($value as item()*) as xs:string
{
  typeswitch($value)
  case xs:double return "xs:double"
  case xs:double* return "xs:double*"
  case xs:integer return "xs:integer"
  case xs:integer* return "xs:integer*"
  case xs:numeric return "xs:numeric"
  case xs:numeric* return "xs:numeric*"
  case xs:string return "xs:string"
  case xs:string* return "xs:string*"
  case xs:boolean return "xs:boolean" 
  case xs:boolean* return "xs:boolean"
  case xs:dateTime return "xs:dateTime" 
  case xs:dateTime* return "xs:dateTime"
  case map(xs:string,item()*) return "map(xs:string,item()*)"
  case map(xs:string,item()*)* return "map(xs:string,item()*)*"
  case map(*) return "map(*)"
  case map(*)* return "map(*)*"
  case function(*) return "function(*)"
  case function(*)* return "function(*)*"
  default return "item()*"
};
        

<xqdoc:xqdoc xmlns:xqdoc="http://www.xqdoc.org/1.0">
  ...
  <xqdoc:module type="library">
    <xqdoc:uri>http://mathling.com/core/utilities</xqdoc:uri>
    <xqdoc:name>this</xqdoc:name>
    <xqdoc:comment end="210" start="23">
      <xqdoc:description>
 Module with functions providing some basic utility operations.</xqdoc:description>
      <xqdoc:since>March 2021</xqdoc:since>
    </xqdoc:comment>
    <xqdoc:body end="75215" start="1" xml:space="preserve">...</xqdoc:body>
  </xqdoc:module>
  ...
</xqdoc:xqdoc>
        

  <xqdoc:imports>
    <xqdoc:import location="../core/callable.xqy" prefix="callable" type="library">
      <xqdoc:uri>http://mathling.com/core/callable</xqdoc:uri>
      <xqdoc:body end="568" start="468" xml:space="preserve">import module namespace callable="http://mathling.com/core/callable"
      at "../core/callable.xqy"</xqdoc:body>
    </xqdoc:import>
  </xqdoc:imports>
        

  <xqdoc:namespaces>
    <xqdoc:namespace prefix="art" uri="http://mathling.com/art"/>
    <xqdoc:namespace prefix="array" uri="http://www.w3.org/2005/xpath-functions/array"/>
    <xqdoc:namespace prefix="math" uri="http://www.w3.org/2005/xpath-functions/math"/>
    <xqdoc:namespace prefix="map" uri="http://www.w3.org/2005/xpath-functions/map"/>
  </xqdoc:namespaces>
        

  <xqdoc:variables>
    <xqdoc:variable>
      <xqdoc:uri>http://mathling.com/core/utilities</xqdoc:uri>
      <xqdoc:name>PRIMES100</xqdoc:name>
      <xqdoc:comment end="1038" start="1008">
        <xqdoc:description>
 The first 100 primes</xqdoc:description>
      </xqdoc:comment>
      <xqdoc:type occurrence="*">xs:integer</xqdoc:type>
      <xqdoc:body end="1565" start="1040" xml:space="preserve">declare variable $this:PRIMES100 as xs:integer* :=
(
  2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109, 113, 127, 131, 137, 139, 149, 151, 157, 163, 167, 173, 179, 181, 191, 193, 197, 199, 211, 223, 227, 229, 233, 239, 241, 251, 257, 263, 269, 271, 277, 281, 283, 293, 307, 311, 313, 317, 331, 337, 347, 349, 353, 359, 367, 373, 379, 383, 389, 397, 401, 409, 419, 421, 431, 433, 439, 443, 449, 457, 461, 463, 467, 479, 487, 491, 499, 503, 509, 521, 523, 541
)</xqdoc:body>
    </xqdoc:variable>
    ...
  </xqdoc:variables>
        

  <xqdoc:functions>
    <xqdoc:function>
      <xqdoc:comment end="6892" start="6781">
        <xqdoc:description>
 is-prime()
 Is the number a prime?</xqdoc:description>
        <xqdoc:param>$i: positive integer</xqdoc:param>
        <xqdoc:return>whether it is prime</xqdoc:return>
      </xqdoc:comment>
      <xqdoc:name>is-prime</xqdoc:name>
      <xqdoc:signature>declare function is-prime($i as xs:integer) as xs:boolean</xqdoc:signature>
      <xqdoc:parameters>
        <xqdoc:parameter>
          <xqdoc:name>i</xqdoc:name>
          <xqdoc:type>xs:integer</xqdoc:type>
        </xqdoc:parameter>
      </xqdoc:parameters>
      <xqdoc:return>
        <xqdoc:type>xs:boolean</xqdoc:type>
      </xqdoc:return>
      <xqdoc:invoked>
        <xqdoc:uri>http://mathling.com/core/utilities</xqdoc:uri>
        <xqdoc:prefix>this</xqdoc:prefix>
        <xqdoc:name>round</xqdoc:name>
      </xqdoc:invoked>
      ...  
      <xqdoc:body end="7140" start="6894" xml:space="preserve">declare function this:is-prime($i as xs:integer) as xs:boolean
{
  if ($i < 6000) then $i = $this:CANVAS-PRIMES
  else (
    every $x in 2 to this:round(math:sqrt($i))
    satisfies $i mod $x != 0
  )
}</xqdoc:body>
    </xqdoc:function>
    <xqdoc:function>
      <xqdoc:comment end="7269" start="7144">
        <xqdoc:description>
 logK()
 Log base k of x: logk(x) = log(x)/log(k)</xqdoc:description>
        <xqdoc:param>$x: number</xqdoc:param>
        <xqdoc:param>$k: base</xqdoc:param>
        <xqdoc:return>log base k</xqdoc:return>
      </xqdoc:comment>
      <xqdoc:name>logK</xqdoc:name>
      <xqdoc:signature>declare function logK($x as xs:double, $k as xs:double) as xs:double</xqdoc:signature>
      <xqdoc:parameters>
        <xqdoc:parameter>
          <xqdoc:name>x</xqdoc:name>
          <xqdoc:type>xs:double</xqdoc:type>
        </xqdoc:parameter>
        <xqdoc:parameter>
          <xqdoc:name>k</xqdoc:name>
          <xqdoc:type>xs:double</xqdoc:type>
        </xqdoc:parameter>
      </xqdoc:parameters>
      <xqdoc:return>
        <xqdoc:type>xs:double</xqdoc:type>
      </xqdoc:return>
      <xqdoc:invoked>
        <xqdoc:uri>http://marklogic.com/xdmp/math</xqdoc:uri>
        <xqdoc:prefix>math</xqdoc:prefix>
        <xqdoc:name>log</xqdoc:name>
      </xqdoc:invoked>
      <xqdoc:body end="7379" start="7271" xml:space="preserve">declare function this:logK($x as xs:double, $k as xs:double) as xs:double
{
  math:log($x) div math:log($k)
}</xqdoc:body>
    </xqdoc:function>
    ...
  </xqdoc:functions>
      

Difficulties

Although the transform goes a long way towards producing an equivalent XSLT package from an XQuery function library module, the sad truth is that some manual editing is still required after the translation step. Some changes are cosmetic, but make the results more readable. Some are more substantial and stem from the key differences between XQuery expressions and XPath expressions allowable in an xsl:sequence select. Some alterations are quite mechanical in nature: others require significant reworking of the code, alas.

Here are some of the key issues:

First up is the cosmetic issue of having the function bodies rendered with a lot of needless entities. Since in my code I tend to use the double quotation mark for string delimiters and the arrow syntax for a lot of function calls, the code looks quite ugly with default output settings. Furthermore, all the indentation is lost with 
 character entities replacing newlines. A character map for > and 
 fixes most of this, and the use of the Saxon serialization parameter saxon:single-quotes takes care of the rest. Having made that fix, that much less post-processing is required.

The second post-processing issue stems from the difference between XQuery FLWOR expressions and XPath 3.1 for and let expressions. This manifests in several ways, all of which I take care of via manual editing.

let $k := ($b - $a) div 3 
let $ts := tail(this:linspace(4, $a, $b, true()))
return (
  $k * (2 * $f($ts[1]) - $f($ts[2]) + 2*$f($ts[3]))
)
      

let $k := ($b - $a) div 3 return
let $ts := tail(this:linspace(4, $a, $b, true()))
return (
  $k * (2 * $f($ts[1]) - $f($ts[2]) + 2*$f($ts[3]))
)
      

sum(
  for $bit at $i in $bits
  return $this:MULTIPLIERS64[$i + $offset] * $bit
) cast as xs:integer
      

sum(
  for $i in 1 to count($bits)
  return $this:MULTIPLIERS64[$i + $offset]*$bits[$i]
) cast as xs:integer
      

for $datum in $data
order by this:size($datum) descending
return $datum
      

for $datum in sort($data, (), function($d) {-this:size($d)})
return $datum
      

for $j in 1 to $N
where ($j idiv math:pow(2, $j - 1) mod 2 = 1)
return $input[$j]
      

for $j in 1 to $N
return
  if ($j idiv math:pow(2, $j - 1) mod 2 = 1)
  then $input[$j]
  else ()
      

let $triangles as map(xs:string,item()*) := this:triangle-mesh($granularity, $n)
      

let $triangles := this:triangle-mesh($granularity, $n)
      

These are fairly mechanical modifications and do not require a great deal of thought or effort. The lack of switch and typeswitch statements is only slightly more tiresome. These need to be converted to conditionals with either equality comparisons or the use of the instance of operator. It is also usually wise to add a let variable. In some cases the determination of what variable to use may need a little thought and analysis.

Often there are multiple ways to convert a problematic construct. Which approach to take is largely a matter of taste and convenience. Adding return to handle a sequence of let clauses is simpler in my case than converting to a list on a single let clause, because it it just repeatedly pasting in the same value (one keystroke in my editor) and makes for easier cross-comparisons. I also prefer the stronger syntactic signal.

switch(this:kind($region))
case "polygon" return path:length($region)
case "ellipse" return ellipse:perimeter($region)
case "quad" return edge:length($region)
case "cubic" return edge:length($region)
case "edge" return edge:length($region)
case "slot" return
  this:length($region=>slot:body())
default return 0
      

let $kind := this:kind($region) return
if ($kind="polygon") then path:length($region)
else if ($kind="ellipse") then ellipse:perimeter($region)
else if ($kind="quad") then edge:length($region)
else if ($kind="cubic") then edge:length($region)
else if ($kind="edge") then edge:length($region)
else if ($kind="slot") then
  this:length($region=>slot:body())
else 0
      

The remaining issues require a great deal more effort and thought. Node construction and try/catch expressions cannot be translated within the context of XPath and need to be converted to XSLT equivalents instead. Often that means pulling apart the whole expression to re-express it in XSLT terms. Consider the example below: as you can see, everything needs to be converted to XSLT. It is quite an involved undertaking. Fortunately node construction plays a relatively limited role in my function libraries and try/catch expressions are only used in a handful of places.

for $node in $nodes return
typeswitch ($node)
case element(svg:feDisplacementMap) return
  element {node-name($node)} {
    $node/(@* except (@scale)),
    attribute scale {
      util:decimal($node/@scale * $rescale, 1)
    },
    this:replace-scale($node/*, $rescale)
  }
case element() return
  element {node-name($node)} {
    $node/@*,
    this:replace-scale($node/*, $rescale)
  }
default return $node
      

<xsl:for-each select="$nodes">
  <xsl:variable name="node" select="."/>
  <xsl:choose>
  <xsl:when test='
      $node instance of element(svg:feDisplacementMap)'
    >
      <xsl:element name="{node-name($node)}">
        <xsl:copy-of select="$node/(@* except @scale)"/>
        <xsl:attribute name="scale" select='
          util:decimal($node/@scale * $rescale, 1)'/>
        <xsl:sequence select='
          this:replace-scale($node/*, $rescale)'/>
      </xsl:element>
    </xsl:when>
    <xsl:when test='$node instance of element()'>
      <xsl:element name="{node-name($node)}">
        <xsl:copy-of select="$node/@*"/>
        <xsl:sequence select='
          this:replace-scale($node/*, $rescale)'/>
      </xsl:element>
    </xsl:when>
    <xsl:otherwise>
      <xsl:copy-of select="$node"/>
    </xsl:otherwise>
  </xsl:choose>
</xsl:for-each>
      

The observant reader here notes the previous example shows a recursive transformation, which is what XSLT is designed to do. A more idiomatic translation is obviously possible. There are definitely tensions here between what is convenient for an initial conversion, what is idiomatic, and what is easier to maintain. I have opted for less idiomatic but easier to maintain in the two forms.

The final difficulty poses conceptual problems as well as technical ones. I categorize named functions using annotations for documentation purposes. For example, all base randomization distribution functions are marked with the annotation %art:distribution. Such annotations can be transferred over to XSLT functions as attributes and serve essentially the same purpose. Other annotations, such as %saxon:memo-function provide performance hints to the processor. These can be translated to the corresponding mechanism in XSLT (if there is one).

Annotations on anonymous functions pose a more difficult problem. XPath 3.1 allows for inline function definitions, but not for function annotations on them. However, certain libraries make heavy use of function annotations on inline functions to give them usable names for metadata capture and debugging. The only solution is a wholesale rewrite: wrapping the function items in a map along with the necessary annotations. Unfortunately that changes the API for the callers, so a rewrite there is also necessary.^[3]

declare function this:sdCappedTorus(
  $angle as xs:double,
  $ra as xs:double,
  $rb as xs:double
) as function(xs:double*) as xs:double*
{
  let $sc := (
    math:sin(math:radians($angle div 2)), 
    math:cos(math:radians($angle div 2))
  )
  let $ra2 := $ra*$ra
  return 
    %art:name("sd3:sdCappedTorus") 
    function($point as xs:double*) as xs:double* {
      let $p := (abs(v:px($point)), tail($point)) 
      let $pxy := (v:px($p), v:pz($p))
      let $k :=
        if (v:determinant($pxy,$sc) > 0)
        then v:dot($pxy, $sc)
        else v:magnitude($pxy)
      return 
        math:sqrt(v:dot($p,$p) + $ra2 - 2.0*$ra*$k) - $rb
    }
};
...
let $sdf := sd3:sdCappedTorus(60, 0.1, 0.4)
return (util:function-name($sdf)||"="||$sdf((0.1, 0.2)))
      

declare function this:sdCappedTorus(
  $angle as xs:double,
  $ra as xs:double,
  $rb as xs:double
) as map(*)
{
  let $sc := (
    math:sin(util:radians($angle div 2)), 
    math:cos(util:radians($angle div 2))
  )
  let $ra2 := $ra*$ra
  return 
    callable:named("sd3:sdCappedTorus", 
    function($point as xs:double*) as xs:double* {
      let $p := (abs(v:px($point)), tail($point)) 
      let $pxy := (v:px($p), v:pz($p))
      let $k :=
        if (v:determinant($pxy,$sc) > 0)
        then v:dot($pxy, $sc)
        else v:magnitude($pxy)
      return 
        math:sqrt(v:dot($p,$p) + $ra2 - 2.0*$ra*$k) - $rb
    })
};
...
let $sdf := sd3:sdCappedTorus(60, 0.1, 0.4)
  =>callable:function()
return (util:function-name($sdf)||"="||$sdf((0.1, 0.2)))
      

This can then be translated into XSLT in the usual way.

A Word on Performance

It is interesting to consider the relative performance of the XQuery functions versus the XSLT functions, and indeed the relative performance of the compiled XSLT functions used in Saxon-JS.

I have not done a detailed performance study, so these results should be taken as suggestive rather than definitive. The numbers were obtained with a simple Unix time command and using Saxon 11.4 or Saxon-JS 2.5 as the processor. The Saxon-JS numbers were run under node.js 16.18.1 over files compiled with Saxon 11.4. All times in seconds unless indicated. These are timings on the running of various unit tests that had been translated into all three contexts. Since Saxon-JS doesn't have a -repeat option and, frankly, because it takes too long to run anyway, these were run just once so Java warm-up time is part of the measurement.

The XQuery and XSLT performance numbers are fairly similar, with the XSLT times generally running about ten percent slower. The Saxon-JS numbers are, frankly, baffling. Some tests run much more quickly, probably because the time to parse and prepare the XQuery or XSLT has been accounted for when we compiled to JSON and the Javascript runtime initializes more quickly than the Java runtime. On the other hand, some of the times are much slower, unusably so. The bulk of the time is taken up in signed distance and certain geometric calculations (particularly intersections and interpolations), both of which do a lot of floating point mathematics and function calls. Still, there is no obvious reason for the numbers to be quite this bad, and it bears further investigation.^[5]

Conclusions and Summary

We have examined several approaches to single-sourcing XQuery and XSLT function libraries starting with an XQuery base.

I was working with Saxon as both my tool and my target. Other alternatives are available with some other processors such as MarkLogic or BaseX that have introspection capabilities. For example, the BaseX inspection module has functions that return structured information about a module either in XQDoc format or its own internal format, the list of functions in a module, structured information about a specific function (including the attached XQDoc), the type of a value, and so forth. The xquery:parse() function could be used to get the detailed parse tree of function bodies for more complete automation instead of modifying the XQDoc processor.

Maintaining the parallel versions and running tests in all three environments (XQuery, XSLT, node.js) has had a quite unexpected benefit of improving the code. Part of this was just the extra proofreading that happens during the fix-up stage: every line of code is examined twice, once to create it and once to translate it. You notice things. In addition, the translation steps essentially introduced compilers into the mix, which have to check every line of code soon after it is written. In an ideal world unit tests would accomplish the same thing, but the truth is, something like "grind on an image for an hour" is not something I choose to have a unit test for, and something like "create a random octopus" is not something particularly amenable to a unit test, so not every module gets run. Bone-head typos and errors have been introduced during major refactorings. In addition, the different processors have slightly different quirks and assumptions, and fixing code to work properly in all of them makes it better for any of them.

With these techniques I have been able to realize my goal of creating XQuery libraries and using them in XSLT and translating them into XSLT as well. Automatic binding generation takes just a few seconds, and is incorporated as part of the build system. Converting a new module to XSLT can generally be accomplished in a minute or so, manual post-processing and all, which is sufficient for my needs. Automatic baselining and comparison has been incorporated into the build and release scripts, piggy-backing off the documentation generation, and has proved manageable. Code overall has improved.