Introduction

Javascript does not just enable applications to perform asynchronous operations, it pretty well requires it. In the browser, a synchronous request to fetch resources from the server will freeze the screen and prevent user interaction until the response is received. In Node.js, a web server operates as a single thread, and synchronous I/O operations will freeze the entire web server and prevent it accepting new incoming requests. In both environments, therefore, synchronous I/O kills performance.

This is not just a feature of the Javascript language, it is a feature of the processing model assumed by the Javascript platform, one that affects any code running on that platform, regardless of programming language.

Saxon-JS implements XSLT on the Javascript platform, both client-side and server-side [Lockett & Kay 2016], and the requirement for asynchrony poses a unique challenge. How can a function like doc() work asynchronously if asynchrony isn't built into the XSLT language, as it is with Javascript itself? Saxon-JS 1.0 addressed this with a new instruction, ixsl:schedule-action, that executes code asynchronously. But it has limitations: once you've forked an asynchronous operation, there's no way of communicating between the two threads or combining their results, except by the rather awkward and unsafe mechanism of updating a mutable document — the HTML page.

With Saxon-JS 2.0, we have had to think about asynchrony on the server as well as the browser, and this is even more challenging, because there's a strong user expectation that XPath functions like doc(), which read and parse an XML document given its URI, will simply work "out of the box". There's also no convenient mutable document in which to assemble the results of multiple asynchronous processes. Saxon-JS 2.0 as released takes some baby steps towards solving the problem, but the development has given us a good opportunity to come up with ideas for a more ambitious solution in the future.

This paper considers how to incorporate asynchrony into the XSLT language, with particular references to the challenges posed by the Javascript environment, but at the same time with an eye to making the solution general-purpose.

It is organised as follows:

  • Section 2 considers the requirements we are trying to address.

  • Section 3 examines previous work in this area, including the facilites offered in the current Saxon-JS product.

  • Section 4 proposes a solution at the level of XSLT and XPath language changes.

  • Section 5 discusses implementation issues.

  • Section 6 analyses the proposed solution, examining its strengths and limitations and suggesting avenues for further work.

Requirements

I'm addressing this problem from the perspective of the Javascript platform: both client-side environments (the web browser), and server-side processing (typified by Node.js).

There are some particular constraints imposed in this environment:

  • For good performance, operations that require network or disk access need to be asynchronous. Other processes need to be given a chance to run while you're waiting for such requests to complete.

  • You can't simply issue a wait request and go to sleep while an asynchronous request is processed; that would require the entire process state to be saved somewhere, and brought back into memory when processing resumes. Rather, the Javascript model is that when you issue the request for a slow operation, you provide some kind of "callback" code to be executed on completion, and this callback code must encapsulate in its closure all the state that is needed to continue.

  • In the browser environment specifically, asynchrony is needed not only to handle external resource requests, but also to enable an interactive user experience. Interactivity implies an incrementally changing external state (specifically, the state of the HTML page), and this means that the requirement for asynchrony is intimately tied up with requirements for mutability.

There are other environments that don't have quite the same constraints, and we would certainly want any solution to be general enough to operate under different processing models from this one. But the Javascript execution model is our current focus.

Saxonica's first foray into XSLT execution in the Javascript world was Saxon-CE [Delpratt & Kay 2013], built by cross-compilation from Java source code, and running in the browser only. The main innovation in Saxon-CE to meet this requirement was the ixsl:schedule-action instruction, and this has been retained, with incremental enhancements, in successor products, of which the latest incarnation is Saxon-JS 2: the current specification is documented at [http://saxonica.com/saxon-js/documentation/index.html#!ixsl-extension/instructions/schedule-action].

The ixsl:schedule-action instruction has a number of limitations, which the current proposal aims to remove:

  • The set of asynchronous requests that can be made is limited and not easily extensible; it is currently limited to requests for documents and files, HTTP requests, and simple timer events.

  • The code that is executed on completion of an asynchronous request is allowed to have side effects (it can make in-situ updates to the HTML page), but it cannot contribute to the principal result of the transformation (which in most cases will be delivered long before the asynchronous request has been satisfied).

  • The syntax is clumsy: and in particular, the only way that the current processing state at the time of the request can be made available to the continuation code executed when the request completes is by explicit parameter passing using xsl:with-param instructions.

Previous Work

An excellent survey of approaches to asynchronous processing, both in the XSLT/XQuery world, and in programming languages more generally, can be found in [Lockett & Retter 2019], and I will not repeat that survey here.

Many programming languages, and Javascript in particular, have settled on the concept of a Promise as a vehicle for managing asynchronous operation, and I shall follow their lead.

Two detailed proposals for adding promises to the XSLT/XQuery programming model have appeared in the literature: [Wright 2016] and [Lockett & Retter 2019].

  • In Wright's xq.promise model, a promise is created using a function call such as p:defer(function, arguments, callback). Here neither the function nor the callback are intrinsically asynchronous. The emphasis in xq.promise is on achieving parallel execution of synchronous tasks, rather than on tasks that are intrinsically asynchronous because they depend on external events.

    To execute the fn:doc() function asynchronously, xq.promise would do p:defer(fn:doc#1, $uri, $onCompletion), where fn:doc#1 is a regular synchronous function, executed in another thread. By contrast, we're working in an environment where we want fn:doc to be an asynchronous function: there is no "thread" that is "waiting". In our model, we could implement p:defer as something like a promise to execute a synchronous function, but that isn't enough; we need asynchronous functions too.

    The promise that p:defer() returns is a zero-arity function, which is invoked to deliver its payload. This model relies on the initiating thread being able to wait until a task that it spawned has finished. In the Javascript concurrency model, this isn't possible; the only thing the initiating thread can do is to nominate another function to be executed when the spawned task has finished.

  • Lockett and Retter's EXPath Tasks model attempts to address the issue of mutability (with a solution based on Haskell's monads) at the same time as tackling asynchrony. The Task concept proposed by Lockett and Retter is something that may both have side-effects on the outside world, and also be executed asynchronously. The focus in this paper, however, is solely on asynchrony, and I believe the two problems are separable, despite the fact that order-of-execution semantics plays a significant role in both.

    Whereas xq.promise models a promise as an XDM function, EXPath Tasks models it as an XDM map. The only real benefit of using maps is the syntactic convenience of using functions local to an object which can be addressed using the "?" operator, for example $task?then($next-task). A disadvantage is that the type signature (map(xs:string, function(*))) conveys nothing about the semantics; this disadvantage could be addressed by using tuple types and type aliases [Kay2020], but still leaves the problem that as far as the type system is concerned, Task objects can be used in places where they are unlikely to be understood.

Proposal

This section proposes changes to the XSLT and XPath languages designed to address the requirements. Notes providing rationale for the proposals are included in italics.

Promises

A Promise is introduced into the XDM data model as a new kind of item. The ItemType syntax is extended to allow promise(X), where X is any sequence type: for example, promise(document-node()) is a promise which, when successfully fulfilled, will deliver a document node. The type promise(*) is equivalent to promise(item()*), that is, a promise that may deliver a value of any type.

In [Wright 2016], a Promise is modelled as a kind of function; in [Lockett & Retter 2019], a promise is modelled as a kind of map. I have made it a primitive item type because I don't want promises to be substitutable for anything else: I want to exploit the type checking machinery to ensure that promises can only be used in a well-defined and restricted set of circumstances.

A promise can be delivered as the result of a number of new asynchronous functions, described in the next section.

A promise can be handled by any operation that is applicable to arbitrary items: for example the comma operator can be used to construct a sequence of promises, and the head() function can be used to select the first promise in such a sequence. You can test whether an item is a promise using the instance of operator, and you can construct a map or array of promises if you find it useful.

A promise has no string value, it cannot be atomized or serialized, it has no effective boolean value, and it cannot be added to the content of a node. There is no equality operator for comparing promises; there are no match patterns that match a promise, and the built-in template rules when applied to a promise throw an error.

Nested promises are collapsed into a single promise: that is, a promise of a promise of X is treated as a promise of X. For orthogonality, however, the type syntax promise(promise(X)) is allowed.

There are a number of standard functions that apply to promises: these are in a namespace (provisionally http://expath.org/ns/promise), represented here by the prefix a: (for "asynchronous"). These functions include:

a:then(promise(X), function(X) as Y) => promise(Y)

Returns a composite promise. The call a:then($promise, $function) will apply $function to the value delivered by $promise, if and when $promise is successfully fulfilled. The result is a new promise. The supplied $function can represent either a synchronous or an asynchronous action to be applied to the result of $promise.

It will often be convenient to use this function with the arrow operator, for example a:doc("request.xml") => a:then(.{//para}). In this example I'm also using a "dot function" as proposed in [Kay2020]; using XPath 3.1 syntax this would be written a:doc("request.xml") => a:then(function($doc){$doc//para}). The effect of the example is to return a promise to (a) fetch the document with URI request.xml, and then (b) select the descendant para elements in that document.

If either $promise or $function fails with a dynamic error, then the composite promise also fails.

a:catch(promise(X), function(map(*)) as Y) => promise(Y)

Returns a composite promise, which will apply $function to the error thrown by $promise, if and when $promise fails with a dynamic error. The result is a new promise. The supplied $function can represent either a synchronous or an asynchronous action to be applied to the error thrown by $promise.

It will often be convenient to use this function with the arrow operator, for example a:doc("request.xml") => a:catch(function($err){if ($err?code eq "FODC0002") then $dummy-doc else error($err)}).

The error information is delivered to $function as a map, whose entries are the values made available in a try/catch expression, such as $err:description and $err:code. The example assumes extensions to the error() function making it easier to re-throw an error.

a:all-of( array(promise(X))) => promise(array(X))

Returns a composite promise, which will deliver an array containing the results of the supplied promises, in corresponding positions, if and when they have all been successfully fulfilled. If any of the supplied promises fails with a dynamic error, the returned promise also fails with a dynamic error.

a:any-of( array(promise(X))) => promise(X)

Returns a promise, which on completion will deliver the result of the first of the supplied promises to successfully complete. If any of the supplied promises fails with a dynamic error, the failure is ignored, unless all of the supplied promises fail, in which case the returned promise fails (the details of the final error are left implementation-defined).

a:await(promise(X)) => X

Waits until the supplied promise is successfully completed, and returns the result that it delivers; or throws a dynamic error if the supplied promise fails.

This function can only be implemented in an environment where the calling thread is able to suspend itself and wait for other threads to complete. It is therefore not available on the Javascript platform.

Asynchronous Functions

A number of asynchronous functions are added to the core function library. These are characterised by the fact that the function returns a promise as its result.

This function library includes asynchronous versions of all the core functions that fetch external resources: a:doc, a:document, a:doc-available, a:unparsed-text, a:unparsed-text-lines, a:unparsed-text-available, a:json-doc, a:collection, a:uri-collection. In each case the arguments and the semantics are the same as the existing function; the only difference is that the returned value is a promise.

Similarly, asynchronous versions of extension functions such as those in the EXPath File module [EXPath-File] and EXPath HTTP module [EXPath-HTTP] are made available, typically in another namespace.

A function a:sleep($duration, $value) is supplied, which completes after a specified time has elapsed, returning $value when it does so.

The function a:promise($value) delivers a trivial promise that completes immediately, returning $value. This is useful in a conditional when one branch is synchronous and another asynchronous. If the argument is a call on the fn:error function then the returned promise fails immediately.

User-supplied asynchronous functions may be written simply by declaring the return type as promise(X) (and actually delivering a promise as the result, of course).

Asynchronous Variables

As discussed above, in some environments it is possible to wait for a promise to be fulfilled using the function a:await, but this doesn't work in the Javascript environment.

In the Javascript environment, we will provide only one way to capture and manipulate the result of an asynchronous function (other than in deferred callback code): namely to use an asynchronous variable. For example:

<xsl:variable name="data" as="document-node()" 
      select="a:doc('data.xml')" async="yes"/>

Asynchronous variables may be either global or local. In either case, the select expression is expected to return a promise, and the value of the variable is the value delivered on successful fulfilment of this promise. (If the select expression delivers something other than a promise, then its value can be used directly; the async attribute then has no effect.) An asychronous variable therefore behaves very like an "await" instruction in Javascript.

  • An asynchronous global variable is evaluated before any code that references its value.

    One possible implementation is to precompute a dependency graph indicating the dependencies of global variables on each other (cycles are not allowed), and then to evaluate the variables in a series of phases: first the asynchronous variables that have no dependencies; then, when these are complete, the variables that depend on these, and so on. Finally, when all global variables have been evaluated, the main transformation can proceed.

  • An asynchronous local variable is evaluated before the rest of the sequence constructor that contains the variable declaration (that is, before the code in which the variable is in scope).

    An asynchronous local variable can only appear in a template that is itself declared to be asynchronous. A named template is declared asynchronous by the presence of the attribute async="yes" on the xsl:template declaration; a template rule (one with a match attribute) is declared asynchronous by the presence of async="yes" on the xsl:mode declaration. Such declarations must be consistent if the template or mode is overridden in another XSLT package, so that the value is always known when the package is compiled.

    Asynchronous local variables can only be used in code that is executed in final output state, as defined in the XSLT 3.0 specification [XSLT 3.0 section §25.2]. The reason for this restriction is explained below.

    An asynchronous local variable can be considered analogous to the await keyword in languages such as Javascript; at one level, it is syntactic sugar that causes the following instructions to execute on successful completion of the promise. There is a difference, however, in that the result of executing these instructions becomes part of the result of the sequence constructor, which means that these instructions may construct nodes that are added to the final result tree.

    This explains the reason for the restriction that this may only happen in final output state. Informally, final output state exists when instructions are writing to a final result tree of the transformation (either the principal result or a secondary result), as distinct from creating the value of a temporary variable. It follows that nothing within the transformation will ever see the results of these instructions; they only have external effect.

    A possible implementation is to ensure that in final output state, the processor is always operating in "push mode", where each instruction writes SAX-like events (startElement, endElement, etc) to a receiving destination, which might be a serializer or a tree builder. With this approach it becomes quite possible to split the execution of the sequence constructor into instructions that are executed before the resource is fetched, and instructions that are executed when the resource becomes available. The state that needs to be passed to the continuation code includes the value of the XPath dynamic context (focus and local variables), plus a handle to the receiving destination.

    Of course, if an asynchronous local variable (an implicit await) occurs in a template, then the invoking instruction (the call-template or apply-templates that invokes this template) also becomes asynchronous, and a sequence constructor containing such an instruction must itself be split into a pre-completion and post-completion part. There is an implicit "await" before such an instruction; the need for this is known at compile time, by virtue of the requirement to declare the named template or mode as asynchronous.

    The most challenging part of the implementation is probably the correct handling of an asynchronous local variable (or asynchronous xsl:call-template or xsl:apply-templates instruction) appearing in the body of a looping construct such as xsl:for-each or xsl:iterate. Such situations demand that the "continuation" object passed to the callback code should include sufficient information to resume the execution of such loops at the correct point. There are two possible strategies to achieve this: one is for the XSLT processor to construct this continuation object itself; the other is to rely on the mechanisms (such as the yield or await constructs) provided by the underlying Javascript engine. It will be necessary to experiment before deciding which approach works best.

Examples

Example 1: Displaying Data on Demand

The demonstration applications for Saxon-JS include a client-side example [cities.xsl] that loads data on demand from JSON files on demand, and updates the HTML page with a rendition of the selected data.

The critical logic of this application is shown below.

First, there is a template rule to process the event that occurs when a user selects the data to be displayed:

    <xsl:template match="span[@class='link']" mode="ixsl:onclick">
        <xsl:call-template name="loadJSON">
            <xsl:with-param name="X" select="string(@id)"/>
        </xsl:call-template>
    </xsl:template>               
                

This invokes the template:

    <xsl:template name="loadJSON">
        <xsl:param name="X" select="'A'"/>
        <ixsl:schedule-action document="{$baseURL || $X || '.json'}">
            <xsl:call-template name="show-all">
                <xsl:with-param name="X" select="$X"/>
            </xsl:call-template>
        </ixsl:schedule-action>
    </xsl:template>             
                

which triggers asynchronous loading of the selected document, calling another template which updates the HTML page (in situ) to show the results:

    <xsl:template name="show-all">
        <xsl:param name="X"/>
        <xsl:result-document href="#title" method="ixsl:replace-content">
            A List of Cities Beginning With {$X}
        </xsl:result-document>
        <xsl:result-document href="#target" method="ixsl:replace-content">
            <table id="cities-table">
                <thead>
                    <tr>
                        <th>City</th>
                        <th>Country</th>
                        <th data-type="number">Longitude</th>
                        <th data-type="number">Latitude</th>
                    </tr>
                </thead>
                <tbody>
                    <xsl:for-each select="json-doc($baseURL || $X ||'.json')?*">
                        <tr>
                            <td>{?name}</td>
                            <td>{?country}</td>
                            <td>{?coord?lon}</td>
                            <td>{?coord?lat}</td>
                        </tr>
                    </xsl:for-each>
                </tbody>
            </table>
        </xsl:result-document>
    </xsl:template>               
              
                

With the new constructs proposed in this paper, we can rewrite the last two templates as follows:

    <xsl:template name="loadJSON">
        <xsl:param name="X" select="'A'"/>
        <xsl:variable name="json" select="a:json-doc($baseURL||$X||'.json')" async="yes"/>

        <xsl:result-document href="#title" method="ixsl:replace-content">
            A List of Cities Beginning With {$X}
        </xsl:result-document>
        
        <xsl:result-document href="#target" method="ixsl:replace-content">
            <table id="cities-table">
                <thead>
                    <tr>
                        <th>City</th>
                        <th>Country</th>
                        <th data-type="number">Longitude</th>
                        <th data-type="number">Latitude</th>
                    </tr>
                </thead>
                <tbody>
                    <xsl:for-each select="$json?*">
                        <tr>
                            <td>{?name}</td>
                            <td>{?country}</td>
                            <td>{?coord?lon}</td>
                            <td>{?coord?lat}</td>
                        </tr>
                    </xsl:for-each>
                </tbody>
            </table>
        </xsl:result-document>
    </xsl:template>
                

Things to note:

  • In place of the ixsl:schedule-action instruction, we now have a call on the asynchronous a:json-doc function. The benefit is that the code is much more extensible and generic; we could call any asynchronous function at this point.

  • Because the result of calling a:json-doc is held in the $json variable, there is now no need for another call on json-doc in the continuation code.

  • The continuation code has access to the full dynamic context for the sequence constructor in which the asynchronous xsl:variable instruction appears. The only part of this that's actually used is the local variable $X; but this is enough to be useful, because it cuts out the clumsy parameter passing.

  • We're still following the typical Saxon-JS paradigm of making in-situ updates to the HTML page. But we're no longer constrained to this model. If we were on the server side, doing a conventional transformation to produce the HTML page as a result document, the continuation code (the code appearing after the asynchronous xsl:variable instruction) could continue writing elements to the result tree in the normal way.

This example also makes an asynchronous call to json:doc right at the start of the transformation:

    <xsl:template name="main">
        <xsl:result-document href="#index" method="ixsl:replace-content">
            ....
        </xsl:result-document>
        
        <xsl:call-template name="loadJSON"/>
    </xsl:template>

(where loadJSON is shown above). This is a case where an asynchronous global variable could be usefully employed. It would require a little restructuring of the code, but would simplify the logic for cases like this where the initial resources to be fetched can be determined without reference to the source document.

Example 2: Animation

Also in the sample applications for Saxon-JS, the knight's tour [tour.xsl] demonstrates timer-based animation.

The existing logic for this is:

    <xsl:template name="next-move">

        <!-- This function takes the board in a given state, decides on the next move to make,
         and then calls itself recursively to make further moves, until the knight has completed
         his tour of the board. It returns the board in its final state. -->

        <xsl:param name="move" as="xs:integer"/>
        <xsl:param name="board" as="array(array(xs:integer))"/>
        <xsl:param name="square" as="map(xs:string, xs:integer)"/>

        <!-- determine the possible squares that the knight can move to -->

        <xsl:variable name="possible-destinations" as="map(xs:string, xs:integer)*"
            select="tour:list-possible-destinations($board, $square)"/>

        <!-- try these moves in turn until one is found that works -->
        
        <xsl:choose>

            <xsl:when test="empty($possible-destinations)">
                <xsl:message>Failed! - the knight is stuck</xsl:message>
            </xsl:when>

            <xsl:when test="ixsl:page()//*:button[@id='stop']/@data-stopped = ('stopped', 'reset')">
                <!--<xsl:message>Stopped/reset at user request</xsl:message>-->
            </xsl:when>

            <xsl:otherwise>

                <xsl:variable name="best-square" as="map(xs:string, xs:integer)"
                    select="tour:find-best-square($board, $possible-destinations, 9, ())"/>

                <!-- update the board to make the move chosen as the best one -->

                <!--<xsl:message>Moving to square <xsl:value-of select="$best-square"/></xsl:message>-->

                <xsl:variable name="next-board" as="array(array(xs:integer))"
                    select="tour:place-knight($move, $board, $best-square)"/>

                <!-- output the board in its new state -->

                <xsl:call-template name="print-board">
                    <xsl:with-param name="board" select="$next-board"/>
                </xsl:call-template>

                <!-- schedule the next move to be made after an interval of time -->

                <xsl:if test="$move != 64">
                    <ixsl:schedule-action wait="tour:speed()">
                        <xsl:call-template name="next-move">
                            <xsl:with-param name="board" select="$next-board"/>
                            <xsl:with-param name="move" select="$move + 1"/>
                            <xsl:with-param name="square" select="$best-square"/>
                        </xsl:call-template>
                    </ixsl:schedule-action>
                </xsl:if>
                
                <xsl:if test="$move = 64">
                    <xsl:result-document href="#title" method="ixsl:replace-content">
                        Completed Knight's Tour: using XSLT 3.0 with maps and arrays
                    </xsl:result-document>
                </xsl:if>

            </xsl:otherwise>

        </xsl:choose>

    </xsl:template>            
                

This code is invoked to find the next move for the knight in its tour of the chessboard. The logic is:

  • First construct a list of all possible destinations (all empty squares to which the knight can move).

  • From these, choose the best (the algorithm need not concern us).

  • Update the board to put the knight on that square.

  • Display the updated board (by mutating the HTML page, as before).

  • After waiting for an adjustable period of time (say one second), make a recursive call to the same template to compute and display the next move.

  • After 64 moves, show a final message.

With the new syntax proposed in this paper, the only change to this code is that the ixsl:schedule-action logic can be simplified to:

                <xsl:if test="$move != 64">
                    <xsl:variable name="next-move" select="a:sleep(tour:speed(), $move+1)" async="yes"/>
                    <xsl:call-template name="next-move">
                        <xsl:with-param name="board" select="$next-board"/>
                        <xsl:with-param name="move" select="$next-move"/>
                        <xsl:with-param name="square" select="$best-square"/>
                    </xsl:call-template>
                </xsl:if>               
                

In addition, the next-move template, and others that call it, need to be declared with async="yes".

In addition, there's a more subtle change to what's going on here. In the original stylesheet, the final logic (the <xsl:if test="$move = 64"> instruction) executes immediately after firing off the asynchronous next-move code. With the new logic, this code executes only when the asynchronous call completes. In this example, this makes no difference, but it's useful to be aware of it.

Implementation Notes

The critical implementation difficulty for implementing asynchronous functions in the Javascript world has been the difficulty of organising the code in such a way that the result of an asynchronous operation can be further processed when the operation completes.

Consider for example the expression count(doc('a.xml')//para).

If we allowed the call on doc('a.xml') to operate asynchronously, then the containing operations (// and count()) would have to be prepared to handle a promise coming back from the evaluation of their operands. Because the language is orthogonal, every expression would need code to cater for the situation where the values of its operands are not immediately available.

Interestingly, this is a similar challenge to making XSLT streamable; in the streaming situation, the effective asynchrony arises because the expression has to wait until its operands are delivered by the streaming parser. Saxon handles streaming by effectively inverting the code to operate in push mode rather than pull mode [Kay 2009}; instead of an expression calling for its operands to be evaluated (pull mode, or top-down evaluation), the evaluation of a parent expression is triggered by the availability of its operands (push mode, or bottom-up evaluation). We could possibly adopt a similar solution here, but it is immensely disruptive; and for the streaming case the inversion is only necessary for expressions that process nodes, whereas here it would apply regardless of data type.

Another possibility would be to rely on the Javascript async/await constructs. This would in effect automate the program inversion (the conversion of pull code too push code). But the problem is that every single operand evaluation would be potentially asynchronous, and we suspect that use of this mechanism would be enormously expensive if applied at that level of granularity.

Instead, we capitalise on the fact that we already use push mode evaluation to an extent, especially at the XSLT level.

XSLT 1.0 had a strong distinction between instructions and expressions. The semantics of expressions were described using the terminology of "evaluating operands", combining their results, and "returning a result": pull language. By contrast, the semantics of instructions were described using push language: an instruction writes something to the result tree. In XSLT 2.0 this was changed to use pull terminology throughout – instructions now return a result, just as expressions do – but of course the internal implementation can still be push-based, and in Saxon it is, mainly because this involves less overhead when constructed elements are attached as children to a new parent. (Saxon-JS 2 achieved significant performance benefits over Saxon-JS 1 by changing tree construction expressions, which originate largely in XSLT rather than XPath, to work in push mode.) In push mode, where the control loop is sending events to an output destination, waiting for something to happen before sending the next event is very straightforward. So long as we follow the discipline that asynchronous code is always executed in push mode, this should all work without problems. And in turn, that matches well the rule we've forced in the language semantics that asynchronous code can only be invoked while writing a final result tree: all instructions that write to a final result tree execute in push mode.

(There's an exception here, not yet addressed in this proposal, concerning try/catch. In fact, error handling generally needs further work; part of the problem is the absence of an explicit error value in the XDM model.)

The XDM promise object described in this proposal can be readily implemented as a simple wrapper around a Javascript promise, and all asynchronous functions for fetching resources can readily be implemented using Javascript primitives that return promises.

Wright's implementation of his promise model in the Java-based BaseX environment [Wright 2016] gives confidence that the language extensions described here are not suitable only for Javascript, and moreover that they have other use cases beyond the ones considered in this paper (Wright is primarily concerned with delivering performance benefits through parallel processing.) Although the promise model described here differs in detail from Wright's, the core ideas are the same.

Analysis

The principal objectives of this project, for the target Javascript environment, were to eliminate the limitations of the ixsl:schedule-action instruction of Saxon-JS. These requirements have been met:

  • The set of asynchronous requests that can be made is now extensible: any asynchronous function can be invoked, and the set of asynchronous functions is infinitely extensible.

  • The code that is executed on completion of an asynchronous request can now contribute to the principal result of the transformation.

  • The syntax is a lot cleaner, resembling the "async/await" extensions to Javascript, which go a long way towards hiding the complexities of asynchrony from the programmer.

Although the proposal is solidly grounded in an underlying mechanism based on promises, most developers will not need to concern themselves with promises as such, and will be able to achieve what they need simply by declaring templates and local variables as asynchronous.

The mechanism works within the constraints of the Javascript model where a thread cannot wait for spawned tasks to complete; but at the same time there is a simple extension (the a:await() function) that can be used in environments where this restriction does not apply.

This proposal does not directly address issues of mutability (in-situ updates to the HTML page). But I believe that by putting asynchronous processing on a much more solid foundation, it makes solutions to that problem a lot more tractable.

References

[EXPath-File] EXPath File Module 1.0. W3C Recommendation, 20 Feb 2015. Ed. Christian Grün. http://expath.org/spec/file

[EXPath-HTTP] EXPath HTTP Candidate Module 1.0. W3C Recommendation, 9 Jan 2010. Ed. Florent Georges. http://expath.org/spec/http-client

[Kay 2009] Kay, Michael. You Pull, I’ll Push: on the Polarity of Pipelines. Presented at Balisage: The Markup Conference 2009, Montréal, Canada, August 11 - 14, 2009. In Proceedings of Balisage: The Markup Conference 2009. Balisage Series on Markup Technologies, vol. 3 (2009). doi:https://doi.org/10.4242/BalisageVol3.Kay01. Available at http://www.balisage.net/Proceedings/vol3/html/Kay01/BalisageVol3-Kay01.html

[Kay 2020] Kay, Michael. A Proposal for XSLT 4.0. Presented at XML Prague, 2020. Available at https://archive.xmlprague.cz/2020/files/xmlprague-2020-proceedings.pdf and at http://www.saxonica.com/papers/xmlprague-2020mhk.pdf

[Lockett & Retter 2019] Lockett, Debbie, and Adam Retter. Task Abstraction for XPath Derived Languages. Presented at XML Prague, 2019. Available at https://archive.xmlprague.cz/2019/files/xmlprague-2019-proceedings.pdf and at http://www.saxonica.com/papers/xmlprague-2019dcl.pdf

[Delpratt & Kay 2013] Delpratt, O'Neil, and Michael Kay. Multi-user interaction using client-side XSLT. Presented at XML Prague, 2013. Available at https://archive.xmlprague.cz/2013/files/xmlprague-2013-proceedings.pdf and at http://www.saxonica.com/papers/prague2013ond1_mhk.pdf

[Lockett & Kay 2016] Lockett, Debbie, and Michael Kay. Saxon-JS: XSLT 3.0 in the Browser. Presented at Balisage: The Markup Conference 2016, Washington, DC, August 2 - 5, 2016. In Proceedings of Balisage: The Markup Conference 2016. Balisage Series on Markup Technologies, vol. 17 (2016). doi:https://doi.org/10.4242/BalisageVol17.Lockett01. Available at http://www.balisage.net/Proceedings/vol17/html/Lockett01/BalisageVol17-Lockett01.html and at http://www.saxonica.com/papers/prague2013ond1_mhk.pdf

[Wright 2016] Wright, James. Promises and Parallel XQuery Execution. Presented at XML Prague, 2016. Available at https://archive.xmlprague.cz/2016/files/xmlprague-2016-proceedings.pdf

[XSLT 3.0] XSL Transformations (XSLT) Version 3.0. W3C Recommendation, 8 June 2017>. Ed. Michael Kay, Saxonica. http://www.w3.org/TR/xslt-30