XSLT Programmer's Reference(now in its fourth edition) and a contributor to many other books.
last()
function. However, streamed evaluation of a single XPath expression during a single pass
of a source document does not help much with streamed evaluation of XSLT, since a
stylesheet contains many XPath expressions, and the starting point for one is typically
dependent on the nodes found by another.xsl:apply-templates instruction, and always select downwards by element name. As a result,
the call hierarchy becomes statically tractable, as in the XQuery case.with-param element within an call-template
element must have a distinct value for its name attribute might be expressed as
follows:In order to reduce the burden on implementers, in particular implementers of streaming processors, only restricted subsets of XPath expressions are allowed in {selector} and {fields}.
WatchManager; each of the listening components is called a Watch. For the example
constraint given, the process is as follows:xsl:call-template element is notified by the
parser, the validator for the xsl:call-template element is fired up. This
creates a SelectorWatch for the uniqueness constraint. The
SelectorWatch maintains a table of key values that have been encountered
(initially empty) so that it can check these for uniqueness.SelectorWatch. For each event, it
checks whether the ancestor path in the document matches the path given in the
xs:selector element. In this case this is simply a test whether the event
is a startElement event for an xsl:with-param element. More generally, it is
essentially a match of one list of element names against another list of element names,
taking account of the fact that the // operator can appear at the start of
the path to indicate that it is not anchored to the root xsl:call-template
element. When the matching startElement event is encountered, the
SelectorWatch instantiates a FieldWatch to process any nodes
that match the expression in the xs:field element.FieldWatch is now notified of all parsing events, and when the
@name attribute is encountered, it informs the owning
SelectorWatch, which checks that its value does not conflict with any value
previously notified. This process is more complex than might appear, because there can be
multiple xs:field elements to define a composite key, and furthermore, the
field value can be an element rather than an attribute, in which case it may be necessary
to assemble the value from multiple text nodes separated by comments or processing
instructions. It is also necessary to check that the FieldWatch fires exactly
once. (A simplifying factor, however, is that XSD requires the element to have simple
content.) startElement event that matches its path expression, the
SelectorWatch must remain alert for further matching events. Before the
corresponding endElement event is encountered, another matching
startElement might be notified. This cannot happen in the above example.
But consider the constraint that within each section of a chapter, the figure numbers must
be unique: xs:field element must have simple
content.) The WatchManager may therefore be distributing events
simultaneously to a large number of Watch instances, even for a single
uniqueness constraint; at the same time, of course, other uniqueness constraints may be
active on the same elements or on different elements at a different level of the source
tree.saxon:read-once attribute, this
code would parse the source document customers.xml and build a tree
representation of the document in memory. It would then search the tree for the elements
matching the path /*/customer, and for each of these in turn it would create a
copy of the subtree rooted at this element, returning it from the function and then applying
templates to it.customers.xml document is unnecessary; it can be bypassed if the elements
matching /*/customer can be recognized in the event stream issuing from the XML
parser. Instead of one large tree, the processor can build a series of smaller trees, each
representing a single customer record. So long as the size of the customer record is kept
within bounds, there is then no limit on the number of customer records present in the input
document. This is sometimes referred to as xsl:copy-of instruction here is significant. In the implementation, there is no physical
copying taking place, because the original whole-document tree is never built. But the result is equivalent to the result
of building the whole-document tree and then copying the sequence of subtrees. In particular, the nodes in one subtree
are not linked in any way to the nodes in other subtrees; there is no way the application can navigate outside the boundaries
of a subtree. Attempting to retrieve the ancestors or siblings of the customer element returns nothing,
just as it would with a true subtree copy.WatchManager machinery
described in the previous section. Having analyzed the select attribute of the
xsl:copy-of instruction to confirm that it satisfies the constraints on
streamable XPath expressions, the document customers.xml is then processed
using a SAX parser which sends parsing events to a WatchManager which in this
case notifies a new kind of Watch, a CopyWatch, of the start and
end of elements matching the path expression; between these start and end events, the
CopyWatch is notified of all intermediate events and uses these to build a
tree representing the customer element./*/customer has a fixed depth. But change the example to //section, and it is clear that
the set of section elements selected by the path can include one section that is a subtree of another. This situation
requires some internal buffering: the language semantics require that the sections are delivered in document order, which means that
the outermost section must be delivered before its nested sections. The trees representing the nested sections must therefore
be held in memory, to be released for processing only when the endElement event for the outermost section is
notified. The code is written so that it is always prepared to do this buffering; in practice, it is very rarely needed, and
no extra costs are incurred in the case where it is not needed. In some cases it would be possible to determine statically
that no buffering will be needed, but this knowledge confers little benefit.saxon:read-once="yes". Although the implementation of the
xsl:copy-of instruction in streaming mode is very different from the
conventional execution plan, the functional behaviour is identical except in one minor
detail: there is no longer a guarantee that if the customers.xml file is read
more than once within the same transformation, its contents will be the same each time. At
the time this feature was first implemented, the XSLT 2.0 conformance rules required
implementations to deliver stable results in this situation. The streaming implementation
necessarily departed from this rule (the only practical way to enforce the rule is to make
an in-memory copy of the tree), so the saxon:read-once attribute was provided
as a way for the user to assert that the file would not be read more than once, thus
licensing the non-conformance in the case where the assertion was not honoured. In the final
version of the XSLT 2.0 specification, there was explicit provision that implementations
were allowed to provide a user option to waive the stability requirement for the
doc() function, thus making this extension conformant.CopyWatch component delivers its results (the sequence of small
customer trees) using a push interface (it calls the next component in the
pipeline to deliver each one in turn), whereas the xsl:apply-templates
instruction that calls the user-defined function expects to use a pull interface (it calls
the XPath engine to deliver each one in turn). There is thus a push-pull conflict, which is
resolved using the design described in customer trees to a cyclic
buffer, which is then read by a parallel thread delivering the trees in response to requests
from the xsl:apply-templates instruction.saxon:stream(), which was also made available in XSLT. This might be
regarded as a pseudo-function: the call
saxon:stream(doc('customers.xml')/*/customer) delivers a copy of the value of
its argument (that is, a sequence of customer subtrees), but it requires its
argument to conform to the syntax of streamable path expressions. saxon:stream(doc('customers.xml')/*/@version) will return the value of the
version attribute from the outermost element of the document, and will read
no further once this has been seen. This makes it possible to obtain information from near
the start of an XML document in constant time, regardless of the document size, which is
especially useful in a pipeline when making decisions on how to route a document for
processing based on information in its header. (It's a moot point whether this is consistent
with the requirement in the XML 1.0 specification that all well-formedness errors in a
document must be reported to the application. But the facility is so useful that we can
ignore the standards-lawyers on this one.)saxon:assign instruction which introduces mutable variables to the language;
but this plays havoc with optimisation).xsl:iterate
instruction defined in the XSLT 2.1 working draft. This was implemented in Saxon 9.2 in the
Saxon namespace (as saxon:iterate), but I will present it using the W3C syntax,
which becomes available in Saxon 9.3.transaction elements, and outputting the
same sequence of elements but with an additional attribute holding the running balance on the account, obtained
by accumulating the values of all the preceding transactions.xsl:iterate, which allows the solution to
be expressed as follows:xsl:for-each instruction with the added ability to set a parameter
after the processing of one value which is then available for use when processing the next.
A key difference compared with the recursive solution is that the set of transactions to be
processed is identified in one place, the select attribute of the
xsl:iterate instruction, rather than being the product of a sequence of
independent calls on following-sibling. A useful consequence of this difference
is that termination is guaranteed.xsl:iterate is as syntactic sugar for the
foldl higher-order function found in many functional programming languages:
this applies a user-supplied function (the body of xsl:iterate) to each item of
an input sequence (the value of the select expression), with each iteration
delivering an accumulated value of some kind, which is made available as a parameter to the
user-supplied function when called to process the next item.select expression to select the stream of subtrees arising from a streaming
copy operation: for example <xsl:iterate
select="saxon:stream(doc('transactions'xml')/*/transaction">. By this means,
information computed while processing one input transaction can be made available while
processing the next. The implementation of xsl:iterate in fact knows nothing
about streaming; it is processing the sequence of subtrees just as it would process any
other sequence of items.xsl:apply-templates instruction to invoke recursive processing of its children); then one or more templates
for specific named elements to process them in a way that differs from the general rule. The effect of this stylesheet
is to copy the source document to the result document unchanged except for the loss of all attributes, and of elements
named note together with their descendants.startElement event and corresponding endElement event
for a note element.xsl:apply-templates instruction with defaulted select
expression, or one of the following expressions or instructions applied to the context
node: xsl:copy-of, xsl:value-of, string(),
data() (or implicit atomization), or one of a small number of other
constructs.xsl:element, literal result elements,
xsl:value-of, xsl:attribute, xsl:comment,
xsl:processing-instruction, xsl:result-document,
xsl:variable, xsl:sequence.local-name() or exists()) that
returns a local property of the node, or of one of its attributes or ancestors, or of an
attribute of an ancestor; (b) an expression that returns the string value or typed value
of an attribute of the node or an attribute of one of its ancestors.<xsl:mode streamable="yes"/>saxon:mode<xsl:value-of select=". + 3"/>, because an addition expression
(+) is not an acceptable ancestor of the implicit drill-down construct
data(.). To get around this restriction, it is possible to bind a variable to
the value of data(.) and then perform the addition using the value of the
variable.getNextInput() calls to the parser, the application becomes
event-driven and is called when new input is available. In consequence, the application has
to maintain its own stack to represent the current state; it can no longer rely on the call
stack maintained by the compiler of the implementation language.startElement events or evaluating complete instructions; the post-descent
actions similarly involve either complete instructions or endElement actions.
These correspond to the instructions that are permitted as ancestors of the drill-drown
construct: mainly instructions such as xsl:element, which are classified as
processLeft() and processRight(). The drill-down action is one
of apply, copy-of, value-of, or skip,
and indicates what is to be done with the content of the matched element (the events that
occur after the startElement that activates the template rule and before the
corresponding endElement). The value skip causes these events to
be skipped, and arises when the template rule contains no drill-down construct. The value
copy-of indicates that a subtree is to be built from these events;
value-of indicates that a string is to be constructed by concatenating the
text nodes and ignoring everything else. Finally, the value apply indicates
that the events should be matched against template rules for the relevant mode; when a match
occurs, the selected template rule will then receive the events until the matching
endElement event occurs.StreamingDespatcher that despatches events
to the relevant template rules. This functions in a very similar way to the
WatchManager described earlier, and in the current Saxon 9.3 code base the
two despatching classes have been combined into one.Pattern object equivalent that used when starting from an XSLT match pattern.
The pattern used in a streamable template rule can be any XSLT pattern provided it does not contain a predicate that is
positional (for example, is numeric or calls position() or last()) or that uses
the child, descendant, descendant-or-self, following, following-sibling, preceding, or preceding-sibling axes.xsl:choose; in this case each branch of the conditional may make downwards
selections. Unlike the XSLT 2.1 draft, Saxon does not currently allow a node in the streamed
input document to be bound to a variable, or passed as a parameter to another template or
function. A further rule is that the template must not return a node in the streamed
document (for example, <xsl:sequence select="."/>) - this is because there
is no way of analyzing what the caller attempts to do with such a node.sum() function supports push evaluation at the item level, which
means that given the expression sum(.//value), each descendant
value element is assembled as a complete node, which is then atomized, and
the resulting atomic values are notified one by one to the sum implementation, which adds
each one in turn to the running total. By constrast, the functions count() and
exists() have implementations that work at the parse-event level, which means
that there is no need to build the nodes being counted as trees in memory: thus
count(.//employee) merely tallies the startElement events that match its
argument expression .//employee without ever building a tree representation of
an employee element.| Input/Output | Composed | Decomposed |
|---|---|---|
| Composed | remove($in, 3) | <e>{$in}</e> |
| Decomposed | data($in) | <xsl:for-each select="$in"/> |
employee), the
xsl:value-of instruction, the distinct-values() function call,
and the path expression skills/skill. It also contains expressions added by the
compiler, reflecting the implicit atomization and type checking performed by the
distinct-values() function, and the implicit call on
string-join() that is performed to insert separator spaces on behalf of the
xsl:value-of instruction. The full expression tree is shown in Figure 3, with
the streaming route highlighted.skills/skill. The data() node composes these events into strings
representing the typed value of the skill elements; these are fed through the operators
found on the streaming route until they reach the <employee> element
constructor, which delivers a stream of events representing the newly constructed element. Typically
these events go straight to the serializer. The parts of the expression tree that are not on the streaming
route (the two attribute constructors) are evaluated in pull-mode as normal.distinct-values() function implicitly atomizes its input. So the first
thing that happens to the incoming data is that it is sent to a TypedValueWatch: this takes
in a decomposed sequence of events and outputs a composed sequence of atomic values
representing the typed values of the skill elements. This is piped into a type
checker that checks that the values in this sequence are all strings. The type-checked
csequence is then piped into the component that evaluates the distinct-values()
function, which again outputs a composed stream (this being identical to its input stream
except that values are removed from the stream if they have been encountered before). The
xsl:value-of instruction is initially compiled into a pipeline of primitive
instructions that first joins adjacent text nodes, then atomizes, then inserts separators
between adjacent strings, then converts the resulting string to a text node; in this case
the optimizer knows from type analysis that the first two steps are superfluous, so we are
left with a pipeline that takes a composed sequence of strings as its input, and produces a
composed sequence comprising a single text node as its output. Finally, the component
representing the literal result element employee takes this composed input, and
produces decomposed output consisting of a startElement event, two attribute events, a text
node event, and an endElement event. Typically these events will be sent directly to the
serializer.(sum(.//value) +
1) is streamable, because although there is no specific support for evaluating the
addition in push mode, the system is capable of constructing the two singleton arguments and
using the conventional pull-mode addition. Many expressions also benefit from a generic
implementation in cases where an argument is a non-singleton sequence. This generic
implementation buffers the argument sequence in memory and then uses the pull-mode
implementation once the sequence is complete. This of course is not pure streaming, but it
provides useful scaffolding to enable less commonly used expressions to appear in a
streaming template while awaiting a pure streaming implementation. Currently this mechanism
is used for comparison expressions such as .//value = 17 (recall that in XPath,
this returns true if at least one of the value elements is equal to 17); there
is no technical reason that prevents the creation of a pure push-mode implementation of a
general comparison expression with respect to either argument, but it has not yet been
implemented, and it is not a top priority because in the vast majority of cases the
arguments actually turn out to be singletons.<xsl:value-of
select="sum(.//value) + 1"/>, the streamable pattern is .//value. The
immediate parent of this pattern on the streaming route must always be an expression that
can be evaluated as a Watch. The number of expressions implemented as a Watch is suprisingly
small:TypedValueWatch handles all expressions that use the atomized value of the
selected nodes.StringValueWatch handles all expressions that use the string value of the
selected nodes.CountWatch handles the count(), exists(), and
empty() functions.CopyWatch (the one we met earlier) handles xsl:copy-of.VoidWatch is used for templates or branches of a conditional that make no
downwards selection.SimpleContentWatch implements the rules for instructions such as
xsl:value-of and xsl:attribute; specifically, it
concatenates adjacent text nodes, removes empty text nodes, and reduces all other nodes
to string values.ApplyTemplatesWatch is used where the downwards selection appears in the select
expression of an xsl:apply-templates instruction, and also supports
xsl:apply-imports and xsl:next-match.ForEachWatch is used where the downwards selection appears in the select
expression of an xsl:for-each instruction.ForEachWatch and the ApplyTemplatesWatch, which
emit a sequence of parse events.CopyWatch earlier in the paper: it is used for a
simple streaming copy, as well as for xsl:copy-of instructions appearing within
a streaming template. When it receives a startElement event representing an element selected
by its controlling pattern (we'll assume to keep things simple that it is watching for
elements), it starts building a tree. All subsequent parse events until the matching
endElement are directed to this tree builder. When the tree is complete, the node at the
root of the tree is emitted to the Feed that implements the parent of the
xsl:copy-of instruction, that is, the parent expression in the streaming
route through the expression tree. StringValueWatch and TypedValueWatch work in a very
similar way, except that instead of building a tree from the incoming events, they construct
the string value or typed value of the node by concatenating the values of those events that
represent text nodes.ApplyTemplatesWatch is similarly notified of the startElement event for
a node that matches the select expression of the xsl:apply-templates
instruction. It responds to this by searching for a matching template rule — this is
possible because the constraints on match patterns in a streamable template ensure that the
pattern can be evaluated when positioned at the startElement event (the object representing
the event, it should be mentioned, provides methods to get local properties of the node such
as the name and type annotation, and also to navigate to the node's ancestors and their
attributes). Having identified the template to be applied, which because it is in a
streaming mode will always have been compiled with template inversion, it then gets the
Watch expression identified during the analysis of the called template, and nominates this
Watch expression to the WatchManager. All events up to the corresponding endElement will
therefore be sent by the WatchManager to the called template, where the same process
continues recursively. At the same time, the same events are being sent to the calling
ApplyTemplatesWatch, because as with xsl:copy-of, it is entirely
possible for the select expression of xsl:apply-templates to
select an element that is a descendant of another element already being processed; the
results of the processing of such nested elements will again be buffered.ForEachWatch operates in a very similar way to the
ApplyTemplatesWatch, except that there is no need to search for a matching
template. Rather, the body of the xsl:for-each instruction is compiled directly
as an inverted template and is invoked unconditionally for each startElement event matching
a selected node.xsl:iterate; none
is yet available for xsl:for-each-group, but it is expected that it will follow
the same principles.xsl:iterate) that make it possible to write powerful
transformations without straying from the streamable subset. The design of the language is
informed by experience with both XSLT 2.0 and STX, and by a large collection of use cases
describing problems that benefit from a streamed implementation.