my:count-words (for some namespace binding of the
prefix my) that counts words in one or more
documents, and you want to test this step. I’ll refer to it as a
my:count-words step, you’re doing two
things:my:count-words step. You also decide
what aspects of the results are useful to verify. You express
each verification in a way that accommodates a
passor
failstatus.
result output port of the
step and that the document equals a particular one you expect. If
exactly one document appears on the result output
port but the content of the document doesn’t match your
expectation, then the first verification passes and the second one
fails.<x:description>, where the
prefix x is bound to the XSpec namespace. The
<x:description> element uses an attribute named
xproc to point to the XProc step file or library
file that contains (directly or through imports) your test
target(s). The xproc attribute is analogous to
attributes named stylesheet for testing XSLT,
query-at for testing XQuery, and
schematron for testing a Schematron schema.<x:description>, you use the
<x:scenario> element to contain elements that
express how you want to call the test target and what you want to
verify. If you have multiple test cases, you use multiple
<x:scenario> elements. To simplify explanations
in this paper, we’ll assume the file has no nesting of scenarios,
so multiple <x:scenario> elements are all siblings
of each other.<x:scenario> element, you use a
child <x:call> element to express how you want to
call the test target. Following <x:call>, you use
a child <x:expect> element for each verification
you want to perform. To make test files and reports easier to
understand, you also label each <x:scenario> and
<x:expect> element with a text description. The
label can be an attribute named label or a child
element named <x:label>; for simplicity, we’ll use
the attribute here.xproc
attribute is the only thing specific to testing XProc. However,
populating the <x:call> and
<x:expect> elements with details about each test
case reveals other ways in which the XSpec vocabulary is tailored
for XProc.type attribute on the
<p:declare-step> element. To configure the
<x:call> element for your test target, you set
the step attribute to the type as a qualified name
(e.g., my:count-words). The step
attribute is analogous to attributes named template
and function for testing XSLT.<x:call> element often needs more
configuration than merely the type of step to call. Your test case
might require non-default documents on any of the step’s input
ports, or it might require non-default values of any of the step’s
options. The <x:call> element supports two kinds
of child elements in tests for XProc only:<x:input>, which specifies zero or more
documents on an input port. The port attribute is
the name of the input port.<x:option>, which specifies a value of a
step option. The name attribute is the qualified
name of the option.<x:input> or <x:option>; point
to an external XML file; provide an XPath expression that derives
something from the embedded content or external XML file; or
provide a standalone XPath expression that constructs a document
or value.<p:document> elements as children of XSpec
<x:input> or <x:option>
elements. Using <p:document> is especially
convenient for specifying multiple documents on an input port,
specifying non-XML documents, or modifying the set of document
properties of a document.<p:document> elements get passed to the XProc
processor for evaluation. By contrast, the other ways of
specifying input documents and option values are evaluated in
XSLT, for reasons I explain later, in the implementation-related
portion of this paper.<x:call> elements with the step type, input
documents, and an option value.<x:input> element in my:count-words step is actually processing: a
<doc> element, or a document node containing
a <doc> element.<x:param>, within an
XSpec test for other languages, then XSpec assumes you want
the parameter value to be the element. If you instead want the
parameter value to be a document node containing that element,
you add select="/" to the
<x:param> element.select="/" to <x:input>
elements in order to get a document node instead of an
element. If you specify a <doc> element as an
input document, the element gets wrapped in a document node,
and the step sees the document node. If you specify five
sibling XML elements as input documents,
<doc> element is why
the as attribute above says
as="document-node(element(doc))", not
as="element(doc)".wrap-eachbehavior is relevant for both new and experienced users of XSpec to know about when writing tests for XProc steps. The behavior will come up again in the discussion of the
<x:expect> element,
next.<x:call> element
accomplishes the first part of the two-part testing process,
namely, running the test target step with your choice of input
documents and/or option values. The second part is to use
<x:expect> elements to verify the results of
running the step. This element can appear in XSpec tests for any
of the supported languages. It can check whether a boolean
condition you specify evaluates to true. An example
is a condition like exists($x:result//p) using a
variable named x:result that XSpec predefines—or
exists(//p) for short, if exactly one document
appears on a particular output port. Alternatively,
<x:expect> can check whether an actual result
equals a corresponding expected result that you specify.<x:expect> element also has a few
ways in which it is tailored for testing XProc. First,
<x:expect> has an optional port
attribute whose value is the name of an output port of the step.
This attribute says which output port you are interested in for
that verification. If you set the port attribute,
XSpec predefines its variable named x:result as the
sequence of output documents on that port only. Specifying the
port also makes certain other aspects of the
<x:expect> element more convenient, as the
next three subsections describe.<x:expect> has a port
attribute, XSpec predefines an additional variable, named
x:document-properties. It stores properties of
the documents on the specified output port. You can use both
$x:result and $x:document-properties
in verification expressions.<x:expect> element without a
port attribute can access the same
information, but it requires lookups in a multiple-level map
that stores results across all the output ports.<x:expect> for XSpec to compare with the
unmodified actual result on the specified port, then XSpec
interprets each child node as a separate document and
automatically wraps each in a document node. Here,
unmodifiedmeans you are not using a
test attribute of <x:expect>
either to drill down into the actual result (e.g.,
$x:result//p) or to derive data from the actual
result (e.g., count($x:result)).result with a pair of expected
documents. One document contains the element
<a/> and the other document contains the
element <b/>. Your intention is probably more
likely to be this pair of unrelated documents than a single
document containing sibling top-level elements
<a/> and <b/>. Also, you are
unlikely to expect the step to produce bare elements on its
output port, because XProc steps that produce XML documents
represent them as document nodes.port attribute
on the <x:expect> element affects the display of
verification failures in the HTML report of test results. For
all the supported languages, the report has a section of details
about each verification failure. If the failure section of the
report shows diffs between actual and expected results, then you
probably want the displayed actual result to be only the result
on the XProc output port that you specified as your port of
interest. Indeed, that’s what XSpec does.false, the failure section of the
report shows an actual result, and it’s up to you to determine
why the condition was false for that result. Here, too, XSpec
shows the actual result on that output port only. Without a
port attribute, the actual result appearing in
this section of the report would be a multiple-level map that
stores results across all the output ports. This map would be
cumbersome to read when investigating the failure.<x:scenario> element for individual test
cases and for grouping<x:import> element for importing from
another XSpec file<x:helper> element for integrating custom
code into the testing process (where the custom code’s
language depends on the language of the test target; tests for
XProc can use custom code written in either XProc or
XSLT)<x:like> element for reusing XSpec
code<x:variable> element for defining
test-specific variables<x:pending> element and
pending attribute for disabling portions of a
test filefocus attribute for enabling only specific
scenarios and disabling all others...) for skipping pieces of
XML content when comparing results<x:scenario> elements<x:call> to have a
type attribute in the step declaration.p:run and
p:file-copy.p:file-copy,
and you want to verify existence of the copied file. Maybe the
copy existed even before your step ran. Maybe calling your step
to copy the file interferes with a subsequent test case. XSpec
doesn’t give you a way to set up the environment before a test
case executes, clean up afterward, or enforce an order of
operations; this is an XSpec limitation across languages, not
specific to testing XProc.compile-xslt-tests.xsl (or
compile-xquery-tests.xsl). The test runner is the box to
which arrow 5 points.Calling steps as functions[
knowing how to accesspart is related to the scope of steps in XProc or the configuration of a non-XProc processor, and you can find those details in the XProc processor’s documentation.
Print test result.)
p:system-property, but how can a test runner
written in XSLT call a function that belongs to an XProc
processor? The p:system-property function is not
callable from XSLT.pf:system-property that wraps around the
p:system-property function. (The prefix
pf binds to an XSpec-related namespace URI,
http://www.jenitennison.com/xslt/xspec/xproc/steps/wrap-standard-functions.)pf:system-property step by calling it as a
function within an XPath expression anywhere in the stylesheet.
To provide a value for the property step option,
the function call provides a map with key property
as a function argument. This function syntax is part of how the
step function feature works. In select
attribute illustrates the function call in an XSLT element taken
from the XSLT-based test runner that XSpec produced.result. As a
function output, the XProc processor returns a map containing
output documents organized by output port. To get the string,
the XPath expression in the stylesheet uses map lookup syntax
?result after the function call.pf:system-property step we
looked at earlier, might be relevant in an arbitrary XProc
test target. When you travel from XSLT to XProc and back, by
calling a step function and then examining the result in XSLT,
the process doesn’t necessarily handle document properties in
the way you need for proper testing.text/csv. The XSLT-based test runner can
provide comma-separated text in an input argument to a step
function but lacks a way to assign a content type. Therefore,
the step sees a more generic content type: plain text. Upon
receiving an output map from the step function, the XSLT-based
test runner can retrieve a document, but its
content-type property is gone. Therefore, the
XSpec <x:expect> element cannot verify the
content type.<x:option name="my-opt" select="1"/>
causes the test runner to set up the option value in
XSLT and pass it to XProc.<p:document> elements
within XSpec <x:input> or
<x:option> elements. The resulting
document has the document properties you expect from an
XProc processor.<x:expect> verification
elements.document.csv file to be loaded by the XProc
processor, not the XSLT processor. The XProc processor also
assigns the document the text/csv content type
as expected, not the more generic text/plain
type. Item 3 runs the test target
(my-csv:test-target-step) using this document
that has content type text/csv. Items 4 and 5
return data to the XSLT-based test runner, and this data
includes all properties of output documents. Tasks 2, 3, 4,
and 5 all happen in the same XProc process. When the
XSLT-based test runner regains control, it evaluates the
<x:expect> elements. In that context, the
XSpec variable named x:document-properties
enables the test author to verify properties of output
documents, including the content-type
property.step runner.It’s a small XProc step that uses the standard
p:run
step to call a dynamically generated XProc step that appears
on the input port. The step runner’s corresponding step
function is what the XSLT-based test runner calls during
test execution. It’s the same function call from XSLT to
XProc for any test target or test case.map-of-inputs are a hint
that this step is not at all customized for a specific test
target or test case.test-case step.The XSpec compiler generates a test-case step for each test case in your XSpec test suite. When generating a test-case step, the XSpec compiler relies on information located in the XSpec document, as indicated in
| Information in XSpec | Resulting code in test-case step |
|---|---|
Location of XProc file containing the test target,
from the <x:description> element’s
xproc attribute |
<p:import> |
Type of step to call, from the
<x:call> element’s step
attribute |
XML element that invokes the step |
Any documents to pass in, from
<x:input> elements |
<p:with-input> |
Any option values to pass in, from
<x:option> elements |
<p:with-option> |
Any input ports and options that should use defaults
that the step declares, implied by the absence of
<x:input> or
<x:option> elements |
Absence of <p:with-input> or
<p:with-option> |
Whether to catch any dynamic errors, from the
optional catch attribute of ancestor
<x:scenario> elements |
<p:try> |
<x:call> element in
x:result
and x:document-properties, among other
things.p:identity and
p:json-merge steps. p:run and that the XSLT-based test runner
can call as an XPath function.<p:document> element within your test suite
for full control over input data; access to document properties;
and verifications that optionally focus on one output port at a
time. XSpec for testing XProc still has an XSpec feel, though,
because the way you do things like organize and label test
cases, express what you want to verify, and examine your test
results is consistent across supported languages.step functionfeature is worth knowing about so that you can apply it in situations where the feature fits well. Such situations might even include an XSpec test suite, where
<x:helper> integrates your custom XProc step
into the testing process (e.g., for normalization, whitespace
handling, or filtering) and you then call the step as a step
function. More generally, an XProc step that wears an XPath hat
can gain entry to—and exhibit its special XProc behaviors
in—XPath expressions that reside within XProc or other languages
that use XPath as an expression language.