ProX

Reasons

XProc pipelines describe step-by-step processing of XML using XML. One step might normalise a bunch of XML modules, the next validate the result and the last convert the normalised XML to something more human-readable. It's easy to add various conditionals, insert XQueries and include additional steps from XProc libraries, and so on.

I think it is a very, very cool spec (id-xproc-spec).

The pipelines are frequently extremely configurable, with options and parameters and outputs and various pipeline engine-specific configuration options, and they might be used as-is with several different stylesheets or other inputs. These in turn may be configurable, defining index generation, TOC generation, paper sizes and other options the stylesheet author has graciously provided as configurable with parameters.

A pipeline is run using a pipeline engine of some description, frequently from a shell script (for Calabash), from inside an XQuery, or some other kind of script. The various configuration options, inputs, etc, are all defined in that script, using whatever format and syntax the script uses. And here lies the problem.

Even though XProc is XML, processes XML, and uses XML as input, you have to write that script. And the more complex or flexible the pipeline is, the more variations there are when writing that script. And that script is not XML.

The pipelines are often part of a larger process that may or may not include other pipelines. The end user might want to choose between several different pipelines, then configure the one chosen with the various options and parameters, choose between several input stylesheets and finally configure the chosen stylesheet. And all this ends up in a script that is supposed to run the pipeline - the process that surrounds the pipeline, actually.

It follows that the pipeline is only as flexible as the means to configure it.

Logic

Enter the process XML. It describes XProc pipelines and their configuration, including any available stylesheets and other input, but also puts them all in the context of processes that surround the pipelines, and it does it all in XML files. There might be a Print process that includes pipelines for producing PDF, and MIF, a Web process that produces ePUB and HTML, and perhaps a Report process with pipelines that output reports.

The structure is roughly this:

Figure 1: ProX Structure

The Process XML, ProX for short[1], is a blueprint that lists all available processes, their associated pipelines, the command lines that configure these pipelines, including the available input files used by the pipelines and the parameters used to configure the inputs. It's a description of what is possible and the choices that need to be made before there can be a specific pipeline to run.

Choosing a process limits the available pipelines to those listed inside that process, choosing a pipeline limits the available command lines to those defined for that pipeline, and so on, like this:

Figure 2: ProX Logic

With all choices made, the blueprint is narrowed down to an instance that describes the running of a specific pipeline, like so:

Figure 3: A Configured Pipeline

The blueprint is an abstraction layer for a generic pipeline engine, setting the contexts in which it can run, how, and using what. The instance describes everything needed to run a specific process in a specific context, which is really useful because the instance can be converted to a suitable scripting format, whatever that format may be.

All that is required, then, is some processing to make the choices available to a user, some more processing to generate a script from the process instance, and finally run that script.

Uses

I'm implementing ProX in a CMS as I write this. The CMS will be able to output various formats and media using XProc pipelines, and quite a few of those pipelines and their input stylesheets are very configurable. Until now, it has not been possible to offer the end users these configuration options; even including a simple validation in a publishing process has been cumbersome at best.

Some rather different requirements come from an eXist-based Publish on Demand solution used to output individualised PDF documents for thousands of users, filtered from a large and infinitely variable content base. The end users are not allowed to change a single parameter, anywhere - their whole publishing user interface consists of a Publish button - but the publishing chain is complex, involving about a dozen pipelines that all gather content from various sources, convert and manipulate it, and validate the results before sending it on to the next step. When things go wrong, it is useful to if the publishing chain can be configured in various ways to spot where the problem is.

ProX will offer configurable publishing processes for system administrators. It will also help describe what processes there are in a given context (the complete blueprint document) and list the specific requirements for running a specific pipeline.

ProX in Some Detail

The ProX schema is not particularly complicated. basically, it describes one or more processes defined in a system. Every such process may use one or more pipelines, and every pipeline may be configured with various command line options, including zero or more input files frequently grouped as packages[2] (more about this below). The packages are usually XSLT stylesheets, and these, again, can be configured in various ways.

Figure 4: Processes

The total XML is a list of things that are possible. The user will need to pick one process, one pipeline, one set of command line options and the appropriate input packages to end up with a specific process.

Processes

The process structure groups related pipelines.

Figure 5: A Single Process

The grouping is intentionally somewhat arbitrary. If a process is defined as Delivery, the associated pipelines might be Print, Web and ePUB, handling those outputs for delivery, but, depending on the situation, a process might just as easily be defined as Print Publishing, leaving the online formats to another process, say, Web Publishing.

Note

Note that the packages are common to all pipelines in this particular process.

Pipelines

A pipeline is an abstraction for a single XProc pipeline[3] and its associated inputs and configuration options.

Figure 6: A Single Pipeline

The pipeline element includes a script element that that points out the actual XProc script, defined in a package, and one or more cmdline groups, that is, related configuration options for the script.

Command Lines

A command line (cmdline) is a group of related configuration options for running the pipeline that group is associated with. A pipeline may include one or more command lines.

Figure 7: A Single Command Line Group

The cmdline started out as the love child of various aspects of the XProc spec and the XProc engine of (many people's) choice, Calabash, but the current version attempts to be more generic in nature. It groups related configuration options for an associated pipeline so that once the listed choices have been made, the resulting cmdline instance is ready to run as-is.

The cmdline contains two basic parts, an engine configuration with engine-specific options from configuration files to Saxon options, and an XProc pipeline semantics-specific part that is more of a reflection of the spec[4].

The pipeline semantics define inputs, outputs, options, parameters, etc, that may be defined either beforehand or at runtime by the system.

The inputs structure defines every choice available for selection through one or more input elements.

Figure 8: A Single Input

The input element defines every kind of input available to the pipeline. Note that not every input is made available as a choice for the user; some are provided at runtime by the system, most notably the XML to be processed by the pipeline[5]. The markup includes attributes for processing user-selectable input (see section “Configuration and Parameter Handling”).

Some inputs may require parameters to be set. Typically, an XSLT stylesheet package will use parameters to define various properties, so these are made available in the input structure that points out the stylesheet. They are also listed with their packages (see section “Packages”).

Packages

A package is a group of related files needed for some aspect of pipeline processing. Commonly, a package is referenced as a single resource by an input port.

Figure 9: A Single Package

The main part of package is a list of links (locator elements) to the files that are part of the package[6], of which one or more may be identified as main file that imports the other files for use.

Packages are used by the process XML as an abstraction layer for an input; an input always points out a package rather than a single file. The system can locate the participating files using the package's file list when needed without having to look elsewhere.

The XProc script is also defined in a package rather than directly as a URL in the script element. For a script comprising several physical files, this is very useful. Similarly, the XProc engine configuration file (such as the one used by Calabash) (and any other such files) is listed in a package so that the system can retrieve everything required by a specific ProX instance before the process runs.

Last but not least, a package may list the parameters that are available for that package. A configurable parameter is marked as such using an attribute on parameter, and also includes the parameter's data type[7]. Some parameters may be required, which is also reflected by the markup.

Note

The parameters defined in package list package options that may be user-configurable. The parameters listed with a specific input in a cmdline are those that the system administrator had actally made available for configuration.

Naming

ProX is being implemented in a URN-based system as I write this. Every resource in it is identified and linked to using URNs rather than URLs - XML, obviously, but also images and other content, as well as stylesheets, schemas, etc. The URNs are unique within the system and include version and localisation information, like so:

urn:x-cassis:cos:00093445:sv-SE:0.19

Every resource is version handled, so it is easy to retrieve a specific version. And here's the neat part: ProX packages in the system identify resources in the exact same way. A package is a list of URNs with with specific versions, meaning that a specific package always identifies specific versions of every participating file. Prox files in the system are identified in the same way so any package version is identifiable and retrievable.

Modularisation

The ProX XML does not need to be a single file. Processes, pipelines, command lines and packages can all be modularised and reused. Note, for example, the inset elements in Figure 5 that are siblings to the pipeline and package elements; these are intended to link to pipeline and package modules, respectively.

Metadata

Every major ProX component (process, pipeline, cmdline, package) includes metadata used to identify the component in a GUI, but also to include context-sensitive help in that GUI.

Writing ProX

It is, of course, possible to write ProX XML in any XML editor. For my current project, I've added an XMetaL-based environment that includes some styling but otherwise uses the same features as the standard authoring environment in the system, with an integration to the database with check-in/out and versioning, URN-based linking, etc.

Figure 10: Adding A Package Locator

Here, authoring ProX is easy. Packages are compiled by including XLinks to the modules and then linked to from the elements that need them, using already implemented XLink- and URN-based linking functionality.

The ProX User Interface

The ProX blueprint was designed to be visualised in a GUI so the right process and pipeline can be selected and configured[8]. Conceivably, the system might allow for several different versions of the basic blueprint, each for its intended user. A power user might have several configuration options available to her while the casual user might only be allowed to choose between the basic processes, leaving the details to the system.

The basic selection procedure is largely sequential. Here's the first concept GUI:

Figure 11: GUI Concept

The pipeline, command line and package steps may all include additional configuration, but the principle should be clear.

Generating the GUI, Pt 1

There's an obvious candidate for creating a dynamic user interface based on XML input: W3C's XForms specification (id-xforms-spec). XForms has a somewhat bad reputation (see, for example, id-mvc-xforms-eric-vdl, Eric van der Vlist's terrific paper on [some of] the problems and some suggested solutions ), with some even claiming it to be dead for all practical purposes, but it is one of the few choices available for the purpose.

My first attempt at creating a ProX user interface used an XSLT stylesheet to generate a FreeMind mind map:

Figure 12: Generated Mind Map

The FreeMind mind map format is XML, and thus easy to grasp and convert to (id-freemind-xsd). The idea here is to insert FreeMind marker symbols to the selected options (nodes) and then convert the resulting FreeMind XML back to ProX, only including the marked nodes.

Of course, this approach is not without its flaws. Nothing stops you from inserting markers everywhere, which would result in useless ProX markup. We need something that allows exactly what the markup allows, so while a tree representation is useful and intuitive, it is only useful if a node can be easily selected and its unselected siblings locked (including a clear visualisation of the changed state).

The FreeMind format may or may not allow this, but I chose instead to have a closer look at XForms.

Design Choices

The process abstraction reflects a pipeline configuration from a process and systems perspective, and the resulting workflow for a user in the above user interfaces mostly reflects this approach. This is not necessarily wrong but the user's view regarding processing her content might actually be very different.

The User's POV

The original idea described a workflow like this:

  1. Select process

  2. Select pipeline

  3. Select the pipeline's command line

  4. Select among the stylesheet packages given as alternatives in the command line

This is really just a formalisation of one way of expressing a pipeline process. Only the last two (command line and stylesheet) included user-configurable options and the basic idea was to have them show up in a subform only when that step was selected.

The user probably doesn't care about the difference between configuring a pipeline and configuring a stylesheet, however. The objective is to run a process without distractions to the extent possible so better is probably:

  1. Select process

  2. Select pipeline

  3. Select and configure stylesheet (or rather, process output)

And depending on the situation, this might be even more appropriate:

  1. Select process

  2. Select and configure output

This is perhaps too simplistic. While the user doesn't necessarily care about the difference between a process and a pipeline, the concept of a process surrounding the pipeline was introduced because the processes might be so different from each other that the abstraction becomes meaningful. The original concept (see Figure 11) lists Delivery, Reviews, Validation and Reports as examples of different processes, the idea being to reflect different work flows with some very different outputs as expected results.

I'd argue that the distinction is meaningful.

But do users need to know or care about the difference between the command line and stylesheet selections? Here, the answer is probably no. The pipeline does something in the defined workflow while the command line and stylesheet options configure the output. The difference between configuring the pipeline and the stylesheet is a subtle one; unless you are an admin, you probably won't care. Better (than the the concept GUI in Figure 11) is something like this:

Figure 13: Pipeline, Cmdline Configuration Mixed

To the end user, this XForm is about configuring output, not what's behind the scenes. The underlying XML does not change; with the GUI is adapted for different user categories, the same XML can provide an admin with a different UI. The difference happens depending on how the initial ProX is processed to generate a form.

The right answer, then, is that these are all possible, simply by preprocessing ProX and by writing appropriate XForms.

The Admin's POV

The underlying XML is in no way changed when simplifying the selection process for a user. What changes is the form, and possibly some preprocessing. The above suggests a simple GUI, with most of the configuration already made. All the admin needs to do is to write the ProX blueprint with complete command lines, with all of the choices split into separate command line groups as listed in the GUI.

The GUI shown in Figure 13 is generated from this example:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-model href="file:/home/ari/mystuff/SGML/DTD/Process-XML/RelaxNG/processes.rnc" type="application/relax-ng-compact-syntax"?>
<processes 
   id="processes-2013-4-9-16-53-8-39562387-" 
   xmlns:xlink="http://www.w3.org/1999/xlink">
   
   <!-- Print Publishing Process -->
   <process id="id-pdf-process">
      <metadata id="metadata-2013-4-9-16-53-8-39562387-">
         <title id="title-2013-4-9-16-53-8-39562387-">Print Publishing</title>
         <description id="description-2013-4-9-16-53-8-39562387-">
            <p id="p-2013-4-9-16-53-8-39562387-">Print publishing for COSML documents</p>
         </description>
      </metadata>
      <pipelines id="pipelines-2013-4-9-16-53-8-39562387-">
         
         <!-- PDF Pipeline -->
         <pipeline id="id-pipeline-pdf-1">
            <metadata id="metadata-2013-4-9-16-53-8-39562387-1">
               <title id="title-2013-4-9-16-53-8-39562387-1">Publish PDF</title>
               <description id="description-2013-4-9-16-53-8-39562387-1">
                  <p id="p-2013-4-9-16-53-8-39562387-1">Normalizes, validates and converts a COSML document to PDF</p>
               </description>
            </metadata>
            <script xmlns:xlink="http://www.w3.org/1999/xlink" type="pkg"
               id="script-2013-4-9-16-53-8-39562387-"
               xlink:href="urn:x-cassis:r1:cos:00002712:sv-SE:0.1#id-xproc-pdf"
               xlink:title=" XProc Pipeline for Normalize, Validate and PDF Normalizes, validates and publishes in PDF a COSML document "/>
            <cmdlines id="cmdlines-2013-4-9-16-53-8-39562387-">
               
               <!-- COSML Internal XSL -->
               <cmdline id="id-cmdline-cos-internal-pdf">
                  <metadata id="metadata-2013-4-9-16-53-8-39562387-2">
                     <title id="title-2013-4-9-16-53-8-39562387-2">COS Internal Template</title>
                     <description id="description-2013-4-9-16-53-8-39562387-2">
                        <p id="p-2013-4-9-16-53-8-39562387-2">Configures the pipeline for the "COS Internal" template</p>
                     </description>
                  </metadata>
                  <engine-config>
                     <config type="pkg" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#id-conf-calabash"/>
                  </engine-config>
                  <inputs id="inputs-2013-4-9-16-53-8-39562387-">
                     <input choice="no" id="input-2013-4-9-16-53-8-39562387-">
                        <port id="port-2013-4-9-16-53-8-39562387-">document</port>
                        <value 
                           xmlns:xlink="http://www.w3.org/1999/xlink" 
                           type="external"
                           xlink:type="simple" 
                           id="value-2013-4-9-16-53-8-39562387-"
                           mimetype="application/xml">DOCUMENT-PLACEHOLDER</value>
                     </input>
                     <input choice="no" id="input-2013-4-9-16-53-8-39562387-1">
                        <port id="port-2013-4-9-16-53-8-39562387-1">stylesheet</port>
                        <value 
                           xmlns:xlink="http://www.w3.org/1999/xlink" 
                           type="pkg"
                           xlink:href="urn:x-cassis:r1:cos:00002712:sv-SE:0.1#id-xslfo-cosml"
                           xlink:type="simple" id="value-2013-4-9-16-53-8-39562387-1"
                           xlink:title=" XSL-FO Package for COSML PDF Converts COSML documents to XSL-FO format for COS PDF layout "/>
                        <params id="params-2013-4-9-16-53-8-39562387-">
                           <!-- Index generation -->
                           <param 
                              choice="yes" 
                              ctype="boolean" 
                              id="param-2013-4-9-16-53-8-39562387-">
                              <port id="port-2013-4-9-16-53-8-39562387-2">xslt-params</port>
                              <name id="name-2013-4-9-16-53-8-39562387-">generate.index</name>
                              <value 
                                 xmlns:xlink="http://www.w3.org/1999/xlink" 
                                 type="string"
                                 xlink:type="simple" 
                                 id="value-2013-4-9-16-53-8-39562387-2">false</value>
                           </param>
                           <!-- XEP Extensions -->
                           <param 
                              choice="no" 
                              id="param-2013-4-9-16-53-8-39562387-1">
                              <port id="port-2013-4-9-16-53-8-39562387-3">xslt-params</port>
                              <name id="name-2013-4-9-16-53-8-39562387-1">xep.extensions</name>
                              <value xmlns:xlink="http://www.w3.org/1999/xlink" type="string"
                                 xlink:type="simple" id="value-2013-4-9-16-53-8-39562387-3"
                                 >0</value>
                           </param>
                           <!-- XSL-FO Bookmark Generation -->
                           <param 
                              choice="yes" 
                              ctype="boolean"
                              id="param-2013-4-9-16-53-8-39562387-2">
                              <port id="port-2013-4-9-16-53-8-39562387-4">xslt-params</port>
                              <name id="name-2013-4-9-16-53-8-39562387-2">xslfo.bookmarks</name>
                              <value 
                                 xmlns:xlink="http://www.w3.org/1999/xlink" 
                                 type="string"
                                 xlink:type="simple" id="value-2013-4-9-16-53-8-39562387-4">true</value>
                           </param>
                           <!-- TOC Generation -->
                           <param 
                              choice="yes" 
                              ctype="boolean"
                              id="param-2013-4-9-16-53-8-39514778-2">
                              <port id="port-2013-4-9-16-53-8-9653444-4">xslt-params</port>
                              <name id="name-2013-4-9-16-53-8-1928364-2">create.toc</name>
                              <value 
                                 xmlns:xlink="http://www.w3.org/1999/xlink" 
                                 type="string"
                                 xlink:type="simple" 
                                 id="value-2013-7-10-16-53-8-764625737-3">true</value>
                           </param>
                           <!-- TOC Depth -->
                           <param 
                              choice="yes" 
                              ctype="list1"
                              id="param-2013-4-9-16-53-8-2385485-2">
                              <port id="port-2013-7-10-16-34-8-9283444-4">xslt-params</port>
                              <name id="name-2013-7-10-16-50-3-1946564-2">toc.depth</name>
                              <value 
                                 xmlns:xlink="http://www.w3.org/1999/xlink" 
                                 type="string"
                                 xlink:type="simple" 
                                 id="value-2013-7-10-16-53-8-764625737-4">2</value>
                              <value
                                 xmlns:xlink="http://www.w3.org/1999/xlink"
                                 id="value-13-07-10-12345-1" 
                                 type="string">1</value>
                              <value
                                 xmlns:xlink="http://www.w3.org/1999/xlink"
                                 id="value-13-07-10-12345-2" 
                                 type="string">3</value>
                           </param>
                        </params>
                     </input>
                     <input choice="no" id="input-2013-4-9-16-53-8-39562387-2">
                        <port id="port-2013-4-9-16-53-8-39562387-5">stylesheet-norm</port>
                        <value 
                           xmlns:xlink="http://www.w3.org/1999/xlink" 
                           type="pkg"
                           xlink:href="urn:x-cassis:r1:cos:00002712:sv-SE:0.1#id-normalize"
                           xlink:type="simple" id="value-2013-4-9-16-53-8-39562387-5"
                           xlink:title=" Normalize XSLT Stylesheet for applics filtering and module normalization for COSML documents "/>
                     </input>
                  </inputs>
                  <options id="options-2013-4-9-16-53-8-39562387-">
                     <option choice="no" id="option-2013-4-9-16-53-8-39562387-">
                        <name id="name-2013-4-9-16-53-8-39562387-3">pdf</name>
                        <value 
                           xmlns:xlink="http://www.w3.org/1999/xlink" 
                           type="external"
                           xlink:type="simple" 
                           id="value-2013-4-9-16-53-8-39562387-6">PDF-PLACEHOLDER.pdf</value>
                     </option>
                  </options>
               </cmdline>
               
               <!-- COSML Formal XSL -->
               <cmdline id="id-cmdline-cos-formal-pdf">
                  <metadata id="metadata-2013-4-9-16-53-8-39562387-3">
                     <title id="title-2013-4-9-16-53-8-39562387-3">COS Formal Template</title>
                     <description id="description-2013-4-9-16-53-8-39562387-3">
                        <p id="p-2013-4-9-16-53-8-39562387-3">Configures the pipeline for the "COS Formal" template</p>
                     </description>
                  </metadata>
                  <engine-config>
                     <config type="pkg" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#id-conf-calabash"/>
                  </engine-config>
                  <inputs id="inputs-2013-4-9-16-53-8-39562387-1">
                     <input choice="no" id="input-2013-4-9-16-53-8-39562387-3">
                        <port id="port-2013-4-9-16-53-8-39562387-6">document</port>
                        <value 
                           xmlns:xlink="http://www.w3.org/1999/xlink" 
                           type="external"
                           xlink:type="simple" 
                           id="value-2013-4-9-16-53-8-39562387-7">DOCUMENT-PLACEHOLDER</value>
                     </input>
                     <input choice="no" id="input-2013-4-9-16-53-8-39562387-4">
                        <port id="port-2013-4-9-16-53-8-39562387-7">stylesheet</port>
                        <value 
                           xmlns:xlink="http://www.w3.org/1999/xlink" 
                           type="pkg"
                           xlink:href="package-xslfo-cosml.xml#id-xslfo-cosml" 
                           xlink:type="simple"
                           id="value-2013-4-9-16-53-8-39562387-8"/>
                     </input>
                     <input choice="no" id="input-2013-4-9-16-53-8-39562387-5">
                        <port id="port-2013-4-9-16-53-8-39562387-8">stylesheet-norm</port>
                        <value 
                           xmlns:xlink="http://www.w3.org/1999/xlink" type="pkg"
                           xlink:href="package-normalize.xml#id-normalize" 
                           xlink:type="simple"
                           id="value-2013-4-9-16-53-8-39562387-9"/>
                     </input>
                  </inputs>
                  <options id="options-2013-4-9-16-53-8-39562387-1">
                     <option choice="no" id="option-2013-4-9-16-53-8-39562387-1">
                        <name id="name-2013-4-9-16-53-8-39562387-4">pdf</name>
                        <value 
                           xmlns:xlink="http://www.w3.org/1999/xlink" 
                           type="external"
                           xlink:type="simple" 
                           id="value-2013-4-9-16-53-8-39562387-10"
                           mimetype="application/pdf">PDF-PLACEHOLDER.pdf</value>
                     </option>
                  </options>
               </cmdline>
            </cmdlines>
         </pipeline>
      </pipelines>
      
      <!-- Packages for Print -->
      <packages 
         xml:base="file:///e:/SGML/DTD/Cassis/Process-XML/"
         id="packages-2013-4-9-16-53-8-39562387-">
         
         <!-- XProc Normalize, Validate, XSLFO Pipeline Package -->
         <package id="id-xproc-pdf">
            <metadata id="metadata-2013-4-9-16-53-8-39562387-4">
               <title id="title-2013-4-9-16-53-8-39562387-4">XProc Pipeline for Normalize, Validate and PDF</title>
               <description id="description-2013-4-9-16-53-8-39562387-4">
                  <p id="p-2013-4-9-16-53-8-39562387-4">Normalizes, validates and publishes in PDF a COSML document</p>
               </description>
            </metadata>
            <!-- publish-cosml-pdf.xpl -->
            <locator 
               type="main" 
               xlink:href="urn:x-cassis:r1:cos:00002715:sv-SE:0.1"
               xmlns:xlink="http://www.w3.org/1999/xlink" 
               id="locator-2013-4-10-10-32-24-12830403-" />
         </package>
         
         
         <!-- COSML Internal XSL-FO Package -->
         <package id="id-xslfo-cosml">
            <metadata id="metadata-2013-4-9-16-53-8-39562387-5">
               <title id="title-2013-4-9-16-53-8-39562387-5">XSL-FO Package for COSML PDF</title>
               <description id="description-2013-4-9-16-53-8-39562387-5">
                  <p id="p-2013-4-9-16-53-8-39562387-5">Converts COSML documents to XSL-FO format
                     for COS PDF layout</p>
               </description>
            </metadata>
            
            <!-- Stylesheet parameters -->
            <params id="params-2013-4-9-16-53-8-39562387-1">
               <!-- Index generation -->
               <param id="param-2013-4-9-16-53-8-39562387-3">
                  <port id="port-2013-4-9-16-53-8-39562387-9">xslt-params</port>
                  <name id="name-2013-4-9-16-53-8-39562387-5">generate.index</name>
                  <value type="string" id="value-2013-4-9-16-53-8-39562387-11">0</value>
               </param>
               <!-- XEP Extensions -->
               <param id="param-2013-4-9-16-53-8-39562387-4">
                  <port id="port-2013-4-9-16-53-8-39562387-10">xslt-params</port>
                  <name id="name-2013-4-9-16-53-8-39562387-6">xep.extensions</name>
                  <value type="string" id="value-2013-4-9-16-53-8-39562387-12">0</value>
               </param>
               <!-- XSL-FO Bookmark Generation -->
               <param id="param-2013-4-9-16-53-8-39562387-5">
                  <port id="port-2013-4-9-16-53-8-39562387-11">xslt-params</port>
                  <name id="name-2013-4-9-16-53-8-39562387-7">xslfo.bookmarks</name>
                  <value type="string" id="value-2013-4-9-16-53-8-39562387-13">1</value>
               </param>
            </params>
            
            <!-- XSLT -->
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000232:sv-SE:0.6" 
               xlink:title="COS Internal XSLT"
               type="main" id="locator-2013-4-9-16-53-8-39562387-1"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000074:sv-SE:0.11"
               id="locator-2013-4-9-16-53-8-39562387-2"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000059:sv-SE:0.2"
               id="locator-2013-4-9-16-53-8-39562387-3"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000070:sv-SE:0.15"
               id="locator-2013-4-9-16-53-8-39562387-4" 
               xlink:title="Layout"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000876:sv-SE:0.2"
               id="locator-2013-4-9-16-53-8-39562387-5" 
               xlink:title="bookmarks.xsl"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000075:sv-SE:0.17"
               id="locator-2013-4-9-16-53-8-39562387-6"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000072:sv-SE:0.10"
               id="locator-2013-4-9-16-53-8-39562387-7" 
               xlink:title="meta-data.xsl"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000078:sv-SE:0.9"
               id="locator-2013-4-9-16-53-8-39562387-8" 
               xlink:title="TOC"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000062:sv-SE:0.9"
               id="locator-2013-4-9-16-53-8-39562387-9"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000233:sv-SE:0.8"
               id="locator-2013-4-9-16-53-8-39562387-10"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000061:sv-SE:0.29"
               id="locator-2013-4-9-16-53-8-39562387-11"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000065:sv-SE:0.6"
               id="locator-2013-4-9-16-53-8-39562387-12"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000071:sv-SE:0.6"
               id="locator-2013-4-9-16-53-8-39562387-13"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000077:sv-SE:0.6"
               id="locator-2013-4-9-16-53-8-39562387-14"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000079:sv-SE:0.7"
               id="locator-2013-4-9-16-53-8-39562387-15"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000060:sv-SE:0.7"
               id="locator-2013-4-9-16-53-8-39562387-16"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000064:sv-SE:0.8"
               id="locator-2013-4-9-16-53-8-39562387-17"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000066:sv-SE:0.2"
               id="locator-2013-4-9-16-53-8-39562387-18"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000069:sv-SE:0.3"
               id="locator-2013-4-9-16-53-8-39562387-19"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000063:sv-SE:0.3"
               id="locator-2013-4-9-16-53-8-39562387-20"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000785:sv-SE:0.6"
               id="locator-2013-4-9-16-53-8-39562387-21"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000076:sv-SE:0.10" type="aux" xlink:title="Strings"
               id="locator-2013-4-9-16-53-8-39562387-22"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000230:sv-SE:0.1" type="aux"
               id="locator-2013-4-9-16-53-8-39562387-23"/>
            <locator xmlns:xlink="http://www.w3.org/1999/xlink"
               xlink:href="urn:x-cassis:r1:cos:00000426:sv-SE:0.1" type="aux"
               id="locator-2013-4-9-16-53-8-39562387-24" xlink:title="tux.jpg"/>
         </package>
      </packages>
   </process>
   
   <!-- Wep PublishingProcess -->
   <process id="id-web-process">
      <metadata id="metadata-2013-4-9-16-53-8-39562387-6">
         <title id="title-2013-4-9-16-53-8-39562387-6">Web Publishing</title>
         <description id="description-2013-4-9-16-53-8-39562387-6">
            <p id="p-2013-4-9-16-53-8-39562387-6">Publishes COSML documents for web output</p>
         </description>
      </metadata>
      
      <pipelines id="pipelines-2013-4-9-16-53-8-39562387-1">
         
         <!-- Pipeline for HTML -->
         <pipeline id="id-pipeline-web-1">
            <metadata id="metadata-2013-4-9-16-53-8-39562387-7">
               <title id="title-2013-4-9-16-53-8-39562387-7">Publish HTML</title>
               <description id="description-2013-4-9-16-53-8-39562387-7">
                  <p id="p-2013-4-9-16-53-8-39562387-7">Normalizes, validates and publishes COSML documents as single-file HTML</p>
               </description>
            </metadata>
            <script id="script-2013-4-9-16-53-8-39562387-1"
               xlink:href="urn:x-cassis:r1:cos:00002712:sv-SE:0.6#package-2013-5-19-11-12-49-71312191-1"
               xlink:title="XProc COSML2XHTMLNormalises, validates and converts COSML to XHTML."
               type="pkg"/>
            <cmdlines id="cmdlines-2013-4-9-16-53-8-39562387-1">
               
               <!-- Single-file HTML Config -->
               <cmdline id="id-cmdline-single-file-HTML-1">
                  <metadata id="metadata-2013-4-9-16-53-8-39562387-8">
                     <title id="title-2013-4-9-16-53-8-39562387-8">COS HTML</title>
                     <description id="description-2013-4-9-16-53-8-39562387-8">
                        <p id="p-2013-4-9-16-53-8-39562387-8">Configures the pipeline for single-file HTML</p>
                     </description>
                  </metadata>
                  <engine-config>
                     <config type="pkg" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#id-conf-calabash"/>
                  </engine-config>
                  <inputs>
                     <input>
                        <port>document</port>
                        <value 
                           id="id-html-docroot"
                           input-type="doc-root" 
                           type="external"/>
                     </input>
                     <input id="input-2013-5-19-11-12-49-71312191-1">
                        <port>stylesheet</port>
                        <value
                           xlink:href="urn:x-cassis:r1:cos:00002712:sv-SE:0.6#package-2013-5-19-11-12-49-71312191-"
                           xlink:title="COSML XHTML XSLTConverts COSML to XHTML" 
                           type="pkg"/>
                     </input>
                     <input id="input-2013-5-19-11-12-49-71312191-">
                        <port>stylesheet-norm</port>
                        <value 
                           xlink:href="urn:x-cassis:r1:cos:00002712:sv-SE:0.6#id-normalize"
                           xlink:title=" Normalize XSLT Stylesheet for applics filtering and module normalization for COSML documents "
                           type="pkg"/>
                     </input>
                  </inputs>
                  <options>
                     <option>
                        <name>htm</name>
                        <value 
                           id="id-htm-out"
                           type="external" 
                           output-type="primary"/>
                     </option>
                  </options>
               </cmdline>
            </cmdlines>
         </pipeline>
      </pipelines>
      
      <!-- Web Publishing Packages -->
      <packages>
         
         <!-- XProc for COSML to XHTML -->
         <package id="package-2013-5-19-11-12-49-71312191-1">
            <metadata>
               <title>XProc COSML2XHTML</title>
               <description>
                  <p>XProc to normalise, validate and convert COSML to XHTML.</p>
               </description>
            </metadata>
            <locator 
               id="id-xhtml-xproc"
               type="main" 
               xlink:href="urn:x-cassis:r1:cos:00002756:sv-SE:0.2"
               xlink:title="publish-cosml-html.xpl" 
               xmlns:xlink="http://www.w3.org/1999/xlink"/>
         </package>
         
         <!-- XSLT for COSML to XHTML -->
         <package id="package-2013-5-19-11-12-49-71312191-">
            <metadata>
               <title>COSML XHTML XSLT</title>
               <description>
                  <p>Converts COSML to XHTML</p>
               </description>
            </metadata>
            <locator 
               id="id-xhtml-xslt"
               xlink:href="urn:x-cassis:r1:cos:00002755:sv-SE:0.1"
               xlink:title="cosml2html-ti.xsl" 
               xmlns:xlink="http://www.w3.org/1999/xlink"
               type="main"/>
         </package>
      </packages>
   </process>
   
   <!-- Content Validation Process -->
   <process id="process-2013-5-19-11-12-49-71312191-">
      <metadata>
         <title>Content Validation</title>
         <description>
            <p>Content validation processes</p>
         </description>
      </metadata>
      
      <!-- Content Validation Pipelines -->
      <pipelines>
         
         <!-- Xref Check Pipeline -->
         <pipeline id="pipeline-2013-5-19-11-12-49-71312191-">
            <metadata>
               <title>Cross-reference Check</title>
               <description>
                  <p>Pipeline for checking cross-references in COSML documents</p>
               </description>
            </metadata>
            <script
               xlink:href="urn:x-cassis:r1:cos:00002712:sv-SE:0.6#package-2013-5-19-11-12-49-71312191-3"
               xlink:title="XProc Xref CheckGenerates a module list for XML in scope, checks the xrefs and produces a report"
               type="pkg"/>
            <cmdlines>
               
               <!-- Xref Validation Config -->
               <cmdline id="cmdline-2013-5-19-11-12-49-71312191-">
                  <metadata>
                     <title>Xref Check Configuration</title>
                     <description>
                        <p>Configures the cross-reference check and report</p>
                     </description>
                  </metadata>
                  <engine-config>
                     <config type="pkg" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="#id-conf-calabash"/>
                  </engine-config>
                  <inputs>
                     <input>
                        <port>map</port>
                        <value 
                           type="external" 
                           input-type="map"/>
                     </input>
                     <input id="input-2013-5-19-11-12-49-71312191-2">
                        <port>stylesheet</port>
                        <value
                           xlink:href="urn:x-cassis:r1:cos:00002712:sv-SE:0.6#package-2013-5-19-11-12-49-71312191-2"
                           xlink:title="XSLT Xref CheckValidates cross-references in publication"
                           type="pkg"/>
                     </input>
                  </inputs>
                  <options>
                     <option>
                        <name>htm</name>
                        <value id="id-value-xref-htm" type="external" mimetype="text/xhtml"/>
                     </option>
                     <option>
                        <name>file-url</name>
                        <value id="id-file-url" type="external" mimetype="text/xml"/>
                     </option>
                  </options>
               </cmdline>
            </cmdlines>
         </pipeline>
      </pipelines>
      
      <packages>
         
         <!-- XProc for Xref Check -->
         <package id="package-2013-5-19-11-12-49-71312191-3">
            <metadata>
               <title>XProc Xref Check</title>
               <description>
                  <p>Generates a module list for XML in scope, checks the xrefs and produces a report</p>
               </description>
            </metadata>
            <locator 
               id="id-xproc-xref"
               xlink:href="urn:x-cassis:r1:cos:00002757:sv-SE:0.2"
               xlink:title="xref-check-cosml.xpl" 
               xmlns:xlink="http://www.w3.org/1999/xlink" 
               type="main"/>
         </package>
         
         <!-- XSLT for Xref Check -->
         <package id="package-2013-5-19-11-12-49-71312191-2">
            <metadata>
               <title>XSLT Xref Check</title>
               <description>
                  <p>Validates cross-references in publication</p>
               </description>
            </metadata>
            <locator 
               id="id-xslt-xref"
               xlink:href="urn:x-cassis:r1:cos:00002754:sv-SE:0.1"
               xlink:title="link-target-check-multifile.xsl"
               xmlns:xlink="http://www.w3.org/1999/xlink" 
               type="main"/>
         </package>
      </packages>
   </process>
   
   <packages id="packages-2013-4-9-16-53-8-39562387-1">
      
      <!-- XSLT for Normalizing COSML -->
      <package id="id-normalize" type="xslt">
         <metadata id="metadata-2013-4-9-16-53-8-39562387-9">
            <title id="title-2013-4-9-16-53-8-39562387-9">Normalize XSLT</title>
            <description id="description-2013-4-9-16-53-8-39562387-9">
               <p id="p-2013-4-9-16-53-8-39562387-9">Stylesheet for applics filtering and module
                  normalization for COSML documents</p>
            </description>
         </metadata>
         <!-- No parameters required. -->
         <locator 
            xmlns:xlink="http://www.w3.org/1999/xlink"
            xlink:href="urn:x-cassis:r1:cos:00000073:sv-SE:0.4"
            id="locator-2013-4-9-16-53-8-39562387-26" 
            type="main" 
            xlink:title="Normalize XSLT"/>
      </package>
      
      <!-- Calabash Engine Configuration File -->
      <package id="id-conf-calabash">
         <metadata id="metadata-2013-5-2-21-40-30-37001288-">
            <title id="title-2013-5-2-21-40-30-37001288-">Calabash Configuration</title>
            <description id="description-2013-5-2-21-40-30-37001288-">
               <p id="p-2013-5-2-21-40-30-37001288-">Configures Calabash</p>
            </description>
         </metadata>
         <locator 
            xlink:href="urn:x-cassis:r1:cos:00002745:sv-SE:0.1"
            xmlns:xlink="http://www.w3.org/1999/xlink" 
            type="main" 
            id="id-loc-calabash-config"/>
      </package>
      
      
      
      <!-- Wrapper ProX Resources -->
      <package id="id-wrapper-resources">
         <metadata id="metadata-2013-5-2-21-40-30-37001288-1">
            <title id="title-2013-5-2-21-40-30-37001288-1">Wrapper Pipeline Processing</title>
            <description id="description-2013-5-2-21-40-30-37001288-1">
               <p id="p-2013-5-2-21-40-30-37001288-1">These files are used for running the wrapper pipeline.</p>
            </description>
         </metadata>

         <locator xlink:href="urn:x-cassis:r1:cos:00002735:sv-SE:0.1"
            xmlns:xlink="http://www.w3.org/1999/xlink" id="id-wrapper-xpl"/>
         
         <locator xlink:href="urn:x-cassis:r1:cos:00002732:sv-SE:0.1"
            xmlns:xlink="http://www.w3.org/1999/xlink" id="id-prox-fix"/>
         
         <locator xlink:href="urn:x-cassis:r1:cos:00002733:sv-SE:0.1"
            xmlns:xlink="http://www.w3.org/1999/xlink" id="id-urn2url"/>
         
         <locator xlink:href="urn:x-cassis:r1:cos:00002731:sv-SE:0.1"
            xmlns:xlink="http://www.w3.org/1999/xlink" id="id-prox2bat"/>
         
         <locator xlink:href="urn:x-cassis:r1:cos:00002734:sv-SE:0.1"
            xmlns:xlink="http://www.w3.org/1999/xlink" id="id-prox2shell-config"/>
      </package>
      
      <!-- XForms -->
      <package id="id-xform">
         <metadata>
            <title>ProX XForms Package</title>
            <description>
               <p>XForms for selecting and configuring a process, based on a ProX blueprint.</p>
            </description>
         </metadata>
         <locator 
            xlink:href="urn:prox:xform:0.1" 
            type="main" 
            id="id-loc-xform"/>
      </package>
   </packages>
</processes>

The point here is that this could easily be (mostly) all that is required; while additional configuration may be useful, it is perfectly feasible to limit the choices to a straight-forward wizard-like behaviour.

Configuration and Parameter Handling

The ProX RNC schema includes a set of attributes used by any element that may offer configuration choices by the user:

choice.att =
    attribute choice { "yes" | "no" }?,  # [ a:defaultValue = "no" ] 
    attribute group { xsd:IDREF }?,
    attribute ctype { "boolean" | "list" | "list1" | xsd:string | xsd:decimal | xsd:date }?,
    attribute req { "yes" | "no" }?

The choice attribute indicates if the parameter is configurable, while ctype indicates the type of user input required.

The group attribute is an IDREF to a related parameter and indicates a dependency to that parameter. For example, a parameter may be used to set the table of contents depth, but it is useless if another parameter has turned off the TOC generation. The first parameter needs to include a group IDREF to the second so only relevant options are made available when configuring a ProX blueprint.

The configuration options available in a cmdline are made available to the user by defining them as choices in the ProX blueprint. For example, logging alternatives might be made available like so:

<engine-config>
    <config 
        type="pkg" 
        xmlns:xlink="http://www.w3.org/1999/xlink" 
        xlink:href="calabash-config.xml"/>
    <log-style 
        id="id-log" 
        choice="yes" 
        ctype="list1" 
        style="off"/>
    <log-style 
        group="id-log" 
        choice="yes" 
        ctype="list1" 
        style="plain"/>
</engine-config>

Here, the logging options are made available as a list1, much like a radio button list for a user interface. Note the id and group attributes in the first and second log-style elements: the group attribute is an IDREF that references the id in the first. The choice that contains the id attribute referenced by the other is the default.

A true/false schema-aware option may only have two values and can therefore be represented like this:

<schema-aware process="true" choice="yes" ctype="boolean"/>

Here, a default value is given in the process attribute. Note the ctype attribute that identifies the type to be used when representing the choice for the user. The XForm template will show this as a checkbox in normal circumstances.

Data types are necessary when representing XSLT parameter alternatives. This one decides if an index should be generated by a PDF publishing pipeline:

<!-- Index generation -->
<param 
    choice="yes" 
    ctype="boolean" 
    id="param-2013-4-9-16-53-8-39562387-">
    <port id="port-2013-4-9-16-53-8-39562387-2">xslt-params</port>
    <name id="name-2013-4-9-16-53-8-39562387-">generate.index</name>
    <value 
        xmlns:xlink="http://www.w3.org/1999/xlink" 
        type="string"
        xlink:type="simple" 
        id="value-2013-4-9-16-53-8-39562387-2">false</value>
</param>

choice="yes" means that this is a user-configurable option, with the default given by the contents. This will render as a checkbox (see Figure 17), as param/@ctype="boolean" is used to bind it to an xs:boolean in the XForm.

These two parameters decide if and how a TOC will be generated:

<!-- TOC Generation -->
<param 
    choice="yes" 
    ctype="boolean"
    id="param-2013-4-9-16-53-8-39514778-2">
    <port id="port-2013-4-9-16-53-8-9653444-4">xslt-params</port>
    <name id="name-2013-4-9-16-53-8-1928364-2">create.toc</name>
    <value 
        xmlns:xlink="http://www.w3.org/1999/xlink" 
        type="string"
        xlink:type="simple" 
        id="value-2013-7-10-16-53-8-764625737-3">true</value>
</param>
<!-- TOC Depth -->
<param 
    choice="yes" 
    ctype="list1"
    id="param-2013-4-9-16-53-8-2385485-2"
    group="value-2013-7-10-16-53-8-764625737-3">
    <port id="port-2013-7-10-16-34-8-9283444-4">xslt-params</port>
    <name id="name-2013-7-10-16-50-3-1946564-2">toc.depth</name>
    <value 
        xmlns:xlink="http://www.w3.org/1999/xlink" 
        type="string"
        xlink:type="simple" 
        id="value-2013-7-10-16-53-8-764625737-4">2</value>
    <value
        xmlns:xlink="http://www.w3.org/1999/xlink"
        id="value-13-07-10-12345-1" 
        type="string">1</value>
    <value
        xmlns:xlink="http://www.w3.org/1999/xlink"
        id="value-13-07-10-12345-2" 
        type="string">3</value>
</param>

The first param is an ON/OFF switch for TOC generation (hence the @ctype="boolean"), while the second allows choosing the number of section levels to be included in the TOC. Since the second param would be meaningless without the first being set to true, the second param includes a group attribute that is an IDREF to the first param, which means that the second value depends on the first. The group reference is used in the XForm to show or hide controls, depending on their relevance.

The second param has ctype="list1", which means that the listed values should be represented as a list1 in the XForm. The first is used as default when processing.

Note

When compiling the package that lists a stylesheet's participating module, the package needs to always include definitions for any parameters that are not explicitly set by the stylesheets.

System Context

The ProX blueprint configuration described above only tells how to configure a process, not what XML if any it processes, other than indirectly. It tells how to do something, not what it should apply the process to. Of course, for any process involving an XML file to be processed, there will be a matching input in the pipeline, but the point here is that the XML is only identified at runtime, if it is identified[9].

The system where proX is being implemented allows for two basic workflows: either the process is configured first and the XML to be processed is pointed out later, or the other way around. The first is useful for new processes and for any process that does not involve an XML file. If an XML file needs to be selected, it can be located by the system using its in-place browsing capabilities after the ProX instance is configured and saved, that is, when a publishing process has been fully configured and saved.

Today, the system uses something called a configuration to point out the XML to be processed. The configuration is an XML file that points out a root XML file, including language and version, along with some system-specific metadata, and then publishes it using an XSL-FO stylesheet[10]:

<?xml version="1.0" encoding="utf-8"?>
<CassisTIConfiguration>
  <Versioning>-1</Versioning>
  <VersionInfo />
  <PortalID>0</PortalID>
  <ConfigURN>urn:x-cassis:r1:cos:00002216:sv-SE:0.2</ConfigURN>
  <ConfigID>2216</ConfigID>
  <ConfigVersionID>-1</ConfigVersionID>
  <ForProcessMgr>true</ForProcessMgr>
  <Name>Balisage 2012 Whitepaper</Name>
  <NoOfNamingFields>0</NoOfNamingFields>
  <NamingFields />
  <Code />
  <Description>Balisage 2012 whitepaper</Description>
  <ThumbNailImage />
  <Modules />
  <XmlModuleID>2112</XmlModuleID>
  <XmlModuleFolderID>83</XmlModuleFolderID>
  <XmlModuleName>Bal2012nord0128.xml</XmlModuleName>
  <XmlModuleURN>urn:x-cassis:r1:cos:00002112:sv-SE:0.43</XmlModuleURN>
  <XmlModuleLanguageID>2</XmlModuleLanguageID>
  <XmlModuleVersionMajor>0</XmlModuleVersionMajor>
  <XmlModuleVersionMinor>43</XmlModuleVersionMinor>
  <LanguageID>2</LanguageID>
  <VersionMajor>0</VersionMajor>
  <VersionMinor>2</VersionMinor>
  <Applicabilities />
  <Applics />
</CassisTIConfiguration>

With ProX-based pipeline processing added, the XSL-FO is just one of several stylesheets run by the pipelines, and so, if a system configuration pointing out a root XML file is opened first, the user must associate a saved process with the configuration or configure a new one before the XML can be processed.

The configuration files that point out the root XML and some system-specific metadata now also list each and every saved ProX instance (basically an instance of the blueprint code in section “The Admin's POV”) associated with that specific configuration, including a default PDF publishing instance, so there will be at least one process to use. New ones can be defined later.

<?xml version="1.0" encoding="utf-8"?>
<CassisTIConfiguration>
  ...
  <Description>Balisage 2012 whitepaper</Description>
  ...
  <Processes>
    <Process>
      <!-- Blueprint -->
      <ProXBlueprint>
        <ID><!-- System ID --></ID>
        <URN><!-- Blueprint URN --></URN>
        <ProXName><!-- Name of blueprint --></ProXName>
      </ProXBlueprint>
      <!-- Instances associated with config, selectable by user -->
      <ProXInstance>
        <ID></ID>
        <URN>urn:x-cassis:r1:cos:00008295:en-GB:0.5</URN>
        <ProXName>PDF Publishing</ProXName>
      </ProXInstance>
    </Process>
    ...
  </Processes>
</CassisTIConfiguration>

The configuration file is used by the system by something called Process Manager as a shortcut for processing XML, including translation handling of the XML. I tend to liken it to a postit note placed on a specific XML document (comprising of several modules in specific versions), describing a specific process such as the PDF publishing for customer delivery of a specific version and translation of the document. For more on this, see id-balVol08-Nordstrom01.

XForms: Generating the GUI, Pt 2

I set out to do the user interface with XForms, but as promising the standard was for me, getting my head around the MVC model was not easy.

My first hypothesis was to read the relevant nodes from the blueprint, list the process metadata's title contents in a select1 itemset, select one and copy it to the target instance, then repeat for the pipeline, cmdline and package choices. This was a wizard-like approach, with every wizard step showing and hiding the appropriate configurations in a switch/case form.

Having banged my head against the wall trying a copy inside an itemset, Mark Lawson pointed out that XSLTForms does not support copy and suggested a far easier way. In a somewhat shortened form:

<?xml-stylesheet href="xsltforms/xsltforms-1.0RC/xsltforms/xsltforms.xsl" type="text/xsl"?>
<?xsltforms-options debug="no"?>
<html 
    xmlns="http://www.w3.org/1999/xhtml" 
    xmlns:ev="http://www.w3.org/2001/xml-events" 
    xmlns:xs="http://www.w3.org/2001/XMLSchema" 
    xmlns:xf="http://www.w3.org/2002/xforms">
    <head>
        <link type="text/css" rel="stylesheet" href="style.css"/>
        <title>Cassis Process Configuration</title>
        
        <!-- Model -->
        <xf:model>
            <!-- Temp data from the selections -->
            <xf:instance id="scratchpad">
                <data xmlns="">
                    <theprocess/>
                    <thepipeline/>
                    <thecmds/>
                    <thepackages/>
                    <commonpackages/>
                </data>
            </xf:instance>
            
            <!-- Target for built process -->
            <xf:instance id="mytarget">
                <data xmlns="">
                    <processes> </processes>
                </data>
            </xf:instance>
            
            <!-- Source ProX Blueprint -->
            <xf:instance 
                src="prox-blueprint.xml" 
                id="mysource"/>
            <xf:bind 
                type="xs:boolean" 
                nodeset="instance('mysource')//param[@ctype='boolean']/value"/>
            
            <!-- Set the list defaults to be first values in the ProX instance -->
            <xf:action ev:event="xforms-ready">
                <xf:setvalue 
                    ref="theprocess" 
                    value="instance('mysource')/process[1]/@id"/>
                <xf:setvalue 
                    ref="thepipeline" 
                    value="instance('mysource')/process[1]/pipelines/pipeline[1]/@id"/>
                <xf:setvalue 
                    ref="thecmds" 
                    value="instance('mysource')/process[1]/pipelines/pipeline[1]/cmdlines/cmdline[1]/@id"/>
                <xf:setvalue 
                    ref="thepackages"/>
                <xf:setvalue 
                    ref="commonpackages" 
                    value="instance('mysource')/processes/packages"/>
            </xf:action>
            
            <!-- Save the configured proX instance -->
            <xf:submission 
                ref="instance('mytarget')/processes" 
                replace="instance" 
                method="put" 
                action="prox-instance.xml" 
                id="save"/>
        </xf:model>
    </head>
    <body>
        <h3>Process Configuration</h3>
        <table>
            <tr>
                <td>
                    <!-- Process -->
                    <fieldset>
                        <legend>Process</legend>
                        <div class="block-form">
                            <xf:select1 
                                appearance="compact" 
                                incremental="false" 
                                ref="theprocess">
                                <xf:label/>
                                <xf:item id="item-proc-dummy">
                                    <xf:label>Select a process</xf:label>
                                    <xf:value>Nothing</xf:value>
                                </xf:item>
                                <xf:itemset 
                                    id="item-proc" 
                                    nodeset="instance('mysource')/process">
                                    <xf:label ref="./metadata/title"/>
                                    <xf:value ref="./@id"/>
                                </xf:itemset>
                                <xf:setvalue 
                                    ref="../thepipeline" 
                                    ev:event="DOMFocusIn" 
                                    value="'Nothing'"/>
                            </xf:select1>
                        </div>
                    </fieldset>
                </td>
                <td>
                    <!-- Pipeline -->
                    <fieldset>
                        <legend>Pipeline</legend>
                        <div class=" block-form">
                            <xf:select1 
                                appearance="compact" 
                                incremental="false" 
                                ref="thepipeline" 
                                id="id-pipe">
                                <xf:label/>
                                <xf:item id="item-pipe-dummy">
                                    <xf:label>Select a pipeline</xf:label>
                                    <xf:value>Nothing</xf:value>
                                </xf:item>
                                <xf:itemset 
                                    id="item-pipe" 
                                    nodeset="instance('mysource')/process[@id = instance('scratchpad')/theprocess]/pipelines/pipeline">
                                    <xf:label ref="./metadata/title"/>
                                    <xf:value ref="@id"/>
                                </xf:itemset>
                                <xf:setvalue 
                                    ref="../thecmds" 
                                    ev:event="DOMFocusIn" 
                                    value="'Nothing'"/>
                            </xf:select1>
                        </div>
                    </fieldset>
                </td>
                <td>
                    <!-- Commands -->
                    <fieldset>
                        <legend>Output</legend>
                        <div class="block-form">
                            <xf:select1 
                                appearance="compact" 
                                incremental="false" 
                                ref="thecmds" 
                                id="id-cmd">
                                <xf:label/>
                                <xf:item id="item-cmd-dummy">
                                    <xf:label>Select output options</xf:label>
                                    <xf:value>Nothing</xf:value>
                                </xf:item>
                                <xf:itemset 
                                    id="item-cmd" 
                                    nodeset="instance('mysource')/process[@id = instance('scratchpad')/theprocess]/pipelines/pipeline[@id = instance('scratchpad')/thepipeline]/cmdlines/cmdline">
                                    <xf:label ref="metadata/title"/>
                                    <xf:value ref="@id"/>
                                </xf:itemset>
                            </xf:select1>
                        </div>
                    </fieldset>
                    
                </td>
            </tr>
            <tr>
                <td class="debug">
                    <xf:output ref="instance('mysource')//process[@id=instance('scratchpad')/theprocess]/metadata/title">
                        <xf:label>Selected: </xf:label>
                    </xf:output>
                </td>
                
                <td class="debug">
                    <xf:output ref="instance('mysource')//pipeline[@id=instance('scratchpad')/thepipeline]/metadata/title">
                        <xf:label>Selected: </xf:label>
                    </xf:output>
                </td>
                <td class="debug">
                    <xf:output ref="instance('mysource')//cmdline[@id=instance('scratchpad')/thecmds]/metadata/title">
                        <xf:label>Selected: </xf:label>
                    </xf:output>
                </td>
            </tr>
        </table>
        <xf:trigger>
            <xf:label>Save</xf:label>
            <xf:action ev:event="DOMActivate">
                <xf:delete nodeset="instance('mytarget')/processes/process"/>
                <xf:delete nodeset="instance('mytarget')/processes/packages"/>
                <xf:insert context="instance('mytarget')/processes" origin="instance('mysource')/process[@id = instance('scratchpad')/theprocess]"/>
                <xf:insert context="instance('mytarget')/processes" origin="instance('mysource')/packages" if="instance('mysource')/packages"/>
                <xf:delete nodeset="instance('mytarget')/processes/process[1]/pipelines/pipeline[@id != instance('scratchpad')/thepipeline]"/>
                <xf:delete nodeset="instance('mytarget')/processes/process[1]/pipelines/pipeline[1]/cmdlines/cmdline[@id != instance('scratchpad')/thecmds]"/>
                <xf:send submission="save"/>
            </xf:action>
        </xf:trigger>
    </body>
</html>

Note the following:

  • The scratchpad instance, containing IDs of the selected ProX components.

  • The target instance and the trigger that writes to it near the end.

  • And, of course, the select1s handling the process, pipeline and cmdline IDs, respectively.

This produces a GUI that writes the selected id to the scratchpad, refreshing the next select1 itemset, until done, like so:

Figure 14: The Basic ProX GUI

The result is saved to a target ProX instance: instance('mysource')/process[@id = instance('scratchpad')/theprocess] is copied using insert. The pipelines and command lines that do not match the IDs in the scratchpad instance, pipeline[@id != instance('scratchpad')/thepipeline] and cmdline[@id != instance('scratchpad')/thecmds], respectively, are then deleted from the target.

Single-Choice Problems

The first working GUI looked like this:

Figure 15: Bal2013nord-2-012812.jpg

With two or more of each choice, it worked perfectly, but if only single choice was available, this resulted in the IDs not being updated in the scratchpad instance:

Figure 16: Pipeline ID Not Updated

Note that while the process is selected and its ID listed, the pipeline ID listed in the scratchpad debug output (below the form) belongs to a previous selection rather than the apparently selected one. The cmdline ID in the scratchpad also belongs to a previous selection, but the current cmdline selection does not show any values at all. If the current state was written to a process instance, the selections would be wrong and the process would fail.

This happened because the selected process contained only a single pipeline and value changed-events do not fire as expected in single-item lists.

Better is to add a static items before the itemsets, like this:

<xf:select1 
    appearance="compact" 
    incremental="false" 
    ref="theprocess">
    <xf:label/>
    <xf:item id="item-proc-dummy">
        <xf:label>Select a process</xf:label>
        <xf:value>Nothing</xf:value>
    </xf:item>
    <xf:itemset ...>
        ...
    </xf:itemset>
    ...
</xf:select1>

This introduces a dummy value, ensuring that the form is updated regardless of the number of items. To make sure that the selection of a parent forces the update of the child, you set the value of the child to a dummy value, forcing the user to actively choose an option. For example, if you change the pipeline selection, this makes sure that the next option in line, the command line list, is updated with a static value:

<xf:select1 ...>
    <xf:label/>
    ...
    <xf:setvalue 
        ref="../thecmds" 
        ev:event="DOMFocusIn" 
        value="'Nothing'"/>
</xf:select1>

This setup has all kinds of advantages: styling becomes easier (a nonexistent list can't be easily fixed in the CSS, as far as I know; see Figure 16), subforms that configure aspects of the selected group can be shown, and conditions for saving the configured process can be imposed, not allowing a Save before all selections have been made.

User-configurable Parameters

The user-configurable stylesheet parameters are set in the input structure (see section “Command Lines”) that links to the package. Here's a complete input for a stylesheet used by a PDF publishing pipeline:

<input choice="no" id="input-2013-4-9-16-53-8-39562387-1">
                        <port id="port-2013-4-9-16-53-8-39562387-1">stylesheet</port>
                        <value 
                           xmlns:xlink="http://www.w3.org/1999/xlink" 
                           type="pkg"
                           xlink:href="urn:x-cassis:r1:cos:00002712:sv-SE:0.1#id-xslfo-cosml"
                           xlink:type="simple" id="value-2013-4-9-16-53-8-39562387-1"
                           xlink:title=" XSL-FO Package for COSML PDF Converts COSML documents 
                                         to XSL-FO format for COS PDF layout "/>
                        <params id="params-2013-4-9-16-53-8-39562387-">
                           <!-- Index generation -->
                           <param 
                              choice="yes" 
                              ctype="boolean" 
                              id="param-2013-4-9-16-53-8-39562387-">
                              <port id="port-2013-4-9-16-53-8-39562387-2">xslt-params</port>
                              <name id="name-2013-4-9-16-53-8-39562387-">generate.index</name>
                              <value 
                                 xmlns:xlink="http://www.w3.org/1999/xlink" 
                                 type="string"
                                 xlink:type="simple" 
                                 id="value-2013-4-9-16-53-8-39562387-2">false</value>
                           </param>
                           <!-- XEP Extensions -->
                           <param 
                              choice="no" 
                              id="param-2013-4-9-16-53-8-39562387-1">
                              <port id="port-2013-4-9-16-53-8-39562387-3">xslt-params</port>
                              <name id="name-2013-4-9-16-53-8-39562387-1">xep.extensions</name>
                              <value xmlns:xlink="http://www.w3.org/1999/xlink" type="string"
                                 xlink:type="simple" id="value-2013-4-9-16-53-8-39562387-3"
                                 >0</value>
                           </param>
                           <!-- XSL-FO Bookmark Generation -->
                           <param 
                              choice="yes" 
                              ctype="boolean"
                              id="param-2013-4-9-16-53-8-39562387-2">
                              <port id="port-2013-4-9-16-53-8-39562387-4">xslt-params</port>
                              <name id="name-2013-4-9-16-53-8-39562387-2">xslfo.bookmarks</name>
                              <value 
                                 xmlns:xlink="http://www.w3.org/1999/xlink" 
                                 type="string"
                                 xlink:type="simple" id="value-2013-4-9-16-53-8-39562387-4">true</value>
                           </param>
                           <!-- TOC Generation -->
                           <param 
                              choice="yes" 
                              ctype="boolean"
                              id="param-2013-4-9-16-53-8-39514778-2">
                              <port id="port-2013-4-9-16-53-8-9653444-4">xslt-params</port>
                              <name id="name-2013-4-9-16-53-8-1928364-2">create.toc</name>
                              <value 
                                 xmlns:xlink="http://www.w3.org/1999/xlink" 
                                 type="string"
                                 xlink:type="simple" 
                                 id="value-2013-7-10-16-53-8-764625737-3">true</value>
                           </param>
                           <!-- TOC Depth -->
                           <param 
                              choice="yes" 
                              ctype="list1"
                              id="param-2013-4-9-16-53-8-2385485-2"
                              group="value-2013-7-10-16-53-8-764625737-3">
                              <port id="port-2013-7-10-16-34-8-9283444-4">xslt-params</port>
                              <name id="name-2013-7-10-16-50-3-1946564-2">toc.depth</name>
                              <value 
                                 xmlns:xlink="http://www.w3.org/1999/xlink" 
                                 type="string"
                                 xlink:type="simple" 
                                 id="value-2013-7-10-16-53-8-764625737-4">2</value>
                              <value
                                 xmlns:xlink="http://www.w3.org/1999/xlink"
                                 id="value-13-07-10-12345-1" 
                                 type="string">1</value>
                              <value
                                 xmlns:xlink="http://www.w3.org/1999/xlink"
                                 id="value-13-07-10-12345-2" 
                                 type="string">3</value>
                           </param>
                        </params>
                     </input>

Note that not all of the parameters are made available as choices (choice="no").

Having the parameter definitions include data types greatly simplifies generating a GUI. As the values are not usually typed in the stylesheets that use them, the proX blueprint author must take care to define data types for every parameter made available as a user-configurable option.

The configuration options are shown or hidden using a grouping of forms in combination with CSS that hides a disabled group in the XForm[11]. The following group generates controls for a selected command line (cmdline). It is hidden if there are no user-configurable parameters for that cmdline.

<xf:group 
    ref="instance('mysource')//cmdline[@id=instance('scratchpad')/thecmds]//param[@choice='yes' 
         and @ctype='boolean']">
    <div class="block-form">
        <fieldset class="config">
            <legend>Configuration</legend>
            
            <xf:repeat 
                id="b-ctrl"
                nodeset="instance('mysource')//cmdline[@id=instance('scratchpad')/thecmds]//param[@choice='yes' and @ctype='boolean']">
                <xf:input 
                    ref="value" 
                    appearance="full">
                    <xf:label>
                        <xf:output ref="../name"/>
                    </xf:label>
                </xf:input>
            </xf:repeat>
            
            <xf:repeat 
                nodeset="instance('mysource')//cmdline[@id=instance('scratchpad')/thecmds]//param[@ctype='list1' and (not(@group) or (@group=//param[@choice='yes']/@id))]">
                <xf:select1 
                    appearance="minimal" 
                    incremental="false" 
                    ref="value">
                    <xf:label>
                        <xf:output ref="../name"/>
                    </xf:label>
                    <xf:itemset nodeset="instance('mysource')//cmdline[@id=instance('scratchpad')/thecmds]//param[@ctype='list1' and (not(@group) or (@group=//param[@choice='yes']/@id))]/value">
                        <xf:label ref="."/>
                        <xf:value ref="@id"/>
                    </xf:itemset>
                </xf:select1>
            </xf:repeat>
        </fieldset>
    </div>
</xf:group>

The current GUI design gathers all configurable options within the cmdline group rather than having any of them appear when the pipeline is selected. This is a design choice rather than a technical one; see section “Design Choices”.

Figure 17: Parameter Configuration

The above is able to generate boolean and single-choice list controls. Of course, other controls may be added, for example, to select a date or enter a text string to be included in the process output.

Initiating and Running ProX

Running the overall process, that is, initiating and running the ProX XForms template, saving the resulting ProX instance and converting it to a shell script or batch file (or some other file executing a pipeline engine) involves the following:

  1. Locate and fetch the ProX blueprint. Normalise, if necessary.

  2. Locate and fetch all files that are part of the processing as defined by the blueprint, including the input XML, stylesheets, XProc, XForms XHTML, configuration files, etc. Map their URNs to temporary URLs in a resource map XML file (see section “The Resource Map”).

  3. Generate any runtime URLs for the target files for the process, as defined by the blueprint. Map these in the resource map XML, adding the ProX IDs (see section “Targets”) where needed.

  4. Preprocess the XForms XHTML, adding the URL to the ProX blueprint and other information required by the XForm.

  5. Open the the XForms XHTML.

  6. Make choices in the XForm as necessary. Save (and close) the a ProX instance.

  7. Replace any URNs in the source XML to be processed with matching temporary URNs.

  8. Add runtime information to the ProX instance (input XML, target URLs, etc).

  9. Convert the ProX instance to a shell script.

  10. Run the script.

  11. Capture the output.

All of the above is handled by a wrapper XProc pipeline (section “The Wrapper Pipeline”).

The Resource Map

When processing an XML file, the system needs to list all files required by the wrapper pipeline process and the resulting child pipeline process in a resource map XML file, mapping their URNs to temporary URLs. This includes the XML to be processed, of course, but also any images and other non-XML data. Every XSLT stylesheet required for the processing (for the wrapper process as well as the child process) must be listed, as must all XProc scripts and whatever files they require.

Also, the system must generate temporary URLs (and, depending on the result, URNs) for any resulting files.

The resource map is then used as the sole input by the wrapper pipeline process.

All of these files are either listed directly in the ProX blueprint (when known), or pointed out indirectly, using the attribute type set to external. A ProX process starts with the generation of a basic resource map file, using a simple XSLT stylesheet applied to the ProX blueprint.

The resource map looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<resource-map>
   
   <!-- Source Modules Listed here -->
   <docs>
      
      <!-- NOTE: The doc lists can contain duplicate modules, with the same URN/URL
           pairs. The ProX wrapper XProc and XSLT will handle them. -->
      
      <!-- Document #1 -->
      <!-- One doc, with root and modules incl images -->
      <doc id="">
         
         <!-- Root document from Process Manager configuration -->
         <!-- ProX instance needs this value -->
         <!-- //*/@type=''external' and //*/@input-type='doc-root' -->
         <root>
            <resource>
               <urn>urn:testroot</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/test-root.xml</url>
               <type>doc-root</type>
               <prox-id>value-2013-4-9-16-53-8-39562387-</prox-id>
            </resource>
         </root>
         
         <!-- All modules linked from root or its descendants -->
         <!-- XML, images, etc -->
         <modules>
            <resource>
               <urn>urn:image1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/image3.jpg</url>
               <type>jpg</type>
            </resource>
            <resource>
               <urn>urn:inset1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/inset1.xml</url>
               <type>xml</type>
            </resource>
            <resource>
               <urn>urn:inset2</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/inset2.xml</url>
               <type>xml</type>
            </resource>
            <resource>
               <urn>urn:inset3</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/inset3.xml</url>
               <type>xml</type>
            </resource>
            <resource>
               <urn>urn:inset4</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/inset4.xml</url>
               <type>xml</type>
            </resource>
            <resource>
               <urn>urn:block-inset1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/block-inset1.xml</url>
               <type>xml</type>
            </resource>
         </modules>    
      </doc>
      
      
      <!-- Document #2 -->
      <doc id="">
         
         <!-- Root document from Process Manager configuration -->
         <!-- ProX instance needs this value -->
         <!-- //*/@type=''external' and //*/@input-type='doc-root' -->
         <root>
            <resource>
               <urn>urn:testroot</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/test-root.xml</url>
               <type>doc-root</type>
               <prox-id>id-html-docroot</prox-id>
            </resource>
         </root>
         
         <!-- All modules linked from root or its descendants -->
         <!-- XML, images, etc -->
         <modules>
            <resource>
               <urn>urn:image1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/image3.jpg</url>
               <type>jpg</type>
               <transl>1</transl>
            </resource>
            <resource>
               <urn>urn:inset1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/inset1.xml</url>
               <type>xml</type>
            </resource>
            <resource>
               <urn>urn:inset2</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/inset2.xml</url>
               <type>xml</type>
            </resource>
            <resource>
               <urn>urn:inset3</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/inset3.xml</url>
               <type>xml</type>
            </resource>
            <resource>
               <urn>urn:inset4</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/inset4.xml</url>
               <type>xml</type>
            </resource>
            <resource>
               <urn>urn:block-inset1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/block-inset1.xml</url>
               <type>xml</type>
            </resource>
         </modules>    
      </doc>
      
   </docs>
   
   
   <!-- Runtime targets -->
   <targets>
      <resource>
         <urn>URN-FOR-OUTPUT</urn>
         <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/my-pdf-internal-file.pdf</url>
         <type>primary</type>
         <prox-id>value-2013-4-9-16-53-8-39562387-6</prox-id>
      </resource>
      <resource>
         <urn>URN2-FOR-OUTPUT</urn>
         <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/my-pdf-formal-file.pdf</url>
         <type>primary</type>
         <prox-id>value-2013-4-9-16-53-8-39562387-10</prox-id>
      </resource>
      <resource>
         <urn>URN-FOR-XREF-XHTML-LOG</urn>
         <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/my-xref-check.htm</url>
         <type>primary</type>
         <prox-id>id-value-xref-htm</prox-id>
      </resource>
      <resource>
         <urn>URN-FOR-FILES-LIST-XML</urn>
         <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/files.xml</url>
         <type>secondary</type>
         <prox-id>id-file-url</prox-id>
      </resource>
      <resource>
         <urn>URN-FOR-HTM-OUT</urn>
         <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/my-xhtml-out.htm</url>
         <type>primary</type>
         <prox-id>id-htm-out</prox-id>
      </resource>
   </targets>
   
   <!-- ProX blueprint and saved instance(s) -->
   <prox>
      <!-- Blueprint used to get instance is here -->
      <blueprints>
         <resource id="id-prox-blueprint">
            <urn>URN-OF-PROX-BLUEPRINT</urn>
            <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/prox-blueprint.xml</url>
            <type></type>
            <prox-id></prox-id>
         </resource>    
      </blueprints>
      
      <!-- Saved instance to run with wrapper is here -->
      <!-- All these are associated with .config files -->
      <!-- Input to wrapper pipeline -->
      <instances>
         <resource id="id-prox-saved-instance">
            <urn>URN-OF-SAVED-PROX-INSTANCE</urn>
            <!--<url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/test-instance.xml</url>-->
            <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/prox-instance.xml</url>
            <type></type>
            <prox-id></prox-id>
         </resource>
      </instances>
   </prox>
   
   <!-- Resources used by ProX Processes -->
   <prox-resources>
      
      <!-- PDF Publishing XProc -->
      <package>
         <name>XProc Pipeline for Normalize, Validate
                  and PDF</name>
         <resources>
            <resource>
               <urn>urn:x-cassis:r1:cos:00002715:sv-SE:0.1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/XProc/publish-cosml-pdf.xpl</url>
               <prox-id>locator-2013-4-10-10-32-24-12830403-</prox-id>
            </resource>
         </resources>
      </package>
      
      <!-- PDF Publishing XSL-FO, Internal -->
      <package>
         <name>XSL-FO Package for COSML PDF</name>
         <resources>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000232:sv-SE:0.6</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/cos-fo-internal.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-1</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000074:sv-SE:0.11</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/param.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-2</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000059:sv-SE:0.2</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/attribute-set.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-3</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000070:sv-SE:0.15</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/layout.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-4</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000876:sv-SE:0.2</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/bookmarks.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-5</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000075:sv-SE:0.17</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/static-content.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-6</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000072:sv-SE:0.10</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/meta-data.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-7</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000078:sv-SE:0.9</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/toc.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-8</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000062:sv-SE:0.9</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/body.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-9</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000233:sv-SE:0.8</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/sections.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-10</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000061:sv-SE:0.29</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/block.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-11</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000065:sv-SE:0.6</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/inline.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-12</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000071:sv-SE:0.6</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/list.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-13</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000077:sv-SE:0.6</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/table.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-14</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000079:sv-SE:0.7</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/xref.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-15</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000060:sv-SE:0.7</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/back.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-16</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000064:sv-SE:0.8</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/index.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-17</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000066:sv-SE:0.2</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/inset.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-18</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000069:sv-SE:0.3</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/l10n.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-19</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000063:sv-SE:0.3</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/extension.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-20</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000785:sv-SE:0.6</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/demo.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-21</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000076:sv-SE:0.10</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/strings.xml</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-22</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000230:sv-SE:0.1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/logotyp.jpg</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-23</prox-id>
            </resource>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000426:sv-SE:0.1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/tux.jpg</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-24</prox-id>
            </resource>
         </resources>
      </package>
      
      <!-- XHTML Publishing XProc -->
      <package>
         <name>XProc COSML2XHTML</name>
         <resources>
            <resource>
               <urn>urn:x-cassis:r1:cos:00002756:sv-SE:0.2</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/XProc/publish-cosml-html.xpl</url>
               <prox-id>id-xhtml-xproc</prox-id>
            </resource>
         </resources>
      </package>
      
      <!-- XHTML Publishing XSLT -->
      <package>
         <name>COSML XHTML XSLT</name>
         <resources>
            <resource>
               <urn>urn:x-cassis:r1:cos:00002755:sv-SE:0.1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/Conversions/cosml2html-ti.xsl</url>
               <prox-id>id-xhtml-xslt</prox-id>
            </resource>
         </resources>
      </package>
      
      <!-- Xref Check XProc -->
      <package>
         <name>XProc Xref Check</name>
         <resources>
            <resource>
               <urn>urn:x-cassis:r1:cos:00002757:sv-SE:0.2</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/XProc/xref-check-cosml.xpl</url>
               <prox-id>id-xproc-xref</prox-id>
            </resource>
         </resources>
      </package>
      
      <!-- Xref Check XSLT -->
      <package>
         <name>XSLT Xref Check</name>
         <resources>
            <resource>
               <urn>urn:x-cassis:r1:cos:00002754:sv-SE:0.1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/Conversions/link-target-check-multifile.xsl</url>
               <prox-id>id-xslt-xref</prox-id>
            </resource>
         </resources>
      </package>
      
      <!-- Standard Normalize XSLT for Publishing -->
      <package>
         <name>Normalize XSLT</name>
         <resources>
            <resource>
               <urn>urn:x-cassis:r1:cos:00000073:sv-SE:0.4</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Cassis/FO/normalize.xsl</url>
               <prox-id>locator-2013-4-9-16-53-8-39562387-26</prox-id>
            </resource>
         </resources>
      </package>
      
      <!-- Calabash Engine Configuration -->
      <package>
         <name>Calabash Configuration</name>
         <resources>
            <resource>
               <urn>urn:x-cassis:r1:cos:00002745:sv-SE:0.1</urn>
               <url>file:///home/ari/xmlcalabash-1.0.9-94/conf-calabash.xml</url>
               <prox-id>id-loc-calabash-config</prox-id>
            </resource>
         </resources>
      </package>
   </prox-resources>
   
   
   <!-- Wrapper stuff -->
   <wrapper-pipeline>
      
      <!-- Wrapper Pipeline Resources -->
      <package>
         <name>Wrapper Pipeline Processing</name>
         <resources>
            <!-- Wrapper Pipeline -->
            <resource>
               <urn>urn:x-cassis:r1:cos:00002735:sv-SE:0.1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/prox-wrapper.xpl</url>
               <prox-id>id-wrapper-xpl</prox-id>
            </resource>
            <!-- ProX Instance Update -->
            <resource>
               <urn>urn:x-cassis:r1:cos:00002732:sv-SE:0.1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/prox-fix.xsl</url>
               <prox-id>id-prox-fix</prox-id>
            </resource>
            <!-- URN2URL for XML Input -->
            <resource>
               <urn>urn:x-cassis:r1:cos:00002733:sv-SE:0.1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/urn2url.xsl</url>
               <prox-id>id-urn2url</prox-id>
            </resource>
            <!-- ProX Instance Conversion to Shell Script -->
            <resource>
               <urn>urn:x-cassis:r1:cos:00002731:sv-SE:0.1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/prox2shell.xsl</url>
               <prox-id>id-prox2bat</prox-id>
            </resource>
            <!-- Engine parameters required by ProX to Shell Script conversion -->
            <resource>
               <urn>urn:x-cassis:r1:cos:00002734:sv-SE:0.1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/prox2shell-config.xml</url>
               <prox-id>id-prox2shell-config</prox-id>
            </resource>
         </resources>
      </package>
      
      <!-- XForm for ProX Process Configuration -->
      <package>
         <name>ProX XForm</name>
         <resources>
            <!-- XForm for proX Blueprint Handling -->
            <resource>
               <urn>urn:prox:xform:0.1</urn>
               <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/prox-xform.xml</url>
               <prox-id>id-loc-xform</prox-id>
            </resource>
         </resources>
      </package>
   </wrapper-pipeline>
</resource-map>

This is a complete resource map example, matching the example ProX blueprint in section “The Admin's POV”. Note that some of the URNs are for testing purposes only, while others come from the actual system.

The resource map is very much like a recipe; it lists every ingredient for every ProX process.

Docs

The docs structure lists the input XML and any linked files. More than one document may be listed, and some of those documents may be duplicates because they originate from different pipelines or processes (as is the case in the above resource map). For example, the same document may be used for both PDF and web publishing in a single resulting process instance, and therefore be listed several times. The wrapper pipeline will only process distinct values, however, and there will be no physical duplicates in the temporary processing folder.

Targets

The targets structure lists the runtime target URLs generated by the system. Every target URL is paired with a ProX ID so that the subsequent processing can place the right URL in the right place in the ProX instance:

<targets>
    ...
    <resource>
        <urn>URN-FOR-FILES-LIST-XML</urn>
        <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/files.xml</url>
        <type>secondary</type>
        <prox-id>id-file-url</prox-id>
    </resource>
    <resource>
        <urn>URN-FOR-HTM-OUT</urn>
        <url>file:///home/ari/mystuff/SGML/DTD/Process-XML/refactor/my-xhtml-out.htm</url>
        <type>primary</type>
        <prox-id>id-htm-out</prox-id>
    </resource>
</targets>

Notable is that ProX defines different types of output: primary (see above) means that the file is something that the system should save, while secondary is a throwaway, a temporary URL that can be discarded. There's also log, that means that the file can be displayed by the system when the process has completed, but what happens to the file later is up to the system to decide.

ProX Resources

The prox and prox-resources structures list the ProX blueprint and the ProX (runtime) instance URN/URL pairs, and the files used by the processes described by the blueprint. The latter include any XSLT, XProc, XML, etc, required for processing, but also any other types of files, whatever they may be.

Wrapper Resources

The wrapper-pipeline structure lists the resources required by the wrapper pipeline, including the XForm and the wrapper pipeline itself.

The Wrapper Pipeline

The wrapper pipeline has but one main task: to configure the child pipeline process. In principle, this involves producing a ProX instance from the blueprint, and then converting that instance to a shell script that is used to run the child pipeline process.

The wrapper pipeline requires a single input, the resource map XML file. It assumes that the process is carried out in a temporary folder, and that every required file is listed in the resource map and moved by the system to the temp folder; currently, there is no way for the wrapper to ask the system for a specific file based on its URN, even though that functionality is planned in a future version.

In a perfect world, the wrapper should be initiated by the system and then take over all of the processing, including configuring[12] and opening the XForm used to configure the Prox blueprint, wait for the user to make her choices and save the resulting instance, and then continue the wrapper pipeline process in preparation for the child process. In reality, there are a few problems, however:

There is currently no wait step defined in the XProc spec (see Kurt Cagle's proposal at id-kurt-cagle-xproc, or my subsequent thread at id-wait-for-user). There is no easy way to have a pipeline wait for user input before continuing. What comes the closest is an XProc hack that looks something like this[13]:

<!-- Open ProX Blueprint in Browser -->
<!-- Opens with an XForms profile in order
         to start a separate browser instance -->
<p:choose name="browse">
    <!-- Linux -->
    <p:when test="$os='linux'">
        <p:exec 
            cx:depends-on="fix-xform"
            command="/usr/bin/iceweasel">
            <p:input port="source">
                <p:empty/>
            </p:input>
            <p:with-option name="args" select="concat('-P &quot;XForms&quot; -no-remote ',$xform-url)"/>
        </p:exec>
        <p:sink/>
    </p:when>
    
    <!-- Mac OS X -->
    <p:when test="$os='osx'">
        ...
    </p:when>
    
    <!-- Windows -->
    <p:when test="$os='win'">
        ...
    </p:when>
</p:choose>

What happens here is basically that while there is no way to tell something like an http-request to wait, at least Calabash will happily wait for the p:exec to complete (meaning in the above example that the browser process is killed) before continuing with the next step, if the XForm is opened in a new thread; if you use your default browser and it happens to be running, the wrapper won't know that it should wait. The hack also requires the use of the cx:depends-on extension step to make sure that they'll all wait until the p:exec is done.

After the ProX instance is saved and the browse process killed, the wrapper continues by preprocessing the input XML (see below) and the ProX instance that was just saved (also see below), before finally converting the instance to a shell script, running that shell script, and handling any logs or reports resulting from the wrapper or child processes.

When the wrapper process ends, it is up to the system to take care of the resulting files and to delete any temporary content, including the temp folder where the action took place.

URN to URL

The input XML is frequently modularised, like so:

Figure 18: Modularised XML

The system uses URNs for all of its linking, which means that whenever an XML document is published, each participating module must first be preprocessed to replace the URN-based links with temporary URL-based ones. Only then can the XML be normalised[14].

The wrapper pipeline runs an XSLT script that maps URNs to URLs using the resource map.

ProX Fixes

A similar preprocessing step is required on the saved ProX instance. It contains a number of URNs that need to be replaced with URLs, but also several empty runtime targets that need values from the resource map.

Converting to a Shell Script

The preprocessed ProX instance is then converted to a shell script (in the case of Calabash) using an XSLT stylesheet:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:xlink="http://www.w3.org/1999/xlink" version="2.0">
    
    <xsl:output method="xml" indent="no"/>
    <xsl:strip-space 
        elements="cmdline script engine-config inputs options params"/>
    
    <xsl:param 
        name="map-url"/>
    
    <xsl:param 
        name="os"/>
    
    <xsl:param 
        name="debug" 
        select="'yes'"/>
    
    <xsl:template match="/">
        <bat>
            <xsl:choose>
                <xsl:when test="$os='win'">
                    <xsl:text>REM Generated for Windows</xsl:text>
                    <xsl:text>&#x0A;</xsl:text>
                </xsl:when>
                <xsl:when test="$os='osx'">
                    <xsl:value-of
                        select="document('prox-xslt2bat-configuration.xml')/config/calabash/shell[@os='osx']/text()"/>
                    <xsl:text>&#x0A;</xsl:text>
                    <xsl:text># Generated for OS X</xsl:text>
                    <xsl:text>&#x0A;</xsl:text>
                </xsl:when>
                <xsl:when test="$os='linux'">
                    <xsl:value-of
                        select="document('prox-xslt2bat-configuration.xml')/config/calabash/shell[@os='linux']/text()"/>
                    <xsl:text>&#x0A;</xsl:text>
                    <xsl:text># Generated for Linux</xsl:text>
                    <xsl:text>&#x0A;</xsl:text>
                </xsl:when>
            </xsl:choose>
            <xsl:apply-templates/>    
        </bat>
    </xsl:template>
    
    <xsl:template match="processes">
        <xsl:apply-templates/>
    </xsl:template>
    
    <xsl:template match="process">
        <xsl:apply-templates select=".//metadata"/>
        <xsl:text>java -classpath </xsl:text>
        <xsl:choose>
            <xsl:when test="$os='win'">
                <xsl:value-of
                    select="document('prox-xslt2bat-configuration.xml')/config/calabash/classpath[@os='win']/text()"/>        
            </xsl:when>
            <xsl:when test="$os='osx'">
                <xsl:value-of
                    select="document('prox-xslt2bat-configuration.xml')/config/calabash/classpath[@os='osx']/text()"/>
            </xsl:when>
            <xsl:when test="$os='linux'">
                <xsl:value-of
                    select="document('prox-xslt2bat-configuration.xml')/config/calabash/classpath[@os='linux']/text()"/>
            </xsl:when>
        </xsl:choose>
        
        <xsl:text> com.xmlcalabash.drivers.Main </xsl:text>
        <xsl:apply-templates select="pipelines/pipeline"/>
        
        <!-- Debug mode -->
        <xsl:if test="$debug='yes'">
            <xsl:choose>
                <xsl:when test="$os='osx' or $os='linux'">
                    <xsl:text>&#x0A;</xsl:text>
                    <xsl:text>read -p "Press [Enter] to continue..."</xsl:text>
                </xsl:when>
                <xsl:when test="$os='win'">
                    <xsl:text>&#x0A;</xsl:text>
                    <xsl:text>pause</xsl:text>
                </xsl:when>
            </xsl:choose>
        </xsl:if>
    </xsl:template>
    
    <xsl:template match="pipeline">
        <!--        <xsl:apply-templates select="metadata"/>-->
        <xsl:apply-templates select="cmdlines/cmdline"/>
        <xsl:text> </xsl:text>
        <xsl:apply-templates select="script"/>
    </xsl:template>
    
    <xsl:template match="script">
        <!-- @xlink:href refers to package -->
        <xsl:choose>
            <xsl:when test="@type='pkg'">
                <xsl:call-template name="fragment-id"/>
            </xsl:when>
            <xsl:otherwise>
                <xsl:value-of select="."/>
            </xsl:otherwise>
        </xsl:choose>
    </xsl:template>
    
    <xsl:template match="cmdline">
        <xsl:apply-templates select="engine-config"/>
        <xsl:apply-templates select="inputs"/>
        <xsl:apply-templates select="outputs"/>
        <xsl:apply-templates select="options"/>
        <xsl:apply-templates select="params"/>
    </xsl:template>
    
    <!-- XProc Engine-specific Configuration -->
    <xsl:template match="enginge-config">
        <xsl:apply-templates select="config"/>
    </xsl:template>
    
    <xsl:template match="config">
        <xsl:text>--config</xsl:text>
        <xsl:text> </xsl:text>
        <xsl:call-template name="fragment-id"/>
        <xsl:text> </xsl:text>
    </xsl:template>
    
    
    
    <!-- Inputs -->
    
    <xsl:template match="inputs">
        <xsl:apply-templates select="input"/>
    </xsl:template>
    
    <xsl:template match="input">
        <xsl:choose>
            <xsl:when test="matches(port,'map')">
                <!-- Standard input for map URL -->
                <xsl:text>--input map=</xsl:text>
                <xsl:value-of select="$map-url"/>
                <xsl:text> </xsl:text>
            </xsl:when>
            <xsl:otherwise>
                <xsl:text>--input </xsl:text>
                <xsl:value-of select="port"/>
                <xsl:text>=</xsl:text>
                <xsl:apply-templates select="value"/>
                <xsl:text> </xsl:text>
                <xsl:apply-templates select="params"/>        
            </xsl:otherwise>
        </xsl:choose>
    </xsl:template>
    
    
    
    <!-- Options -->
    
    <xsl:template match="options">
        <xsl:apply-templates select="option"/>
    </xsl:template>
    
    <xsl:template match="option">
        <xsl:value-of select="name"/>
        <xsl:text>=</xsl:text>
        <xsl:apply-templates select="value"/>
        <xsl:text> </xsl:text>
    </xsl:template>
    
    
    
    <!-- Parameters for XSLT -->
    
    <xsl:template match="params">
        <xsl:apply-templates select="param"/>
    </xsl:template>
    
    <xsl:template match="param">
        <xsl:text>--with-param </xsl:text>
        <xsl:value-of select="port"/>
        <xsl:text>@</xsl:text>
        <xsl:value-of select="name"/>
        <xsl:text>=</xsl:text>
        <xsl:apply-templates select="value"/>
        <xsl:text> </xsl:text>
    </xsl:template>
    
    <xsl:template match="value">
        <xsl:choose>
            <xsl:when test="@type='pkg'">
                <xsl:call-template name="fragment-id"/>
            </xsl:when>
            <xsl:when test="@type='external'">
                <!-- "ti" previously -->
                <!-- External value -->
                <xsl:value-of select="." exclude-result-prefixes="#all"/>
            </xsl:when>
            <xsl:when test="@type='uri'">
                <!-- Single-resource URI -->
                <xsl:value-of select="."/>
            </xsl:when>
            <!-- Fallback: single file assumed -->
            <xsl:otherwise>
                <xsl:value-of select="."/>
            </xsl:otherwise>
        </xsl:choose>
    </xsl:template>
    
    <xsl:template name="fragment-id">
        <xsl:variable name="href" select="./@xlink:href"/>
        <xsl:choose>
            <xsl:when test="contains(@xlink:href,'#')">
                <xsl:value-of
                    select="//package[@id=substring-after($href,'#')]/locator[@type='main']/@xlink:href"/>
            </xsl:when>
            <xsl:otherwise>
                <xsl:value-of select="//package[@id=$href]/locator[@type='main']/@xlink:href"/>
            </xsl:otherwise>
        </xsl:choose>
    </xsl:template>
    
    
    
    <!-- Metadata Handling -->
    
    <xsl:template match="metadata">
        <xsl:apply-templates/>
    </xsl:template>
    
    <xsl:template match="title">
        <xsl:text>echo </xsl:text>
        <xsl:value-of select="normalize-space(.)"/>
    </xsl:template>
    
    <xsl:template match="description">
        <xsl:apply-templates select="p"/>
    </xsl:template>
    
    <xsl:template match="p">
        <xsl:text>echo </xsl:text>
        <xsl:value-of select="normalize-space(.)"/>
    </xsl:template>
    
    
</xsl:stylesheet>

This does not process any engine-specific options, nor does it handle data-inputs.

Note that the XSLT uses a configuration file that lists the CLASSPATHs and other OS- and system-specific strings.

Child Process and Capturing Output

The shell script (or batch file) that results from the conversion is saved and then run using a p:exec step:

<!-- Store generated shell script and run it -->
<p:choose>
    <!-- Linux -->
    <p:when test="$os='linux'">
        <!-- Save shell script -->
        <p:store method="text" name="save-bat">
            <p:with-option name="href" select="'out2.sh'"/>
        </p:store>
        <!-- Run -->
        <p:exec 
            source-is-xml="false" 
            result-is-xml="false" 
            name="run-bat" 
            cx:depends-on="save-bat">
            <p:with-option name="command" select="'sh'"/>
            <p:with-option name="args" select="'./out2.sh'"/>
            <p:with-option name="cwd" select="substring($tmp-url,6, string-length($tmp-url)-1)"/>
            <p:log port="errors" href="error.txt"/>
            <p:input port="source">
                <p:empty/>
            </p:input>
        </p:exec>
        <p:sink/>
    </p:when>
    
    <!-- OS X -->
    <p:when test="$os='osx'">
        ...
    </p:when>
    
    <!-- Windows -->
    <p:when test="$os='win'">
        ...
    </p:when>
</p:choose>

And yes, again, there are OS-specific hacks in there. While the system is purely Windows, I much prefer developing on Linux or Mac, and so made the necessary changes for at least a rudimentary OS independence.

XForms Engine

The XForm currently runs on XSLTForms. The decision to use XSLTForms was a practical one; I'm running it locally for ProX user interfaces, without a server, but XSLTForms is just as easy to get to run on the server.

There are some mostly minor but noteworthy issues:

  • xf:code is not implemented so xf:insert must be used instead. No biggie, but it helps to know about it.

  • For local use (seems to be the same on Windows, OS X and Linux), the stylesheet PI pointing out the XSLT not only needs to be relative; the XSLT needs to be in a descendant directory. This is very strange and it took me some time before I noticed what was going on.

  • Local submissions using relative URLs (for example, as described at https://en.wikibooks.org/wiki/XForms/Submit) fail silently on a Windows machine. It seems that Microsoft never introduced a standard way of expressing their relative file paths, but action="file://myfile.xml" works.

  • Local submits, even with the right relative URLs, will always enforce a Save As dialogue. This is annoying but only a problem locally where the file system is, in fact, at risk.

  • And, depending on the platform, the Java applet run when submitting can cause endless grief, from warnings when running it to failing, either silently or with a bang. OS X is particularly difficult in this respect.

  • And lastly, XSLTForms converts XForms to HTML and JavaScript. Running this locally can cause some unpredictability, depending on the browser. On Windows, Internet Explorer can frequently refuse to run code that works without a hitch in Firefox and Safari. On the other hand, on that same Windows machine, Safari then quit running XSLTForms altogether, following a Java update.

Run from a server, XSLTForms works like a charm.

XProc Engine

Currently, I'm using Calabash (id-xmlcalabash) to run the pipelines configured with ProX. This is unlikely to change any time soon for the system being implemented now; there aren't that many viable alternatives that aren't part of a competitor's product. ProX started out as a reflection of the Calabash way of doing things.

There are other systems, though, where another engine might better match the system's requirements. eXist, for example, includes xprocxq (id-xprocxq), an XQuery-based XProc engine that is configured using an XML-based set of parameters. Converting a ProX instance to the xprocxq format should be uncomplicated but the engine's current state in eXist makes it difficult to test. A new version for MarkLogic was announced recently, and presented (id-jimf-xmllondon) at XML London in June 2013.

End Notes

What The Future Holds

ProX is still a work in progress, even though it's now running locally and on a pre-release system. Here are some of my future plans:

  • Add (and expand?) metadata where needed. The main structures (process, pipeline, cmdline) include metadata used to generate help for these sections, but just as useful would be to add it to all user-configurable structures. Stylesheet parameters, for one, would greatly benefit from help texts, but also from better GUI display names (see Figure 17).

  • GUI localisation. XForms is not easily modularised in reusable components (it's not, at all), but it would be useful to move any GUI labels and help texts to a file that can be localised.

  • A ProX implementation for eXist. XProc is not currently well supported in eXist itself. xprocxq is more or less broken in it, as is the Calabash module, but it is perfectly feasible to run XProc pipelines outside eXist itself using James Sulak's XProc extension library (id-sulak, id-xmlprague-2013-existential).

  • Various XForms additions and fixes, specifically a standardised XForm preprocessing step in the wrapper script that might be used to handle a modularised XForms GUI.

  • To lessen the dependency on the resource map XML: Resource retrieval in the system based on a known URN (something like getUrl(Urn)). Also, target URL generation and better handling of the temporary folder in the system.

  • Prepare and release an open source version of the ProX package. A few of the scripts in ProX are system-specific, but it should be straight-forward to do a generic version.

Last But Not Least

Huge thanks must go to Mark Lawson, who not only pointed out that xf:copy is currently not supported by XSLTForms, but also wrote the XForm that is the basis for the ProX GUI (see section “XForms: Generating the GUI, Pt 2”). That's another way of saying that he provided all of the basic XForms logic and I only had to add to it.

Thanks also to Norman Walsh, without whom I certainly wouldn't be writing a paper involving XProc, to Jim Fuller, who has provided me with valuable XProc hints and tips on numerous occasions, and to my friend Henrik Mårtensson who patiently helped me get ProX to run on my Mac.

Thanks must also go to the Balisage program committee and their brilliant blurb. If you read this, the blurb is probably why.

Finally, any errors and omissions on these pages should be attributed to me, and me only. You can lead a horse to the water but you can't make it drink.

References

[id-balVol08-Nordstrom01] Using XML to Implement XML, Ari Nordström. http://www.balisage.net/Proceedings/vol8/html/Nordstrom01/BalisageVol8-Nordstrom01.html. doi:https://doi.org/10.4242/BalisageVol8.Nordstrom01

[id-xproc-spec] XProc: An XML Pipeline Language, Recommendation. http://www.w3.org/TR/xproc/

[id-xmlcalabash] XML Calabash. http://xmlcalabash.com/

[id-calabash] XML Calabash Reference, Norman Walsh. http://xmlcalabash.com/docs/reference/

[id-xprocxq] xprocxq in eXist, James Fuller. Documentation currently missing at http://exist-db.org/exist/apps/doc/xproc/xproc.xml

[id-freemind-xsd] FreeMind XML format. http://freemind.cvs.sourceforge.net/viewvc/freemind/freemind/freemind.xsd?revision=1.2&view=markup

[id-xforms-spec] XForms 1.1 Recommendation. http://www.w3.org/TR/xforms/

[id-mvc-xforms-eric-vdl] When MVC becomes a burden for XForms, Eric van der Vlist. http://eric.van-der-vlist.com/blog/2013/06/17/when-mvc-becomes-a-burden-for-xforms-xml-london-2013/

[id-kurt-cagle-xproc] Re: Immediate Market Needs for XProc. http://lists.w3.org/Archives/Public/xproc-dev/2009May/0030.html

[id-wait-for-user] Wait for User Input (xproc-dev thread). http://lists.w3.org/Archives/Public/xproc-dev/2013May/0008.html

[id-jimf-xmllondon] Architecture of xproc.xq an XProc processor. http://es.slideshare.net/jimfuller2009/xml-london-2013-architecture-of-xprocxq-an-xproc-processor

[id-sulak] eXist XProc Extension Library. https://github.com/jsulak/eXist-XProc-Library

[id-xmlprague-2013-existential] eXistential Issues in Farming, XML Prague 2013. http://archive.xmlprague.cz/2013/files/xmlprague-2013-proceedings.pdf



[1] Turns out XProc was already taken, as was XPipe and some other exciting variations.

[2] Note that the packages structure in the illustration is common to all processes. Packages may also be included at process level, in which case they only apply to that process.

[3] Or, conceivably, a single step.

[4] ProX is very much a work in progress, and the design currently reflects the requirements of the system it is being implemented in.

[5] Not every pipeline needs the XML to be defined at runtime, either. For example, the system might use a pipeline to produce reports.

[6] A package may include binary files.

[7] This is necessary when generating the user interface that allows the user to configure the parameter.

[8] How much of the process is configurable is decided by the author of the blueprint.

[9] Some processes may not need an XML file to apply the process to.

[10] The system will fetch any XML modules linked by the root XML, after wich a normalisation process is carried out, and only then are the FO stylesheets applied.

[11] Provided that they are made available to the user in the first place.

[12] The XForm needs to know the URL of the ProX blueprint to be used, and it needs to be handed a temporary URL for the runtime ProX instance that results (and a permanent new URN, if the process is saved for later).

[13] And yes, the hack does include conditionals for OS X and Windows, in addition to Linux. Know that OS X is a pain if you want to try this at home.

[14] If the child pipeline normalises it; the wrapper does not.

Ari Nordström

Ari Nordström is the resident XML guy at Condesign AB in Göteborg, Sweden. His information structures and solutions are used by Volvo Cars, Ericsson, and many others, with more added every year. His favourite XML specification remains XLink so quite a few of his frequent talks and presentations on XML focus on linking.

Ari spends some of his spare time projecting films at the Draken Cinema in Göteborg, which should explain why he wants to automate cinemas using XML. He now realises it's too late, however.