XML Pipeline Processing in the Browser
Copyright © 2010 EMC Corporation. All rights reserved.
Table of Contents
Copyright © 2010 EMC Corporation. All rights reserved.
Table of Contents
Looking back at the past couple of years, it is clear that web browsers have come a long way and have evolved tremendously. With increasing support for open standards and rapid performance improvements, modern web browsers are no longer just tools for viewing web content - they have become complete platforms for developing complex applications.
But integrating and orchestrating XML processes is exactly what XProc: An XML Pipeline Language W3C XProc is trying to address. The declarative, pipeline-oriented approach to XML processing in XProc provides a flexible integration layer on top of other XML technologies (such as XQuery, XSLT or, for instance, schema validation) that makes developing complex XML processing flows easier and more transparent.
To our knowledge, all currently available XProc implementations are essentially server-side applications. This is definitely the case with XML Calabash XML Calabash and EMC's Calumet EMC Calumet, which are both Java-based. Other projects that we are aware of also make use of server-side technologies.
Porting Calumet to GWT turned out to be mostly a mechanical process: refactoring out dependencies on features not supported by GWT from the original Java code, and implementing adapters for accessing the browser DOM functionality. Especially the latter turned out to be critical for the overall function of the XProc engine and its interoperability with different browsers.
The result is an XProc processor with a standard
Calumet. The demo was presented at the XML Prague '10 DemoJam event.
Figure 1: GWT Calumet Demo Application
A large part of the code base is shared between the Java and GWT versions. This not only reduces code duplication, but also ensures that both versions of Calumet are aligned in terms of functionality and the level of XProc support. As of July 2010, the GWT version supports 34 out of the total 41 steps from the standard XProc step library W3C XProc (the Java version supports all steps); Table I provides an overview of the missing functionality.
|p:http-request||Yes||Only simple GET requests|
|p:load||Yes||DTD validation not supported|
|p:xinclude||Yes||XPointer not supported|
|p:xslt||Yes||Only XSLT 1.0 supported (browser-native)|
|p:hash||Yes||CRC32 not supported|
The GWT version has most of the features of the original Java version: from high compliance with the XProc specification to extensibility and customizability. For instance, it is possible to register extension steps or custom URI handlers with both the Java and GWT versions.
Required tests (that all conformant XProc processors must pass)
Optional tests (that conformant XProc processors are not required to pass)
Serialization tests (that test the XML Serialization W3C Serialization features)
Extension tests (that test the support for XProc extensibility)
Regarding the performance of the GWT version of the processor, there are noticeable (and expected) differences between different browsers, especially with more complex pipelines that involve evaluating many XPath expressions or creating large numbers of intermediate XML documents. But surprisingly, the most expensive part of running an XProc pipeline turned out to be the initial phase: parsing the pipeline source, resolving imports, performing the static analysis, and establishing the evaluation order of the steps in the pipeline. While executing the pipeline itself generally requires only milliseconds or tens of milliseconds (in Gecko- or WebKit-based browsers on a 2.33 GHz dual-core workstation with 4 GB of RAM), preparing and statically checking the pipeline often takes considerably more time (from 50 to 300 milliseconds depending on the web browser and the complexity of the pipeline); a clear area for further performance optimizations.
interacting with the server-side. In the XML-based model, the result of an AJAX request
is an XML document that needs to be processed in some way; most often by traversing
the XML structure using the DOM API. For example, in a web shop application, an AJAX
callback might be used to call a
productlist service and to display the results in a dynamically constructed HTML table. This
would typically involve iterating over the result elements in the returned XML document
and creating a table row for each result.
The example below attempts to translate the usual
AJAX request - process response - update host page pattern to XProc. The XProc pipeline starts with an HTTP request to the
service. The XML document returned by the service is then processed by creating an
HTML table row for each
product element in the document. After that, all table rows are inserted into a table wrapper
which is then injected into the host page. In the example, a custom extension is used
that makes it possible to address elements of the host page using their ID. In our
p:store step effectively replaces the element with the ID
by the generated HTML table.
<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" xmlns:c="http://www.w3.org/ns/xproc-step" xmlns:xhtml="http://www.w3.org/1999/xhtml" version="1.0"> <p:http-request> <p:input port="source"> <p:inline> <c:request method="GET" href="/productlist"/> </p:inline> </p:input> </p:http-request> <p:for-each name="for"> <p:iteration-source select="//product"/> <p:output port="result" sequence="true"/> <p:variable name="product-name" select="/product/@name"/> <p:string-replace match="xhtml:td/text()" replace="concat('"', $product-name, '"')"> <p:input port="source"> <p:inline> <xhtml:tr> <xhtml:td>tmp</xhtml:td> </xhtml:tr> </p:inline> </p:input> </p:string-replace> </p:for-each> <p:insert match="xhtml:tbody" position="first-child"> <p:input port="source"> <p:inline> <xhtml:table id="search-results"> <xhtml:tbody/> </xhtml:table> </p:inline> </p:input> <p:input port="insertion"> <p:pipe step="for" port="result"/> </p:input> </p:insert> <p:store href="domid:search-results"/> </p:declare-step>
So is there a benefit in adopting the inherently
heavier XProc approach where a simpler and more efficient alternative exists? We argue there
is, although it depends strongly on the particular use case. XProc is not a hammer
for everything: it is first and foremost an XML processing language, and it should
be used as such. Client-side XProc therefore makes most sense in user interfaces that
are XML-driven, consume or produce XML data, or require non-trivial XML processing.
In other situations, other approaches may be more appropriate.
The XProc pipeline above is admittedly very minimal, and from the dynamic web applications perspective it does not show anything new that could not be done with existing approaches. The interesting part lies in what XProc can offer beyond this point. While the example pipeline may seem as unnecessary overhead for the simple problem (populating an HTML table based on data returned from the server), the perspective begins to shift when further processing of the server-side results is needed. The XProc pipeline can query or transform the data easily, enrich it, or combine it with data obtained from other services; all using a unified, declarative, and flexible XML processing model.
A common task in XML-based web applications is presenting the XML data (either static or dynamically generated) to the user. Typically, this is done by transforming the XML data on the server-side to HTML or some other format understood by the browser. Most of these transformations can be expressed in the form of XProc pipelines, and with a reasonably compliant client-side XProc implementation, it should be possible to move the processing to the client-side completely. Delegating the rendering to the client-side can not only reduce the load on the server, but in many cases, it can also simplify the server-side functionality in general and make it less coupled with the front-end technology.
The pipeline below takes an XML document, resolves possible XInclude references, and applies an XSLT stylesheet to the resolved document; all with standard XProc and completely in the browser.
<p:pipeline xmlns:p="http://www.w3.org/ns/xproc" version="1.0"> <p:xinclude/> <p:xslt> <p:input port="stylesheet"> <p:document href="style.xsl"/> </p:input> </p:xslt> </p:pipeline>
The previous pipeline is obviously a very simple one, but the potential of using XProc as a client-side XML rendering tool is obvious: one can imagine using XProc for bringing popular XML vocabularies like, for instance, DocBook DocBook or DITA DITA to the browser.
The example below shows a pipeline that takes a DITA topic and returns its HTML rendition.
While the pipeline itself is trivial, the
dita:topic-to-xhtml step will most likely perform rather complex XML processing: from resolving the various
DITA link types to content filtering to applying an XSLT stylesheet. Or... maybe not:
the black-box nature of XProc steps provides great freedom by allowing different implementations
of the same step
interface - which is not only convenient when writing (and testing) the pipelines, but it also
makes it possible to adapt the pipelines to the needs of a particular user audience
or to different browser environments. Thus, the pipeline below can, for instance,
do full-blown client-side DITA processing in web browsers that are known to be fast
(or compliant) enough, and delegate to the server-side in other cases.
<p:pipeline xmlns:p="http://www.w3.org/ns/xproc" xmlns:dita="http://example.org/ns/dita" version="1.0"> <p:import href="dita-lib.xpl"/> <dita:topic-to-xhtml/> </p:pipeline>
An interesting application of XProc pipelines in web browser is using them as a driver for interactive client-side procedures. In such an obvious user interface-oriented platform as a web browser, it is possible to imagine XProc extension steps that would add interaction with the end-user to the XML processing logic - for example, by displaying dynamically generated dialogs or messages on the screen. This would make XProc a simple (yet powerful) alternative to other approaches for representing interactive processes: from simple data collection procedures to complex and often highly dynamic maintenance and diagnostic procedures found, for instance, in the military and in the aerospace industry.
In the area of interactive processing, client-side XForms is a technology that combines exceptionally well with XProc: as an XML-based standard, it is very easy to load - or even dynamically generate - XForms-based dialogs in XProc pipelines. Similarly, the XForms submissions, which are XML documents as well, can be processed naturally in XProc.
The pipeline below shows how XForms could be used with XProc in an imaginary aircraft
maintenance system; in this case, the example is a simple
circuit breaker check procedure. The pipeline has two options, an aircraft model number and its variant.
Depending on a specific combination of the model and the variant, the maintenance
mechanic is presented with a dialog (an XForm) that allows him to enter information
about the state of the circuit breaker. When the mechanic submits the dialog, the
pipeline displays a message that tells the mechanic to switch on the breaker if it
was in the OFF position.
<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" xmlns:am="http://example.org/ns/aircraft-maintenance" xmlns:xf="http://www.w3.org/2002/xforms" version="1.0"> <p:option name="model" required="true"/> <p:option name="variant" required="true"/> <p:import href="aircraft-maintenance-lib.xpl"/> <p:choose> <p:when test="$model = '777' and ($variant='200' or $variant='200ER')"> <am:dialog href="forms/777-200-circuit-breaker-check.xfm"/> </p:when> <p:when test="$model = '777'"> <am:dialog href="forms/777-circuit-breaker-check.xfm"/> </p:when> <p:when test="$model = '787'"> ... </p:when> ... </p:choose> <p:choose> <p:when test="/state = 'OFF'"> <am:message> <p:input port="source"> <p:inline> <p xmlns="http://www.w3.org/1999/xhtml"> Switch the circuit breaker to the ON position. </p> </p:inline> </p:input> </am:message> </p:when> <p:otherwise> <p:sink/> </p:otherwise> </p:choose> </p:declare-step>
Although XProc is still a relatively new technology, it is already finding its way into the XML application developers' tool set. Extending and complementing the family of established XML processing languages such as XSLT and XQuery, XProc provides a unifying and flexible integration layer that makes orchestration of XML processes easier.
In the web applications world, XML processing has traditionally been done primarily on the server-side. However, with the recent advances in modern web browsers and the growing interest in end-to-end XML application architectures, we can see that more and more XML processing is being done on the client-side. We believe that the problems that XProc is attempting to address - the impedance mismatch between different XML processing models, and the need to write plumbing code - apply equally to the server-side and the client-side (although on the client-side this may not be that visible - yet).
The current GWT port of Calumet is still very much work-in-progress. While it is reasonably
stable already and supports a relatively large portion of the XProc specification,
the code still needs to be optimized and fine-tuned for different web browsers. Also,
In GWT-based applications, the
we are considering is to support embedding XProc pipelines in the HTML
script element; however, that direction still needs to be researched.
[DITA] Michael Priestley, JoAnn Hackos, eds. Darwin Information Typing Architecture (DITA) Architectural Specification v1.1. OASIS Standard. 1 August 2007. http://docs.oasis-open.org/dita/v1.1/OS/archspec/archspec.html
[W3C Serialization] Scott Boag, Michael Kay, Joanne Tong, Norman Walsh, and Henry Zongaro, eds. XSLT 2.0 and XQuery 1.0 Serialization. W3C Recommendation. 23 January 2007. http://www.w3.org/TR/xslt-xquery-serialization/
[W3C XForms] John M. Boyer, ed. XForms 1.1. W3C Recommendation. 20 October 2009. http://www.w3.org/TR/xforms11/