How to cite this paper

Davis, Cornelia. “Programming Application Logic for RESTful Services Using XML Technologies.” Presented at Balisage: The Markup Conference 2011, Montréal, Canada, August 2 - 5, 2011. In Proceedings of Balisage: The Markup Conference 2011. Balisage Series on Markup Technologies, vol. 7 (2011). https://doi.org/10.4242/BalisageVol7.Davis01.

Balisage: The Markup Conference 2011
August 2 - 5, 2011

Balisage Paper: Programming Application Logic for RESTful Services Using XML Technologies

Cornelia Davis

Senior Technologist

EMC Corporation

Cornelia Davis is a Senior Technologist in the Architecture group of the Office of the CTO, focusing RESTful Service Oriented Architectures. Areas of expertise include XML and Atom, and she frequent speaker on RESTful SOA. Cornelia holds a B.S. and an M.S. in Computer Science from California State University, Northridge.

Copyright © 2011 EMC Corporation. All rights reserved.

Abstract

XRX, XForms on the client, RESTful services and XQuery on the server, introduced a development paradigm that could avoid the use to procedural code in the implementation of RESTful services.  With the standardization of XProc, the XML pipelining language, and the availability of several XProc engines, we have an even more powerful mechanism for RESTful services construction. In this paper we briefly introduce an XML REST Framework that allows a developer to define resources and provide an XML-centric implementation.  Then the main focus of the paper is on how XQuery, XSLT and XProc together form a powerful set of tools with which RESTful services can be developed, effectively redifining XRX to stand for XForms, RESTful services and XProc on the server. We illustrate the benefits each technology brings to this service construction by incrementally building up a RESTful service for a patient medical records registry.

Table of Contents

Introduction
Integrated Health Exchange
The REST Architectural Style
The XML REST Framework
The RESTful Service Interface
Starting with XQuery
XSLT
XProc – the XML Pipelining Language
Getting More Sophisticated
Multi-part Responses
XProc and Transactions
Binding HTTP Processing to the XML-Centric Implementation
Conclusions
Limitations
Future Work

Introduction

The term XRX XRX , stands for XForms XForms on the client, RESTful services and XQuery XQuery on the server. At its core it is a design approach that uses XML as the model for the application entities, and other XML technologies, specifically XForms and XQuery, for the application UI and for interface to the persistence layer, respectively. At the extreme, XRX can be seen as a no-transformation approach, where resource representations accepted and served by RESTful services closely match the form stored in an XML database; that is, there is no difference between the logical model for entities and the physical one. While our work has been inspired by XRX, and we embrace the notion of XML-centric implementations, we have found that RESTful services require transformations and other sophisticated operations that are not particularly well suited to an XQuery implementation alone.

First, RESTful services must serve resource representations that are hyperlink rich, containing links to related resources as well as to URLs that can drive application state.  These links are generated only when the resource representation is served and they reflect runtime and deployment contexts such as host names.  As such, these links must be added to the content that is retrieved from the XML database, and while XQuery is clearly the right technology for database access, it is, at best, awkward when used to insert these hyperlinks.  

Another key tenet of the REST architectural style is content negotiation, the ability for a service to accommodate various formats for the data they serve.  For example, a resource representation may be served in some simple XML format (that may indeed closely resemble that which is stored in the database), and it may alternatively serve an Atom Atom entry.  Those well versed in the XML-technology stack would likely agree that this is a task ideally suited to XSLT XSLT.

When we begin to address the implementation details of our RESTful services we see common patterns, such as the need to assign identifiers to new resources and the need to return from the service values beyond the resource representation.  We have found it rather easy to implement these patterns using XProc XProc

It is these RESTful services requirements that have driven us to an interpretation of XRX that stands for XForms on the client, RESTful services and XProc on the server.  Our work has focused on the RESTful services implementation with little attention given to the XForms or other consumer-side user interface.

In this paper we will show the value that various XML technologies bring to the implementation of RESTful services, with a focus on XProc, XQuery and XSLT.  We will demonstrate each of their strengths by incrementally building up a simple service implementation that is a part of a larger set of services that implement a patient medical records registry.  This implementation was taken to the IHE Connectathon IHEConnect event in January 2011 where EMC received certification.  That is, the use case presented in this paper is real. 

After briefly introducing the IHE XDS IHEXDS  registry usecase, outlining the key principles of REST and also briefly introducing an XML REST Framework we have constructed, we spend the bulk of the remainder of the paper squarely in the XML space.  We begin by establishing the base implementation which uses XQuery to write resource data to the persistence layer, an XML database.  We then address the hyperlink constraint of RESTful services with the construction of an XSLT and build a simple XProc pipeline to sequence these operations.  This solution does not yet address the generation of identifiers or construction of data elements beyond the resource representation, which we then add.  Finally we bind all of the pieces together within Spring Framework Spring configurations.

While we have found some prior work on XML-centric application development Wilde, we have found little that addresses how the XML technology stack addresses the unique needs of RESTful service construction. This is the focus of this piece.

Integrated Health Exchange

Integrating the Healthcare Enterprise (IHE) IHE is a consortium that develops interoperability standards for health care delivery systems.  They publish specifications that address how a wide range of devices and systems should communicate, allowing them to be easily connected in a variety of settings. It is in one of these specifications that they define interfaces that medical records registries and repositories must provide in order to be easily connected to document suppliers and consumers. A Cross-Enterprise Document Sharing (XDS) repository stores documents that make up a patient medical record including such things as images (e.g. x-rays, CT scans), text files (e.g. doctors notes) and documents of any other format. An XDS registry augments this content both by associating metadata with the documents stored in the repository and by establishing additional organizational structures, such as folder hierarchies, around that content. The solution we describe in this paper is a portion of the document registry implementation which earned EMC a certification at the IHE Connectathon event in January 2011.

The abstractions defined by the IHE for an XDS registry include the following:

  • A document entry holds metadata for a single document.

  • A folder is a container that may hold multiple document entries.

  • An association captures a binary, unidirectional relationship between document entries, folders, submission sets and other associations.

  • A submission set represents a collection of document entries, folders and associations that together form a set that, when written to the registry, must be handled atomically.

While submission sets are written as a whole, the individual parts of a submission set may be consumed in different combinations.  For example, within a single submission set, a folder may have been created and a document entry placed therein, however, the document entry may be accessed independently or even in combination with other document entries that arrived in different submission sets. 

It is the atomicity requirements on the writing of submission sets, and the granular consumption model, that contribute to making the XDS registry RESTful services an interesting case-study.  We will examine the implementation of the service for creating submission sets in detail in the subsequent sections of this paper.  This is relatively complex operation that may involve the assignment of identifiers, necessitates validation of some of the input XML against both other portions of the submission set as well as to content already existing in the database, and requires that the submission set be decomposed for storage in the database.

The REST Architectural Style

Roy Fielding was one of a group of individuals who developed the architecture of the World Wide Web and in 2000 he formalized the key architectural principles in his PhD dissertation IHE, coining the term REpresentational State Transfer (REST). The REST architectural style is characterized by the following four key tenets:

  • Identification and addressability of resources: All interesting bits of information are identified with URIs and are usually accessed via URL.

  • The uniform interface: Interaction with resources is through a standardized set of operations, with well understood and agreed upon semantics.

  • Manipulation of resources through representations: Clients are not operating directly on resources, rather resource representations are transfered between the server and the client.

  • Hypermedia constraint: Resource representations include hyperlinks that can be used to drive application state transitions.

Each of these principles has played an important role in the success of the World Wide Web. Resource centricity and the hierarchical, global address space of URLs provides for limitless scale by allowing resources to be continually added to the domain of discourse while maintaining linear scale through the use of DNS and a cache-rich infrastructure. Having a uniform interface allows the layered web to perform optimizations as a part of a resource operation. For example, because the HTTP PUT operation is idempotent (meaning it can be executed 1 or more times with the same result), an actor in the web infrastructure may perform automatic retries on PUT operations that may have failed to complete. The transfer of resource representations between the client and server allows those interactions to be entirely stateless, further providing scale-out characteristics. And having hyperlink-rich resource representations not only provides a means for relationships between resources to be presented, it also supports the construction of less-brittle interfaces and looser coupling between clients and servers.

The XML REST Framework

We have produced a framework that allows a developer to create a set of RESTful services with most of the implementation achieved using XML-based technologies. We have found these technologies to be very effective at addressing many requirements specific to RESTful services. XQuery is used to persist resource state into an XML database. Content negotiation is straight-forward via the declarative, XSLT programming model, and resource hyperlinks are generated using the same declarative approach. Common patterns for resource operations are effectively captured in XProc pipelines and processing of composite resources is also well accomplished using this XML pipelining approach.

The only portion of the implementation not done with an XML-based technology is the interface to the RESTful service. Here we have elected to use Plain Old Java Objects (POJOs), annotated with information about resource URLs and the uniform interface. We chose to keep this part of a RESTful service implementation in Java primarily for two reasons. First, while there is a technology, Servlex Servlex, that does provide a capability for producing web applications with only XML technologies, we were concerned that the uptake of the EXPath Webapp EXPath Webapp approach has been slow and the community activity is marginal. And more importantly, having RESTful services executing within an environment such as the Spring Framework allows additional services (such as security) to be wrapped around the core RESTful services we implement; it was unclear how the Servlex technology could be leveraged within Spring.

Figure 1 depicts the basic construction of RESTful services using our framework.

Figure 1

XML-Centric Services Implementation

In the next sections I cover each of the blocks shown in this figure.

The RESTful Service Interface

Because we are specifically addressing RESTful services implementations, we must address more than just the construction and delivery of XML data.  In particular, we must be able to accept HTTP requests [1] , parse URLs, read and write headers and return errors appropriately.  Because several frameworks addressing these HTTP-specific needs, such as Spring MVC Spring MVC, Apache CXF Apache CXF and Jersey Jersey, are already in widespread use, we embrace those and offer an extended framework that allows an XML-centric implementation to be wired in.  These RESTful services frameworks share common development paradigms where REST resources are implemented as Java classes, operations on the resource are implemented with class methods, and annotations are used to express RESTful service specifics such as URI templates and uniform interface operations.  The following code snippet shows the skeleton Java class for the submission sets resource, with a method that will fulfill the POST operation; this is the operation we will use to create new submission sets.

package com.emc.cto.healthcare;
            
    // … imports omitted for brevity
            
    @Controller
    @RequestMapping("/submissionsets")
    public class SubmissionSets {
            
    @RequestMapping(method = RequestMethod.POST)
    @ResponseStatus(HttpStatus.CREATED)
    public String addPatient(HttpServletRequest request, 
                             HttpServletResponse response, 
                             Model model) throws XProcException, IOException, URISyntaxException, TransformerException {
            
        …
            
    }
            
    public SubmissionSets() {		
    }
            
}

It is within this method that we will invoke the XML-based services implementation.  In a later section we will see exactly how the XProc pipeline, which forms the core of the implementation, is bound into this service dispatcher class.

Starting with XQuery

XQuery serves the role of interfacing with the solution’s persistence layer, the XML database.  Each resource operation of the RESTful service will require one or more XQueries to map the logical to the physical model.  Figure 2 shows a table representing the logical model at left, a pictorial view of the physical database model on the right and shading that indicates how the logical resources are mapped to physical ones.

Figure 2

Logical to Physical Mapping

The darker shaded objects in the tree structure represent folders and the lighter shaded objects represent XML documents.  The yellow and blue shadings show a correspondence between entities in the logical model and the physical model. Note that the submission set resource maps to many entities in the physical model, whereas document entry resource mappings are far more constrained.

For this first stage of the implementation we will assume that the submission set coming in is entirely valid and has identifiers properly set for each of the elements within it.  The following shows an excerpt of the resource representation supplied to a POST operation.

<?xml version="1.0" encoding="UTF-8"?>
<CompoundSubmissionSet>
    <SubmissionSet>
        <comments>
            <lang>en-US</lang>
            <charset>UTF-8</charset>
            <value>Annual physical</value>
        </comments>
        <entryUuid>0324990a-00a4-4b3f-be6d-0a98267d1d28</entryUuid>
        <patientId>
            <id>RED7020</id>
            <assigningAuthority>
                <universalId>1.3.6.1.4.1.21367.13.20.1000</universalId>
                <universalIdType>ISO</universalIdType>
            </assigningAuthority>
        </patientId>
        <title>
            <lang>en-US</lang>
            <charset>UTF-8</charset>
            <value>Physical</value>
        </title>
        ... additional submission set fields
    </SubmissionSet>
    <folders />
    <documentEntries>
        <DocumentEntry>
            <entryUuid>788ccf31-f9fe-409a-91c4-8983e1c8ae14</entryUuid>
            <patientId>
                <id>RED7020</id>
                <assigningAuthority>
                    <universalId>1.3.6.1.4.1.21367.13.20.1000</universalId>
                    <universalIdType>ISO</universalIdType>
                </assigningAuthority>
            </patientId>
            <title>
                <lang>en-US</lang>
                <charset>UTF-8</charset>
                <value>Physical</value>
            </title>
            ... additional document entry fields
        </DocumentEntry>
    </documentEntries>
    <associations>
        <Association>
            <targetUuid>788ccf31-f9fe-409a-91c4-8983e1c8ae14</targetUuid>
            <sourceUuid>0324990a-00a4-4b3f-be6d-0a98267d1d28</sourceUuid>
            <associationType>HAS_MEMBER</associationType>
            <label>ORIGINAL</label>
            <entryUuid>cb297ac4-22f7-4303-81f5-76bb0befd8cc</entryUuid>
        </Association>
    </associations>
</CompoundSubmissionSet>
        

In the XQuery, then, we fundamentally do two things: we split this larger XML document into several smaller ones and write each of those to the database. The following XQuery code shows this implementation.

declare variable $input external;

declare updating function local:storeSubmissionset($elem as element()) {
    let $docFileName := concat("/SubmissionSets/", data($elem/entryUuid), ".xml")
    return if (doc-available($docFileName))
           then replace node doc($docFileName)/SubmissionSet with $elem
           else xhive:insert-document($docFileName, document{$elem})
};

declare updating function local:updateTime($folder as element()) {
        let $time := replace(substring-before(xs:string(adjust-dateTime-to-timezone(current-dateTime(),xs:dayTimeDuration("-PT0H"))),"."), "[-:T]", "")
        return <Folder>{$folder/*[name() != "lastUpdateTime"]} <lastUpdateTime>{$time}</lastUpdateTime></Folder>
};
declare updating function local:storeFolders($elem as element()) {
        for $folderRaw in $elem/Folder
                                    let $folder := local:updateTime($folderRaw)
        let $docFileName := concat("/Folders/", data($folder/entryUuid), ".xml")        
        return if (doc-available($docFileName))
                 then replace node doc($docFileName)/Folder with $folder
                else xhive:insert-document($docFileName, document{$folder})
};

declare updating function local:storeDocEntries($elem as element(), $assocs as element()) {
        for $entry in $elem/DocumentEntry
        let $entryUuid := normalize-space(data($entry/entryUuid))
        let $docFileName := concat("/DocumentEntries/", $entryUuid, ".xml")
        let $assoc := $assocs/Association[sourceUuid = $entryUuid and (associationType='TRANSFORM_AND_REPLACE' or associationType='REPLACE') ]
        let $oldDocUuid := normalize-space(data($assoc/targetUuid))
        return  if (doc-available($docFileName))
               then replace node doc($docFileName)/DocumentEntry with $entry
               else (
                      xhive:insert-document($docFileName, document{$entry}),
                      if ($oldDocUuid) 
                      then
                            replace value of node doc("DocumentEntries")/DocumentEntry[entryUuid=$oldDocUuid]/availabilityStatus
                            with "DEPRECATED"
                         else ())
};

declare updating function local:storeAssociations($elem as element()) {
        let $time := replace(substring-before(xs:string(adjust-dateTime-to-timezone(current-dateTime(),xs:dayTimeDuration("-PT0H"))),"."), "[-:T]", "")
        let $res := for $association in $elem/Association
        let $docFileName := concat("/Associations/", data($association/entryUuid), ".xml")
        let $sourceFileName := concat(normalize-space(data($association/sourceUuid)),".xml")
        return (if (doc-available($docFileName))
               then replace node doc($docFileName)/Association with $association
               else xhive:insert-document($docFileName, document{$association}),
               (: if new doc placed into existing folder, update the lastUpdateTime of that folder :)
               if (doc-available(concat("/Folders/", $sourceFileName)))
               then replace value of node doc(concat("/Folders/", $sourceFileName))/Folder/lastUpdateTime with $time
               else (),
               (: if the new doc replaces another we need to create an assoc between the new doc and all of the folders
                  that the orignal doc is in :)
               if ($association/associationType = "REPLACE")
               then local:addAssocsForReplacementDocToFolders($association)
               else ()
               )
return $res
};

declare updating function local:addAssocsForReplacementDocToFolders($association as element()) {
    let $orgDocUuid := normalize-space(data($association/targetUuid))
    let $newDocUuid := normalize-space(data($association/sourceUuid))
    let $newAssocs := for $assoc in doc("Associations")/Association[targetUuid=$orgDocUuid and associationType = "HAS_MEMBER"]
                          let $folder := doc("Folders")/Folder[entryUuid=$assoc/sourceUuid]
                          let $newAssoc := if ($folder)
                               then <Association><targetUuid>{$newDocUuid}</targetUuid>
                               <sourceUuid>{normalize-space(data($folder/entryUuid))}</sourceUuid>
                               <associationType>HAS_MEMBER</associationType>
                               <entryUuid>{xs:string(uuid:random-uuid())}</entryUuid></Association>
                               else ()
                       return $newAssoc
return local:storeAssociations(<associations>{$newAssocs}</associations>)
};

let $compoundsubmissionset := if (not(empty(.)))
              then .
              else xhive:parse($input)
return (local:storeSubmissionset($compoundsubmissionset/CompoundSubmissionSet/SubmissionSet),
        local:storeFolders($compoundsubmissionset/CompoundSubmissionSet/folders),
        local:storeDocEntries($compoundsubmissionset/CompoundSubmissionSet/documentEntries,$compoundsubmissionset/CompoundSubmissionSet/associations),
        local:storeAssociations($compoundsubmissionset/CompoundSubmissionSet/associations), 
        $compoundsubmissionset/CompoundSubmissionSet)

This implementation is relatively crisp and XQuery serves the needs rather well, however there are RESTful service requirements that this implementation has not yet met.

Notice that the XQuery responds with XML that represents the resource that has been newly created.  This forms the basis of the resource representation that will be returned by the RESTful service, however, it is not yet complete.  The hypermedia constraint in RESTful services Hypermedia Constraint requires that resource representations contain hyperlinks to other resources, as well as hyperlinks that can otherwise drive the state of the application.  In the case of a submission set, for example, the representation should contain links to the documents and folders comprising it.

While it is possible to achieve this augmentation of the XML using XQuery, we prefer XSLT for two primary reasons. First, implementations of RESTful services using the approaches described in this piece, follow a Model-View-Controller pattern and we intentionally have the XML returned from the XQuery represent the application model objects. The hyperlinks presented in a resource representation are the responsibility of the controller portion of the implementation, so having hyperlinks inserted as a part of the XQuery that interacts with the database would conflate the responsibilities of the model and the controller. Of course, a separate XQuery could be used for hyperlink insertion, yet this brings us to the second reason for a different choice, and that is simply that we prefer the declarative approach that XSLT affords.

XSLT

Inserting hyperlinks into an XML document involves two things; one must define the points of insertion and then express what is to be inserted, and the <xsl:template> is ideally suited to the task.  Our XML REST Framework aims to make this task easy, even for the non-XSLT expert, by providing a simple XSLT template that traverses the document tree seeking those points of insertion, and by providing samples of the <xsl:template> definitions that insert links.

The following code snippet shows the simple recursive template that simply copies each of the source nodes into the result tree and applies a template with an “insert-here” mode at each element node.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="2.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:fn="http://www.w3.org/2005/xpath-functions" xmlns:xdt="http://www.w3.org/2005/xpath-datatypes"
    xmlns:pat="http://www.emc.com/cto/PMR" xmlns:atom="http://www.w3.org/2005/Atom">
    <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes" />

    <!-- 
        The templates in this xslt simply traverse the input XML and for each element apply any insertion templates 
        that are defined for the particular type of object
     -->

    <xsl:template match="*">
        <xsl:copy>
            <!-- bring all attributes over -->
            <xsl:apply-templates select="@*" />
            <!-- insert any hyperlinks -->
            <xsl:apply-templates select="." mode="inserthere" />
            <xsl:apply-templates select="node()" />
        </xsl:copy>
    </xsl:template>

    <xsl:template match="@*">
        <xsl:copy />
    </xsl:template>

    <xsl:template match="text()" mode="inserthere" />

</xsl:stylesheet>

The RESTful services developer then need only define the <xsl:template> definitions for the elements that should have child link nodes inserted. The following stylesheet inserts hyperlinks into the submission set resource representation that is returned from the creation (POST) operation.

<xsl:stylesheet version="2.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:fn="http://www.w3.org/2005/xpath-functions" xmlns:xdt="http://www.w3.org/2005/xpath-datatypes"
    xmlns:atom="http://www.w3.org/2005/Atom"
    xmlns:functx="http://www.functx.com" exclude-result-prefixes="xs xsl fn xdt functx">
    <xsl:import href="classpath:insertHyperlinks.xslt" />
    <xsl:import href="classpath:utils.xslt" />
    <xsl:output method="xml" version="1.0" encoding="UTF-8"    indent="yes" />

    <xsl:param name="baseURL" />
    
    <!--
        This XSLT defines insertion templates that will be driven by the
        imported stylesheet
    -->

    <xsl:template match="SubmissionSet" mode="inserthere">
        <atom:link rel="self">
            <xsl:attribute name="href"><xsl:value-of
                select="concat($baseURL,'/',entryUuid)" /></xsl:attribute>
        </atom:link>
        <xsl:apply-templates select="/CompoundSubmissionSet/documentEntries/DocumentEntry" mode="inserthere" />
        <xsl:apply-templates select="/CompoundSubmissionSet/folders/Folder" mode="inserthere" />
    </xsl:template>

    <xsl:template match="DocumentEntry" mode="inserthere">
        <atom:link rel="document">
            <xsl:attribute name="href"><xsl:value-of
                select="concat(functx:substring-before-last($baseURL,'/'),'/documents/',entryUuid)" /></xsl:attribute>
        </atom:link>
    </xsl:template>
    
    <xsl:template match="Folder" mode="inserthere">
        <atom:link rel="folder">
            <xsl:attribute name="href"><xsl:value-of
                select="concat(functx:substring-before-last($baseURL,'/'),'/folders/',entryUuid)" /></xsl:attribute>
        </atom:link>
    </xsl:template>
    
    <xsl:template match="CompoundSubmissionSet">
        <xsl:apply-templates select="SubmissionSet"/>
    </xsl:template>
    
    <xsl:template match="folders"/>
    <xsl:template match="documentEntries"/>
    <xsl:template match="associations"/>
    
</xsl:stylesheet>

Another type of translation that is often needed for RESTful services provides support for additional media types.  In general, the media type for the content returned from an XQuery will be application/xml or text/xml, and this may be directly returned by the RESTful service. Other XML formats, such as application/atom+xml, are also in widespread use and therefore a RESTful services framework that makes it easy to perform such a transformation is valuable.  Just as with the insertion of hyperlinks, such transformations are well executed with an XSLT stylesheet.  Note that JSON is gaining popularity as a media type for resource representations.  Our recommendation is to keep the application model entities in XML and perform the transformation to JSON, again, via XSLT transformation, at the outer edge of that implementation.

At this point we have seen the value in XQuery to interface with the underlying XML database, and XSLT for the insertion of hyperlinks and support for alternate media types.  The next question to answer, then, is how to tie these two things together.

XProc – the XML Pipelining Language

In May 2010 the XML Processing Model Working Group of the World Wide Web Consortium standardized XProc, an XML Pipelining language.  XProc is a high-level programming language in which XML processing steps are wired together in such a way that the outputs of one step are routed to the inputs of another step; all content flowing between steps is XML.  An XProc Engine executes a pipeline, accepting external inputs and producing zero or more outputs. The XProc language includes several dozen processing steps that when used together allow for sophisticated capabilities to be implemented with very few lines of code.

Before looking at some of these more complex examples, let us first look at a very simple pipeline that ties together the two solution components we previously discussed – the XQuery for database access and the XSLT for hyperlink insertion.  Figure 3 shows a pictorial representation of this pipeline. 

Figure 3

Base Resource-creation Pipeline

In this case where we are creating a submission set, the source document passed into the pipeline is the submission set representation.  A second input to the pipeline is the base URL, a parameter that reflects the deployment particulars of the service (such as hostname), is used in the later XSLT step.  The source document is passed into the XQuery, which, as we saw above, splits the input document and writes several XML documents to the XML database; the XQuery step produces an XML document as the response.  The XSLT step then accepts the result from the XQuery, as well as the base URL that was passed into the pipeline, and generates a hyperlink rich XML document on the output.

Getting More Sophisticated

Recall that earlier we made some simplifying assumptions; one being that the XML document supplied to the XQuery would be valid, including having identifiers already assigned to the various portions of the submission set. While the IHE specifications do require that all recorded document entries, folders, submission sets and associations have UUIDs assigned, they allow for a case where the registry receives a submission set containing only locally scoped identifiers (within the submission set). In this case, the registry implementation responsible for assigning UUIDs. The following shows and example of such a resource representation.

<?xml version="1.0" encoding="UTF-8"?>
<CompoundSubmissionSet>
    <SubmissionSet>
        <comments>
            <lang>en-US</lang>
            <charset>UTF-8</charset>
            <value>Annual physical</value>
        </comments>
        <entryUuid>SubmissionSet01</entryUuid>
        ... additional submission set fields
    </SubmissionSet>
    <folders />
    <documentEntries>
        <DocumentEntry>
            <entryUuid>Document01</entryUuid>
            ... additional document entry fields
        </DocumentEntry>
    </documentEntries>
    <associations>
        <Association>
            <targetUuid>Document01</targetUuid>
            <sourceUuid>SubmissionSet01</sourceUuid>
            <associationType>HAS_MEMBER</associationType>
            <label>ORIGINAL</label>
            <entryUuid>Association01</entryUuid>
        </Association>
    </associations>
</CompoundSubmissionSet>
                

In our implementation we use XProc to replace the local identifiers with UUIDs and to do so we need only loop over each of the non-UUID identifiers and invoke the UUID XProc step. Our main XProc pipeline will now include a step which performs this insertion, and the output, which fulfills that validity constraint, will be wired to the XQuery step. The following XProc step definition shows the recursive application of the UUID step.

<p:declare-step name="main"
    xmlns:p="http://www.w3.org/ns/xproc" 
    xmlns:c="http://www.w3.org/ns/xproc-step" 
    xmlns:emc="http://www.emc.com/cto/xds"
    type="emc:replaceNonUuids"
    version="1.0">
  <p:input port='source'/>
  <p:output port='result' primary='true'/>

  <p:choose name="processUuids">
      <p:xpath-context>
          <p:pipe step="main" port="source"/>
      </p:xpath-context>
      <p:variable name="nonUuid" select="//entryUuid[not(contains(.,'-')) and (position() = 1)]/text()">
          <p:pipe step="main" port="source"/>
      </p:variable>
      <p:when test="$nonUuid">
          <p:uuid name="uuid">
              <p:with-option 
                  name="match" 
                  select='concat("//*[(name(.)='entryUuid' or name(.)='targetUuid' or name(.)='sourceUuid')and .='",
                                 $nonUuid,
                                 "']/text()")'/>
              <p:input port="source">
                  <p:pipe step="main" port="source"/>
              </p:input>
          </p:uuid>
          <emc:replaceNonUuids>
              <p:input port="source">
                  <p:pipe step="uuid" port="result"/>
              </p:input>
          </emc:replaceNonUuids>
      </p:when>
      <p:otherwise>
          <p:identity>
              <p:input port="source">
                  <p:pipe step="main" port="source"/>
              </p:input>
          </p:identity>
    </p:otherwise>
  </p:choose>
</p:declare-step>

Figure 4 shows how this step is wired into the original pipeline.

Figure 4

Pipeline Augmented with Content Enrichment Step

To complete the pipeline for the submission set creation operation we also include some additional validation steps (which, for brevity, we will not show here), as well as some steps that produce a multi-part response.

Multi-part Responses

Because we are implementing our RESTful services almost exclusively in XProc pipelines, those pipelines must accommodate the need to generate more than just a single result. For example, a best practice in RESTful services is that resource creation operations return not only the resource representation, but also a URL to the newly created resource; this URL is to be returned in the HTTP Location header.  Furthermore, RESTful services MUST respond with a status code indicating success or failure, and since that outcome is largely determined in the XProc pipeline, that status information must similarly be returned.

Fortunately, XProc provides for multi-part responses with multiple output ports, where any output port can carry more than one value.  We implement the pipelines for our RESTful services in a consistent manner, providing three output ports: the primary contains the resource representation, another called “headers” contains key/value pairs and one called “error” outputs an XML document that carries information on any errors that occurred during the processing of the pipeline.  The following is the final implementation of our submission set creation pipeline, including error handling and the creation of the location header.

<p:declare-step name="main" 
    xmlns:p="http://www.w3.org/ns/xproc"
    xmlns:c="http://www.w3.org/ns/xproc-step"
    xmlns:emc="http://www.emc.com/cto/xds" 
    version="1.0">
    <p:input port="source"/>
    <p:input port='xqueryscript' />
    <p:input port="stylesheet"/>
    <p:input port="stylesheetParameters" kind="parameter"/>
    <p:input port="xqueryParameters" kind="parameter"/>
    <p:output port='result' sequence='true' primary='true'>
        <p:pipe step='checkXquery' port='result'/>
    </p:output>
    <p:output port='error' sequence="true">
        <p:pipe step='checkXquery' port='error' />
    </p:output>
    <p:output port='headers' sequence="true">
        <p:pipe step='checkXquery' port='headers' />
    </p:output>
    <p:import href="classpath:replaceNonUuids.xpl"/>
    
    <!-- This pipeline will assign identifiers whereever needed and then will
         execute the xquery against the source passed in.  Finally, it will
         take the result and enhance it with hyperlinks to related resources. 
         The xquery and xslt are both passed into the pipeline. -->

    <!-- Replace local identifiers with uuids. -->
    <emc:replaceNonUuids name="replaceNonUuids">
        <p:input port="source">
            <p:pipe step="main" port="source"/>
        </p:input>
    </emc:replaceNonUuids>

    <!-- execute xQuery against the input source -->
    <p:xquery name="xquery">
        <p:input port='source'>
            <p:pipe step="replaceNonUuids" port="result" />
        </p:input>
        <p:input port="query">
            <p:pipe step="main" port="xqueryscript" />
        </p:input>
        <p:input port="parameters">
            <p:pipe step='main' port='xqueryParameters'/>
        </p:input>
    </p:xquery>

    <!-- check the result of the xQuery to make sure there was no error -->
    <p:choose name="checkXquery">
        <p:variable name="error" select="/error/code">
            <p:pipe step="xquery" port="result" />
        </p:variable>
        <!-- in case of error, return error xml out of pipeline -->
        <p:when test="$error">
            <p:output port="error">
                <p:pipe step="genError" port="result" />
            </p:output>
            <p:output port="result" sequence='true' primary="true">
                <p:empty />
            </p:output>
            <p:output port="headers" sequence='true'>
                <p:empty/>
            </p:output>
            <p:string-replace name="genError" match="/error/code/text()">
                <p:input port="source">
                    <p:inline>
                        <error><code>err</code><description>description</description></error>
                    </p:inline>
                </p:input>
                <p:with-option name='replace' select="$error" />
            </p:string-replace>
        </p:when>
        <p:otherwise>
            <p:output port="error" sequence="true">
                <p:empty />
            </p:output>
            <p:output port="result" sequence='true' primary="true">
                <p:pipe step="xslt" port="result"/>
            </p:output>
            <p:output port="headers" sequence='true'>
                <p:pipe step="locXML" port="result"/>
            </p:output>
            <!-- insert hyperlinks -->
            <p:xslt name="xslt">
                <p:input port='source'>
                    <p:pipe step='xquery' port='result'/>
                </p:input>
                <p:input port='stylesheet'>
                    <p:pipe step='main' port='stylesheet'/>
                </p:input>
                <p:input port='parameters'>
                    <p:pipe step='main' port='stylesheetParameters'/>
                </p:input>
            </p:xslt>

            <!-- generate the URL for the newly created resource -->
            <p:variable name="baseU" select="/c:param-set/c:param[@name='baseURL']/@value">
                <p:pipe step="main" port="stylesheetParameters"/>
            </p:variable>
            <p:string-replace name="locXML" match="/Location/text()">
                <p:input port="source">
                    <p:inline>
                        <Location>here</Location>
                    </p:inline>
                </p:input>
                <p:with-option name='replace' select="concat('"',$baseU, '/', //entryUuid/text(),'"')">
                    <p:pipe step='xslt' port='result'/>
                </p:with-option>        
            </p:string-replace>

        </p:otherwise>
    </p:choose>
</p:declare-step>

XProc’s support for multiple outputs is a good fit for the requirements of RESTful services and avoids having the developer design a multi-part model themselves.

XProc and Transactions

One might note that while in our current implementation we have split our submission set into more fine-grained parts within the XQuery, that we could also have done so in the XProc pipeline. Doing the split as a step in the XProc pipeline could result in greater reuse of portions of the implementation and code that is easier to produce and maintain.  But then what would come of our requirement for atomicity in the construction of the submission set, if pipeline were to execute several updating XQuery steps?

Fortunately, the Xproc engine that we are using in house provides transaction boundaries at the start and end of a pipeline.   This allows us to construct pipelines that execute multiple XQueries, yet have them succeed or fail as one single unit.  This powerful capability is quite welcome in the construction of RESTful services where compound resources are common.

Binding HTTP Processing to the XML-Centric Implementation

At this point we’ve seen how an XProc pipeline allows us wire up a set of steps to provide the core implementation of our RESTful services.  The XProc pipeline accepts inputs and returns a multi-part response, including what can be thought of as a body, headers and error/status information.  What remains is how an XProc pipeline is bound to the Java classes we’ve created for each of our resources, as well as how to bind portions of the HTTP request/response to the pipeline inputs/outputs.

To bind an XProc pipeline to a resource (Java) method we inject the XProc pipeline into the method via a Spring configuration.  Each method in the Java class will have an XMLProcessingContext object associated with it, which encapsulates the pipeline as well as any design time bindings to the pipeline.  To understand why design time bindings are valuable, consider that the pipeline for most GET operations will be very similar; an XQuery will be used to obtain XML from the database and an XSLT will be applied to those results.  The only thing that differs from one GET operation to another is the specific XQuery and the specific XSLT stylesheet; hence, we will parameterize a single GET pipeline with those values and reuse that pipeline over many resource implementations.  The following excerpt from a Spring configuration file shows the injection of XQueries and XSLTs into the XMLProcessingContext for the createSubmissionSet operation, and the injection of that XMLProcessingContext into the resource class.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="http://www.springframework.org/schema/beans 
                        http://www.springframework.org/schema/beans/spring-beans.xsd
                        http://www.springframework.org/schema/context
                        http://www.springframework.org/schema/context/spring-context-3.0.xsd">

    <!-- other bean definitions omitted for brevity -->

    <!-- Patients and Patient Resource -->
    <bean id="Patients" class="com.emc.cto.healthcare.SubmissionSets">
        <property name="addSubmissionSet" ref="addSubmissionSetXMLProcessingContext" />
    </bean>

    <bean id="addSubmissionSetXMLProcessingContext" class="com.emc.cto.xproc.XProcXMLProcessingContext">
        <property name="xprocPool" ref="xprocPool" />
        <property name="pipelineSource"><value>classpath:xqueryXsltUuid.xpl</value></property>
        <property name="inputs">
            <map>
                <entry key="xqueryscript" value="classpath:registerObjectsMinimal.xq" />
                <entry key="stylesheet" value="classpath:hyperlinksSubmissionSet.xslt" />
            </map>
        </property>
        <property name="options"><map/></property>
        <property name="parameters"><map/></property>
    </bean>

</beans>

Finally, we must bind additional XProc parameters at run time, using the Java-based framework to access parts of the HTTP request and supplying those values to the pipeline in the Java method.  Following the execution of the pipeline the results are mapped back to the HTTP response.  This is done by our framework in a Spring MVC view that knows how to convert the multi-part pipeline output into an HTTP response; this relieves the services developer of these concerns.  The following code shows the full implementation of the method corresponding to the POST operation on the submission sets resource.

@RequestMapping(method = RequestMethod.POST)
@ResponseStatus(HttpStatus.CREATED)
public String addPatient(HttpServletRequest request, 
                         HttpServletResponse response, 
                         Model model) throws XProcException, IOException, URISyntaxException, TransformerException {
    try {
        PipelineInputCache pi = new PipelineInputCache();

        // supply http body as the source for the resource Create pipeline
        pi.setInputPort("source", request.getInputStream());
            
        // supply current resource URL as the base URL to craft hyperlinks
        String baseUrl = request.getRequestURL().toString();
        if (baseUrl.endsWith("/"))
            baseUrl = baseUrl.substring(0, baseUrl.length()-2);
        pi.addParameter("stylesheetParameters", new QName("baseURL"), baseUrl);

        PipelineOutput output = m_addSubmissionSet.executeOn(pi);

        model.addAttribute("pipelineOutput", output);
        return "pipelineOutput";
    } finally {
        ; 
    }
}

Conclusions

Our experience implementing an XDS Registry using the XML REST Framework shows that the XML Technology stack not only provides capabilities adequate for implementation of sophisticated RESTful web services, but also results in an elegant solution that is arguably easier to construct and maintain than a purely Java-based counterpart. We have not done any quantitative analysis comparing developer productivity between the two approaches, or comparing the use of our framework to other XRX-based approaches, however we do have some anecdotal evidence that supports this claim. In one instance, a development team was able to complete the work for two project sprints within the time frame of a single sprint; they credited the XML-centric approach to RESTful services construction for that acceleration. Some work has been done by Syntactica to quantify the productivity gains of an XRX-based approach XRX Value

While there is clearly some overlap between the capabilities in XQuery, XSLT and XProc, there is a clean mapping from requirements on RESTful services to the tools that best address them.  Limiting an XQuery to interactions with the database results in queries that are simple to write and unit test, and effectively encapsulates the logic that creates application model objects (XML structures). The controller portion of our implementation is implemented with XProc pipelines and XSLT stylesheets.  With its mechanism for identifying insertion points, and a powerful language for expressing what should be inserted, XSLT is ideally suited to perform hyperlink insertion.  It is also the de facto standard for doing XML to XML[2] transformations.  The new XProc standard which defines a high level language for wiring together a set of XML-centric steps, provides a means for addressing many of the requirements presented by RESTful services: implementations of compound resources, a multi-part data model and a large set of out of the box XML processing steps.

One of the most significant, positive results of our work has been that with the availability of our XML REST Framework we have enabled developers to more readily understand the important elements of RESTful services.  Rather than simply providing guidance in the form of a reference architecture, providing a set of tools and samples along with that guidance has proven very effective.

Limitations

We have employed our XML REST Framework in several projects within EMC with a good deal of success.  The biggest barrier, however, continues to be a reluctance by developers to embrace XML as the development model.  The XML-based approach that we espouse in this paper, while quite powerful, still suffers from a lack of tooling and, even more importantly, a learning curve issue.  Most developers are quite familiar with Java programming approaches, IDEs, testing frameworks, and so on, and have limited experience with the XQuery and XSLT.  Most have never heard of XProc. While the XProc processing model is powerful and conceptually allows pipelines to be easily expressed, the syntax is unwieldy and will turn most developers completely off.  It would behoove us in the XML community to address these issues to expand the utilization of XML technologies in such development scenarios.

One of the things we have found most useful in this XML-centric approach to RESTful service construction is the use of XProc to operate on compound resources. We noted in a section above that the XProc implementation we are using in our work creates transaction boundaries at the beginning and end of a pipeline. This feature does not exist in all XProc engines, limiting the generality of the approach we describe here.

Future Work

Our work in this area continues.  We are exploring the use of XProc as the core engine in the View portion of a Spring MVC implementation. We are exploring the use of finite state machines (FSM) to model application flows and are building an engine that interprets these FSMs, inserting hyperlinks in the resource representations. We are also investigating how other elements of RESTful service implementations (such as feed paging and eTag eTags support) can be integrated into the framework so as to continue lessening the load on the services developer.

The XML REST Framework is available, along with a sample application, from the EMC Developer Network XMLRESTFW.  The distribution includes all source code.

References

[XRX] McCreary, Dan. XRX: Simple, Elegant, Disruptive. May 2008. http://www.oreillynet.com/xml/blog/2008/05/xrx_a_simple_elegant_disruptiv_1.html.

[XForms] Boyer, John M., XForms 1.1. October 2009. http://www.w3.org/TR/xforms11/.

[XQuery] Boag, Scott, et. al. XQuery 1.0: An XML Query Language (Second Edition). December 2010. http://www.w3.org/TR/xquery/.

[XSLT] Kay, Michael. XSL Transforamtions (XSLT) Version 2.0. January 2007. http://www.w3.org/TR/xslt20/.

[Atom] Nottingham, M. and R. Sayre. The Atom Syndication Format. December 2005. http://tools.ietf.org/html/rfc4287.

[XProc] Walsh, Norman, Alex Milowski and Henry S. Tompson. XProc: An XML Pipeline Language. May 2010. http://www.w3.org/TR/xproc/.

[IHEConnect] IHE Connectathon. http://www.ihe.net/Connectathon/.

[IHEXDS] IHE Cross Enterprise Document Sharing (XDS). http://www.ihe.net/Profiles/index.cfm#IT.

[Spring] Johnson, R., et. al. Spring Framework Reference Documentation 3.0. 2004-2010. http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/.

[Wilde] Wilde, Erik. XML-Centric Application Development. February 2006. http://dret.net/netdret/docs/wilde-tikrep242.pdf.

[IHE] Integrating the Healthcare Enterprise Technical Frameworks. http://www.ihe.net/Technical_Framework/.

[IHE] Fielding, Roy Thomas, Architectural Styles and the Design of Network-based Software Architectures. 2000. http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm.

[Servlex] Servlex: an implementation of the EXPath Webapps frameworkhttp://code.google.com/p/servlex/.

[EXPath Webapp] EXPath Webapp. http://expath.org/wiki/Webapp.

[Spring MVC] Haines, Steven. Mastering Spring MVC. April 2009. http://www.javaworld.com/javaworld/jw-04-2009/jw-04-springmvc.html.

[Apache CXF] Apache CXF: An Open Source Services Framework. http://cxf.apache.org/.

[Jersey] Glassfish: Jersey. http://jersey.java.net/.

[Hypermedia Constraint] Fielding, Roy. REST APIs must be hypertext-driven. Blog post, October 2008. http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven.

[eTags] Hypertext Transfer Protocol -- HTTP/1.1 - ETag Section. June 1999. http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.19.

[XRX Value] Syntactica Solutions for XRX Developers. http://www.syntactica.com/solutions/xrx.xq.

[XMLRESTFW] Davis, Cornelia, XML REST Framework: Spring MVC and XProc. https://community.emc.com/docs/DOC-10494.



[1] Note that while REST is an architectural style that does not require HTTP, in practice most RESTful services are offered over HTTP and we will focus on those here.

[2] Note that JSON resource representations are increasingly popular.  The approach we describe in this paper does not preclude such representations and XSLT is also an excellent choice for XML to JSON transformations.

×

McCreary, Dan. XRX: Simple, Elegant, Disruptive. May 2008. http://www.oreillynet.com/xml/blog/2008/05/xrx_a_simple_elegant_disruptiv_1.html.

×

Boyer, John M., XForms 1.1. October 2009. http://www.w3.org/TR/xforms11/.

×

Boag, Scott, et. al. XQuery 1.0: An XML Query Language (Second Edition). December 2010. http://www.w3.org/TR/xquery/.

×

Kay, Michael. XSL Transforamtions (XSLT) Version 2.0. January 2007. http://www.w3.org/TR/xslt20/.

×

Nottingham, M. and R. Sayre. The Atom Syndication Format. December 2005. http://tools.ietf.org/html/rfc4287.

×

Walsh, Norman, Alex Milowski and Henry S. Tompson. XProc: An XML Pipeline Language. May 2010. http://www.w3.org/TR/xproc/.

×

IHE Cross Enterprise Document Sharing (XDS). http://www.ihe.net/Profiles/index.cfm#IT.

×

Johnson, R., et. al. Spring Framework Reference Documentation 3.0. 2004-2010. http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/.

×

Wilde, Erik. XML-Centric Application Development. February 2006. http://dret.net/netdret/docs/wilde-tikrep242.pdf.

×

Integrating the Healthcare Enterprise Technical Frameworks. http://www.ihe.net/Technical_Framework/.

×

Fielding, Roy Thomas, Architectural Styles and the Design of Network-based Software Architectures. 2000. http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm.

×

Servlex: an implementation of the EXPath Webapps frameworkhttp://code.google.com/p/servlex/.

×

Haines, Steven. Mastering Spring MVC. April 2009. http://www.javaworld.com/javaworld/jw-04-2009/jw-04-springmvc.html.

×

Apache CXF: An Open Source Services Framework. http://cxf.apache.org/.

×

Glassfish: Jersey. http://jersey.java.net/.

×

Fielding, Roy. REST APIs must be hypertext-driven. Blog post, October 2008. http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven.

×

Hypertext Transfer Protocol -- HTTP/1.1 - ETag Section. June 1999. http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.19.

×

Syntactica Solutions for XRX Developers. http://www.syntactica.com/solutions/xrx.xq.

×

Davis, Cornelia, XML REST Framework: Spring MVC and XProc. https://community.emc.com/docs/DOC-10494.

Author's keywords for this paper:
REST; XML; XProc; XQuery; XSLT; XML Database; XRX; HATEOAS