Balisage Paper: Including XSLT stylesheets testing in continuous integration process

XSLT test framework

This is the core of the solution. Writing tests should be easy, intuitive and frictionless. XSpec framework meets these conditions[xspec-01]. BDD approach, tool quality, authors and recent project activities have lead to its selection from among various other projects like Juxy, XSLTUnit or UTF-X for instance.

The general idea is to write XSpec documents to describe stylesheet behaviours. XSpec documents are XML documents which adhere to the XSpec RELAX NG schema. XSpec scenarios are straightforward to write for XSLT developers. Stylesheet templates and functions can be tested against contextual data (XML documents, fragments or mock object) by creating test scenarios. Here is a sample XSpec file.

<?xml version="1.0" encoding="UTF-8"?>

<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec" stylesheet="toHtmlWithIds.xsl">

    <x:scenario label="When processing a list of books">

        <!-- apply template rules to this element -->
        <x:context href="xsltestengine-data/books-a.xml/>

        <!-- check the result -->
        <x:expect label="There must be 12 button to books" test="count(//button) = 12"/>

    </x:scenario>

</x:description>

            

The precise way to write XSpec document is described in the official documentation [xspec-02].

Systematically writing XSpec test scenarios for each XSLT stylesheets produced is a first step towards improving bug detection, guaranteeing compliance to specifications, business and technical requirements, safer refactoring, etc. But XSpec documents alone are not sufficient. Running tests is still under the sole responsibility and good will of the developer. Tests should be run on a regular basis to allow early error detection. Such systems exist and are widely used for other programming languages. How could XSpec files be integrated in those existing solutions?

Java unit testing wrapper

In order to be able to benefit from other languages testing tools availability, XSpec test executions should be wrapped up in one of these languages. This is one of the goals of the Jxsl project [jxsl-01]: to offer Java wrapper objects for XSpec test configuration and executions.

Jxsl provides method to execute XSpec tests directly from Java code, one at a time (XspecTestScenarioRunner) or in batch mode (XspecTestSuiteRunner). This is useful for embedding test executions in custom Java code. But Jxsl also contains wrapper for the JUnit unit testing framework (TestNG support is coming). That means that an existing Java project can add XSpec file testing to its standard JUnit test suites.

To enable XSLT unit testing with XSpec in an existing Java project, it is as simple as creating a new class in the project test package that extends XspecScenarioJUnitTest and to provide a Spring bean configuration file called xspec-context.xml. Here is a complete and functional implementation.

package com.mycompany.test.xspec;

import com.servicelibre.jxsl.scenario.test.xspec.XspecScenarioJUnitTest;
import java.io.File;

public class XspecUnitTesting extends XspecScenarioJUnitTest {

    public XspecUnitTesting(File xspecFile) {
       super(xspecFile);
    }
}

Jxsl offers a Maven archetype to get started quickly. This archetype is also very useful for non Java developers. Java 1.5+ JRE and Maven 2.9+ are required. The archetype can be generated with the following command:

mvn archetype:generate -DarchetypeGroupId=com.servicelibre  \
                       -DarchetypeArtifactId=xspec-test \ 
                       -DarchetypeVersion=0.1.5 \ 
                       -DarchetypeCatalog=http://jxsl.googlecode.com/svn/trunk/archetypes                
            

The only things to configure to get started is the XSpec file locations . This is achieved by editing the src/test/resources/xspec-context.xml file. Tests can now be run with the following command:

mvn test

XSpec test executions have now be integrated into some Java unit testing code in order to benefit from all the tools available for triggering and monitoring standard Java unit tests. The Maven project build from the archetype could also be used to integrate XSpec testing in XML editor like Oxygen XML Editor (through External tools configuration).

Build tool and continuous integration server

XSpec tests wrapped up in Java unit tests can now be triggered as part of a standard Java project build process. Build tool like Maven does this automatically by convention. Ant can be configured to run JUnit tests as follows :

<?xml version="1.0" encoding="UTF-8"?>
<project name="jxsl" default="test" basedir=".">

	<target name="init">
		<property name="resources" location="${basedir}/../resources"/>
	</target>

	<target name="test" depends="init">
		<echo>basedir=${basedir}</echo>
		<junit dir="${basedir}" fork="true" haltonerror="true" >
			<test name="com.servicelibre.jxsl.scenario.test.xspec.XspecScenarioJUnitTest"/>

			<classpath>
				<pathelement path="${resources}/jxsl/jxsl-with-dependencies.jar"/>
				<pathelement path="${user.home}/.m2/repository/net/sf/saxon/saxon/9.3.0.2j/saxon-9.3.0.2j.jar"/>
				<path path="${basedir}"/> <!-- for xspec-context.xml Spring bean configuration file -->
			</classpath>

			<formatter type="brief" usefile="false"/>
		</junit>
	</target>

</project>

            

Today, not having a CI server in place when doing software development is almost like not using a version control system: a risky bet. CI server ensures, at least, that code are regularly built and tested successfully in a clean environment (understand « not on the developer computer ») . There are several popular CI servers: Hudson/Jenkins, Continuum, Bamboo, TeamCity, etc.

XSpec tests should be run regularly by a CI server.

In order to catch errors even more early, it is suggested to trigger test executions on commit in the version control system. This is usually done via server-side hook scripts. Resources on hook scripts for common version control systems:

Here is a sample Subversion hook script that triggers a project rebuild by Hudson

#!/bin/sh

# POST-COMMIT HOOK

USER=hudson
PASSWORD=*****
HUDSON_SERVER=hostname:8081
HUDSON_COMMAND=polling

HTTP_PREFIX=http://${USER}:${PASSWORD}@${HUDSON_SERVER}

wget -b ${HTTP_PREFIX}/job/hudson_project_name/${HUDSON_COMMAND} > /dev/null