Introduction

Objectives and Requirements

XTemp is an acronym for 'XML Testing and Event-driven Monitoring of [business] Processes' also suggesting a strong 'temporal' aspect. The monitoring and testing of business processes and transactions serves anyone or a combination of these objectives:

  • Compliance with agreements. These agreements may be business agreements such as SLAs, or regulatory compliance rules. They may also be infrastructure configuration agreements (e.g. ebXML CPA, WS-Policy). This category of testing includes SLA and SLO monitoring (service level agreements, objectives).

  • Business operations intelligence and improvement. This involves monitoring of operations, either internal business processes or external message exchanges (eBusiness). The primary intent is to help improve or predict operations by enhancing visibility, generating reports, alarms and various metrics of dynamic business activities. This includes analysis of logs of processes and of business transactions.

  • Compliance with specifications. This could be seens as a particular form of agreement compliance, involving the use of IT standards. Such specifications may be of a business transaction, business process definition, documents exchanged, or of infrastructure behavior (e.g. messaging protocol). This includes verifying that the execution trace of a business process conforms to the process definitions (e.g. XPDL, BPMN, UML). In the B2B area, the actual message exchanges must obey specified choreographies (e.g. RosettaNet/GS1, OAGI) whether defined in a machine-processable manner (e.g. using ebXML ebBP) or not. Compliance to profiles on how to combine existing specifications - as in WS-I (http://www.ws-i.org/) - also falls in this category.

In all these cases, it is assumed a business process generates a trace of its behavior - either as part of its business communicating function (messages) or as a log of its activities. It is also assumed that this trace can be modeled as events of various kinds, that are represented in - or may be mapped into - an XML format. This event format may be either a proprietary vocabulary or may follow a standard such as XES (http://www.xes-standard.org/), SAF (Symproms Automation Framework from OASIS), CEE (http://cee.mitre.org/), DMTF/CIM or XDAS. For example, a trace of exchanges between a Web client application and a Web service can be formatted in XML as done in [WSI10], including HTTP headers.

More detailed XTemp requirements are:

  • The same test scripts should be executable either in a “log analysis” mode where all events to be processed are already logged, or in a real-time monitoring mode where test script processing is driven by events as they occur, while having the ability to correlate these to past events. A mix of both situations must be handled as well. XTemp assumes events to be stored in an “event board” that allows for past events selection and correlation.

  • The script language must support a notion of concurrent execution proper to the monitoring and analysis of interleaved “threads” (or sequential traces) of events, e.g. mimicking the concurrency of business processes or of B2B exchanges.

  • The language must not be tied to a particular event format and content. Events in a log can be alarms, eBusiness policies, system configuration files, or B2B messages containing business documents. Any XML representation is acceptable provided that these events are timestamped in the same way, which may be done by using a thin common wrapper for all events.

  • XTemp must be an XML vocabulary itself, so that XTemp scripts can be generated easily from templates representing monitoring or testing design pattern – e.g. using user input provided via forms and wizards. Another reason to design XTemp as an XML vocabulary is to support easy generation of XML test reports, by an intuitive mix of in-line externally-namespaced XML fragments with XTemp-computed values, in the way XSLT allows.

  • Finally a significant subset of XTemp (“core”) must be entirely implementable in XSLT2.0, and usable at least for non real-time executions (log analysis use cases). This makes it easy to develop and debug scripts over sample logs, before deploying for real-time monitoring. This is a major usability requirement, as debugging concurrent programs or workflow programs is not an easy task.

General Positioning of XTemp

XTemp is workflow-capable in the sense that it provides a minimal set of operators that enable most workflow patterns, of which a good account has been provided in [RHAM06] (see also http://www.workflowpatterns.com/). Compared with workflow languages that are also XML-based and support a notion of event (BPMN / XPDL, WS-BPEL), XTemp has more of an analytical focus. While BPMN and XPDL have restricted the use of events to include only those types of events that will affect the sequence or timing of activities of a process, XTemp is accessing past events for correlation and analysis. Consequently its concept of Event Board is more than an event Source, but also a queryable repository of events that can be used off-line or live.

Because its focus is the analysis of processes and of transactions via the events they generate, the general execution model of XTemp is time-aware and event-centric. Time awareness is based on a notion of virtual present date/time (VP-time) associated with a script execution, and decoupled from the actual present time (AP-time). The VP-time which can be set as part of the execution context - i.e. independently from the script logic - allows for the same script to execute either in a live mode (consuming events as they occur) or deferred mode (i.e. for static analysis of past events) or yet in a way that combines consumption of both real-time and logged events. As explained in the execution model overview, concurrency in XTemp is based on the notion of VP-time.

XTemp may be viewed as “yet another concurrent programming language”, but it is the first one that leverages XML in three ways: (a) for input (event) processing, (b) for in-line report generation, and ( c) for its own syntax which allows for generating scripts from script templates using transforms. Conversely, XTemp could be defined as “yet another XML processing language” close to XSLT, but one with built-in support for event processing support and concurrency capability. How XTemp relates to XSLT is discussed in more details in the Related Works section.

Some Aspects of the Execution Model

Event Processing

Because events often reflect on concurrent, distributed processes, XTemp supports a form of concurrency. However it is not intended for general concurrent programming and lacks many of the features found in such languages (e.g. advanced execution control as in coroutines, resource access control as semaphores or mutex). There are however two features that distinguish XTemp and highlight its analytic focus:

  • (1) Event consumption is mediated via one or more event board(s) that provide more than conventional pub-sub functions. The event board serves both as event source and/or event sink, but more importantly supports access to past events (e.g. event ordering and access based on date, indexed access based on a provided 'key'). It is a queryiable event history, yet querying is not distinguished from live consumption as both operations are performed using the same primitive ('catch'). Thus an event may be consumed (accessed) more than once by a script. In other words there is no difference between event querying and event listening.

  • (2) Time handling in XTemp revolves around the notion of 'virtual present time' (VP-time), a value that is contextual to the execution of every statement, yet a value that can be manipulated and modified intentionally before and during execution, allowing some distance with the actual present (or execution time). The VP-time is conditioning event consumption and governs the notion of concurrency in XTemp: two script executions are 'concurrent' if their VP-times overlap, meaning they can catch the same events - or at least events occurring over the same period, as consumed events may be 'masked' by a script so that they will not be consumed twice.

The notion of VP-time combined with the access allowed by event boards makes it possible to synchronize script execution with events regardless of the actual time of their occurrence. This in turn allows for executing the same script 'live' or 'deferred' (i.e. for static analysis of past events) by simply modifying the initial VP-time value.

Time Control

An XTemp script is composed of several script units called 'scriplets' that can invoke each other, with functional semantics with regard to the output they may or may not produce - XML fragments called 'X-effect' - the aggregation of which represents the complete script output called 'report'. Scriplet execution follows a (virtual) thread model: scriplets may be concurrent to each other if their VP-times overlap, but more important is how they influence each other's VP-times. By default, a scriplet invocation is 'synchronized' as it is blocking the invoking scriplet ('blocking' invocation) until completion of the invoked scriplet. This blocking means that the next statement after the invocation is set to the ending VP-time of the invoked scriplet - in which case both scriplets can still execute in a single thread implementation-wise as they cannot consume (or produce) same or co-occurring events. A 'non-blocking' scriplet invocation will decouple the VP-times and allow both scriplets to access same or co-occurring events, with the effect of a parallel execution more aligned with a conventional and stronger meaning of concurrency.

Even for static analysis over a log of past events, it may be convenient and more intuitive to script for concurrent executions. As illustrated later, a scriplet specialized for capturing a particular sequence of events independently from the main execution thread (i.e. represented by the invoking scriplet), should be scripted as fully concurrent to the invoking scriplet, whether the execution is live or not.

XTemp concurrency features are minimal but sufficient to track (either live or deferred) most workflow execution patterns. Non-blocking scriplet invocation allows for matching conventional workflow forking (or 'and-split'). The merging (or 'and-join') of one or more concurrent workflow paths is supported in XTemp by using a single primitive ('wait') that allows for synchronizing with all kinds of clock events - e.g. a date or a scriplet completion.

In addition to the scriplet invocation mode (blocking vs. non-blocking), the VP-time of a scriplet execution may be set at its invocation: a VP-time setting anterior to the VP-time of the invoking scriplet is called a backward invocation, while a VP-time setting future to the VP-time of the invoking scriplet is called a forward invocation. The invocation mode combined with VP-time setting allows for convenient expression of the logic behind various patterns of events:

  • A backward, non-blocking scriplet invocation will allow for querying past sequences of events, relative to a (current) event, without affecting the timing of the invoking scriplet.

  • A backward, blocking scriplet invocation will allow for querying past sequences of events, relative to a (current) event, and to synchronize with the end of such a sequence in case it goes past the invoking VP-time.

  • A forward, non-blocking scriplet invocation will allow for capturing future sequences of events without affecting the timing - i.e. concerning access to 'current' events - of the invoking scriplet (in which case of course the 'current' events will now be 'past', once future events have occurred). This, when assigning the X-effect of scriplets to variables, supports the resolution of 'futures' as defined in concurrent languages.

  • A forward, blocking scriplet invocation will allow for capturing future sequences of events, relative to a (current) event, and to synchronize the invoking scriplet with the end of such a sequence for subsequent operations.

  • A current, non-blocking scriplet invocation is equivalent to a conventional spawning of a thread in concurrent languages, or to a workflow forking ('and-split') of concurrent (generally different) paths of execution.

  • A current, blocking scriplet invocation is equivalent to a conventional function invocation inside the same path (or thread) of execution, where the invoking scriplet will synchronize with the ending VP-time of the invoked scriplet.

In terms of concurrency semantics, XTemp cannot be considered an Actor language: scriplets are not aware of each other and do not communicate explicitly with each other. The model is rather of CSP (Communicating Sequential Processes), where scriplets behave as processes, consuming events and communicating via events only - except when a scriplet is starting another scriplet, in which case parameters may be passed. When scriplets do not communicate with each other e.g. by posting events, a single-threaded implementation of XTemp (such as most XSLT engines provide) is sufficient. When intended for static analysis of complete event boards (e.g. event logs), the subset of XTemp language identifed as 'core' - which excludes posting of new events - can be implemented in XSLT using conventional XSLT engines.

An Illustrated Overview of XTemp

This article does not pretend to provide an exhaustive view of XTemp, but will emphasize some XML-related aspects, and how XML and related tools are leveraged. A more complete definition of XTemp canbe found in [XTEMP10] along with an XSLT translator for "core" XTemp and a set of use cases on the Web page from the OASIS TaMIE TC (http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tamie).

General Execution Paradigm

An XTemp script is consuming events – either in real time or from a log. XTemp is not entirely event-driven though: a script makes use of conventional algorithmic statements (branching, loops, exits, serial or concurrent sub-script invocations) in order to control the flow of execution of testing/monitoring units called “scriplets”. These scriplets may (or may not) synchronize with events. This combination of workflow-style control (allowing concurrency) and event synchronization is most appropriate for process monitoring and the analysis of related event patterns. Scriplets provide a parameterized, reusable unit of execution and are assigned a date/time (their “virtual present time”) when executing. The script - or script package - is the largest unit of execution. The binding between the script and an actual event source (event board), is defined inside an “execution context” element separate from the script logic.

The XML practitioner take-away: XTemp has constructs close to XSLT constructs, however its execution is not driven by XML patterns or templates. Its controls are workflow-like with a built-in concept of time and event, as it was designed to be familiar to business process designers and programmers. Yet, the “core” conformance level of XTemp is implementable in XSLT2.0 or above, as this was a key requirement. It is also designed for the XML practitioner: XPath has a central role for expression logic and event selection, and inline XML fragments of foreign namespaces allow for an intuitive generation of XML reports – in a way similar as XSLT does.

EXAMPLE 1: Periodic tallying of Purchase Order messages

The abbreviated monitoring script below illustrates some features of XTemp.

  • The script is intended to produce a report showing how many Purchase Order events occurred every period of time ($period ) from a date/time ($initdate).

  • The execution context <execution-context> at the beginning associates an event board - here a simple log file - with a symbolic name (POlog), and declares some global variables.

  • The scriplet named “mainloop” is iterating over each period of time, incrementing date ($currentdate) and counter ($mycounter) mutable variables.

  • A monitoring report is initiated inline ( myreport:monitoringreport) and dynamically filled in a loop.

<script-package 
...
		xmlns="http://docs.oasis-open.org/tamie/xtemp/200909"
		xmlns:myreport="http://www.abc.com/TBD/report"
		xmlns:xs="http://www.w3.org/2001/XMLSchema"
		name="CUC-2">

  <execution-context>
    <start-with scriplet="mymainloop"/> <!-- starting point -->
    <!-- global variables -->
    <var name="maxiter" type="integer">5</var> 
    <var name="initdate" type="dateTime">2009-10-08T11:12:00</var> 
    <var name="period" type="duration" expr="xs:dayTimeDuration('P1DT1H')"/> <!-- 1 day + 1 hour -->
    <event-board name="POlog" event-store="myeventlog" mode="source"/>
  </execution-context>

  <!-- ======= scriplet: mymainloop ======= -->
 
  <scriplet name="mymainloop">
    <myreport:monitoringreport period="{$period}"> 

      <loop>
      …
	<until expr="$mycounter ge $maxiter"/>
	<var name="currentevts">
	  <start scriplet="doperiod" vptsync="true" vptset="$currentdate">
            <with-param name="startperiod" expr="$currentdate"/>
            <with-param name="lengthperiod" expr="$period"/>
	  </start>
	</var>
      …
	<myreport:currentevts period="{$mycounter}" startdate="{$currentdate}">
	  <eval expr="$currentevts"/>
        </myreport:currentevts>
	<lvar-next name="mycounter" expr="$mycounter + 1 "/>
	<lvar-next name="currentdate" expr="($currentdate + $period)"/>
	...
      </loop>
    </myreport:monitoringreport>
  </scriplet> <!-- mymainloop -->

  <!-- ======= scriplet: doperiod ======= -->
  <!-- The scriplet "doperiod" gathers various metrics for the period: 
average response time of Purchase Orders, total amount ordered, etc.  -->

</script-package>
	

  • The scriplet “doperiod” below is catching all events related to a particular time period, which is bounded with a date/time ending specified in catch/@vptend. At each iteration, the current VP-time is implicitly incremented to the time the last event was caught. The following partial excerpt only illustrates event catching and makes abstraction of the computation of various metrics, illustrated in subsequent examples.

  <!-- ======= scriplet: doperiod ======= -->

  <scriplet name="doperiod">
    <param name="startperiod" type="dateTime"/>
    <param name="lengthperiod" type="duration"/>

    <var name="endperiod" type="dateTime">xsd:dateTime($startperiod + $lengthperiod)</var>
    <loop>
      ...
	<var name="po-event">
          <catch eb=”POlog” vptend=”{$endperiod}”>
            <match>
	      <condition>xtemp:content//myapp:PurchaseOrder</condition>
            </match>
          </catch>
	</var>
	<until expr="not($po-event/xtemp:event)"/>

      <lvar-next name="count1" expr="$count1 + 1"/>
      ...
    </loop>
  </scriplet> <!-- doperiod -->
	

An example of generated report from running this script over a sample event board is:

<myreport:monitoringreport       
  xmlns:xtemp="http://docs.oasis-open.org/tamie/xtemp/200909"
  xmlns:myreport="http://www.abc.com/TBD/report"
  ...
  period="P1DT1H">
   <myreport:currentevts period="0" startdate="2009-10-08T11:12:00" >2</myreport:currentevts>
   <myreport:currentevts period="1" startdate="2009-10-09T12:12:00" >0</myreport:currentevts>
   <myreport:currentevts period="2" startdate="2009-10-10T13:12:00" >0</myreport:currentevts>
   <myreport:currentevts period="3" startdate="2009-10-11T14:12:00" >1</myreport:currentevts>
   <myreport:currentevts period="4" startdate="2009-10-12T15:12:00" >2</myreport:currentevts>
   <myreport:finaldate>2009-10-13T16:12:00</myreport:finaldate>
   <myreport:elapsedtime>P5DT5H</myreport:elapsedtime>
</myreport:monitoringreport>
	

Some Convenient Algorithmic Constructs

Three features are particularly helpful for testing and monitoring scripts:

  • Conditional branching: Two separate conditional statements are provided as in XSLT: simple condition and switch. For the latter, XTemp uses intuitive algorithmic keywords (if... else-if... else) found in WS-BPEL, but opts for the cleaner switch packaging found in XSLT that puts all its conditions at the same level, the definition of a switch statement ('decide') distinct from a simple condition being the price to pay for this.

  • Loops with mutable variables: XTemp supports a general loop construct covering the two main iteration styles: for-each (element in a collection) and until (condition) or both. An important feature for monitoring functions, is the introduction of mutable variables – even if their mutability is strongly constrained. Such mutable variables are called “loop variables” and are declared differently from other immutable variables. They can only be modified once in the body of a loop, and only just before the next iteration (i.e. their new value is effective at the next iteration only).

  • Exit: Exiting from a scriplet while gracefully closing a report element, is done with the <exit> statement. This allows for simplified cascading test conditions as a sequence of distinct conditions. Depending whether the scriplet was started with the attribute @bubble-exit=”true” or not, an exit may propagate up to the invoking scriplet.

Using both regular variables and mutable loop variables, is illustrated in the loop example below:

<loop>
  <on-start>  <!-- some mutable variables -->
    <lvar name="myitems" expr="$PO/lineItems/item"/>
    <lvar name="item" type="int">1</lvar>
    <lvar name="pototal" type="int">0</lvar>
  </on-start>
  <until expr="$item gt fn:count($myitems)"/>
  <var name="myItemAmount">
    <eval expr="$myitems[$item]/unitprice * $myitems[$item]/quantity"/>
  </var>
  <var name="itemRebate">
    <decide>
      <if expr="$myItemAmount gt 1000">
        <eval expr="$myItemAmount * 0.10"/>
      </if>
      <else>
        <eval expr="0"/>
      </else>
    </decide>
  </var>
  <lvar-next name="pototal" expr="$pototal + ($myItemAmount -
$itemRebate) "/>
  <lvar-next name="item" expr="$item + 1"/>
  <on-final>
    <abc:mypototal><eval expr="$pototal"/></abc:mypototal>
  </on-final>
</loop>
	

The XML practitioner take-away: Variables can be made mutable only where this capability is most needed: in loops (loop variables). They are distinguished from regular (immutable) variables by a different declaration syntax ( <lvar> vs. <var>), and are otherwise referenced similarly. It was acceptable to severely restrict the scope of mutable variables (local to a loop) and their mutation (only at the end of each iteration using <lvar-next>), while these restrictions made an XSLT implementation possible. An initialization loop section <on-start> inside the loop statement was then necessary to initiate their state, while a closing section <on-final> inside the loop was needed to extract the final values of mutable variables e.g. to report it as part of the X-effect.

Event Processing

A single statement <catch> is controlling event input, either for catching a single event or a pattern of events. An event pattern is a set of events correlated based on content, and often following some time-ordering constraint or time window constraint. An event pattern may be a well-correlated conversation such as a multi-message eBusiness transaction, or a more loosely defined co-occurrence of events e.g. close to each other in time but without well-established causality or common cause.

In XTemp an event pattern may be expressed in two ways: (a) by using one catch statement for each event, and by controlling the sequence(s) of <catch> statements using other XTemp controls (scriplets, conditionals, loops), or (b) by using a combination of filters (<match> statements) inside a single catch. The former (a) is more appropriate when the pattern occurs over a significant time window (such as for some long-lasting business transactions) or when the pattern is itself defined dynamically depending on some event content, requiring some processing as it unfolds (e.g. when a business transaction may unfold differently based on its initial event content). The latter (b) allows for a compact expression of deterministic patterns but more importantly allows for delegating the <catch> execution to specialized event processors that can optimize such processing (equivalent of query optimization in databases), e.g. by delegating to a complex event processing (CEP) module.

EXAMPLE 2: Catching some event pattern

The following catch statement selects a sequence of three events representing an exchange of three messages:

  • (E1) a Purchase Order request

  • (E2) a PO response (acceptance or rejection)

  • (E3) a generic receipt to the PO response

These three events are ordered in time: E3 after E2 after E1. The correlation between these events is based on different content:

  • Between E1 and E2: the same value for the PO reference ("order-ref" element in message headers)

  • Between E2 and E3: the MessageID value present in E2, must equal the RefMessageID in E3.

<catch>
  <match event="E1">
    <condition>content/soap/Header/msgData/action = 'PORequest'</condition>
  </match>
  <match event="E2" after="E1">
    <condition>(content/soap/Header/msgData/action = 'POAccept' | 'POReject') and 
content/soap/Header/msgData/property[@name = 'order_ref'] = 
$E1/content/soap/Header/msgData/property[@name = 'order_ref']</condition>
  </match>
  <match event="E3" after="E2">
    <condition>content/soap/Header/msgData/action = 'Receipt' and 
content/soap/Header/msgData/property/RefMessageID = $E2/content/soap/Header/msgData/messageID
    </condition>
  </match>
</catch>
	

<catch> plays two roles: (1) synchronization operation for real-time execution, and (2) event board querying for past events analysis (deferred processing). In both cases, <catch> is operating using an event board as event source: the only difference between live and deferred is the VP-time of execution compared with the timestamp of events in board. “Catching” an event pattern 'live' often involves a combination of real-time event consumption with querying past events.

In contrast, WS-BPEL event catching <pick> is designed to capture and process single events at a time, as its use of events is more as a process driver, than for analytical and diagnosis purpose.

The XML practitioner take-away:Xpath2.0 proved sufficient as expression language for event querying, selection and correlation. Because event pattern expressed in <catch> are usually well time-constrained and/or strongly correlated, large logs of (time-ordered) events are manageable without penalty using conventional XPath and XSLT processors. Any XML-based event format - e.g. XES, SAF (Symproms Automation Framework from OASIS), CIM Events/Indications from DMTF - can be accommodated, provided that these events are timestamped so they can be ordered by the event board. In the baseline “core” implementation of XTemp with XSLT2.0, the event log is a plain file of time-ordered event records. Logs of events that have intermediate structures – e.g. events are grouped in “traces” under a log in XES – can also be handled without transformation. The event content is quite open, allowing for parts using binary encoding or attachments of various formats (e.g. MIME or SOAP attachments). However only the XML parts are queryable.

Concurrency

The notion of concurrency is as necessary to the monitoring and validation of business transactions or processes as it is to the actual programming of these processes. The concurrency of threads or activities in the latter translates into the concurrency of related event sequences in the former. A set of typical workflow patterns, several of them involving concurrency can be found in [RHAM06]. workflow language like WS-BPEL [WSBPEL07] uses an advanced set of constructs to manage concurrency and synchronization (<flow>, <links>, <joinCondition>, <sources> and <targets>).

However, the monitoring of concurrent sequences of activities does not require the same sophistication in concurrency control as the actual execution of these activities, provided the events associated with these sequences have a known correlation logic, which is generally the case in choreographed eBusiness transactions (e.g. messages relating to the same procurement action will report the same PO reference) and business process instances (e.g. a process instances initiated by an insurance claim, will generally report the claim number or originator ID across all its activities and events). Consequently XTemp has a minimal set of constructs for concurrency and synchronization: (1) the concurrent – or “non-blocking” - invocation of a scriplet, (<sart scriplet=”abc” vptsync=”false”>) for tracking overlapping sequences of related events, and (2) a synchronization operator <wait>, for joining a group of concurrent scriplets if necessary. Most other forms of synchronization simply rely on event catching <catch>.

The VP-time assigned to a scriplet invocation determines which events are accessible to the scriplet. This makes it possible to execute the same script either live or deferred over a log of events with same result, just by setting its initial VP-time to the same date/time relative to these events. Two scriplets are concurrent if their VP-time windows overlap. Concurrency is “virtual” in XTemp to the extent that the notion of “present time” (of execution) is manipulated as a data item. Even when the objective is only the analysis of past logs, concurrency actually simplifies the test scripting by allowing event processing to follow the logic of a particular sequence regardless of its context (e.g. the processing of interleaved logs of PO transactions can be done by concurrent scriplets tracking each transaction).

The XML practitioner take-away:“Core” XTemp, including virtual concurrency, is XSLT2.0-implementable for non-live executions over logs of events: this makes it very easy to verify the correctness of concurrent script over various events scenarios, before executing live. When a script has been validated over a log of events, the only modification needed before executing it live is a change in the @vptset value of its <start-with> statement in he execution context. The implementation of virtual concurrency is greatly simplified when the concurrent threads (here scriplets) do not communicate with each other. This is the case for the “core” conformance level of XTemp (no event posting). In such cases, an XSLT implementation is possible.

EXAMPLE 3: Monitoring interleaved PO transactions

The following script is monitoring PO transactions. Each PO transaction is comprised of a PO request, and a returned PO response. PO transactions are generally intertwined, as many PO requests may be sent before the first response is obtained. The monitoring of each individual PO transaction is handled by an invocation of the “get-PO-response” scriplet. These invocations are non-blocking (@vptsync="false") meaning they execute in (virtual) concurrency with the main script and with each other. The concurrency is here virtual in the sense that these invocations may actually be executed serially, provided their concurrency semantics is preserved by setting the VP-time of each invocation to the time the transaction is started (PO request occurrence). Virtual concurrency makes scripting of such test cases quite intuitive, when distinct sequences of events need be analyzed that can be intertwined in their actual occurrence.

In the script below:

  • The scriplet “main” is looping over each Purchase Order catch. For each PO it invokes concurrently the “get-PO-response” scriplet, for tracking this PO transaction. The scriplet initiates a top report element <myreport:results>

  • The “get-PO-response” scriplet is waiting no more than 10mn (@tryfor=”PT600S”) for a PO response, and generates an error report if no response is obtained in this delay.

  • The outcome of the “get-PO-response” scriplet is captured by the enclosing (non-mutable) variable $poresponse. This outcome is an XML fragment reflecting on the status of the PO transaction, and containing metrics (elapsed time).

  • Although the assignment of the $poresponse variable in the main loop has (virtually) “waited” for the completion of the current PO transaction, the next PO to be caught in the loop may occur before the end of the previous PO transaction. This is made possible by the concurrent invocation of the “get-PO-response” scriplet, i.e. not blocking the course of the main loop. Yet, an interesting twist is that the actual execution of the main loop needs to wait the end of get-PO-response in order to consolidate its results in summary values (e.g. for a final report on average response time for all Pos). But “virtual concurrency” is still achievable here as the scriplets “main” and “get-PO-response” have different VP-time values.

These three events are ordered in time: E3 after E2 after E1. The correlation between these events is based on different content:

<script-package xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" schema-version-id="0.4"
                ...
		xmlns="http://docs.oasis-open.org/tamie/xtemp/200909"
		xmlns:xs="http://www.w3.org/2001/XMLSchema"
		xmlns:myreport="http://www.abc.com/TBD/report"
		xmlns:myapp="http://www.abc.com/TBD/myapp"
		>
<scriplet name="main">
    <myreport:results>
      <loop>
        <on-start>
          <lvar name="count1" type="integer">0</lvar>
          <lvar name="successcnt" type="integer">0</lvar>
          <lvar name="elapsedtotal" type="dayTimeDuration">PT0H</lvar>
        </on-start>

        <var name="po-event">
          <catch>
            <match><condition>xtemp:content/myapp:PurchaseOrder</condition></match>
          </catch>
        </var>

        <until expr="not($po-event/xtemp:event)"/>

        <var name="poresponse">
          <start scriplet="get-PO-response" vptsync="false">
            <with-param name="ref" expr="$po-event/xtemp:event/xtemp:content/myapp:PurchaseOrder/myapp:poref"/>
            <with-param name="time0" expr="$po-event/xtemp:event/@timestamp"/>
          </start>
        </var>
	
        <eval expr="$poresponse/myreport:error"/>
        <eval expr="$poresponse/myreport:messg"/>

        <lvar-next name="count1" expr="$count1 + 1 "/>
        <lvar-next name="elapsedtotal" expr="if ($poresponse//myreport:elapsed ) then 
          $elapsedtotal + xs:dayTimeDuration($poresponse//myreport:elapsed ) else $elapsedtotal "/>
        <lvar-next name="successcnt" expr="if ($poresponse//myreport:elapsed ) then 
          $successcnt + 1 else $successcnt "/>
        <on-final>
          <myreport:summary msgtype="{concat('TOTAL PO selected: ', $count1)}"/>
          <myreport:summary msgtype="{concat('TOTAL successful transactions: ', $successcnt)}"/>
          <myreport:summary msgtype="{concat('TOTAL successful elapsed time: ', $elapsedtotal)}"/>
	  <if expr="$successcnt gt 0">
	    <myreport:summary>AVERAGE RESPONSE TIME: 
              <eval expr="minutes-from-duration($elapsedtotal) div $successcnt"/> MINUTES, 
              <eval expr="seconds-from-duration($elapsedtotal) div $successcnt"/> SECONDS.
            </myreport:summary>
	  </if>
        </on-final>

      </loop>
    </myreport:results>
  </scriplet>

  <scriplet name="get-PO-response">
    <param name="ref" type="string"/>
    <param name="time0" type="dateTime"/>

    <var name="confirm-event">
      <catch tryfor="PT600S">
        <match>
          <condition>xtemp:content/myapp:confirm[myapp:poref = $ref]</condition>
        </match>
      </catch>
    </var>
    <if expr="not($confirm-event/xtemp:event)">
      <myreport:error>No confirm event for <eval expr="$ref"/> 
        received within 10 minutes of the PO</myreport:error>
      <exit/>
    </if>

    <var name="resp-time" type="dateTime" expr="$confirm-event/xtemp:event/@timestamp"/>
    <myreport:messg>Confirmation received for PO <eval expr="$ref"/>
      <eval expr="concat('-SUCCESS: message:', $ref, 
		  ' Response received within:', 
		  minutes-from-duration($resp-time - $time0), ' MIN ',
		  seconds-from-duration($resp-time - $time0), ' SEC'
		  )"/>
      <myreport:elapsed><eval expr="$resp-time - $time0"/></myreport:elapsed>
    </myreport:messg>
  </scriplet>
</script-package>
	

The XML monitoring report produced by this script over a sample event log is:

<myreport:results xmlns:translator="translator"
           xmlns:xtemp="http://docs.oasis-open.org/tamie/xtemp/200909"
             xmlns:myreport="http://www.abc.com/TBD/report"
          ...
             xmlns="http://docs.oasis-open.org/tamie/xtemp/200909">
   <myreport:error>No confirm event for Name1 received within 10 minutes of the PO
     </myreport:error>
   <myreport:messg>Confirmation received for PO Name2 -SUCCESS: message:Name2 
     Response received within:5 MIN 40 SEC<myreport:elapsed>PT5M40S</myreport:elapsed>
   </myreport:messg>
   <myreport:error>No confirm event for Name3 received within 10 minutes of the PO</myreport:error>
   <myreport:messg>Confirmation received for PO Name4 -SUCCESS: message:Name4 
     Response received within:6 MIN 0 SEC<myreport:elapsed>PT6M</myreport:elapsed>
   </myreport:messg>

   <myreport:summary msgtype="TOTAL PO selected: 4"/>
   <myreport:summary msgtype="TOTAL successful transactions: 2"/>
   <myreport:summary msgtype="TOTAL successful elapsed time: PT11M40S"/>

   <myreport:summary>AVERAGE RESPONSE TIME: 5.5 MINUTES, 20 SECONDS.</myreport:summary>

</myreport:results>
	

Generating Test Reports

The primary output of a script execution is called an X-effect (for "XML side-effect"). The X-effect is a text output, which is generally well-formed serialized XML but may also just be plain text. The notion of X-effect is a generalization of the output produced by XML script languages such as XSLT, that can be a combination of in-line XML fragments with external namespaces and of computed values or fragments.

The X-effect of a script can be recursively defined as follows:

  • The X-effect of a scriplet execution is the concatenation of the X-effects of its child elements.

  • The X-effect of an XML fragment that does not contain any embedded XTemp statement (i.e. not using the xtemp namespace), is itself.

  • The X-effect of an XML fragment that contains some embedded XTemp statement, is the same fragment after substituting the XTemp statement with its X-effect.

  • The X-effect of an xtemp:eval statement, is the result of the evaluation of the embedded expression.

  • The X-effect of an XTemp statement other than xtemp:eval ( e.g. <if>, <loop>, <start>... ) is evaluated as specified in the definition of each statement.

  • The X-effect of a loop is the concatenation of the X-effect of each iteration (if any), with the X-effect of the <final> statement (if any).

  • The X-effect of a variable declaration is nil, as the X-effect of the contained statements is simply assigned to the variable as its value.

Two XTemp features could be seen as challenging for the production of X-effects, but ended up with intuitive X-effect semantics:

Exits: The execution of an <xtemp:exit> statements will simply cancel the production of X-effects of its following siblings (as these are not executed), meaning the X-effect of the embedding statements is preserved. The rule also applies to the following siblings of any conditional statement that executes <xtemp:exit> at any level inside.

Concurrent executions:The rule for X-effect generation of a scriplet invoking another scriplet, is exactly the same regardless whether this invocation was blocking or non-blocking (i.e. serial or concurrent): The combined X-effect of scriplets where one is started by the other, is following these rules: When scriplet S1 "starts" (either blocking or non-blocking ) scriplet S2, the X-effect of S1 is obtained by : (1) capturing the X-effect of S2 at the end of S2 execution, (2) substituting the <starts scriplet="S2" vptsync="..." /> statement in S1 with the X-effect of S2. (3) generating the final X-effect of S1 at the end of its execution, after substitution in (2). This rule applies recursively to all scriplet invocation trees, possibly combining blocking and non-blocking invocations. The rationale for this, is that the tree-like structure of an XML report must generally match the hierarchy and order of scriplets *invocations* regardless of the actual course of their execution which may overlap or be concurrent in various ways. For example, the report produced by the previous script example is naturally listing the results for each PO transaction in the order these PO were initiated – as a sequence of transaction reports, as a human reader would expect -, regardless of how these transactions executions were intertwined, e.g. even if the first PO transaction is also the last one to complete.

The XML practitioner take-away: The production of XML test reports is most intuitive and readable when combining in-line XML fragments of foreign namespaces and computed fragments or values, as done in XSLT. Concurrent executions in XTemp, as well as breaking statements such as <exit> are not an obstacle to the use of these report generation techniques familiar to XSLT developers.

Related Works

XTemp and XSLT

XSLT is computationally sufficient to process events in a 'deferred' mode (static analysis), assuming a representation of event boards as static XML files. However, when considering whether XSLT alone should be used - e.g. by defining appropriate design patterns for event handling and timing - the issue of usability and developer-friendliness arise. The declarative, functional design of XSLT based on scripts driven by template matching does not make it an intuitive language for workflow practitioners or more generally for analysts and developers familiar with the scripting of business processes designed along a state machine model with clearly defined state transitions for underlying data. It is the experience of the authors that it is critical to provide constructs familiar to target users and at the right level of abstraction, for a language to be well accepted. Consequently XTemp defines or imports concepts familiar to business process and workflow developers such as event catching, concurrent paths control and synchronization, timing of state transitions, exiting, mutable variables. The experience of the authors in analyzing and testing event traces is also that a balance must be reached between pattern-driven language with strong functional design such as XSLT and workflow languages like WS-BPEL.

On the other hand, XSLT and XPath have reached a level of maturity and expressiveness that cannot be ignored, and XTemp is leveraging both to a large extent. XTemp1.0 is using XPath as expression language and relies extensively on its functions and operators library, in particular for its time and duration processing (e.g. XPath2.0 library). Shouldn't then XTemp be defined as an XSLT extension? Although such an approach is not ruled out in future versions, a couple of reasons weight in favor of keeping its syntax separate for now:

  • It is not clear to the authors whether and how all aspects of XSLT language semantics can be compatible with real-time processing of live events, concurrency, and other timing aspects. Whether future evolutions of XTemp can be defined as extensions to XSLT, will rely on an execution semantics more ambitious than the one presented here, and able to encompass all XSLT features in a time-sensitive, event-driven context.

  • Another reason for decoupling XTemp design from XSLT syntax, is the option - not yet investigated by the authors - of leveraging XQuery instead of XSLT for a scalable implementation. XQuery has powered XML content servers - e.g. MarkLogic server (see http://gcdnmirror.paultan.org/markmail.org/docs/faq.xqy), proving adequate for processing large repositories of relatively small items such as XML-formatted emails for searching and querying. Such repositories of XML content may provide a scalable implementation for event boards.

In spite of these open questions the XTemp design is still influenced by XSLT features - i.e. it matches feature-wise a subset of XSLT and its new features (variable mutability, event catching and time handling) are designed to be XSLT-implementable, even if not in an optimal way, at least for deferred executions - or analysis of static event boards. Implementability using XSLT - even for a restricted ('deferred') usage and for a subset of XTemp features called 'core' XTemp - is an important aspect of usability of the language.

Among the features of XTemp beyond the 'core' subset (as defined by the 'core' conformance level in [XTEMP10]) and considered challenging or impossible using conventional XSLT engines, are:

  • Event posting, as this provides ways for concurrent paths to depend on each other (event communication) that would require a level of concurrency control beyond what XSLT allows to emulate.

  • Definition of a complex event pattern inside a single 'catch' statement, which requires advanced processing of event boards (e.g. as supported by a dedicated query language). As explained later, the value of this feature is precisely in its delegation to more specialized processors, while it can be replaced by a flow of simpler 'catch' statements, even if less efficient.

  • Event masking, which implies some event state management associated with a script more than with the event board.

  • Concurrent path joining (as in workflow) using the 'wait' statement, which implies the ability to keep track of several concurrent paths and their execution status.

Another approach in analyzing and testing the trace of business processes, is to rely on a simpler and more declarative 'rule' language (or 'pattern' language) usually implemented in XSLT, such as Schematron or Tamelizer (the latter being a recent implementation - see: http://code.google.com/p/tamelizer/ - of a test assertion language proposed in [TAG09] by one of the authors.) Such declarative languages in turn rely heavily on the expressive and processing power of XPath for the testing logic. But declarative test assertions or patterns are of little help when expressing complex test logic spanning sequences of events: although Xpath2.0 and above has a remarkable ability for expressing such correlation and validation logic in a single predicate expression, the readability and maintenability of such tests is poor. For example, the test suites for WS-I [WSI10] were based on the above test assertion design and related analyzer tool. In these test suites XPath2.0 alone proved indeed able to handle complex choreographies of timestamped inputs such as Web services requests and responses, combined with related low-level messages such as resends or receipts and various hand-shaking procedures. But the resulting large XPath expressions are hardly readable and difficult to evolve or to update, requiring a level of skills and efforts in XPath programming beyond what most test engineers are ready or willing to exercise. The ability of above languages to chain test assertions or to compose rules, does not really help to break down this kind of complexity. It became clear that a more powerful yet more intuitive combination of logical expressions and procedural flow definition such as in XTemp, was needed.

WS-BPEL and Workflow XML vocabularies

Workflow languages such as WS-BPEL [WSBPEL07] and others XML vocabularies or notations (XPDL, BPMN) are intended to describe and drive business processes, more than to analyze and verify their behavior. To this extent, even if they share several seemingly equivalent algorithmic and synchronization constructs with XTemp, their purpose is rather complementary. The notion of event is used in WS-BPEL mostly as a dynamic input to business process instances, and not as an input for a more general analysis that may span several processes. The notion of correlation set is used for helping the selection of the next event that belongs to the same process, which is expected to depend on a predefined set of event attributes that define some event equivalence class. To this end, correlation sets are sufficient and efficient but unable to express more general event patterns needed in analysis and live monitoring. The concept of event handler is intended in WS-BPEL as a device for helping drive process instances, typically as these events occur. It does not assume the capability of processing past events. Finally, the focus on analysis makes it possible for XTemp to only use a minimal set of constructs, compared with WS-BPEL.

XProc and Pipelining

Although XSLT, XProc and XTemp share the same general purpose of advanced processing of XML inputs, and share several similar algorithmic constructs, it is interesting to stress the different nature of the entities behind these XML inputs and of their processing. The target unit for processing in XProc is a “document”, while it is an event in XTemp. Clearly, this distinction is not as much about content (a document can be represented as an event and vice-versa) as it is about the general nature of the processing of these items – as well as how these items relate to each other. In short, documents are to be validated, transformed, compared, merged. Events have to be matched, correlated, waited for or queried.

A combination of documents that has to be part of the same processing flow is statically known in general (e.g. a business document + its possible includes + its XML schema + a set of codelists + some semantic rules). Their role is well established as these documents will be assigned specific ports in an XProc pipeline. In comparison, the set of events of interest for processing as a unit is generally unknown beforehand, determined dynamically by a tentative match with a pattern definition that may or may not succeed. The document itself has a business value, and the value-add from Xproc is in its (efficient) validation, transform, or assembly. Events generally have value not through individual processing, but in relation with each other. Consequently, the ability to correlate efficiently over a potentially large event repository is key in XTemp, while advanced, multistep processing and related pipeline optimization is not its goal.

Similarly, although XSLT is computationally sufficient by itself to drive a chain of document processing or to analyze a log of events, its constructs do not support these paradigms in a way that is specific enough for practitioners to feel comfortable with the language, or for implementors to use appropriate modules or architectures.

Testbeds using XML vocabularies for Test Suite syntax

XML vocabularies have been used in B2B testbeds for test case and test suite representation. Two of the authors have been involved in an earlier, very limited version of XTemp [ETSL07] that did not have the same algorithmic capabilities nor leveraged XPath to the same extent. Implementations proved sufficient for the driving and verification of many B2B test cases, but lacked the ability to generate advanced reports that require data consolidation over several B2B exchanges. Other testbeds have also used XML vocabularies for test suites, that were developed at NIST [WSTB2B06] and also in [TESTHL709]. These testbeds represent the state of the art in XML-based test suites. However they do not leverage fully the notions of event and test execution concurrency as means to a powerful analysis of B2B logs. Their focus is more on driving test cases than on test analysis.

Conclusion

An XML language for event processing and analysis is presented, that fully leverages XML in three ways: as event format, as test script format, and as test report format. The concurrency semantics and execution required by a well-rounded event processing has proved to be quite compatible with the XML focus both as script format and incremental report generation, while a core version of the language could be implemented using XSLT2.0 which was a major requirement in terms of ease of deployment and convenience of script development and testing. The authors are working on deploying XTemp in their B2B testbed work and for advancing it as analysis and test language for eBusiness exchanges in various user consortiums. However, although the initial motivation was business process analysis and validation, the resulting event-centric design and general execution model appears to apply to more diverse event-driven monitoring and analysis applications.

References

[RHAM06] N. Russell, A.H.M. ter Hofstede, W.M.P. van der Aalst, and N. Mulyar. Workflow Control-Flow Patterns: a Revised View Technical report, BPM Center Report BPM-06-22 , http://www.workflowpatterns.com/documentation/documents/BPM-06-22.pdf BPMcenter.org, 2006.

[WSI10] WS-I Testing Tools V2 for Basic Profiles 1.2 and 2.0, Web Services Interoperability, 2010, http://www.ws-i.org/

[TAG09] Durand, J., Green, S., Kulvatunyou, S., and Rutt, T.; Test assertions on steroids for XML artifacts proceedings of "Balisage the markup conference" http://www.balisage.net/2009/Program.html August 2009. doi:https://doi.org/10.4242/BalisageVol3.Durand01

[XTEMP10] XTemp: XML Testing and Event-driven Monitoring of Processes, Committee Specification Draft, OASIS, http://www.oasis-open.org/committees/download.php/41676/XTemp-1.0-csd01.pdf, March 2011.

[ETSL07] Durand, J., Kulvatunyou, S., Woo J.,and Martin, M. ; Testing and Monitoring E-Business using the Event-driven Test Scripting Language ; proceedings I-ESA (Interoperability of Enterprise Systems and Applications), April 2007

[WSTB2B06] Jeong, B.; Woo, J.; Cho, H.; Kulvatunyou, B.; Lee, J. ; A Web Service-based Reconfigurable Testbed for Business-to-Business (B2B) Integration International Conference on Web Services (ICWS), http://www.nist.gov/manuscript-publication-search.cfm?pub_id=822715 September 2006

[TESTHL709] Namly, T., Aluc, G., Dogac, A. ; An interoperability test framework for HL7-based systems ; IEEE Trans Inf Technol Biomed. 2009 May;13(3):389-99. doi:https://doi.org/10.1109/TITB.2009.2016086

[WSBPEL07] Web Services Business Process Execution Language Version 2.0; OASIS standard, http://docs.oasis-open.org/wsbpel/2.0/OS/wsbpel-v2.0-OS.pdf, April 2007.

Author's keywords for this paper:
Document Testing; Test Assertion

Jacques Durand

Senior architect, R and D dir.

Fujitsu America, Inc.

Jacques Durand is software architect at Fujitsu America, Inc. with a long-time involvement in XML standard organizations. He has extensive experience in XML-related testing, is chair of the Test Assertions Guideline OASIS committee and of the Testing and Monitoring of Internet Exchanges (TaMIE) committee in OASIS, where he is also member of the Technical Advisory Board. He has been leading testing activities for years in the WS-Interoperability consortium and in the ebXML technical committee, while contributing to XML user consortiums such as RosettaNet, OAGI.Prior to this he was a development lead in a Business Process Management company for 5 years. He earned a Ph.D. in rule-based systems and logic-programming from Nancy Univ., France.

Hyunbo Cho

Professor

Pohang University of Science and Technology

Hyunbo Cho is a professor in the department of industrial and management engineering at the Pohang University of Science and Technology, South Korea. He received his B.S. and M.S. degrees in Industrial Engineering from Seoul National University in 1986 and 1988, respectively, and his Ph.D. in Industrial Engineering with a specialization in Manufacturing Systems Engineering from Texas A and M University in 1993. He was a recipient of the SME 1997 Outstanding Young Manufacturing Engineer Award. His areas of expertise include B2B integration, Testing and monitoring of B2B solutions, and Supply chain management. He is co-chair of the Testing and Monitoring of Internet Exchanges (TaMIE) committee in OASIS.

Dale Moberg

Senior Scientist

Axway

Dale Moberg has been involved in defining or enabling reliable, secure systems for business collaboration since 1993. He has worked in product development, architecture, strategy, and research. Many of his activities have been concerned with integrating security standards for B2B such as digital signatures for nonrepudiation of origin and nonrepudiation of receipt. He has chaired or co-chaired ebXML TCs in OASIS working on eBusiness transactions choreography and agreements. He currently works on designs and functional requirements for products in areas of application integration, B2B gateways, business activity monitoring, and business intelligence. He holds a M.A. and Ph.D from Northwestern University, and a M.S. from Ohio State University and has worked in academic and commercial organizations.

Jungyub Woo

Researcher

National Institute of Standards and Technology (NIST)

Jungyub Woo is a researcher at the Software and System Division of the National Institute of Standards and Technology (NIST), the technology laboratory of the US Department of Commerce. He received his B.S. and M.S. degrees in Industrial Engineering from Postech (Pohang University of Science and Technology) in 2001 and 2003, respectively, and his Ph.D. in Industrial Engineering with a specialization in Manufacturing Systems Engineering from Postech (Pohang University of Science and Technology) in 2007. Dr. Woo has spearheaded development of the Business-to-Business test bed at NIST, which has been used by a number of industry and standards development organizations. Also he is researching for the Health Level Seven International (HL7) interoperability. His areas of expertise include Supply Chain Management, Manufacturing Management and Strategy, and Semantic Web.