Balisage Paper: Where XForms Meets the Glass

Introduction to Web 2.0 Application Frameworks

With the rise of Web 2.0 applications, the balance of processing for end-user interaction represented by the server vs. the browser has shifted increasingly toward the browser. There are two benefits of this shift. First, interactions that can be handled without submission to the server can decrease latency when processing is entirely related to the UI and located on the client. As an example, adding and removing a “zip code” form element based upon a selection of “United States” in another “Country” form element can be done entirely in the browser without a server round trip for a new page. Second, data load/save operations can run asynchronously while users interact with a web page. As an example, an initial page can load with partial data in a table, but as the user interacts with that table, more data could be loaded from the server without requiring the user to wait for an entire web page refresh.

Many of the above examples are powered by a programming style in the browser called AJAX (Asynchronous JavaScript and XML). As will be explained later, despite the name, XML is not required and more frequently than not the data is JSON encoded.

There are multiple frameworks that build upon JavaScript and AJAX that help with abstraction of page interaction and events and provide cross browser portability. Many of these frameworks provide “widget” libraries that build upon HTML, CSS, and JavaScript to provide commonly known UI controls such as trees, tables, input form elements. The widgets provide the View in a typical MVC (Model View Controller MVC) architecture. These widgets read and write data (for example the text “90210” in the “zip code” form element) either to the page DOM data or to abstracted data stores. This storage provides the Model in the MVC pattern. Manipulation of the data is typically done in JavaScript-based event handlers (the Controller in MVC). As part of this manipulation, when the data is read and written to the server asynchronously, it is common to encode this data as JSON data. Encoding the data as JSON data is typical as JSON data natively loads into JavaScript and aligns well with the JavaScript programming model skills the programmer is already using within the framework.

Given this set of Web 2.0 languages, programming models, and components, most developers work to create an optimal user experience within the natural affordances of the available technologies. We believe this often implies following a "glass to data" design process where the UI implies a model that is close to its structure but likely differs from back-end data designs. Also, the encoding for data in this style will likely be JSON based upon prevelant development experience. As this Web 2.0 interface needs to be integrated with existing enterprise data on the server, there will be a collision both in data structures and in encoding technologies. Typically this collision is worked around by writing transformation routines on the server between JSON, Web 2.0 model and XML, enterprise data model. This transformation then needs to be maintained and adjusted whenever either the Web 2.0 model or enterprise data model changes which is costly.

A design process that starts rather from back-end data structures and works toward front-end UI design we refer to as "data to glass" design. There are AJAX data stores that partly support this design style and allow XML data sources to be the backing store for widgets (such as Dojo’s dojo.data interface Dojo Toolkit). While these stores can handle some XML data, the data needs to adhere to the exact XML format as required by the data store. This can be problematic if the XML instance was authored without consideration of the data store. Some examples of the problems are shown in Andrew Spyker’s blog post on “The Pain of XML in Web 2.0” XML in Web 2.0.

There are two ways to solve this cost of transformation. One option, which has been gaining traction, is to throw away the enterprise model and XML. One can take the JSON objects and work with them via JavaScript on the server via Rhino Rhino. One can also take the JSON objects and persist them with JSON document stores such as CouchDB CouchDB. Additionally there are efforts to create schema definition languages and query languages for JSON. With all of these JSON facilities on the server and strong support for JSON in the browser, it is easy to consider replacing the XML enteprise model with a JSON enterprise model especially for quickly developed applications.

A second option will be presented in this paper. We introduce a more complete MVC design pattern in frameworks such as Dojo to enable authors to converge front- and back-end models logically, perform required transformations to adapt data for more effective user interaction, and interact with both JSON and XML models using Web 2.0 widgets. In this way, a choice of Web 2.0 interfaces doesn’t collide with the enterprise XML data model.

Overview of XForms and Lightweight Runtimes

XForms is an XML technology that defines a Model-View framework for data-centric applications delivered over the web or installed clients. XForms model elements include support for data "instances", for content being presented to or obtained from the user. Instances may have associated "bind" elements which define validation, read-only, or relevance constraints as well as calculations that propagate across related values through the application of an XPath-based dependency graph. XForms has been implemented in a variety of client-based and server-based platforms as described on the W3C Forms Working Group site XForms Implementations.

XForms is not a complete language for interactive applications but is designed to be embedded in a "host" language such as HTML. In addition to host languages for interactive applications, XForms is also included in the Open Document Format ODF 1.0 specification as the data model for form-related content in text, spreadsheet, and presentation documents. Such formats are increasingly being seen as valid formats for user interactivity with a blurring between web, office, and "rich" client platforms under way. Other host languages have included vendor-specific languages such as IBM's XML Forms Definition Language XFDL.

XForms has traditionally required the implementation of native platform support or platform extensions (for example in the Mozilla Firefox browser Mozilla XForms). Such implementations are an excellent means of delivering XForms function with typically a level of performance and function comparable to native implementations. Obstacles to the rapid adoption of XForms have resulted, however, from the need for and development time/expense of multiple implementations for adequate platform coverage and the attendant user burden of installing and maintaining current configurations of the extension.

With the rapid performance improvements in JavaScript runtimes on most current browsers, script-based implementations of XForms are becoming increasingly popular. Some implementations transcode XForms into (X)HTML and JavaScript XSLT XForms while others maintain the original source document structure by attaching script-based "behaviors" to original XForms elements thus providing the runtime processing model for each element type Ubiquity XForms. In either case there is a need to bind XForms data models, with their included instances, binds, actions, and submission elements, to appropriate view elements to create a complete model/view interactive application.

In the remainder of this paper, we explore two options for constructing the view elements of a complete XForms application such as the sample loan application shown in Figure 1.

Figure 1: An example loan form in the Ubiquity AjAX XForms framework

The Ubiquity XForms library is an AJAX implementation of the XForms language using JavaScript functions on the client to attach behaviors to each XML element in an XForms page. Unlike prior implementations of XForms based on browser plug-ins or extensions, the Ubiquity library is loaded dynamically with the page and requires no installation or update maintenance. Existing AJAX libraries, such as Dojo or the Yahoo User Interface library (YUI) are used to provide concrete presentation controls such as the calendar widget shown in the figure.

In the first approach, which we argue is more natural for application authors starting from a data or process centric point of view, we outline briefly the two-layer view consisting of abstract controls for user intent such as "select", "repeat", or "trigger" coupled with concrete widgets or controls for actual presentation and interaction such as "button", "check box", or "radio button". The Ubiquity AJAX implementation of XForms Ubiquity XForms provides a very complete example of this approach and readers are directed to the samples for information beyond the scope of this paper.

The second option maintains the model/view separation of XForms but adopts only a single layer view consisting of the existing widgets in an AJAX library such as Dojo augmented to support model/view binding. We believe the architectural principles illustrated here with Dojo are applicable to other AJAX libraries, albeit likely with changes to the specific interfaces used to support data binding and property change notification, and we have used Dojo as an illustrative example of a desirable broader evolution in Web 2.0 programming models.

To assure a natural design experience for current AJAX developers, the key concept in this approach is to first introduce a general model/view binding mechanism in Dojo which can apply to an extensible set of model technologies, whether XForms, other XML, or simple JavaScript-based models. Having provided Dojo with a first-class MVC framework, XForms can then serve as a preferred model technology for those wishing an XML-based or more declarative authoring approach than conventional AJAX design.

Model-View-Controller Patterns in UI Design

MVC design in XForms consists of a data model with instances, binds, and submissions as shown in the example loan model fragment here:

	<xforms:model id="loan_model">
	  <xforms:instance id="loan">
		<loanrecord xmlns="">
		  <startdate>2008-08-08</startdate>
		  <borrower>
			<name>John Q. Public</name>
			<addr>123 Main St. Tinyville</addr>
		  </borrower>
		  <principal currency="USD">10000</principal>
		  <duration>12</duration>
		  <interestrate>5</interestrate>
		  <monthly-payment></monthly-payment>
		  <totalpayout></totalpayout>
		</loanrecord>
	  </xforms:instance>
	  
	  <xforms:instance id="rate">
		<rate  xmlns=""></rate>
	  </xforms:instance>
	  
	  <xforms:bind nodeset="instance('rate')" 
		calculate="instance('loan')/interestrate div 1200.0"/>
	  
	  <xforms:bind nodeset="totalpayout" 
		calculate="../monthly-payment * ../duration" 
		relevant="../monthly-payment > 0 and ../duration > 0"/>
		
	  <!-- other model elements omitted for brevity...
		See http://code.google.com/p/ubiquity-xforms/source/browse/trunk/samples/Loan/loan.html
		for the full source. -->
		
	</xforms:model>
          
      

Model data are presented to the user for interaction via a set of abstract view controls such as select, repeat, group, trigger, and input or output. These controls are "abstract" in the sense that they express interaction intent but not the specific look and feel of the user interface. Thus there are controls for single and multiple selection vs. radio buttons and check boxes. An additional mapping mechanism associated concrete presentation components, such as Dojo widgets, with their corresponding abstract XForms controls. See XForms 1.1 for a complete definition of the XForms language. The following fragment shows the abstract controls for a subset of the example loan application above:

    <html:body>
	
	  <xforms:input ref="startdate" datatype="xforms:date"> 
	    <xforms:label>Agreement Date: </xforms:label>
	  </xforms:input> 
  
	  <xforms:select1 ref="principal/@currency" appearance="minimal" incremental="true">
	    <xforms:label>Choose currency:</xforms:label> 
	    <xforms:item>
		  <xforms:label>US Dollars</xforms:label>
		  <xforms:value>USD</xforms:value>
	    </xforms:item>
	    <xforms:item>
		  <xforms:label>Canadian Dollars</xforms:label>
		  <xforms:value>CDN</xforms:value>
	    </xforms:item>
	    <xforms:item>
		  <xforms:label>Euros</xforms:label>
		  <xforms:value>EUR</xforms:value>
	    </xforms:item>
	  </xforms:select1>
	
	  <xforms:output class="regular" >
	    <xforms:label>Monthly Payment: </xforms:label>
	  </xforms:output>
  
    </html:body>
    

XPath-based binding expressions (@ref in the example) connect abstract controls to data in the model. UI changes are written to the corresponding model element text content while model changes are signalled to the UI with a set of defined events indicating changes in model element values, validity, relevance, R/O status, or required status. Atomic elements such as those shown bind to single data elements in the model. Repeat controls expand their child content as a template for each entry in their nodeset-based binding. See XForms 1.1 for a more complete discussion of single and multiple node model-view binding and update notification events.

Of particular interest to our discussion here is the manner in which Birbeck refines XForms abstract controls with presentation specific elements in the Ubiquity framework to complete their graphic and interaction design Ubiquity XForms. The first input control in the example above is refined in this way using a calendar widget from the Yahoo User Interface (YUI) Yahoo UI library library as shown in Figure 2.

Figure 2: Model-View-Controller design in XForms web applications

XForms data models contain instance elements for data values, bind elements for calculations and Model-Item-Properties indicating data relevance, R/O status, and validity, and submission elements to interact with the server. XForms views contain elements for abstract user intent such as single or multiple selection, grouping, switch/case, repeated content, and atomic controls such as input, output, and secret for passwords. Concrete presentations are bound to abstract controls and map between physical interaction and display events and their corresponding abstract XForms controls.

The manner in which abstract to concrete control refinement takes place in the Ubiquity library (using the MVC design in Figure 2) combines the three-layers of model, abstract view, and concrete widgets into a single coherent runtime structure. Each layer is actually present at runtime and participates in event handling and synchronization with the other layers. Alternative approaches transcode source XForms markup into dynamic HTML, maintaining the end-user functionality of the form but losing the original document structure at runtime in the client.

In the design shown in Figure 2, the XML programming model is therefore maintained and visible from the model through to abstract user interface. AJAX widgets such as the calendar in Figure 2 are plugged into their abstract controls algorithmically (under rule based or stylesheet control). An automated mapping from abstract to concrete control offers significant productivity gains, eliminating the need for an additional manual authoring stage at the concrete presentation level, but with the potential for loss of control or expressivity at that final "on the glass" layer. Depending on the sophistication of the mapping machinery this loss of control can be potentially eliminated and the rules for associating abstract and concrete controls refined and customized as needed.

Supporting MVC Design in AJAX: Case Study Using Dojo

In the above example, we have shown how data in XForms models -- usually obtained from back-end data stores or business processes -- may conveniently be projected out to the client and bound to abstract controls for user interaction. This design process is largely data-driven, i.e. begins with a definition of data schemas and process models and works from there out toward user interaction. While it is indeed possible to use XForms in other ways, it is our impression that in most cases it is not the specifics of on-the-glass interaction or graphical design that predominates but rather the functional data-driven aspects of the design that come first.

Rich Internet Applications (RIAs), by contrast, are often driven initially by user experience and visual design considerations. Exactly what constitutes an RIA is hard to pin down, but often they are a means to access and interact with a variety of media types including video, images, animations, and formatted text intermixed in composite user interfaces. While it is rare that data-oriented (e.g. CRUD CRUD) operations do not appear in an RIA, they are typically hidden or implicit behind the "flashier" aspects of rich media interaction.

It is our belief, nonetheless, that RIAs architecturally share many if not most of the features of data-centric "forms" applications as outlined above - and indeed, that the architectural features in forms runtimes, including separation of concerns between model, view, and control (both abstract and concrete) are as important to media-rich RIAs as to data-rich ones. The question, then, is how to extend popular RIA frameworks with these features in a way that preserves their appeal for presentation and interaction oriented design methods but still allows for binding to data and reuse of model-based artifacts behind the facade of rich media.

Taking Dojo as an example of a mature AJAX framework, we have reworked the sample loan application from Figure 1 and implemented it in a rigorous model-view pattern using native Dojo constructs. Our approach is to establish a first-class MVC pattern in Dojo either by convention in the usage of current componentry or by extensions to existing Dojo libraries to fully support the separation of concerns and behavioral patterns present in a pure XForms-based application.

Figure 3: The loan application using Dojo Model-View-Controller design patterns

A near functional equivalent of the XForms loan application implemented using pure Dojo/AJAX components. The data model is implemented as a Dojo package with JS objects for instance data and JS functions for constraints such as rate conversions and relevance (and other Model-Item-Property) calculations. The view consists of Dojo widgets extended with Model-View binding behavior to receive data and Model-Item-Property updates as model values change.

We require separation of model data and presentation controls at either an abstract or concrete layer. We require the ability within the model to observe changes in data elements and bind those changes to functions to compute derived values. The model should provide a degree of encapsulation so that internal operations are shielded from observation and allowed to complete before external observers in the view or elsewhere are notified. Finally, all of these aspects of modularity, encapsulation, within-model bindings, and model-view bindings should be expressable with a minimum of application level machinery -- rather, the framework itself should support the core notions of modularity, dependency, and binding and not require authors to emulate them with lower-level APIs.

Figure 4: Extending Dojo with Model-View-Controller design patterns

MVC design in Dojo extends Dojo widgets to bind to data models and receive value and Model-Item-Property updates. Any data model technology, whether script or XML-based, that can be wrappered with the extended widget MVC APIs may be used. A single layer view is adopted in contrast to the two-layer abstract vs. concrete view in traditional XForms. Controls in the view are Dojo widgets (shown in green) which expand into HTML fragments replacing their original markup elements under template-based control. In the figure an HTML fragment for the calendar is shown as one such subtree generated by its managing Calendar widget.

Figure 4 outlines the means to realize these goals in the existing Dojo framework. We have reimplemented the key aspects of the loan application data model as a Dojo class as shown to the right of the figure. Dojo has introduced a new abstraction for property change notification (the dojo.Stateful API introduced in the Dojo 1.5 release Dojo Toolkit). Those elements of the data model intended for eventual binding to UI controls are therefore implemented as such dojo data sources to simplify UI updates as well as allow for an approximation to the data-dependency graph in XForms within the script-based Dojo data model. Other data elements, used strictly for internal model record keeping and not subject to user alteration, do not require this notification support and hence may be implemented by author's choice in a variety of other JavaScript types.

      var loan_init = {
        // Initial values for the loan form are defined in this JS variable to be passed
        // into the model constructor at the end of this fragment.
        
        // We have extended Form widgets in Dojo to bind to data having both "value" and 
        // other properties such as required, validity, and read/only status...
        
        "StartDate": { "value": "2008-08-08", "required": false, "valid": true }, 
        
        // Data fields can be assigned values directly and will be remapped into the above structure when
        // initializing the model, avoiding the need for authors to specify properties when their 
        // default values are acceptable...
        
        "Borrower" : "John Doe",    
        "Street"   :  "",
        "City"     :  "",
        "County"   :  "",
        "State"    :  "",
        "Zip"      :  "",
        "Country"  :  "US",
      
        "Principal":  "1000",
        "Currency" :  "USD",
        "Duration" :  "12",
        "InterestRate" : "6"
      };
      
      // The loanModel variable will have properties for each of the above data fields grouped as
      // "loanModel.loanInstance.Borrower"  etc.  
      
      // Derived loan values introduced by the model, such as monthly interest rate, appear as
      // "loanModel.infoInstance.MonthlyRate"
      
      // This separation into two groups of fields is an application design decision intended
      // to illustrate separating model inputs and outputs from fields introduced to 
      // support user interaction or temporary values during model execution.
      
      var loanModel = new patterns.loan.StatefulModel( { 'loan': loan_init } );
    

The code fragment below uses the standard Dojo JavaScript class mechanism to define a set of internal functions for computing monthly interest rates and payments, and total loan payouts given loan principal, duration, and interest rate. Such internal functions are written by convention with an "_" prefix, but are still attached as visible properties of the loanModel variable given the lack of formal encapsulation in the Dojo class mechanism. In this example, the only public properties required on the loanModel are the data instances themselves which are provided to allow for binding from UI widgets.

      
      dojo.provide("patterns.loan.StatefulModel");    // name of the module being exported from this file
      dojo.require("mvc.stateful.StatefulItems");     // dependencies on other Dojo modules
      dojo.require("mvc.data._base");     
      
      // our StatefulModel class extends the _base class
      
      dojo.declare("patterns.loan.StatefulModel", mvc.data._base, {   
        loanInstance: null,	 // loan parameters, becomes public property
        infoInstance: null,	 // record keeping info form and derived loan values, also public property
      
        info_init: {
          "MonthlyRate"    : { "value": "0.005", "relevant": true },  // data fields have "value" and
          "Monthly_Payment": { "value": "0",     "relevant": true },  // other properties bound to form
          "TotalPayout"    : { "value": "0",     "relevant": true }   // widgets for validity, relevance, etc
        },
      
        constructor: function (args) {
          // StatefulItems is just a container object for a set of fields, each of which implements 
          // the dojo.Stateful interface for get/set and property change notifications...
          
          this.loanInstance = new mvc.stateful.StatefulItems( { 'data': args.loan } );
          this.infoInstance = new mvc.stateful.StatefulItems( { 'data': this.info_init } );
      
          // simple dependence of monthly rate on APR...
          this.bindValue( this.infoInstance.MonthlyRate,      // the target field being assigned a value
            this.loanInstance.InterestRate,                   // the input field whose change will trigger
            this._APR2MonthlyRate );                          // return value from this function sets target
      
          // monthly_payment = f( principal, monthlyrate, duration )
          this.bindInputs( [                                  // when we have multiple input fields to monitor
            this.loanInstance.Principal,                      // they are just given as an array
            this.infoInstance.MonthlyRate,                    // and we don't specify a single output field
            this.loanInstance.Duration                        // since in general more than one value may be set
            ],
            this._computeMonthlyPayment );                    // Then the callback function sets model directly
      
          // totalpayout = f( monthly_payment, duration )
          this.bindInputs( [ 
            this.infoInstance.Monthly_Payment, 
            this.loanInstance.Duration 
            ],
            this._computeTotalPayout );
      
          this.bindRelevance( this.infoInstance.MonthlyRate,  // bindRelevance is similar to bindValue above
            this.loanInstance.InterestRate,                   // but the boolean return value is assigned to
            this._monthlyRelevance );                         // the "relevant" property of the target field
            
          this.bindRelevance( this.infoInstance.Monthly_Payment, 
            this.loanInstance.InterestRate, 
            this._monthlyRelevance );
            
          this.bindRelevance( this.infoInstance.TotalPayout, 
            this.loanInstance.InterestRate, 
            this._monthlyRelevance );			
        },
      
      _APR2MonthlyRate: function(newValue) {			
        // recompute the monthly rate as APR/12 then /100 for percentage
        // just need to return the value, will be assigned to the target field for us...
      
        var monthly = parseInt( newValue ) / 1200.0;  
        return monthly+="";
      },
      
      _computeMonthlyPayment: function() {
        // form of callback used in bindInputs does not provide single input value
        // and the function must set its outputs directly...
        
        var principal = parseFloat( this.loanInstance.Principal.get("value") );
        var duration = parseFloat( this.loanInstance.Duration.get("value") );
        var rate = parseFloat( this.infoInstance.MonthlyRate.get("value") );
        var monthly;
      
        if ( rate == 0 ) monthly = 0;
          else monthly = principal * rate / ( 1 - Math.pow( 1.0 + rate, -1.0 * duration ) );
        this.infoInstance.Monthly_Payment.set("value", monthly+="" );
      },
      
      _computeTotalPayout: function() {
        var monthly = parseFloat( this.infoInstance.Monthly_Payment.get("value") );
        var duration = parseFloat( this.loanInstance.Duration.get("value") );
      
        var total = monthly * duration;
        this.infoInstance.TotalPayout.set("value", total+="" );
      },
      
      _monthlyRelevance: function(newValue) {
        // callbacks for metadata like relevance get the new value of their
        // monitored field and return a boolean for its relevance...
        
        if ( newValue >= 0 ) return true;
        else return false;
      }
      
    });
      
    

The existing dojo.Stateful interface provides a simple abstraction above event listeners that combines a getter/setter pattern along with property change registration using a watch method. We have extended this library to provide for simple binding functions ("bindValues", "bindInputs" above) where multiple incoming fields are input to a composing function whose return value is then assigned to an output field. These bind functions seen in the model code above are intended to stand-in for the first-class dependency graph provided by XForms and its bind expressions.

    <body class="tundra">
      <label for="principalInput">Principal:</label>
      <!-- the mvc.form library extends the usual dijit.form package with Model-View bindings.
      
          The refID attribute binds to a data model field implementing the dojo.Stateful interface
          allowing the widget to listen to changes in the value, relevant, required, readonly
          and valid properties.  
          
          These flags are merged with corresponding view states to determine the overall state of the control.  
          For example, an input field must be valid both at the view level (has correct lexical value) 
          and at the model level (semantic validity).
      -->
      
      <input id="principalInput" dojoType="mvc.form.NumberTextBox"
        refID="loanModel.loanInstance.Principal">
      </input>
        
      <label for="durationInput">Duration:</label>
      <input id="durationInput" dojoType="mvc.form.NumberTextBox"
        refID="loanModel.loanInstance.Duration">
      </input>
                  
      <!-- etc -->
    </body>
      
    

Data binding uses the same mechanism both within the model, to watch model changes and compute derived values, and to watch changes in the model to refresh relevant controls (Dojo widgets) in the view. The dojo.Stateful interface is used in an extended widget library to register for changes in model values as well as a set of form-related properties such as relevance, readonly, required, and valid -- the familiar Model Item Properties from XForms. Widget implementations handle both data and property bindings and updates implicitly, based on the value of the refID attribute, freeing authors from the need to write significant amounts of wiring as application code and hence we hope stimulating the use of Model-View patterns by AJAX authors more widely in the future.

Where XForms Data Meets Dojo Widgets

Having prepared Dojo for a first-class model-view separation, we are now ready to re-introduce an XML-based data model. such as the one from XForms, as one of an extensible set of model technologies that can play within this extended MVC pattern We have implemented a wrapper for XForms models which allows widgets to bind to models using the wrapper's support for the change notification interfaces described above. The internal operation of the XForms model, including its dependency graph for spreadsheet-like value propagations, constraints for validity, relevance, and the other Model Item Properties, and the model submission and action features may be used directly behind such a Dojo-based view. This model-view separation should allow for cleaner and simpler views while at the same time packaging validation and calculation logic in the model for better reuse.

Figure 5: Dojo MVC design patterns using XForms models

Having introduced a first-class MVC design into Dojo form widgets, XForms models with an appropriate API wrapper can now bind directly to common Dojo widgets for presentation and interaction. The standard XForms bind constructs can be used as a fully declarative dependency graph for value and Model-Item-Property calculation replacing the script-based functions in the Dojo models in earlier figures. Common coding conventions for UI design, however, may continue to be used for interaction design at the view layer.

Figure 6 is our first example of using Dojo widgets for presentation while delegating validation to the XForms model. A Dojo "NumberTextBox" widget is bound to the "Principal" data element in the XForms model using the extended APIs discussed above -- the "ref" attribute on the widget gives the XPath query expression into the model as it would appear on an XForms control. Internally, the widget uses the dojo.Stateful APIs to connect to the specified data element and to respond to changes in its value, validity, or other Model Item Property XForms 1.1. The widget provides alternative presentations for the focus and non-focus states of the control -- inserting formatting syntax to conform to conventions for comma or period separated numeric or currency values. While focused, the pure numeric value is drawn to allow for easier editing.

Figure 6: Dojo NumberTextBox widget bound to XForms models

      <body>
        <label for="dojoPrincipal">Principal:</label>
        <div id="dojoPrincipal" dojoType="mvc.form.NumberTextBox" 
          model="loan_model" 
          ref="instance('loan')/principal">
        </div>
      </body>
    

Validation in vanilla Dojo is handled entirely on the widget. There are validating subclasses of many of the forms-related widgets which allow for regular expressions or script-based validation handlers to be attached to value change events on the widget itself. Through the use of the XForms model we are able to support a more refined lifecycle of validation in which "lexical" and "syntactic" levels of validation remain attached to the widget while "semantic" validation occurs behind the view in the model layer. Validation thus may fail early at the view layer, giving immediate feedback to the user, or succeed lexically or syntactically but still fail semantically in the model. Such "semantic" failures will typically be due to factors that we want outside the scope of the widget to handle.

As an example, a phone number may consist correctly only of numeric digits and syntactically be valid in terms of a set of area code, exchange, and extension string of the right length, but still fail model validation if that number is not in actual use. The latter validation will require in general a back-end data base lookup or minimally a comparison with a set of valid phone numbers cached in the model. Managing this lookup is more complex, and has more potential for reuse, than would be supported by leaving this logic at the widget level.

"Vanilla" Dojo also has the capability to compute dynamic widget values using the existing dojo.data APIs. Selection boxes, for example, may query data sources in a variety of alternative formats using a common interface and populate pull-down lists or provide keystroke-level autocompletion based on the dynamic context of the application. Figure 7 shows such a "FilteringSelect" widget with an XPath query into the XForms model to extract the currently valid set of currency types for the loan application. The value-added feature obtained by use of the XForms model is the ability for such queries to operate not only over static XML data in the model, but to reference computed values derived using the XForms dependency graph -- transparently to the view author.

Figure 7: Dojo "filtering" select using dynamic data queried from XForms models

    <xforms:model id="loan_model">
      <xforms:instance id="loan">
        <LoanRecord xmlns="">
          <principal currency="USD">10000</principal>
          <currencies>
            <currency>USD</currency>
            <currency>CDN</currency>
            <currency>Euro</currency>
            <currency>Pound</currency>
            <currency att="calc"></currency>
          </currencies>
        </LoanRecord>
      </xforms:instance>
        
      <xforms:bind nodeset="instance('loan')/currencies/currency[@att = 'calc']" 
          calculate="concat(../currency[1], ' or ', ../currency[2])">
      </xforms:bind>
    </xforms:model>
      
    <body>
      <label for="combo">Currency:</label>
      <select id="combo" dojoType="mvc.form.FilteringSelect" 
        model="loan_model" 
        nodeset="instance('loan')/currencies/currency"
        ref="instance('loan')/principal/@currency">
      </select>
    </body>
    

The ability for xforms:bind elements to maintain a dependency graph of dynamic data provides a convenient transformation layer between data structures appropriate to the backend business process or web service and those suitable for user interaction. Values may be aggregated or disaggregated using XForms on the way out to the user and inversely transformed and validated on the way back in toward the database or business process engine. Figure 8 and Figure 9 show chart and tabular views of a set of data from the XForms model, giving hypothetical revenue streams over time from 4 different geographies. In the views, we want to see just the total trends, summing over geography. The XForms model can do this sum easily using an XForms bind expression as shown in the code fragment below.

Figure 8: Dojo charting using data queried from XForms models

Figure 9: Dojo grid view using data queried from XForms models

The Dojo chart component requires input data to provide "x" and "y" properties which are computed by the two xforms:bind expressions shown below. The "x" position is simply the index of the month in the input data set, and the "y" value is the sum for each month over the regions where there is contributing revenue. The grid component is a flexible Dojo widget able to take input from data sources and generate HTML tables under the given div element with incremental updates as data elements change or rows are added and deleted.

As an additional affordance to AJAX developers, we have extended the Ubiquity data model runtime to populate the XPath-based data model by import and export of JSON values using a convention where properties are mapped to element names, structures to child content, and arrays to repeating elements. There are no standards yet controlling XML to JSON mapping (see JSON2XML for various examples) so this approach, while useful, is not exhaustive.

    <script id="json_sales" type="application/json">
    {
      "sales" : {
        "month" : [
          {
            "name"  : "January",
            "region" : [ "1", "2", "4", "1"],
            "comments" : "This was a really good month"
          },
          {
            "name"  : "February",
            "region" : [ "2", "1", "3", "1"],
            "comments" : "This was another good month"
          },
          {
            "name"  : "March",
            "region" : [ "1", "1", "1", "1"],
            "comments" : "Not so great..."
          },
          {
            "name"  : "April",
            "region" : [ "1", "2", "4", "1"],
            "comments" : "This was a really good month"
          },
          {
            "name"  : "May",
            "region" : [ "2", "2", "3", "2"],
            "comments" : "This was ok"
          }                             
        ]
      }
    }
    </script>
      
    <xforms:model id="loan_model">
      <xforms:instance id="sales" src="#json_sales"/>
      
      <xforms:bind nodeset="instance('sales')/month/x" calculate="position()"/>
      <xforms:bind nodeset="instance('sales')/month/y" calculate="sum(../region)"/>
    </xforms:model>
      
    <body>
      <div dojoType="dojox.charting.DataChart" 
        type="dojox.charting.plot2d.Columns" 
        store="loan_model"
        query="instance('sales')/month"
      </div>
        
      <div dojoType="dojox.grid.DataGrid" 
        store="loan_model"
        query="instance('sales')/month"
      </div>
    </body>
      
    

Next steps

Apart from providing an abstract set of UI controls, the XForms view layer specifies a means for relative data binding where XPath locators on child controls are evaluated relative to their parent control's context. As shown in Figure 2, control grouping can be used to define reusable content where child elements can bind anywhere in the model consistent with the relative offsets from their parent locators. Further, child bindings are reevaluated whenever parent bindings change. In this way, complex patterns such as Master/Detail UI Patterns views may be coordinated with a single change to the detail grouping control. We intend to explore further extensions to widget behavior in Dojo to support this hierarchical data binding behavior. In parallel, we will likely implement a set of simple container widgets which can provide grouping context and repetition over child content similar to the xforms:group and xforms:repeat elements.

Discussion and Conclusions

We reviewed briefly the Ubiquity XForms Ubiquity XForms AJAX implementation which embeds XML forms in web pages supported entirely by JavaScript -- i.e. without requiring any plug-in or extension technology. We described the use of AJAX widgets such as YUI calendars Figure 2 as custom controls rendering the concrete presentation and interaction with abstract XForms controls. We argued this "data to glass" style of authoring has considerable benefits in terms of simplicity and end-to-end consistency of an XML programming model. Nonetheless, we considered an inverse "glass to data" approach to design starting with native Dojo constructs familiar to current developers and working back towards implied data structures. As a preparatory step, we defined a rigorous model-view separation in Dojo and then used this pattern to support XForms-based data models with native Dojo controls as the view layer.

While not strictly necessary for integrating XML-based models into Dojo, we have focused on understanding and documenting model-view patterns in JavaScript-based Dojo programming as a key end in itself. We believe that defining and adopting such patterns will both benefit existing developer communities as well as speed the adoption of emerging XML-based technologies in the client over time. Given greater acceptance of the underlying architectural patterns in languages like XForms, albeit with script-based implementations as alternatives, the rate of the XML-based implementations of these same patterns should increase.

In real design, there is most likely a mixture of design styles which argue for both approaches -- data first and user experience first. In addition, there will likely remain a mixture of preferences for modeling technologies, using both XML and script-based programming models. Probably the key factor is to move to a more rigorous separation of concerns, in the direction attempted by this paper, such that a diversity of design methods and technologies can be adopted as fitting the requirements of the problem domain and skills or goals of designers and developers.

We see folks going "data to glass" that are working from existing enterprise application and expanding them to new web channels. They struggle with the new world, not only dealing with bridging their existing data to the web 2.0 world, but also moving from a server centric presentation technology to a client centric presentation technology. While it might seem likely that these customers would avoid the issue and let the data dictate the approach, they cannot as they have hired skills in Web 2.0 that may never fully understand the data. They need to ensure the data used in the Web 2.0 client has full fidelity with the enterprise data it represents as they cannot afford to maintain the mappings between multiple data formats. Data to glass design occurs frequently in behind the firewall applications focused on internal users. Its tempting to avoid using Web 2.0 constructs in such internal facing applications, but even internal users assume they will have Web 2.0 responsive and feeling applications as they use Web 2.0 consumer focused applications for mail, collaboration, travel booking, etc.

Contrarily, the more consumer focused applications tend to start "glass to data". While it might be tempting to discount such applications, one should realize the impact of presentation technology on server side implementations as well. Many times these glass data format of JSON is well understood by Web 2.0 developers and it's easy and convenient for them to justify extending this data concepts onto the server replacing the use of XML altogether. It is important to create this bridge from the glass to the data to allow the best technologies to be used in each tier. Without the bridge, technologies will be choosen out of what is easiest to get done the most quickly - not choosen upon technical merit.

Acknowledgements

We are grateful to our colleagues Mark Birbeck and John Boyer at the Ubiquity XForms open source project Ubiquity XForms for their innovation on the underlying AJAX-based implementation of XForms reviewed at the top of this paper. In addition, we have patterned the Dojo loan application after one written originally for Ubiquity by John Boyer and reused the style sheet work for the loan as implemented by Erik Johnson. We thank the reviewers for their careful and thoughtful comments on the paper content and style, all faults in executing on their suggestions remaining of course with the authors.