Generating Schema-Aware XML Editors in XForms

Mustapha Maalej

Technische Universität München

Anne Brüggemann-Klein

Technische Universität München

Copyright © 2013 by the authors. Used with permission.

expand Abstract

Balisage logo

Proceedings

expand How to cite this paper

Generating Schema-Aware XML Editors in XForms

International Symposium on Native XML User Interfaces
August 5, 2013

Introduction

In his PhD work [Maa13], the first author of this paper, under the second author's supervision, is designing and implementing a system called XFGen, that generates a schema-aware XML editor XFGen(s) for each XSD schema s. The key achievement of this work is that the editor XFGen(s) is an XForms document that is capable of creating or loading, editing and saving any XML instance that conforms to the schema s; each user interaction with XFGen(s) preserves validity of the XML instance against schema s. XFGen(s) is much more than a form editor. It goes beyond letting users fill out data fields for a structurally static XML document in that it enables editing of structure, too. Each editor XFGen(s) is indeed a fully functional schema-aware XML editor. Here are some crucial properties of the editors that XFGen generates:

  • Strictly schema-aware (schema instances always in schema-conformant state).

  • Implemented as an (extended) XForms document.

  • Supporting a large part of XML Schema.

  • Completely independent of XML instances, capable of loading or creating from scratch, editing and saving any schema-conformant document.

The generator XFGen covers nearly the complete XML Schema specification. It handles, most importantly:

  • Elements declared as complex types

  • Elements declared with simple content

  • Recursive type definitions

  • Mixed-content declarations

  • Multiple potential top-level elements (elements declared globally)

  • Identity constraints

  • Attribute declarations

  • Predefined simple types

  • A wide range of facets in simple type restrictions

  • Union of simple types

  • Lists of simple type

Other parts of XML Schema are also supported, but did not require great effort, since their support mostly rests on standard schema validation. They are: inheritance, substitution groups, namespaces, inclusions, attribute and element groups. Furthermore, we wish to emphasize that element and attribute declarations as well as type definitions can all be local or global.

The remainder of this paper is organized into five sections as follows: The next section is about architecture; it illustrates the interplay between components and briefly describes the architecture of the editors that XFGen generates. The main part of the paper is a tour of principles that we have used with XFGen; we cover editing of data values, editing of non-recursive structures and editing of mixed content. After that, we have sections about implementation and related work, before we conclude. For further illustration, we provide an appendix with editor screen shots for the purchase order example from the XML Schema Recommendation WF04.

The complete XForms generator XFGen is ready for demo at Balisage 2013. In this paper, we point out some of the challenges of XForms as an implementation technology for a schema-aware editor and demonstrate some of the principles and techniques that make such editors possible. A complete description can be found in Maalej's PhD thesis [Maa13].

Architecture

Components and their interactions are illustrated in Figure 1.

Figure 1: Components and their interaction

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-001.png

The editors (XForms documents) that XFGen produces are complex pieces of software that conceptually follow the Model-View-Controller (MVC) architecture. They are realized as extended XForms.

The data of each editor consist of the XML instance, that is to be edited and can be loaded, freshly created, replaced and saved on demand, and of some auxiliary state, comprising among others prototype structures that can be copied into the XML instance during editing under action control. The editor's data are contained in the data-model section of the form. The form's data model also holds declarative constraints for the editor's data, expressed as XForms bindings, and actions that will be triggered during the form's life, for example for editing. Conceptually, with respect to MVC, some of the latter are part of the controller.

The view of each editor is defined by templates of XForms widgets, that are embedded into a host language, most commonly HTML, but we have also tested XUL. The view is generated by XFGen.

The controller of each editor consists of XForms actions and custom scripts that XFGen inserts into XForms; they are executed by the XForms engine in the XForms client.

A tour of principles for XForms editors

What are the principles that govern how the XFGen-generated XForms editors are built? In this section we illustrate them incrementally by example. We show the XForms code that is generated and how it is executed in a browser, for a series of XSD examples. And we explain systematically the underlying principles.

Displaying structures, editing data values

We first present simplified editors that allow editing of purely textual element content, but just display the document structure, not yet enabling structural editing. These editors could be used as classical form editors for a structurally-fixed XML-encoded form. At first, we restrict ourselves to elements. This is not a severe restriction; it will be easy to add attributes and editing support for them. Furthermore, we do not allow mixed content yet. Finally, we exclude recursive type definitions for now; this feature of XML Schema requires further techniques, that fall outside the scope of this section.

The challenge of this section is to generate a static structure of XForms widgets that only depends on the input schema but that is able to bind to any XML instance, that conforms to the schema, and to display it. The key idea is to generate a liberal structure of XForms widgets that is capable of displaying a superset of the required XML instances. For example, if the schema uses a choice operator, we generate widgets for all alternatives. Each widget tries to bind to some element in the XML instance, but only some of them succeed, depending on the choice that the current instance realizes. We rely on the fact that "superfluous" widgets do not display when the referenced nodes in the current XML instance do not exist. We call this principle Liberal Inputs.

Let us look at the schema liberalInputsS, see Figure 2. It provides one root element, xyz, that can have one or two subsequences of one or two subelements a and one or two subelements b. Element a is typed as xsd:int and element b is typed as xsd:boolean. Now we are looking at the editor that we have generated, the XForms document liberalInputsF, see Figure 3: It has a repeating group of one widget refering to an element a and one widget refering to an element b, each of which enables editing of the element's text content. Finally, let us load the editor with the XML instance liberalInputsI in Figure 4 that conforms to the schema liberalInputsS. Then, the XForms element xforms:repeat iterates over all children of element xyz in the current XML instance, displaying the appropriate input widget for each element a or b that is met, as illustrated in Figure 5. This form could in principle display any sequence of elements a and b, even those that do not conform to the type of element xyz, depending on the XML instance. But we may assume that the XML instance is valid with respect to the schema when it is loaded; and the user interactions that we will later introduce, always preserve validity. Hence, it does not matter that the widget structure is more liberal than the schema.

Figure 2: XML Schema liberalInputsS

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-002.png

Figure 3: XForms liberalInputsF

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-003.png

Figure 4: XML instance liberalInputsI

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-004.png

Figure 5: Editor screen shot liberalInputsB

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-005.png

Some further points and easy generalizations:

  • XForms processors provide some processor-specific type-aware editing support and complete validation for data values that are typed with pre-defined XML Schema simple types. Hence, the example editor liberalInputsF in Figure 3 supports XForms processor-dependent type-aware editing and complete validation of the contents of elements a and b out of the box, as seen in Figure 5. Custom simple types, which are defined using restriction, union or list, require special treatment that is described in Maalej's PhD thesis [Maa13].

  • If a schema has several globally declared elements, each of them is allowed as the top-most element of a schema-conformant XML instance. Following the Liberal Inputs principle, we generate a set of widgets for each of these elements, and only one of them will display for any given XML instance that is loaded into the editor, because only one of them will successfully bind to the unique top-level element of the current XML instance. Consequently, we can load the editor with any schema-conformant document, regardless of its root element, and we can also switch to a different root element during an edit session without changing the editor.

  • The same idea as in the previous item is used when a type definition contains the choice operator.

  • What if the schema allows for deeper element hierarchies? We must ensure that bindings from widget elements into the XML instance can be expressed in XPath, without consideration for expressive features of XML Schema such as context-dependent element declarations. We achieve that by having the hierarchy of XForms widgets mirror the element hierarchy of the schema, following the XForms pattern Stepwise XPath [D2003]]. Then bindings into the XML instance are always defined relatively to the parent level, by element name only. There is a catch, though: If two top-level elements of a type definition have the same name but different types, then we have to generate two different widget structures for them and cannot bind them into the XML instance by the (ambiguous) element name. Fortunately, XML Schema forbids this type of ambiguity.

  • Why did we preclude recursion in type definitions for the simple schemas that we can handle in this section? Recursion in a type definition would lead to an infinite, non-halting generation of widget structures. To stop recursion when generating XForms widget structures, widgets would have to refer to and reuse previously defined structures, a feature not provided by XForms. Our solution is to expand XForms with a new type of reference control and support it with an extension to the XForms processor. This principle, Reference Control, is further explained in Maalej's PhD thesis [Maa13].

  • It is easy to extend our approach so far to attributes, by adding a set of input widgets to the group of each element declaration, one for each potential attribute. Once more, the Liberal Inputs principle applies.

  • Just for purposes of display, we could also handle mixed content in element instances, by letting the xforms:repeat iterate not only over sub-elements but also over text node children. We use a different technique, though, for supporting insert and delete of text nodes, which we explain later in this paper.

We can now algorithmically describe how to generate an XForms editor that allows editing of text content but only displays structure [Maa13].

Editing structures: The downwards-facing perspective

One novelty of this work is that our editors support editing not only of data values, as demonstrated in the previous section, but also of structures. XForms provides some basic support for insertion and deletion of nodes in an XML instance, with its actions xforms:insert and xforms:delete that can be triggered under user control. The challenge is to make sure that editors only allow for schema-conformant changes of XML instances.

In this section, we take the downwards-facing perspective. We demonstrate our Prototype principle, which guarantees that only such element structures are inserted into an XML instance that by themselves conform to their declared type. We'll address the upward-facing perspective, how deletions and insertions of children nodes can be forced to respect their parent's type, in the next section.

If we wish to insert an element that conforms to some type into an XML instance, we'll insert a whole structure, with subelements and attributes as required by the type. We precompute in XFGen one minimal structure that conforms to the type and call it the element's prototype [Maa13].

The xforms:insert action cannot create new structures; rather, it can only clone structures that are already present somewhere in the form's model. Therefore, XFGen builds an auxiliary instance that holds all prototype element structures and attributes of the schema, which can then be cloned and inserted into the form's XML instance under action control.

Once a prototype has been inserted into the XML instance, the user can further edit it. Right now, XFGen computes some arbitrary prototype that conforms to the required type definition and cannot be further reduced by omitting attributes or subelements. The schema author can influence which prototype is generated by the order of choice operands in a type definition. Currently, XFGen always instantiates the first choice operand for a prototype.

The auxilliary instance with the prototypes for Schema liberalInputsS in Figure 2 is in Figure 6.

Figure 6: Auxilliary instance liberalInputsP

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-006.png

Editing structures: The upward-facing perspective

We now address the problem how to support insertion and deletion of children nodes such that the result necessarily conforms to the parent node's type.

Let us consider schema editControlsS in Figure 7, that allows top-level element xyz to have subelements a, b and c with the following additional constraints: The subelements of xyz either form a nonempty sequence of a, followed by a nonempty sequence of b, or consist just of a single c. In compact regular-expression notation, that is (a+b+)|c.

Figure 7: Schema editControlsS

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-007.png

Our goal is to offer a finite number of primitive edit operations, so that any schema-conformant sequence u1...um of children of xyz can be transformed into any other schema-conformant sequence v1...vn by applying a finite number of the primitive edit operations, one after another, in such a way that each intermediate step leads also to a schema-conformant intermediate sequence.

In our example, we generate the following set of primitive edit operations:

  • P1: insert a

  • P2: insert b

  • P3: delete a

  • P4: delete b

  • P5: delete ab, insert c

  • P6: delete c, insert ab

  • P7: insert c

  • P8: delete c

  • P9: insert ab

  • P10: delete ab

For simplicity's sake, we include "P7: insert c" in our supply of primitive edit operations, although it can never be used in a schema-conformant transformation. We can only insert a c if we also delete every a and b that might be present. And we can break down such a combined operation into a number of primitive edits, first deleting any single a and b except one, respectively, with P3 and P4, and then deleting the last remaining sequence ab and simultaneously inserting c with P5. Following the same argument, P8, P9 and P10 can also never be used in a schema-conformant transformation.

We cannot do without primitive P5 simulating P5 by the sequence P3 P4 P7, since intermediate states would not be schema-conformant.

Of course, unwise application of primitive edit operations can lead to non-schema-conformant states. If we apply P5 in the middle of sequence aab, between a and b, we erroneously get the invalid sequence ac.

Our claim is the following: For any complex type definition, we can compute a finite number of primitive edit operations such that we can transform any conformant sequence into any other conformant sequence using the primitive edit operations and having only conformant intermediate states. We stress once more that this is an "existential" claim. We do not care at this point, that our primitive edit operations can also generate non-conformant sequences when applied at wrong positions. Maalej [Maa13] has the algorithm to generate a sufficiently large but finite set of primitive edit operations. Further research is needed to investigate if one can compute a minimal such set or if a minimal set would be unique.

Following a principle that we call Liberal Edits, we include any of the primitive edit operations as one button in the editor's user interface at any position in the sequence of children of xyz, see editControlsUI.xml.

We can do this generically, without refering to a specific instance, by utilizing xforms:repeat, as previously. Note that we insert one set of buttons outside the xforms:repeat to handle the beginning of the sequence.

Finally, we still need to control which of the edit buttons that we have so liberally included in our editor are actually active and which are passive (not shown), to preclude illegal edits that lead to non-conformant documents.

In our example, we could do this with XPath bindings into the instance. We can, for example, express that P3 is only active if there is another a among the children of xyz, apart from the one that is to be deleted.

However, an educated guess leads us to believe that, in the general case, the expressive power of XPath is too weak compared to the power of regular expressions in XML Schema complex types. Hence, we offer a different approach that we call Try and Tell.

We bind to each edit button a script that experimentally performs the primitive edit operation on a copy of the current instance, validates the result and makes the button visible only if the result is in fact valid. These scripts are triggered by XForms refresh events that are activated after each user interaction, ensuring up-to-date visibility status of each edit button. The scripts are implemented in a scripting language for which the XForms processor provides an interface, in our case in XBL (see also the section on implementation below.

We illustrate the effect of Try and Tell for schema editControls in Figure 7 with a browser screenshot in Figure 8.

Figure 8: Editor screen shot editControlsB

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-008.png

Editing mixed content

In XML Schema, elements that are typed to have element content can orthogonally be declared to be of mixed content, allowing text-node children to be interspersed at any position in an element's instantiation, at the beginning, the end and between element nodes. This is in contrast to the more powerful Relax NG, where the appearance of text nodes can be constrained by regular-expression rules in the same way as subelements. The orthogonal approach of XML Schema opens up the opportunity to decouple handling of text nodes in mixed content from the handling of structured content, in a simpler and platform-independent way.

In a first attempt, we apply our principle Liberal Inputs to elements that are declared to be of mixed content, iterating in the form's user interface not only over the elements' child elements but also the child text nodes, generating input widgets in the user interface for display and editing of these text nodes. This will display and make editable any text nodes that are present in the current document instance but does not handle positions in the document instance where text nodes are allowed but do currently not exist. For this case, we introduce a new principle, Automatic Text Insertion. When the document instance is loaded into the form, we automatically trigger a script (written in JavaScript) that inserts dummy empty text nodes at any position where a text node is allowed by the schema but none is present in the instance. Then, input widgets appear at any position in the user interface, where text nodes are allowed by the schema, displaying an empty input field for the dummy text nodes and the original text for text nodes that were already present in the document instance. These text nodes can be edited via the input widgets; they can also be "deleted" by resetting their content to the empty string.

Unfortunately, there is a glitch with this approch: The iteration in the form's user interface will also produce the set of buttons for primitive edit operations that would consequently also have to be managed. There is, however, a slightly different way to deal with text nodes without considering buttons for primitive edit operations: We couple the widgets for text nodes to the groups for element nodes within an xforms:repeat and insert a further widget for a text node outside the xforms:repeat, to handle the text node that appears before the first element. We call this principle Coupled Inputs and demonstrate its application with schema coupledInputsS in Figure 9, XForms coupledInputsF in Figure 10 and browser screen shot coupledInputsB in Figure 11.

Figure 9: Schema coupledInputsS

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-009.png

Figure 10: XForms coupledInputsF

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-010.png

Figure 11: Editor screen shot CoupledInputsB

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-011.png

The last point to consider is how structural edit operations deal with text nodes. Here we modify the edit operation in such a way that empty text nodes are inserted as needed and that the contents of text nodes that are deleted together with a sequence of neighboring element nodes are copied into the remaining text node in front of them.

This solution for mixed content is essentially platform independent. It uses a custom JavaScript script that is triggered by a built-in XForms event and accesses the document instance through the standard DOM interface. Furthermore, it utilizes the capability of XForms to associate the script with the event. It is worth mentioning, though, that XForms makes it really hard to insert text nodes in the middle of a sequence of nodes, demanding to sequentially build up of the sequence from front to back.

Implementation

The system XFGen itself is a standard Java program. it uses Xerces to process XML Schema. The XForms documents that XFGen generates require extensions for some XSD features (recursive type definitions) or editing tasks (experimental evaluation to determine admissible edit operations dynamically). These extensions utilize XBL scripts that interact with certain data structures and methods of the XForms processor [Hic12]

XBL is a scripting language that was introduced by Mozilla and submitted to W3C, but has not been standardized by W3C. Some XForms processors, among them Orbeon Forms, offer interfaces for XBL scripts that can access internal data structures and functions of the XForms processor. These interfaces are not standardized. We have extended the Firefox plugin XForms processor, which incidentally is programmed in XBL itself, with XBL functions. Consequently, the more complex XForms documents that XFGen produces run only on this custom extension of the XForms Firefox plugin on old versions of Firefox. We refer to Dubinko's text on extending XForms [D2003].]

Some XSD features can be more fully supported with an XForms processor that allows for XPath 2.0. Unfortunately, the Firefox plugin XForms processor that we use only supports XPath 1.0, as required by XForms. Consequently, support for some XSD features such as identity constraints is more cumbersome to define or even more limited in practice than conceptually necessary.

Related work

We briefly discuss three papers that are related to our work [RRK05][SL07][WKdW04].

Despite its title and stated intent, the paper by Radha [RRK05] and others does not really contribute anything specific to user interfaces. It is mainly concerned with semantic interpretation of XML Schema, given a generic DOM representation that sees the schema just as any XML document. A Java Swing user interface is presented without discussion how it was generated.

Song and Lee [SL07] specifically address the XForms target platform. Their goal is to support user interfaces for Web Services. They also briefly address the task of semantic interpretation of XML Schema. As to schemas, we don't see that they support recursive type definitions. The editors that are generated can only generate new XML documents from scratch, not load existing documents. Editing of structures and of mixed content is supported, but only in a "one-way" approach; that is, edit decisions, for example for one alternative when a choice is given, cannot be revised. There is some support for custom simple data types, for some facets in restrictions and for union. The paper mentions lists but means presumably the enumeration facet, since only a finite number of items in the base type can be supported with xforms:select1.

De Wolf and others [WKdW04] discuss problems that need to be solved when XForms is used as an implementation platform for an XML editor. They seek solutions mostly in extensions to the XForms standard. Some but not all of their proposed extensions have in fact found their way into the XForms 1.1 specification.

Our system XFGen supports a far greater range of XML Schema features than comparable systems. We delegate the semantic interpretation of XML Schema to Xerces. We have explicitly delineated some principles and algorithms of XFGen; further capabilities of XFGen are covered by Maalej in his PhD work [Maa13].

Discussion, conclusions and further work

Maalej in his PhD thesis [Maa13] covers further features of XML Schema:

  • Recursion in type definitions.

  • Custom simple types (restriction, list, union).

  • Identity constraints.

In principle, the XML editors that are generated by XFGen could be platform independent, but currently, they are not. We would need an alternative XForms processor that allows the necessary extensions and also runs in current browser. One candidate would be xf.js [NBK13] in a more fully functional version, where we could use JavaScript to support the extensions.

In this paper, we have not discussed the user-interface aspects of the editors that XFGen generates. We follow a template approach as indicated in Figure 1. For the examples in this paper, we have defined basic templates with the HTML fieldset element. Obviously, more sophisticated custom templates that combine HTML with CSS need to be tried out.

Although XFGen handles mixed content, the generated editors presumably work better for data-driven XML instances than for text-driven ones. Particularly with text-driven XML documents, we run into the largely unsolved usability problems of general XML editors. Our generated editors work probably best in cases in which a classical form-based interface is appropriate. Investigate further.

Further documentation of this work will appear in Maalej's PhD thesis [Maa13].

Acknowledgement

The comments of the anonymous referees have been extraordinarily helpful. Thank you!

Appendix A. The purchase order example

For further illustration, we include the purchase order example from the XML Schema Recommendation WF04 with the schema in Figure 12, an instance in Figure 13 and two browser screen shots in Figure 14.

Figure 12: Schema purchaseOrderS

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-012.png

Figure 13: XML instance purchaseOrderI

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-013.png

Figure 14: Editor screen shots purchaseOrderB

png image ../../../vol11/graphics/Bruggemann-Klein01/Bruggemann-Klein01-014.png

References

[Boy09] John M. Boyer, XForms 1.1, W3C Recommendation, W3C, October 2009, http://www.w3.org/TR/2009/REC-xforms-20091020/.

[BPM04] Paul V. Biron, Kaiser Permanente, and Ashok Malhotra, XML Schema Part 2: Datatypes Second Edition, W3C Recommendation, W3C, October 2004, http://www.w3.org/TR/2004/REC-xmlschema-2-20041028/.

[Dub03] Micah Dubinko, XForms Essentials, O’Reilly & Associates, Inc., Sebastopol, CA, USA, 2003.

[Hic12] Ian Hickson, XBL 2.0, W3C Candidate Recommendation, W3C, Mai 2012, http://www.w3.org/TR/xbl/.

[Maa13] Mustapha Maalej, Generieren von XML-Editoren in XForms aus XML Schema, Ph.D. Thesis, TU München, 2013, In preparation.

[NBK13] Tobias Niedl and Anne Brüggemann-Klein, Processing XForms in HTML5-Enabled Browsers, Balisage 2013, 2013. doi:10.4242/BalisageVol10.Niedl01.

[RRK05] V. Radha, S. Ramakrishna, and N. Pradeep Kumar, Generic XML Schema Definition (XSD) to GUI Translator., ICDCIT, Lecture Notes in Computer Science, vol. 3816, Springer, 2005, pp. 290–296. doi:10.1007/11604655_33.

[SL07] Kisub Song and Kyong-Ho Lee, An Automated Generation of XForms Interfaces for Web Services, 2012 IEEE 19th International Conference on Web Services (2007), 856–863. doi:10.1109/ICWS.2007.35

[TBMM04] Henry S. Thompson, David Beech, Murray Maloney, and Noah Mendelsohn, XML Schema Part 1: Structures Second Edition, W3C Recommendation, W3C, October 2004, http://www.w3.org/TR/2004/REC-xmlschema-1-20041028/.

[WF04] Priscilla Walmsley and David C. Fallside, XML Schema Part 0: Primer Second Edition, W3C Recommendation, W3C, October 2004, http://www.w3.org/TR/2004/REC-xmlschema-0-20041028/.

[WKdW04] Koen De Wolf, Frederik De Keukelaere, and Rik Van de Walle, Generic XForms-Based User Interface Generation for XML Schema, Proceedings of the IADIS International Conference e-Society 2004, 7 2004, pp. 773–782.

Author's keywords for this paper: XML editor; XForms; XML Schema