Chakkera, Vasu. “Documentation of XSLTs with Code Intelligence.” Presented at Balisage: The Markup Conference 2018, Washington, DC, July 31 - August 3, 2018. In Proceedings of Balisage: The Markup Conference 2018. Balisage Series on Markup Technologies, vol. 21 (2018). https://doi.org/10.4242/BalisageVol21.Chakkera01.
Balisage: The Markup Conference 2018 July 31 - August 3, 2018
Balisage Paper: Documentation of XSLTs with Code Intelligence
Vasu Chakkera is an XML XSLT enthusiast and has been working with the XML and
related technologies since 2001.
His initial introduction to XSLT was in BBC worldwide, where he joined as an
XSLT developer for their TV Data system. He is currenty a Sr.Platform Specialist
(XML Technologies ) with Sapient Corporation.
We benefit more from documenting why certain functionality was implemented, or
coded in a particular way in an XSLT stylesheet, than from the typical “what the
code does” comment. K7:XSLTDocEngine is a personal project (non-commercial) to
create XSLT stylesheet documentation from both inline stylesheet comments and
documentation living outside the stylesheet. The external documentation lives in XML
files, written in a variant of DocBook, that are generated by script and populated
by XSLT analysts. These files are then used to generate configurable HTML
documentation that provides the text as well as 1) hyperlinks to named templates,
global variables and functions, imported/included templates and 2) reports of code
violations such as potentially overridden functions, single-expressions, unused
variables, and the like. Code violation criteria are defined in user-configurable
rule sets.
A big set of business functionalities and complexities is often hidden inside a XSLT
code base. As time goes by, it is very difficult to remember the background of why
certain functionality was built in and why the said functionality was translated into
the XSLT code in a way that it was coded. Here is where the documentation of the code
comes in very handy to revisit the history of the code and the business functionalities
that governed the code.
As a common practice, most developers tend to write the business within the comments
in the XSLT file itself. These are OK, if the comments are small, but not so legible,
if
we have to write some background information. We benefit more from
documenting why certain functionality was implemented, or coded
in a particular way in an XSLT stylesheet, than from the typical “what the
code does” comment. K7: XSLTDocEngine is a personal project
(non-commercial) to create XSLT stylesheet documentation from both inline stylesheet
comments and documentation living outside the stylesheet. The external documentation
lives in XML files, written in a variant of DocBook, which are populated by XSLT
analysts. The framework that generates the HTML documentation files pulls information
from these files and puts them in the appropriate place in the documentation (for
example, the documentation regarding a specific function will be pulled from the
external document config file, into the body of the function documentation).
Although this presentation has some parts that describes the generation of the HTML
documentation out of XSLT files, the HTML documentation will be used to highlight
some
of the techniques that I had devised to make documentation more effective.
The following topics will be the focus of this presentation.
Keeping the XSLT documentation out of the XSLT file
Code quality check based on predetermined set of rules
Hyperlinking variables and functions to their definitions
The above features will be shown using the HTML Documentation Pages that K7:
XSLTDocEngine creates. Specifically, the HTML Documentation will
Hyperlink all imported/ included XSLT files
Document every template and function
Document variable definition
Hyperlink the variables, functions and templates to where they are defined.
This will help in easy lookup, when the XSLT file is big and searching for
variable definition is time consuming.
To summarize, we need a code documentation system, which can
• Allow for extensive verbose documentation, keeping the code legible
• Provide brows ability in the code by linking the variables to their definitions
• Provide Code Violation reports, for better code management
Although there are other XSLT Documentation tools available, some of the features
mentioned above, are special in K7: XSLTDocEngine.
While the HTML document is used to display some important documentation techniques,
the HTML document itself is not the focus of the presentation. The look and feel can
be
a lot better and a lot of advanced UI APIs can be used to make documentation look
sleeker.
Document Generation Process
The following illustration shows the document generation
process
Figure 1: Hyperlinking Imported and Included templates
K7: XSLTDocEngine works in three parts:
1. Doc Config XML: Create the XSLT doc config xml for every XSLT file in the code
base
2. XSLT HTML Doc: Create the XSLT doc HTML
3. Code Violations Report: Create Code Violations report, organized with files and
folders. It also reports where the code violations occurred and number of code
violations in each file.
Doc Config XML
Objective: Allow for extensive verbose
documentation, keeping the code legible
The following sections describe how the above objective is achieved.
A script first creates skeleton document config XML files for additional
documentation of Stylesheets,
Templates, Functions and Variables.
What are document config XML files?
document-config XML files are the placeholders for the documentation of the XSLTs.
The documentation in these sections will be written using variant of docbook xml
format. If the analyst wanted to write some history about some function or template,
then this is the place. A script runs through all the XSLT files in the XSLT
codebase and creates skeleton XSLT documentation config files with file name
convention of xsltfilename.doc.stub.xml. The
contents of this file will essentially have sections to document every template in
the XSLT, every function in the XSLT and every variable in the XSLT.
Where will this file be used?
These XSLT doc config files will be used by the XSLT file that creates the
document HTML Files. While documenting the templates, it will check if there was a
documentation defined in the config xml file. If the analyst has defined a
documentation, then the Documentor XSLT will pull the documentation information into
the HTML it produces.
Why shouldn’t I write documentation within the XSLT itself
Mainly, because writing big stories within the XSLT makes the XSLT big and
sometimes obstructs the reader from following the code. In addition, this step
ensures separating the code from its documentation. Documentations here can be
verbose
Example of doc config file:
Figure 2: Hyperlinking Imported and Included templates
XSLT HTML Doc
This is the actual documentation of each XSLT in the code base. A script runs the
Documentation XSLT through all the files in the code base and creates doc HTMLs for
all the XSLT files in the codebase
➢ Within each Document HTML the following will be taken care of.
Every import and include is hyperlinked to the corresponding documentation
HTML
Table of contents will be created for easy navigation to
Named Templates
Matched Templates
Global Variables
Version Control
If a certain variable, template or function is used within an XPATHS or
anywhere in the code, then it will be hyperlinked to the place in the XSLT
documentation, where the variable is defined. This will provide better code
navigation and browsability
The documentation of the Templates, Functions and Variables will include
the documentation added in the external document config file
Summary of process flow
“Doc config file generator” (batch file) runs on all the XSLT files and
creates doc-config-xml files. The file naming convention will be “xsltfilename.doc.stub.xml” within the doc.config folder. The batch file runs the
DocumentConfigStubCreator.xsl
The analyst opens this file in an XML editor and fills in the information
regarding the XSLT file, its functionality and any important background
information that may be important for understanding the context of why the
template of function was written. This file is generated in a variant of
docbook schema. Once the necessary information has been added to the
doc-config xml, it is then saved without the “stub” in the filename for
example: xsltfilename.doc.xml. This
renaming is done to avoid any accidental overwriting of the edited
file.
HTML file generator scripts is then run on the XSLT codebase, which runs
the XSLTDocumentationBuilder.xsl.
DocumentationBuilder.xsl does the following:
Creates a HTML file for the XSLT file
Within each HTML file, it creates individual sections for
variables, templates, functions, import/include sections.
While documenting the functions templates and variables,
DocumentationBuilder.xsl hyperlinks all the variables to where they
are defined.
Since the DocumentationBuilder.xsl knows the XSLT it is currently
documenting, it also has a handle to the previously generated
doc-config-xml file of the XSLT file being processed.
While processing the functions and templates within the XSLT file,
DocumentationBuilder.xsl looks into the external doc config XML file
for a template or function with the same name as the one it is
currently processing
If it finds one, then it brings the contents of the external doc
config XML file into the HTML documentation.
You can see how the documents are arranged. You can also get some insight into
some of the functionalities described above.
The following images show some of the features mentioned
above.
Bringing in Code documentation into the HTML, from external document
config file
Hyperlinking global variables and functions to where they are
defined.
Code Violations Report
As with any development team, increase in team size brings complexity of code
management and introduces inconsistencies in the code base. Some of these
inconsistencies and bad coding practices creeps in the code base are:
Namespaces that are declared but not being used in the code
Choose expressions with just one condition
Variables that are never used
XPaths or parts of XPaths that are repetitively being called in the
stylesheets, without effective use of variables.
Inconsistent naming conventions
These are some of the issues that can be caught in the code violations
report.
To address the above issues, the Code Violation Report Generator goes through
every file in the code base and creates an index HTML with File name and number of
code violations. The code violations are caught using XSLTs with rule set defined
to
address the code violations.
Each file in the above report is also hyperlinked to corresponding Code-Violations
html, which would give more detailed information on what the code violations are and
what needs to be done to fix them.
Files used in documentation process
create-doc-config.bat : Runs on the XSLT
codebase and applies DocumentConfigStubCreator.xsl
on the XSLT files in the codebase
DocumentConfigStubCreator.xsl: Creates
the document config xml.
create-html-documentation.bat: Runs on
the XSLT codebase and applies XSLTDocumentationBuilder.xsl on the XSLT files in the
codebase
XSLTDocumentationBuilder.xsl: Creates the
actual HTML documentation files.
Running K7: XSLTDocEngine
Following are the steps to run K7: XSLTDocEngine
1. Run create-doc-config.bat: This will create a
folder with all the skeleton document config xmls for every XSLT in the codebase.
This is a onetime process. There are options in the script to run selected set of
files only, for any future newly added XSLT files
2. Run create-html-documentation.bat. This will
create HTMLs for every XSLT in the codebase
Conclusion
The process described in this paper is currently being used and roughly, 600 to
800 XSLTs are auto generated and maintained. Taking large chunks of documentation
out of the XSLTs and into a config file has given the developers the freedom to
start writing extended stories about certain functionalities.
The Code Violations are being chased regularly, to make sure the coding standards
are followed
The Triage Analysts are finding it extremely useful to browse through
documentation files, instead of going through the code.
As with any process, the single most important thing that will make system work is
the discipline and resolve to keep following the best practices.
Reference
For the illustration of functionalities described in the paper, I needed some
example XSLT functions and found them in the following resources.
