Balisage Paper: Looking for Rumpelstiltskin

Unstyled Microsoft Word Documents

Microsoft Word documents present the biggest challenge. One of the product teams authors and maintains content as Microsoft Word documents (System D in Table I). The collection includes short articles, Q & A, Recipes, and other wellness information that is combined into newsletters, magazines, e-blasts, or other custom publications for their clients. The Word files are often shared with clients beforehand, who may request changes to the content prior to incorporation in the final deliverable.

The articles themselves are short – fewer than 3 double-spaced pages in length – and inconsistently styled (see Figure 1 through Figure 5). The most commonly accepted approach when dealing with Microsoft Word is to develop a set of named styles (as opposed to the toolbar icons or the generic 'Heading 1' through 'Heading n') that mimic, to a greater or lesser extent, the XML markup that should be applied to a particular piece of content. There are a number of products in the marketplace that act as add-ins to Microsoft Word that customize the toolbar, restrict style usage, and through some behind-the-scenes coding, detect various bits of content and automatically apply the appropriate style. This is coupled with an embedded transformation to a given XML vocabulary; the end result being a valid – and hopefully correct – XML instance. In this particular case, neither the expense nor time needed to configure such applications was deemed worthwhile; instead it was decided that the first set of documents to be converted (approximately 3500 files) would be styled manually.

The DITA for Publishers Toolkit provides a set of utilities to transform Microsoft Word styles into DITA markup. The only pre-requisite is that your Microsoft Word document has been styled using named styles and is saved in Word's XML format (.docx).

Step 1: Since styles weren't used to create the original documents, a set of styles needed to be created and then applied to the documents. Samples covering the various document types were provided and Microsoft Word templates were created.

Step 2: A style to tag map file was created and tested with the sample set (see Figure 6.

Step 3: After several review sessions, a training session was held (and recorded), and the team set about styling the initial batch for conversion.

<paragraphStyle level="1" structureType="topic" styleName="TOPIC - Section" tagName="title" outputclass="Section">
<topicProperties bodyType="body" format="topic" topicDoc="no" topicType="topic" outputclass="TOPIC-Section"/>
</paragraphStyle>
   

Step 4: Run the transformations and QC the results.

The styling was fairly straightforward, however, since no toolbar customizations or other developent was done, users weren't prevented from using toolbar shortcuts for bold, italic, lists, etc., or from choosing character styles rather than an appropriate heading style. In some instances, Microsoft Word-supplied styles were used rather than the custom styles. This resulted in character styling being dropped (while text was not lost, bold or italic was not carried forward), or hierarchical structures not being created (a paragraph styled as body text with the character style "bold" applied to the entire content is not the same as the paragraph style "Topic - Section"), and in the worst case scenario, mis-styled content being dropped. While the resulting XML is valid, it isn't correct.

Result: The content will need some additional cleanup – either by fixing the Microsoft Word document and re-executing the transform, or editing the resulting XML. In this particular case, since the Microsoft Word documents will continue to be used until the project team is migrated off of their existing platform, it makes sense to restyle the Word files.

HTML

Another product team creates and manages content related to diseases and conditions, tests and procedures, prevention, management, and care, targeted to the patient, their caregiver, or family members. This content is often presented as a collection of articles, integrated into your health insurance provider’s, medical association’s, or practice’s website, discharge instructions received after a visit to the ER or hospital procedure, or hand-outs after a medical appointment. All content in this group must be reviewed at least every two years, or whenever a change in protocol is recommended. There are three different systems currently in use – one used to track the review cycles, support an online medical review process, and maintain historical versions of record, and two other systems that are customer-facing and support a number of product offerings. Two of the three systems are scheduled to be retired (See Systems A, B, and C in Table I).

The two customer-facing systems serve much of the same content to different audiences. The older system (B) uses a custom schema to describe article content (see Figure 7 – it's mostly HTML, but limits certain functionality and incorporates several non-HTML elements to support the downstream applications.

Figure 7: Content Schema

While it's still possible to create a heading by applying <bold> to an entire paragraph, users are much more likely to use the well-known <h#> elements to identify the start of a section. What isn't typical, however, is for users to follow the logical progression from <h1> to <h2> to <h3>, etc. Instead, depending on how they think the heading should be styled, they might skip a level or two, progressing from <h1> to <h3>, or skipping <h1> entirely and starting with <h2> or <h3>. Throw the <Section> element into the mix, and the task becomes much more complex. Thankfully, the <div> element was excluded from the schema.

The newer system (C) does not use a schema; instead, it uses the HTML5 tag set, and supports modular content types not possible in the older system. The markup is much more complex, including the use of custom data attributes (<data-*) and an expanded use of @class attributes to further identify the content. The entire archive needed to be searched to discover all of the data attributes and class attribute values to ensure they were accounted for in the transform and that no information would be overlooked.

This system also supports a segmented content model that explicitly identifies sections that can be eliminated or revised at the client’s discretion. The <div> element is supported as is <section>, and segments may be further broken down by any one of <h#>, <div>, or <section> elements, in any combination. These articles may have been in existence for quite some time; there is a requirement that they be medically reviewed at least once every two years and updated in accordance with the latest primary source literature. At least three different individuals are involved in sequentially reviewing the content and incorporating changes. The end result is inconsistent application of markup to the content.

The initial set of documents to be converted consisted of 20 content types, including four segmented types. As each type was added to the testing pool, new use cases were encountered that needed to be addressed in the transforms. In one group of documents, unordered lists contained nested unordered lists (ul/ul) rather than the sub-list being tagged as a child of an individual list item (ul/li/ul). In another group, images were tagged as bold (b/img). It was also very recently discovered that while optional/required indicators were maintained on individual segments within the customer-facing system, they weren't reflected in the HTML being used as the source content for the transforms; in that instance, the export utility needed to be modified to include an attribute on each segment noting required or optional and then added to the transform, and the four segmented content types reprocessed.

This content also makes extensive use of images (an average of 3 per article), external links, and on rare occasions, links to other articles. The image references were all hard-wired to IDs and locations in the existing system; the transform needed to resolve the IDs to image names and remove the hard-wired paths.

While more effort was required, the quality of the results is much higher and the XML is now the version of record. It should also be mentioned that more analysis, testing, and refinement of the processes involved went into this content set, given its usage (and accreditation status) within the medical community.

JSON

Another team is creating e-learning content (System F in Table I). for patient and wellness education. The content is highly interactive, and designed to be viewed online from any type of device. The design itself is modular, with each "unit" consisting of a shell index.html file, several javascript files, css files, images, a single PDF file that can be downloaded for future reference, and a set of three JSON files that contain the actual prose.

Figure 8: Interactive MicroLearning Unit - Style 1

Figure 9: Interactive MicroLearning Unit - Style 2

There are currently 10 different styles in use, each one portraying information in a different format; several incorporate interactive quizzes. In Figure 8, as the slider is progressed from left to right, the image changes and the corresponding text appears in the circle below. In Figure 9, hotspot locations have been identified and as the user clicks on each location, descriptive text is displayed either below or beside the illustration, depending on the size of the screen. This content was originally created by subject matter experts who reviewed existing article content and extracted relevent bits to be incorporated into a learning object; Word files were then sent to the developer and the text from the Word document was transferred and tagged in the JSON files. The long-term goal was for the subject matter experts to use styled Microsoft Word documents that would then be transformed to create the necessary JSON components.

This has been the most challenging of the transforms to accomplish both due to the newness of the project and the inconsistent structure of the JSON templates. When the project was originally conceived, the plan was for content to be copied from the Word document into the JSON file, giving the user the opportunity to incorporate any additional markup or changes that might be needed. Over time, the learning unit templates were refined and additional CSS rules added to facilitate automated transformations. When we first began developing the transforms, we only had one or two samples of each of the styles; as more content was developed, the requirements grew more complex. Once we reached critical mass, we were able to go back and refine the work that had been done. While all of the styling is intended to be handled in the css files, there are valid use cases where the defaults do not work; formatting instructions are slowly creeping into the Word styles and transforms. There are additional content types in the pipeline that will provide yet more challenges.

Figure 10: Word -> DITA -> MLUXML -> JSON

On any given day there is at least one learning unit created that doesn't function properly; this is most commonly due to improper use of the designated Microsoft Word styles. The plan going forward is to impement a forms-based content creation process that will eliminate the reliance on user-applied styles and more closely align with the desired output.

Metadata

Metadata is maintained in at least 4 different systems, not including metadata embedded in images. Different product teams use different taxonomies to identify similar content. When transforming the articles, the associated metadata can't be left behind; it's an integral piece of the content. Most of this metadata is managed in relational tables in the legacy systems and is exported as XML. For those systems managing content as HTML, the metadata is prepended to the HTML file and extracted; for systems managing content as Microsoft Word files, a separate XML file is associated with the Word document so that the metadata can be maintained along with the resulting DITA files.

Since all of the content is intended to be managed in a single repository going forward, the taxonomies needed to be harmonized; metadata that was required in one system needed to be added to content from another; discrepencies in how metadata was entered needed to be resolved. Take age groups, for example. One product team simply used "child," "adult," or "senior," while another team used a more detailed breakdown "infant," "child," "teen," "young adult," "middle adult," "mature adult," and "senior." In another example, a fairly sophisticated medical taxonomy was matched up with a more consumer-friendly identification scheme. In yet another example, while the terms used were fairly consistent across product groups, different identifiers were used – serviceline vs category; and the reverse – audience used to identify two very different sets of values. Decisions needed to made not only on the values but on the identifers as well, and then transforms written.

In addition to classification metadata, information relating to when an article was last reviewed and by whom also needed to be retained, as well as, for the product team using Microsoft Word, where each article was used – including the name of the client, the product, and the date.

In this particular instance, the content will be managed in a repository that sits on top of a MarkLogic database; the metadata will be maintained separately from the articles themselves. In other use cases, some if not all of the metadata might be embedded in the actual XML content. The planned environment is such that metadata can be extracted from or incorporated into the XML articles as needed.