Ng, Kin, Lisandro Gonzalez and Stacy Lathrop. “Validation-Driven Automation in a Federated XML Ingest Pipeline: The NCBI Bookshelf
Case.” Presented at Balisage: The Markup Conference 2026, Washington, DC, August 3 - 7, 2026. In Proceedings of Balisage: The Markup Conference 2026. Balisage Series on Markup Technologies, vol. 31 (2026). https://doi.org/10.4242/BalisageVol31.Lathrop01.
Balisage: The Markup Conference 2026 August 3 - 7, 2026
Balisage Paper: Validation-Driven Automation in a Federated XML Ingest Pipeline: The NCBI Bookshelf
Case
Kin Ng
National Center for Biotechnology Information, National Library of
Medicine, National Institutes of Health
Lisandro Gonzalez
National Center for Biotechnology Information, National Library of
Medicine, National Institutes of Health
Stacy Lathrop
National Center for Biotechnology Information, National Library of
Medicine, National Institutes of Health
Abstract
The NCBI Bookshelf, managed by the National Center for Biotechnology Information (NCBI),
National Library of Medicine (NLM), ingests and makes publicly accessible a wide range
of biomedical books and reports contributed by diverse submitters who deposit heterogeneous
source formats, including Word, PDF, and XML. Ensuring data integrity across this
content has historically required substantial manual intervention due to variability
in input formats, metadata quality, and distributed system dependencies.
This paper describes the design and implementation of a
validation-driven automation framework for Bookshelf that supports scalable conversion,
ingestion, processing, and release of content within a federated architecture. The
system integrates rule-based validation, workflow orchestration, and identifier-based
reconciliation across multiple systems, including content management, XML processing
pipelines, PubMed indexing, and Open Access dataset services.
Central to the approach is the use of layered validation, including Schematron/XPath
and business-rule enforcement, to control workflow transitions, prevent duplicate
or inconsistent content, and ensure the completeness and consistency of metadata.
Validation is applied at multiple stages in the lifecycle—from submission and conversion
through preview and release—and is exposed to submitters and informaticists through
actionable feedback mechanisms.
The pipeline supports multiple conversion and ingestion pathways, including direct
XML submission, PDF-to-XML conversion via tagging vendors, and Word-based authoring
workflows. Despite this diversity, all workflows converge on a common processing model
governed by validation rules and state transitions tracked in an external workflow
system.
The paper also describes mechanisms for controlled self-service publishing, automated
generation of derived content structures, and cross-system integration through identifier
reconciliation. Operational reporting and provenance tracking further support auditability
and continuous improvement.
This work demonstrates how validation can serve not only as a quality assurance mechanism,
but as a central organizing principle for workflow automation, enabling scalable and
reliable content management across a distributed XML ecosystem.
Managed by the National Library of Medicine, the NCBI Bookshelf
serves as a platform for the free dissemination of biomedical books,
reports, and continuously updated reference works. Unlike
traditional publishing systems that operate within tightly
controlled pipelines, Bookshelf as an institutional repository must
accommodate contributions from a wide range of submitters, including
large publishers, government agencies, contractors, editorial teams,
and individual authors. These contributors provide content in
multiple source formats—primarily Word documents, PDFs, and XML—each
with different levels of structure, consistency, and metadata
completeness.
Historically, the conversion and ingestion of this content required
substantial manual intervention. Staff were responsible for
validating submissions, coordinating with tagging vendors,
reconciling metadata across systems, and ensuring that content was
accurately represented on Bookshelf and in downstream services such
as PubMed and the LitArch Open Access Subset. As submission volume
increased and workflows diversified, this approach became
increasingly difficult to scale. The absence of a centralized,
standardized processing model resulted in fragmented workflows,
inconsistent processing practices, and increased operational
overhead, making it more difficult to maintain data quality, ensure
timely content availability, and efficiently accommodate the growing
volume and diversity of submissions across the repository’s content
management, ingest, and public access ecosystem.
In response, Bookshelf developed an automated submission and
processing pipeline designed to operate across a federated
architecture. Rather than centralizing all processing in a single
system, the pipeline coordinates multiple loosely coupled
components, including a content management system, XML conversion
and ingest pipelines, workflow tracking tools, and indexing and
programmatic services. Within this environment, maintaining data
integrity requires not only validating individual documents against
DTDs, but also ensuring consistency across systems and over time, as
well as that NLM meets policy agreements, such as permissions.
This paper describes the resulting system, with a particular focus
on how validation is used not merely as a final quality check, but
as a mechanism for controlling workflow transitions, enforcing
business rules, and coordinating cross-system integration.
System Context: A Federated Publishing Architecture
Bookshelf operates within a distributed environment in which
multiple systems store and process content and metadata.
Responsibility for making this content findable, accessible,
interoperable, and reusable, or FAIR, is divided across several
systems, among them:
FTP-based bulk submission and ingestion points for publishers
and vendors
A content management system (BCMS) used for submission, editing,
and previewing
XML processing pipelines responsible for conversion, validation,
and packaging
PMCBook as the primary content repository
PMC Publisher Database as the source of participant agreements
PMC Deposit Tracking System to store processing errors
PubMed as the primary indexing and discovery system
LitArch Open Access Subset as the primary programmatic dataset
Jira as a workflow tracking and state management system
Content and metadata move between these systems through a
combination of scheduled jobs, scripts, and API calls. This
architecture provides flexibility and scalability, but it also
introduces challenges: metadata must be reconciled across systems,
identifiers must remain consistent, compute load must be managed,
and processing must be coordinated despite asynchronous operations.
Most publishing models are optimized for one styled content type and
submission workflow; however, Bookshelf allows for different content
types through a single federated architecture. Variably styled
content enters the system through one of several pathways: direct
XML submission via FTP or submitter upload to BCMS, PDF submission
to BCMS for vendor tagging, or Word-based authoring in BCMS. Each
pathway reflects the needs and capabilities of different submitters,
but all converge on a shared pipeline that supports the conversion
and ingestion of either individual book parts or complete books.
Once content is received, the data is checked for completeness and
conformity to supported schemas. Bookshelf supports source XML in
multiple schemas. Next the system attempts to match the content to
an existing domain or create a new domain in Bookshelf’s PMC Book
database assisted through various repository business-rules for
validating and matching the data. Based on metadata in the
submission, such as publisher, collection, or book title, Bookshelf
selects a database clone template to copy publisher-specific
database properties and attributes that control display and
downstream processes, such as sending metadata automatically to
PubMed for indexing and permissions to add the content to the NLM
LitArch Open Access FTP service.
Once the database domain is matched or created, content is then
converted as necessary into a canonical Bookshelf BITS XML
representation, followed by validation using a combination of
schema-based and rule-based checks. During this conversion,
Bookshelf adds a PMC Publication ID in the book metadata that is the
same as the unique persistent PMC Book domain name. If validation
succeeds, a preview is generated to allow for optional review. Upon
approval—or automatically, where appropriate—the content is released
in the production repository after additional validations to ensure
data integrity and propagated to downstream systems if the data
meets business rules and permissions.
Workflow status is tracked using Jira, which functions as a status
machine for the pipeline. Transitions such as “Submitted,”
“Converting,” “Loading,” “Previewing,” “Releasing,” and “Released”
provide both visibility and control to NCBI, enabling automated
processes to proceed while allowing staff intervention when
necessary.
Figure 2: Simplified processing and validation workflow diagram
Data Integrity Challenges in a Federated Pipeline
Operating across heterogeneous inputs and distributed systems
introduces several classes of data integrity challenges.
First, input variability requires normalization. Word documents,
PDFs, and XML differ significantly in structure and metadata
richness, necessitating different preprocessing and validation
strategies. The Bookshelf team supports NIH-funded editorial teams
by developing Word macros that automated editorial style validation
and document preparation. These tools enable editorial teams to
verify that manuscripts conformed to their own editorial standards,
as an automated copyeditor, while also transforming
publisher-specific house styles into the NCBI Bookshelf Word
template styles required for reliable conversion to BITS XML. Here
is an example of the steps in a simple macro created by Bookshelf
that imports the Bookshelf styles template into a document:
Start subroutine
Define the path to the template file from which the styles will
be imported
If the file cannot be found, provide the user with an error
message
Copy the styles from the specified template into the active
document
End subroutine
Second, large cross-organization teams may interact with the same
content at different stages. Authors, editors, tagging vendors,
publishers, and Bookshelf staff may each contribute to or modify
content, increasing the risk of divergence between representations.
For example, vendors might be responsible for uploading, validating,
and correcting tagging issues with the source XML, and editors and
publishers then be responsible for reviewing and approving previews,
or reporting and addressing content issues requiring resubmitted XML
by the vendors of software updates by the Bookshelf team. All along
this workflow, automated validations ensure that accurate and
required metadata and full text structure is maintained in the final
content made publicly accessible via the Bookshelf repository.
Third, versioning is inherently complex, starting with
differentiating between file versions of a document, versus content
iterations, such as when minor substantive edits are made to the
content or the content is fully re-reviewed and updated. We support
both file and content versioning in our content management system to
ensure a transparent audit trail. Furthermore, clinical content in
our repository may be corrected and updated at the chapter level or
book level, replaced entirely, or archived if no longer current,
such as a summary of a drug no longer approved by the FDA. Having a
way to track these statuses according to each participants’
editorial policies and then communicate them on the Bookshelf
website is necessary according to our agreements with these editors
and publishers. Thus, we rely on persistent identifiers and
consistent metadata to correctly match file versions of the same
content and communicate substantive content changes in status.
Finally, the federated architecture itself introduces consistency
challenges. Systems such as PMCBook, PubMed, the LitArch OA Subset,
and other internal databases responsible for different use cases
must remain synchronized despite asynchronous processing and
independent update cycles. Bookshelf supports its submitters by
automatically submitting metadata to PubMed for indexing and
Linkouts to the full text on publisher sites and Bookshelf.
Automated QA and Validation Framework
To address these challenges, Bookshelf employs a layered validation
framework that operates throughout the content lifecycle.
Validation is not confined to a single stage; rather, it is applied
at multiple points, including before domain creation, before
conversion, before preview generation, and before public release.
This ensures that errors are detected as early as possible and
prevents invalid content from progressing through the pipeline.
At the core of the system are rule-based validations implemented
using Schematron/Xpath and other custom logic triangulating data
stored in various databases against the XML being processed. Xpath
is an XML query language, and Schematron is a rule-based validation
language that uses XPath to check for the presence or absence of
patterns. These rules enforce both structural requirements and
domain-specific business constraints. For example, content is
checked for duplicate titles, missing or duplicate identifiers such
as DOI or ISBN, and missing required metadata such as publication
dates, series information, funding information, and permissions. In
some cases, similarity thresholds are used to detect near-duplicate
titles, reflecting the need to identify potential duplication even
when exact matches are not present.
Validation outcomes are classified as either warnings or errors.
Errors prevent content from advancing to the next stage, while
warnings are reported to users but do not block processing. This
distinction allows the system to balance strict enforcement of
critical constraints with flexibility for less critical issues.
Figure 3: Validation layers
Schematron Examples
Check for required publication date
<rule context="pub-date">
<assert test="@date-type and @publication-format">This pub-date does not include both @date-type and @publication-format</assert>
</rule>
Check for required DOI
<assert test="book-id[@book-id-type = 'doi']">ERROR: This report is missing the DOI. Add a DOI to the book-meta, tagged in a 'book-id' element with @book-id-type='doi'.</assert>
Check that content has a valid agreement before making it
public in Bookshelf
<xsl:template name="t:agree-exists">
<xsl:variable name="book-agree-no" select="count(agreement[contains($main-agreement-types, concat('|', @type,'|'))])"/>
<xsl:variable name="coll-agree-no" select="count(collection/agreement[contains($main-agreement-types, concat('|', @type,'|'))])"/>
<xsl:choose>
<xsl:when test="$book-agree-no = 0 and $coll-agree-no = 0">
<xsl:call-template name="error">
<xsl:with-param name="msg">Neither the book nor the parent collection is linked to an agreement.</xsl:with-param>
</xsl:call-template>
</xsl:when>
<xsl:when test="$book-agree-no > 1 or ($book-agree-no = 0 and $coll-agree-no > 1)">
<xsl:call-template name="error">
<xsl:with-param name="msg">
<xsl:text>The book is linked to more than one agreement of the following types: </xsl:text>
<xsl:variable name="all-comma" select="translate($main-agreement-types, '|', ', ' )"/>
<xsl:value-of select="substring($all-comma, 2, (string-length($all-comma) - 2))"/>
</xsl:with-param>
</xsl:call-template>
</xsl:when>
</xsl:choose>
</xsl:template>
Check that if a book is set to be added to the LitArch Open
Access subset it has a license describing its terms of use
<xsl:template name="t:agree-oa-license">
<xsl:if test="Properties/oa = 'yes' and permissions/copyright-statement and not(permissions/license or (has-chapters = 'yes' and has-chapters-without-license = 'no'))">
<xsl:call-template name="error">
<xsl:with-param name="msg">The book has DomainState property 'DomainState property 'entire journal is open access' checked and contains a copyright-statement, but no license </xsl:with-param>
</xsl:call-template>
</xsl:if>
</xsl:template>
Validation-Driven Self-Service Publishing
To improve throughput and reduce manual intervention, Bookshelf
introduced a controlled self-service publishing mechanism within its
content management system for its NIH-funded editorial teams to use.
Users can initiate publication directly by invoking a “Publish”
action, which triggers automated packaging and transmission of
content into the ingest pipeline.
This mechanism does not bypass validation. Instead, it shifts
validation into the automated pipeline, where the same checks are
applied as in staff-mediated workflows. Content is packaged with
structured metadata, transmitted via FTP, and processed
asynchronously. Validation results are returned through email and
system notifications, allowing users to correct errors as needed.
Validation not only gatekeeps whether content can proceed to the
next workflow step or not as in traditional publishing, but it may
improve the quality of the content by keying off metadata or other
structurally tagged parts of the content itself to run additional
quality checks and add or correct missing data for completeness,
such as collection information or permissions.
This approach of matching content like chapters to the book or
collection they belong by way of persistent identifiers and metadata
also enables continuous, automated publication of regularly updated
database records authored and reviewed independently. This
eliminates the need for batch processing of all the records for that
domain at once, thereby significantly reducing processing time. What
once may have taken more than a day to load to the site once each
week can now be updated within minutes after a document has been
finalized. To support this effort some tasks were shifted to
asynchronous nightly scheduled automated jobs, such as the
generation of table of contents landing pages to add new and updated
content released that day. These jobs are possible by relying on
consistent metadata in each document, such as labels, publication
history, or processing instructions that indicate what external data
to crawl and integrate as necessary each time it has been updated.
Automation also enables detailed reporting and tracking of content
throughout the pipeline. Reports can be automatically generated to
capture tagging activity, production timelines, and release status,
drawing data from multiple systems including Jira, PMCBook, and XML
metadata.
The need to support such reporting exposed gaps in existing
metadata, leading to the introduction of additional fields to
capture key events such as submission dates, tagging delivery dates,
and release dates. These enhancements support auditability and
provide insights into pipeline performance.
Results and Impact
The introduction of automation has significantly improved the
scalability and reliability of Bookshelf workflows. Manual
intervention has been reduced by 75%, particularly in routine
validation and release processes. Turnaround times have decreased by
80%, especially for high-volume content such as continuously updated
resources.
Lessons Learned
Several lessons emerged from this work. Validation must be both
rigorous and user-facing, providing actionable feedback that enables
submitters to correct errors efficiently, or more innovative, to
update content automatically according to business rules.
Triangulation of database SQL queries and XML tagging and
Schematron/XPath proved effective for encoding complex business
rules, particularly those involving conditional logic.
For example, some projects require the table of contents to be
ordered by the label of the chapter. The submitter includes the
label element in their XML submission. XPath is used to check for
the presence or absence of the label. If a label is not found, an
email is sent to reviewers, and the Jira ticket for that submission
is updated for tracking and reporting. If a label is found, then the
SQL database is queried to see if there is another chapter with the
same label. A duplicate label will trigger an email to reviewers, as
well as updating the Jira ticket.
At the same time, enabling self-service publishing requires careful
balancing of flexibility and control. While automation reduces
bottlenecks, it also necessitates robust validation to prevent
errors from propagating into production systems and to be able to
rigorously reuse the content to build downstream content such as
table of contents pages.
Finally, operating within a federated architecture requires explicit
mechanisms for maintaining consistency across systems.
Identifier-based reconciliation and comprehensive reporting are
essential components of this effort. Besides validating labels,
persistent IDs such as DOIs and ISBNs are validated against the
database to ensure there are no duplicate submissions, to reliably
match submissions across resubmissions, and to block submissions
that fail to provide a valid persistent ID.
Conclusion
The Bookshelf automation framework demonstrates how validation,
workflow orchestration, and cross-system integration can be combined
to support scalable content management, public access, discovery,
and preservation in a distributed environment. By treating
validation as a central organizing principle, the system ensures
data integrity across diverse inputs and workflows while enabling
increased automation and user autonomy. These enhancements exist to
support the continuous publication of evolving biomedical knowledge.
Acknowledgements
We wish to credit our current and past colleagues at the National
Center for Biotechnology Information and the National Library of
Medicine for their contributions to the Bookshelf project,
especially members of the Bookshelf team, as well as Evgeny Kireev,
Jeff Beck, Marie Collins, Martin Latterner, Denis Karamyshevd, and
the PMC team of developers. This work was supported by the National
Center for Biotechnology Information of the National Library of
Medicine (NLM), National Institutes of Health (NIH). The
contributions of the NIH author(s) are considered Works of the
United States Government. The findings and conclusions presented in
this paper are those of the author(s) and do not necessarily reflect
the views of the NIH or the U.S. Department of Health and Human
Services.
