Introduction

Managed by the National Library of Medicine, the NCBI Bookshelf serves as a platform for the free dissemination of biomedical books, reports, and continuously updated reference works. Unlike traditional publishing systems that operate within tightly controlled pipelines, Bookshelf as an institutional repository must accommodate contributions from a wide range of submitters, including large publishers, government agencies, contractors, editorial teams, and individual authors. These contributors provide content in multiple source formats—primarily Word documents, PDFs, and XML—each with different levels of structure, consistency, and metadata completeness.

Historically, the conversion and ingestion of this content required substantial manual intervention. Staff were responsible for validating submissions, coordinating with tagging vendors, reconciling metadata across systems, and ensuring that content was accurately represented on Bookshelf and in downstream services such as PubMed and the LitArch Open Access Subset. As submission volume increased and workflows diversified, this approach became increasingly difficult to scale. The absence of a centralized, standardized processing model resulted in fragmented workflows, inconsistent processing practices, and increased operational overhead, making it more difficult to maintain data quality, ensure timely content availability, and efficiently accommodate the growing volume and diversity of submissions across the repository’s content management, ingest, and public access ecosystem.

In response, Bookshelf developed an automated submission and processing pipeline designed to operate across a federated architecture. Rather than centralizing all processing in a single system, the pipeline coordinates multiple loosely coupled components, including a content management system, XML conversion and ingest pipelines, workflow tracking tools, and indexing and programmatic services. Within this environment, maintaining data integrity requires not only validating individual documents against DTDs, but also ensuring consistency across systems and over time, as well as that NLM meets policy agreements, such as permissions.

This paper describes the resulting system, with a particular focus on how validation is used not merely as a final quality check, but as a mechanism for controlling workflow transitions, enforcing business rules, and coordinating cross-system integration.

System Context: A Federated Publishing Architecture

Bookshelf operates within a distributed environment in which multiple systems store and process content and metadata. Responsibility for making this content findable, accessible, interoperable, and reusable, or FAIR, is divided across several systems, among them:

  • FTP-based bulk submission and ingestion points for publishers and vendors

  • A content management system (BCMS) used for submission, editing, and previewing

  • XML processing pipelines responsible for conversion, validation, and packaging

  • PMCBook as the primary content repository

  • PMC Publisher Database as the source of participant agreements

  • PMC Deposit Tracking System to store processing errors

  • PubMed as the primary indexing and discovery system

  • LitArch Open Access Subset as the primary programmatic dataset

  • Jira as a workflow tracking and state management system

Content and metadata move between these systems through a combination of scheduled jobs, scripts, and API calls. This architecture provides flexibility and scalability, but it also introduces challenges: metadata must be reconciled across systems, identifiers must remain consistent, compute load must be managed, and processing must be coordinated despite asynchronous operations.

Figure 1: Validation-driven federated architecture

End-to-End Workflow Automation

Most publishing models are optimized for one styled content type and submission workflow; however, Bookshelf allows for different content types through a single federated architecture. Variably styled content enters the system through one of several pathways: direct XML submission via FTP or submitter upload to BCMS, PDF submission to BCMS for vendor tagging, or Word-based authoring in BCMS. Each pathway reflects the needs and capabilities of different submitters, but all converge on a shared pipeline that supports the conversion and ingestion of either individual book parts or complete books.

Once content is received, the data is checked for completeness and conformity to supported schemas. Bookshelf supports source XML in multiple schemas. Next the system attempts to match the content to an existing domain or create a new domain in Bookshelf’s PMC Book database assisted through various repository business-rules for validating and matching the data. Based on metadata in the submission, such as publisher, collection, or book title, Bookshelf selects a database clone template to copy publisher-specific database properties and attributes that control display and downstream processes, such as sending metadata automatically to PubMed for indexing and permissions to add the content to the NLM LitArch Open Access FTP service.

Once the database domain is matched or created, content is then converted as necessary into a canonical Bookshelf BITS XML representation, followed by validation using a combination of schema-based and rule-based checks. During this conversion, Bookshelf adds a PMC Publication ID in the book metadata that is the same as the unique persistent PMC Book domain name. If validation succeeds, a preview is generated to allow for optional review. Upon approval—or automatically, where appropriate—the content is released in the production repository after additional validations to ensure data integrity and propagated to downstream systems if the data meets business rules and permissions.

Workflow status is tracked using Jira, which functions as a status machine for the pipeline. Transitions such as “Submitted,” “Converting,” “Loading,” “Previewing,” “Releasing,” and “Released” provide both visibility and control to NCBI, enabling automated processes to proceed while allowing staff intervention when necessary.

Figure 2: Simplified processing and validation workflow diagram

Data Integrity Challenges in a Federated Pipeline

Operating across heterogeneous inputs and distributed systems introduces several classes of data integrity challenges.

First, input variability requires normalization. Word documents, PDFs, and XML differ significantly in structure and metadata richness, necessitating different preprocessing and validation strategies. The Bookshelf team supports NIH-funded editorial teams by developing Word macros that automated editorial style validation and document preparation. These tools enable editorial teams to verify that manuscripts conformed to their own editorial standards, as an automated copyeditor, while also transforming publisher-specific house styles into the NCBI Bookshelf Word template styles required for reliable conversion to BITS XML. Here is an example of the steps in a simple macro created by Bookshelf that imports the Bookshelf styles template into a document:

  1. Start subroutine

  2. Define the path to the template file from which the styles will be imported

  3. If the file cannot be found, provide the user with an error message

  4. Copy the styles from the specified template into the active document

  5. End subroutine

Second, large cross-organization teams may interact with the same content at different stages. Authors, editors, tagging vendors, publishers, and Bookshelf staff may each contribute to or modify content, increasing the risk of divergence between representations. For example, vendors might be responsible for uploading, validating, and correcting tagging issues with the source XML, and editors and publishers then be responsible for reviewing and approving previews, or reporting and addressing content issues requiring resubmitted XML by the vendors of software updates by the Bookshelf team. All along this workflow, automated validations ensure that accurate and required metadata and full text structure is maintained in the final content made publicly accessible via the Bookshelf repository.

Third, versioning is inherently complex, starting with differentiating between file versions of a document, versus content iterations, such as when minor substantive edits are made to the content or the content is fully re-reviewed and updated. We support both file and content versioning in our content management system to ensure a transparent audit trail. Furthermore, clinical content in our repository may be corrected and updated at the chapter level or book level, replaced entirely, or archived if no longer current, such as a summary of a drug no longer approved by the FDA. Having a way to track these statuses according to each participants’ editorial policies and then communicate them on the Bookshelf website is necessary according to our agreements with these editors and publishers. Thus, we rely on persistent identifiers and consistent metadata to correctly match file versions of the same content and communicate substantive content changes in status.

Finally, the federated architecture itself introduces consistency challenges. Systems such as PMCBook, PubMed, the LitArch OA Subset, and other internal databases responsible for different use cases must remain synchronized despite asynchronous processing and independent update cycles. Bookshelf supports its submitters by automatically submitting metadata to PubMed for indexing and Linkouts to the full text on publisher sites and Bookshelf.

Automated QA and Validation Framework

To address these challenges, Bookshelf employs a layered validation framework that operates throughout the content lifecycle.

Validation is not confined to a single stage; rather, it is applied at multiple points, including before domain creation, before conversion, before preview generation, and before public release. This ensures that errors are detected as early as possible and prevents invalid content from progressing through the pipeline.

At the core of the system are rule-based validations implemented using Schematron/Xpath and other custom logic triangulating data stored in various databases against the XML being processed. Xpath is an XML query language, and Schematron is a rule-based validation language that uses XPath to check for the presence or absence of patterns. These rules enforce both structural requirements and domain-specific business constraints. For example, content is checked for duplicate titles, missing or duplicate identifiers such as DOI or ISBN, and missing required metadata such as publication dates, series information, funding information, and permissions. In some cases, similarity thresholds are used to detect near-duplicate titles, reflecting the need to identify potential duplication even when exact matches are not present.

Validation outcomes are classified as either warnings or errors. Errors prevent content from advancing to the next stage, while warnings are reported to users but do not block processing. This distinction allows the system to balance strict enforcement of critical constraints with flexibility for less critical issues.

Figure 3: Validation layers

Schematron Examples

  1. Check for required publication date 

    <rule context="pub-date">
     <assert test="@date-type and @publication-format">This pub-date does not include both @date-type and @publication-format</assert>
</rule>

  2. Check for required DOI 

    <assert test="book-id[@book-id-type = 'doi']">ERROR: This report is missing the DOI. Add a DOI to the book-meta, tagged in a 'book-id' element with @book-id-type='doi'.</assert>

  3. Check that content has a valid agreement before making it public in Bookshelf 

    <xsl:template name="t:agree-exists"> 
    <xsl:variable name="book-agree-no" select="count(agreement[contains($main-agreement-types, concat('|', @type,'|'))])"/> 
    <xsl:variable name="coll-agree-no" select="count(collection/agreement[contains($main-agreement-types, concat('|', @type,'|'))])"/> 
    <xsl:choose> 
      <xsl:when test="$book-agree-no = 0 and $coll-agree-no = 0"> 
        <xsl:call-template name="error"> 
          <xsl:with-param name="msg">Neither the book nor the parent collection is linked to an agreement.</xsl:with-param> 
        </xsl:call-template> 
      </xsl:when> 
      <xsl:when test="$book-agree-no > 1 or ($book-agree-no = 0 and $coll-agree-no > 1)"> 
        <xsl:call-template name="error"> 
           <xsl:with-param name="msg"> 
              <xsl:text>The book is linked to more than one agreement of the following types: </xsl:text>  
              <xsl:variable name="all-comma" select="translate($main-agreement-types, '|', ', ' )"/>  
              <xsl:value-of select="substring($all-comma, 2, (string-length($all-comma) - 2))"/> 
           </xsl:with-param> 
        </xsl:call-template> 
      </xsl:when> 
    </xsl:choose> 
</xsl:template>

  4. Check that if a book is set to be added to the LitArch Open Access subset it has a license describing its terms of use 

    <xsl:template name="t:agree-oa-license">
    <xsl:if test="Properties/oa = 'yes' and permissions/copyright-statement and not(permissions/license or (has-chapters = 'yes' and has-chapters-without-license = 'no'))"> 
        <xsl:call-template name="error"> 
            <xsl:with-param name="msg">The book has DomainState property 'DomainState property 'entire journal is open access' checked and contains a copyright-statement, but no license </xsl:with-param> 
        </xsl:call-template> 
    </xsl:if> 
</xsl:template>

Validation-Driven Self-Service Publishing

To improve throughput and reduce manual intervention, Bookshelf introduced a controlled self-service publishing mechanism within its content management system for its NIH-funded editorial teams to use. Users can initiate publication directly by invoking a “Publish” action, which triggers automated packaging and transmission of content into the ingest pipeline.

This mechanism does not bypass validation. Instead, it shifts validation into the automated pipeline, where the same checks are applied as in staff-mediated workflows. Content is packaged with structured metadata, transmitted via FTP, and processed asynchronously. Validation results are returned through email and system notifications, allowing users to correct errors as needed. Validation not only gatekeeps whether content can proceed to the next workflow step or not as in traditional publishing, but it may improve the quality of the content by keying off metadata or other structurally tagged parts of the content itself to run additional quality checks and add or correct missing data for completeness, such as collection information or permissions.

This approach of matching content like chapters to the book or collection they belong by way of persistent identifiers and metadata also enables continuous, automated publication of regularly updated database records authored and reviewed independently. This eliminates the need for batch processing of all the records for that domain at once, thereby significantly reducing processing time. What once may have taken more than a day to load to the site once each week can now be updated within minutes after a document has been finalized. To support this effort some tasks were shifted to asynchronous nightly scheduled automated jobs, such as the generation of table of contents landing pages to add new and updated content released that day. These jobs are possible by relying on consistent metadata in each document, such as labels, publication history, or processing instructions that indicate what external data to crawl and integrate as necessary each time it has been updated.

Figure 4: Validation-driven self-service publishing

Reporting, Provenance, and Operational Tracking

Automation also enables detailed reporting and tracking of content throughout the pipeline. Reports can be automatically generated to capture tagging activity, production timelines, and release status, drawing data from multiple systems including Jira, PMCBook, and XML metadata.

The need to support such reporting exposed gaps in existing metadata, leading to the introduction of additional fields to capture key events such as submission dates, tagging delivery dates, and release dates. These enhancements support auditability and provide insights into pipeline performance.

Results and Impact

The introduction of automation has significantly improved the scalability and reliability of Bookshelf workflows. Manual intervention has been reduced by 75%, particularly in routine validation and release processes. Turnaround times have decreased by 80%, especially for high-volume content such as continuously updated resources.

Lessons Learned

Several lessons emerged from this work. Validation must be both rigorous and user-facing, providing actionable feedback that enables submitters to correct errors efficiently, or more innovative, to update content automatically according to business rules. Triangulation of database SQL queries and XML tagging and Schematron/XPath proved effective for encoding complex business rules, particularly those involving conditional logic.

For example, some projects require the table of contents to be ordered by the label of the chapter. The submitter includes the label element in their XML submission. XPath is used to check for the presence or absence of the label. If a label is not found, an email is sent to reviewers, and the Jira ticket for that submission is updated for tracking and reporting. If a label is found, then the SQL database is queried to see if there is another chapter with the same label. A duplicate label will trigger an email to reviewers, as well as updating the Jira ticket.

At the same time, enabling self-service publishing requires careful balancing of flexibility and control. While automation reduces bottlenecks, it also necessitates robust validation to prevent errors from propagating into production systems and to be able to rigorously reuse the content to build downstream content such as table of contents pages.

Finally, operating within a federated architecture requires explicit mechanisms for maintaining consistency across systems. Identifier-based reconciliation and comprehensive reporting are essential components of this effort. Besides validating labels, persistent IDs such as DOIs and ISBNs are validated against the database to ensure there are no duplicate submissions, to reliably match submissions across resubmissions, and to block submissions that fail to provide a valid persistent ID.

Conclusion

The Bookshelf automation framework demonstrates how validation, workflow orchestration, and cross-system integration can be combined to support scalable content management, public access, discovery, and preservation in a distributed environment. By treating validation as a central organizing principle, the system ensures data integrity across diverse inputs and workflows while enabling increased automation and user autonomy. These enhancements exist to support the continuous publication of evolving biomedical knowledge.

Acknowledgements

We wish to credit our current and past colleagues at the National Center for Biotechnology Information and the National Library of Medicine for their contributions to the Bookshelf project, especially members of the Bookshelf team, as well as Evgeny Kireev, Jeff Beck, Marie Collins, Martin Latterner, Denis Karamyshevd, and the PMC team of developers. This work was supported by the National Center for Biotechnology Information of the National Library of Medicine (NLM), National Institutes of Health (NIH). The contributions of the NIH author(s) are considered Works of the United States Government. The findings and conclusions presented in this paper are those of the author(s) and do not necessarily reflect the views of the NIH or the U.S. Department of Health and Human Services.