Posts Tagged ‘LilyXu’
[PyDataGlobal2025] Using Traditional AI and Large Language Models to Automate Complex and Critical Documents in Healthcare
Lecturer
Lily Xu is a Data Science Director in the corporate data-science and AI team at Vertex Pharmaceuticals, where she has worked for approximately seven years. She leads interdisciplinary groups of data scientists, data engineers, software engineers, and operations specialists focused on clinical-area solutions. She holds a doctorate in bioengineering from the Massachusetts Institute of Technology and an undergraduate degree from the University of California, Berkeley. Her earlier research produced publications on virtual microfluidics and the human microbiome; at Vertex she has driven projects spanning generative AI for clinical documentation, predictive patient modeling, large-scale claims analytics, protocol design, and centralized site intelligence.
Abstract
Informed consent forms constitute high-stakes, patient-facing, heavily regulated documents that must be tailored to jurisdictional requirements, local ethics boards, and plain-language standards. Their manual production across dozens of countries and hundreds of sites creates substantial operational bottlenecks in clinical-trial start-up. This article examines a production system developed at Vertex Pharmaceuticals that combines classical document-processing pipelines with large language models to auto-draft informed consent forms at scale. Emphasis is placed on architectural choices that minimize hallucination risk, rigorous measurement of end-to-end time savings, the centrality of change management, and the longer-term strategy of constructing a connected document network rather than isolated point solutions.
Clinical-Trial Operations Context and the Dual AI Portfolio
Clinical-trial operations span design, planning, execution, and monitoring phases, each generating or consuming large volumes of structured and unstructured documents. Failure to recruit patients, suboptimal site selection, or protracted regulatory review can each cost tens to hundreds of millions of dollars. Beginning in 2019 the Vertex data-strategy and solutions team—functioning as an internal SWAT unit—built trust through small, measurable pilots that combined public and private data into AI-ready assets. Over successive years the portfolio matured from ad-hoc analytics into standardized offerings for site identification, patient finding, enrollment forecasting, and, more recently, generative document automation.
The team deliberately distinguishes analytical AI (predictive modeling, Bayesian enrollment forecasts, rare-disease patient identification) from generative AI (first-draft document creation, knowledge-base chat, brand-copy generation). Business partners often approach the group believing a problem requires generative technology when structured data and classical machine learning would suffice; conversely, generative methods unlock previously intractable free-text tasks. Framing the two categories helps both data scientists and operational stakeholders select the appropriate tool. A foundational data layer aggregates site performance metrics, physician databases, claims, and census information; disease-specific analytic views and predictive models sit atop this foundation. Parallel generative pipelines extract structured content from lengthy protocols and feed downstream document generators, with embedded quality-control workflows so that extraction errors are corrected before they propagate into patient-facing material.
Architecture of the Informed-Consent-Form Auto-Drafting System
An informed consent form must convey risks, procedures, and rights in plain language while satisfying country-specific and sometimes site-specific regulatory requirements. A single multi-country trial may therefore require dozens of distinct variants. The solution developed at Vertex treats the clinical protocol as the primary source of truth, a blank regulatory template as the structural skeleton, and an approved language library as the repository of standardized phrasing.
Custom Python modules parse the protocol into logically coherent sections rather than arbitrary token chunks. Section-specific prompts and deterministic extraction routines pull the necessary facts. User-supplied answers to questions that cannot be parsed from the protocol are collected through a controlled interface. The resulting structured payload is inserted into the template; approved language snippets are retrieved via API from a purpose-built library that replaced earlier Excel spreadsheets and now maintains full audit trails and disease-area tagging.
The application is implemented in Flask and Dash, hosted on AWS behind single-sign-on, and calls a private Microsoft OpenAI endpoint for the generative steps. A monitoring dashboard continuously compares newly generated drafts against ground-truth forms produced by human experts, allowing the team to detect drift in accuracy over time. Because the generative component constitutes only a minority of the code base, the majority of engineering effort is devoted to robust parsing, template management, and workflow orchestration—skills that remain essential even as language models improve.
The design philosophy is “AI in the human loop” rather than “human in the AI loop.” Regulatory and patient-safety constraints demand that every draft undergo expert review; the system’s value lies in accelerating the initial drafting phase so that reviewers begin from a high-quality baseline rather than a blank page.
Measuring Impact, Change Management, and Scaling Strategy
Early controlled experiments compared pure manual drafting (one to three hours depending on trial complexity) with auto-draft generation (under ten minutes). Drafting-time reduction approached 90 percent. When subsequent editing and quality-control effort was included, net end-to-end time savings settled near 40 percent—still substantial given the volume of forms required across a growing portfolio. Because operational teams are chronically time-constrained, such measurements were performed on only two trials; the results nevertheless provided the quantitative foundation for continued investment.
Technology alone does not guarantee adoption. Change-management activities therefore received equal attention: standardization of templates and language libraries, transparent communication of model assumptions and known failure modes, and staged training that enabled business users to generate drafts independently. Treating free-text language assets with the same governance rigor applied to numerical data proved essential.
The longer-term vision is a connected document network rather than a collection of isolated point solutions. Clinical protocols and clinical study reports function as central hubs; mapping the full input–output relationships among start-up documents reveals opportunities for shared extraction components and cascading automation. The same platform is being extended to site budgets, case-report-form specifications, training materials, and other protocol-derived artifacts. Country-level templates are already linked so that a single protocol upload can spawn multiple jurisdiction-specific drafts simultaneously. Site-level customization remains outside the automated scope because the return on investment diminishes rapidly at that granularity; country-level guidance is instead provided to local teams.
Broader Lessons for Generative Applications in Regulated Environments
Several observations travel beyond the specific use case. First, impact measurement must be designed from the outset; without side-by-side timing studies and accuracy tracking, claims of productivity gain remain anecdotal. Second, the majority of engineering effort in production document systems continues to reside in classical software and data-engineering practices; large language models occupy a focused niche once reliable extraction and templating are in place. Third, alignment with business ownership is decisive: projects lacking motivated operational sponsors are deferred in favor of those with clear accountability and enthusiasm. Finally, the cumulative benefit of a systematically constructed document network can outweigh the initial development cost provided the organization persists past the early pilots.
Ambient listening, internal retrieval-augmented generation over institutional knowledge bases, and protocol optimization via real-world data are complementary initiatives already underway at Vertex and peer organizations. Collectively they illustrate a measured trajectory in which generative and analytical methods remove routine cognitive load while leaving critical reasoning and final accountability with domain experts.