Posts Tagged ‘HealthcareAI’
[PyDataGlobal2025] Using Traditional AI and Large Language Models to Automate Complex and Critical Documents in Healthcare
Lecturer
Lily Xu is a Data Science Director in the corporate data-science and AI team at Vertex Pharmaceuticals, where she has worked for approximately seven years. She leads interdisciplinary groups of data scientists, data engineers, software engineers, and operations specialists focused on clinical-area solutions. She holds a doctorate in bioengineering from the Massachusetts Institute of Technology and an undergraduate degree from the University of California, Berkeley. Her earlier research produced publications on virtual microfluidics and the human microbiome; at Vertex she has driven projects spanning generative AI for clinical documentation, predictive patient modeling, large-scale claims analytics, protocol design, and centralized site intelligence.
Abstract
Informed consent forms constitute high-stakes, patient-facing, heavily regulated documents that must be tailored to jurisdictional requirements, local ethics boards, and plain-language standards. Their manual production across dozens of countries and hundreds of sites creates substantial operational bottlenecks in clinical-trial start-up. This article examines a production system developed at Vertex Pharmaceuticals that combines classical document-processing pipelines with large language models to auto-draft informed consent forms at scale. Emphasis is placed on architectural choices that minimize hallucination risk, rigorous measurement of end-to-end time savings, the centrality of change management, and the longer-term strategy of constructing a connected document network rather than isolated point solutions.
Clinical-Trial Operations Context and the Dual AI Portfolio
Clinical-trial operations span design, planning, execution, and monitoring phases, each generating or consuming large volumes of structured and unstructured documents. Failure to recruit patients, suboptimal site selection, or protracted regulatory review can each cost tens to hundreds of millions of dollars. Beginning in 2019 the Vertex data-strategy and solutions team—functioning as an internal SWAT unit—built trust through small, measurable pilots that combined public and private data into AI-ready assets. Over successive years the portfolio matured from ad-hoc analytics into standardized offerings for site identification, patient finding, enrollment forecasting, and, more recently, generative document automation.
The team deliberately distinguishes analytical AI (predictive modeling, Bayesian enrollment forecasts, rare-disease patient identification) from generative AI (first-draft document creation, knowledge-base chat, brand-copy generation). Business partners often approach the group believing a problem requires generative technology when structured data and classical machine learning would suffice; conversely, generative methods unlock previously intractable free-text tasks. Framing the two categories helps both data scientists and operational stakeholders select the appropriate tool. A foundational data layer aggregates site performance metrics, physician databases, claims, and census information; disease-specific analytic views and predictive models sit atop this foundation. Parallel generative pipelines extract structured content from lengthy protocols and feed downstream document generators, with embedded quality-control workflows so that extraction errors are corrected before they propagate into patient-facing material.
Architecture of the Informed-Consent-Form Auto-Drafting System
An informed consent form must convey risks, procedures, and rights in plain language while satisfying country-specific and sometimes site-specific regulatory requirements. A single multi-country trial may therefore require dozens of distinct variants. The solution developed at Vertex treats the clinical protocol as the primary source of truth, a blank regulatory template as the structural skeleton, and an approved language library as the repository of standardized phrasing.
Custom Python modules parse the protocol into logically coherent sections rather than arbitrary token chunks. Section-specific prompts and deterministic extraction routines pull the necessary facts. User-supplied answers to questions that cannot be parsed from the protocol are collected through a controlled interface. The resulting structured payload is inserted into the template; approved language snippets are retrieved via API from a purpose-built library that replaced earlier Excel spreadsheets and now maintains full audit trails and disease-area tagging.
The application is implemented in Flask and Dash, hosted on AWS behind single-sign-on, and calls a private Microsoft OpenAI endpoint for the generative steps. A monitoring dashboard continuously compares newly generated drafts against ground-truth forms produced by human experts, allowing the team to detect drift in accuracy over time. Because the generative component constitutes only a minority of the code base, the majority of engineering effort is devoted to robust parsing, template management, and workflow orchestration—skills that remain essential even as language models improve.
The design philosophy is “AI in the human loop” rather than “human in the AI loop.” Regulatory and patient-safety constraints demand that every draft undergo expert review; the system’s value lies in accelerating the initial drafting phase so that reviewers begin from a high-quality baseline rather than a blank page.
Measuring Impact, Change Management, and Scaling Strategy
Early controlled experiments compared pure manual drafting (one to three hours depending on trial complexity) with auto-draft generation (under ten minutes). Drafting-time reduction approached 90 percent. When subsequent editing and quality-control effort was included, net end-to-end time savings settled near 40 percent—still substantial given the volume of forms required across a growing portfolio. Because operational teams are chronically time-constrained, such measurements were performed on only two trials; the results nevertheless provided the quantitative foundation for continued investment.
Technology alone does not guarantee adoption. Change-management activities therefore received equal attention: standardization of templates and language libraries, transparent communication of model assumptions and known failure modes, and staged training that enabled business users to generate drafts independently. Treating free-text language assets with the same governance rigor applied to numerical data proved essential.
The longer-term vision is a connected document network rather than a collection of isolated point solutions. Clinical protocols and clinical study reports function as central hubs; mapping the full input–output relationships among start-up documents reveals opportunities for shared extraction components and cascading automation. The same platform is being extended to site budgets, case-report-form specifications, training materials, and other protocol-derived artifacts. Country-level templates are already linked so that a single protocol upload can spawn multiple jurisdiction-specific drafts simultaneously. Site-level customization remains outside the automated scope because the return on investment diminishes rapidly at that granularity; country-level guidance is instead provided to local teams.
Broader Lessons for Generative Applications in Regulated Environments
Several observations travel beyond the specific use case. First, impact measurement must be designed from the outset; without side-by-side timing studies and accuracy tracking, claims of productivity gain remain anecdotal. Second, the majority of engineering effort in production document systems continues to reside in classical software and data-engineering practices; large language models occupy a focused niche once reliable extraction and templating are in place. Third, alignment with business ownership is decisive: projects lacking motivated operational sponsors are deferred in favor of those with clear accountability and enthusiasm. Finally, the cumulative benefit of a systematically constructed document network can outweigh the initial development cost provided the organization persists past the early pilots.
Ambient listening, internal retrieval-augmented generation over institutional knowledge bases, and protocol optimization via real-world data are complementary initiatives already underway at Vertex and peer organizations. Collectively they illustrate a measured trajectory in which generative and analytical methods remove routine cognitive load while leaving critical reasoning and final accountability with domain experts.
Links:
[AWSReInvent2025] Transforming Integrated Diagnostics: Philips’ AI-Driven Evolution on AWS
Lecturer
Sam Cool is a Director and Global Lead for Healthcare Solutions at Amazon Web Services (AWS), where he focuses on accelerating digital transformation for global health organizations. With extensive experience in cloud architecture and clinical workflows, Sam works with industry leaders to dismantle data silos and implement scalable AI solutions. Jared Nicks is a Principal Solutions Architect at AWS, specializing in medical imaging and Health-IT. His work is instrumental in developing the AWS HealthImaging service, which provides high-performance storage and retrieval for large-scale medical datasets. Wilson Toe serves as a Senior Product Manager at AWS, focusing on the intersection of Generative AI and healthcare analytics. Dr. Praeloski is a Senior Clinical Scientist at Philips, bringing decades of expertise in diagnostic imaging, pathology, and cardiology. He leads Philips’ efforts to integrate multi-modal data into a unified platform that enhances clinical decision-making. Together, these experts have pioneered a collaboration that leverages cloud-native technologies to redefine the diagnostic landscape.
Abstract
Modern healthcare is characterized by an explosion of diagnostic data, yet this information remains largely fragmented across disparate systems for radiology, cardiology, and pathology. This fragmentation hampers the ability of clinicians to form a holistic view of the patient, leading to diagnostic delays and suboptimal treatment planning. This article examines the strategic journey of Philips in transforming integrated diagnostics through its partnership with AWS. By shifting from on-premises infrastructure to a cloud-native architecture, Philips has successfully integrated diverse data streams, with a particular focus on the emerging frontier of digital pathology. The discussion explores the technical implementation of AWS HealthImaging, the transition to standardized DICOM formats for pathology, and the application of Generative AI to streamline clinical reporting. Ultimately, this framework enables global collaboration and real-time diagnostic consensus, moving the needle toward truly personalized and precise medicine.
The Paradox of Fragmented Diagnostic Intelligence
The clinical diagnostic process is the cornerstone of patient care, influencing over 70% of medical decisions. However, the current infrastructure supporting these decisions is often a patchwork of “black boxes.” A patient’s journey typically involves multiple diagnostic touchpoints: an X-ray in radiology, an ECG in cardiology, and a tissue biopsy in pathology. Historically, each of these domains has operated in a silo, utilizing proprietary data formats and isolated storage systems. Sam observes that while the volume of data is increasing—driven by higher-resolution imaging and molecular diagnostics—the “intelligence” derived from that data remains localized.
For a clinician, this fragmentation means navigating multiple interfaces and manually correlating reports, a process prone to error and inefficiency. The transition to integrated diagnostics is not merely a technical upgrade; it is a clinical necessity. By centralizing these streams in the cloud, healthcare providers can move from a reactive, department-centric model to a proactive, patient-centric one. Philips’ vision for integrated diagnostics centers on breaking down these silos to provide a “single source of truth” for every patient, regardless of where the data was generated.
Digital Pathology: The Final Frontier of Digitalization
While radiology and cardiology have been digital for decades, pathology—the study of tissue samples—has remained stubbornly analog. For over a century, pathologists have relied on glass slides and manual microscopy. The sheer scale of the data involved has been the primary barrier; a single high-resolution digital slide can exceed several gigabytes in size, and a single patient case may involve dozens of slides.
Dr. Praeloski highlights that digital pathology represents the next great shift in clinical innovation. By digitizing these slides, Philips enables pathologists to work in an environment that is “born digital,” allowing for the application of computer vision and machine learning. This transition is facilitated by the adoption of the DICOM (Digital Imaging and Communications in Medicine) standard for pathology images. Standardizing these massive datasets allows them to be treated with the same rigor and interoperability as traditional radiological images, enabling them to be stored, shared, and analyzed within the same AWS-backed ecosystem.
Architecting for High-Throughput Imaging with AWS HealthImaging
The technical challenge of managing millions of high-resolution pathology slides requires an infrastructure that can handle extreme throughput and low-latency retrieval. Standard object storage, while durable, often struggles with the specific access patterns required for medical imaging, where a clinician needs to “zoom and pan” through a multi-gigabyte image in real-time.
To solve this, Philips leverages AWS HealthImaging. This purpose-built service allows for the ingestion of medical images at scale while providing sub-second access to specific image frames. By decoupling storage from the viewing application, AWS HealthImaging ensures that clinicians can access images from any device, anywhere in the world, without the need for high-powered local workstations.
'''# Conceptual example of fetching metadata for a DICOM image set'''
import boto3
health_imaging = boto3.client('healthimaging')
def get_image_metadata(datastore_id, image_set_id):
response = health_imaging.get_image_set_metadata(
datastoreId=datastore_id,
imageSetId=image_set_id
)
return response['metadata']
Jared emphasizes that this architecture is foundational for “high-throughput” clinical environments. In a traditional setup, moving a slide from storage to a viewer could take minutes; with HealthImaging, it takes milliseconds. This efficiency is critical in pathology, where time-to-diagnosis directly impacts patient outcomes in oncology and acute care.
Empowering Clinicians through Generative AI and Automated Reporting
Once diagnostic data is centralized and accessible, the next challenge is synthesis. Pathologists and radiologists spend a significant portion of their day dictating and transcribing findings. Generative AI offers a transformative solution by automating the creation of structured reports and summarizing complex longitudinal patient histories.
Wilson explains how Philips integrates Amazon Bedrock to assist in the “last mile” of the diagnostic process. By analyzing the metadata and AI-detected features of an image, the system can draft a preliminary report that the clinician then reviews and validates. This doesn’t replace the expert; rather, it removes the “blank page” problem and ensures that reports follow a standardized, high-quality format. Furthermore, LLMs (Large Language Models) can scan years of a patient’s prior records to highlight relevant changes—such as the growth of a lesion over time—that might be missed in a manual review.
Global Collaboration and the Future of Consensus
One of the most profound impacts of shifting integrated diagnostics to the cloud is the enablement of global collaboration. In the analog world, seeking a second opinion on a rare pathology case required physically shipping glass slides across borders—a process that was slow, expensive, and risky.
Through Philips’ cloud-native platform, a specialist in New York can consult on a case in London in real-time. The digital platform supports “shared view” sessions where multiple clinicians can annotate the same slide simultaneously. Dr. Praeloski notes that in recent surveys, 100% of pathologists using the digital system reported that it facilitated reaching a diagnostic consensus more effectively than manual methods. This democratization of expertise is particularly vital for underserved regions, where access to specialized sub-pathologists is limited.
Conclusion: A Paradigm Shift in Precision Medicine
The journey of Philips and AWS illustrates that the future of healthcare is not just about “better machines,” but about “smarter data.” By integrating radiology, cardiology, and pathology into a unified cloud-native framework, they have laid the groundwork for the next generation of precision medicine. This evolution reduces clinical burnout by automating administrative tasks, improves diagnostic accuracy through AI assistance, and accelerates the pace of care through global collaboration. As the system continues to scale, the data captured today will become the training ground for the cures of tomorrow, proving that when diagnostic intelligence is integrated, the potential for clinical innovation is limitless.
Links:
[GoogleIO2024] Under the Hood with Google AI: Exploring Research, Impact, and Future Horizons
Delving into AI’s foundational elements, Jeff Dean, James Manyika, and Koray Kavukcuoglu, moderated by Laurie Segall, discussed Google’s trajectory. Their dialogue traced historical shifts, current breakthroughs, and societal implications, offering profound perspectives on technology’s evolution.
Tracing AI’s Evolution and Key Milestones
Jeff recounted AI’s journey from rule-based systems to machine learning, highlighting neural networks’ resurgence around 2010 due to computational advances. Early applications at Google, like spelling corrections, paved the way for vision, speech, and language tasks. Koray noted hardware investments’ role in enabling generative methods, transforming content creation across fields.
James emphasized AI’s multiplier effect, reshaping sciences like biology and software development. The panel agreed that multimodal, long-context models like Gemini represent culminations of algorithmic and infrastructural progress, allowing generalization to novel challenges.
Addressing Societal Impacts and Ethical Considerations
James stressed AI’s mirror to humanity, prompting grapples with bias, fairness, and values—issues societies must collectively resolve. Koray advocated responsible deployment, integrating safety from inception through techniques like watermarking and red-teaming. Jeff highlighted balancing innovation with safeguards, ensuring models align with human intent while mitigating harms.
Discussions touched on global accessibility, with efforts to support underrepresented languages and equitable benefits. The leaders underscored collaborative approaches, involving diverse stakeholders to navigate complexities.
Envisioning AI’s Future Applications and Challenges
Koray envisioned AI accelerating healthcare, solving diseases efficiently worldwide. Jeff foresaw enhancements across human endeavors, from education to scientific discovery, if pursued thoughtfully. James hoped AI fosters better humanity, aiding complex problem-solving.
Challenges include advancing agentic systems for multi-step reasoning, improving evaluation beyond benchmarks, and ensuring inclusivity. The panel expressed optimism, viewing AI as an amplifier for positive change when guided responsibly.