Accuracy and Efficiency of Deep-Learning–Based Automation of Dual Stain Cytology in Cervical Cancer Screening

Nicolas Wentzensen, MD; Bernd Lahrmann, PhD; Megan A. Clarke, PhD; Walter Kinney, MD; Diane Tokugawa, MD; Nancy Poitras, BS; Alex Locke, MD; Liam Bartels, BS; Alexandra Krauthoff, BS; Joan Walker, MD; Rosemary Zuna, MD; Kiranjit K. Grewal, MS; Patricia E. Goldhoff, MD; Julie D. Kingery, MD; Philip E. Castle, PhD; Mark Schiffman, MD; Thomas S. Lorey, MD; Niels Grabe, PhD


J Natl Cancer Inst. 2021;113(1):72-79. 

In This Article

Abstract and Introduction


Background: With the advent of primary human papillomavirus testing followed by cytology for cervical cancer screening, visual interpretation of cytology slides remains the last subjective analysis step and suffers from low sensitivity and reproducibility.

Methods: We developed a cloud-based whole-slide imaging platform with a deep-learning classifier for p16/Ki-67 dual-stained (DS) slides trained on biopsy-based gold standards. We compared it with conventional Pap and manual DS in 3 epidemiological studies of cervical and anal precancers from Kaiser Permanente Northern California and the University of Oklahoma comprising 4253 patients. All statistical tests were 2-sided.

Results: In independent validation at Kaiser Permanente Northern California, artificial intelligence (AI)-based DS had lower positivity than cytology (P < .001) and manual DS (P < .001) with equal sensitivity and substantially higher specificity compared with both Pap (P < .001) and manual DS (P < .001), respectively. Compared with Pap, AI-based DS reduced referral to colposcopy by one-third (41.9% vs 60.1%, P < .001). At a higher cutoff, AI-based DS had similar performance to high-grade squamous intraepithelial lesions cytology, indicating a risk high enough to allow for immediate treatment. The classifier was robust, showing comparable performance in 2 cytology systems and in anal cytology.

Conclusions: Automated DS evaluation removes the remaining subjective component from cervical cancer screening and delivers consistent quality for providers and patients. Moving from Pap to automated DS substantially reduces the number of colposcopies and also achieves excellent performance in a simulated fully vaccinated population. Through cloud-based implementation, this approach is globally accessible. Our results demonstrate that AI not only provides automation and objectivity but also delivers a substantial benefit for women by reduction of unnecessary colposcopies.


Advances in digital imaging and machine learning can revolutionize cancer screening, diagnosis, and treatment by improving accuracy and reproducibility of image assessment and streamlining clinical workflow.[1–4] With its requirement for high throughput and fast turnaround and its dependence on microscopic and visual technologies, automation can play a major role in improving the efficiency of cervical cancer screening. Many countries are currently switching from Pap cytology to high-risk human papillomavirus (HPV) screening.[5–7] Although a negative HPV test provides great reassurance of low cervical cancer risk over the next decade,[8–10] only a small subset of women with a positive HPV test require further evaluation. To avoid overburdening the system with HPV-positive women, additional triage is required for colposcopy referral.[11,12] Current triage strategies include partial HPV genotyping and Pap cytology.[7,13] The limited sensitivity and reproducibility of cytology require laborious quality control procedures and frequent retesting.[14,15] Improving the efficiency of cervical cancer screening is particularly important for vaccinated populations due to lower disease prevalence and higher demands for screening test performance.

A promising triage strategy is concomitant detection of p16 and Ki-67 in the same cell (p16/Ki-67 dual stain [DS]), 2 markers that are closely linked to cervical carcinogenesis and HPV oncoprotein actions. The HPV oncoprotein E7 interrupts cell cycle control by releasing E2F, activating p16 expression. The coexpression of p16 and Ki-67, a cell proliferation marker, in the same cell is specific to HPV-related carcinogenesis. DS has shown greater accuracy for detection of HPV-related precancers compared with cytology.[16–21] Currently, artificial intelligence (AI) algorithms mostly try to match manual reading accuracy to improve automation but do not offer a substantial improvement for patients. Automated scanning and deep-learning evaluation of DS slides can improve throughput, reproducibility, and accuracy of the assay for better risk stratification and a direct benefit to women.[8,22,23] To achieve this, we developed the CYTOREADER system that combines whole-slide scanning with automated evaluation of DS cytology slides. Cloud-based evaluation provides ample computational capacity and storage space and can provide diagnostic procedures where sufficient personnel, expertise, or infrastructure is lacking. We evaluated the clinical performance of CTYOREADER in 4253 slides from 3 epidemiological studies of HPV-positive cervical and anal precancers.