Supplement 1 Supplementary methods.
COVID-19 Incidence Data
County-level COVID-19 incidence data was obtained from a database developed by the New York Times that was based on data from public health agencies using methods defined by the Centers for Disease Control and Prevention for reporting aggregated cumulative counts of COVID-19 cases at the county level over time.1–3 Data were incorporated into a choropleth map also depicting the location of participating hospitals.
For some patients, data were missing or unavailable for a subset of the prespecified candidate risk factors for mortality and/or respiratory failure. Before applying L1 (lasso) penalized regression,4–6 we therefore employed multiple imputation using chained equations to avoid bias in the resulting analysis.7–9 Multiple imputation ensures validity of inference if data is missing at random (missingness pattern and data values are independent conditional on observed data), the most general data-centric missing data assumption.10
Selection of Risk Factors for Inclusion in Multivariable Models
A team of experienced physicians, clinical researchers, and epidemiologists prespecified candidate risk factors for each outcome on the basis of reported association, plausibility, data availability (missingness < 20%), and clinical utility. We maintained a ratio of >10:1 outcome events to candidate risk factors to reduce instability in variable selection and estimation.11,12 Mortality and late-onset respiratory failure candidate risk factors were restricted to data available before or within 24 hours of hospital arrival (eg, highest heart rate in the first 24 hours). Candidate risk factors for early-onset respiratory failure included only data available before or at hospital arrival (eg, first-measured heart rate).
To select lasso tuning parameters λ, multiply imputed datasets were stacked and cross-validation was performed patient-wise.13–15 Analyses were based on 10 multiply imputed datasets and 10-fold cross-validation across 100 values of the tuning parameter. Each cross-validated model included a random effect for admission hospital.16 For each value of the tuning parameter, the average cross- validated area under the receiver operating characteristic curve (AUC) was calculated as the mean AUC across all iterations. The relationship between the tuning parameter and the associated models' average cross-validated AUC was smoothed via locally weighted regression (lowess) curves (Supplemental Figure 1),17 the highest average cross-validated AUC tuning parameter was identified, and the most parsimonious model with an average AUC within 1 standard deviation of the maximal average AUC was selected.18
Patient Inclusion in Risk Factor Analyses
For analysis of risk factors for mortality, we excluded patients (n = 10) suspected to have been moribund at presentation based on survival <24 hours from hospital arrival. To avoid misclassification of the outcome, evaluation of early respiratory failure risk factors excluded patients (n = 43) who were transferred to the study hospital from an inpatient unit of another hospital. Patients transferred from the emergency department of another hospital were included. Finally, analysis of late respiratory failure risk factors was restricted to patients (n = 1196) who did not die or develop respiratory failure within 24 hours of arrival at the study hospital.
The New York Times Github. Coronavirus (COVID-19) data in the United States. Accessed February 2, 2021. https://github.com/nytimes/covid-19-data
Turner K, Davidson SL, Collins J, et al. Council of State and Territorial Epidemiologists. Standardized surveillance case definition and national notification for 2019 novel coronavirus disease (COVID-19). 2020. Accessed February 18, 2021. https://www.cste.org/resource/resmgr/ps/positionstatement2020/Interim-20-ID-02_COVID-19.pdf
Centers for Disease Control and Prevention. Cases, Data, and Surveillance: About CDC COVID-19 Data. 2020. Accessed December 17, 2020. https://www.cdc.gov/coronavirus/2019-ncov/cases-updates/about-us-cases-deaths.html
Efron B, Hastie T, Johnstone I, et al. Least angle regression. Ann Stat. 2004;32(2):407–499.
Zou H, Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc B. 2005;67(2):301–320.
Buuren SV, Groothuis-Oudshoorn K. MICE: Multivariate imputation by chained equations in R. J Stat Soft. 2011;45(3).
Rubin DB. Inference and missing data. Biometrika. 1976;63(3):581–592.
White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2010;30(4):377–399.
van der Heijden GJMG, T Donders AR, Stijnen T, et al. Imputation of missing values is superior to complete case analysis and the missing-indicator method in multivariable diagnostic research: a clinical example. J Clin Epidemiol. 2006;59(10):1102–1109.
Rubin DB. Multiple imputation after 18+ years. J Am Stat Assoc. 1996;91(434):473.
Peduzzi P, Concato J, Kemper E, et al. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373–1379.
Wynants L, Bouwmeester W, Moons KGM, et al. A simulation study of sample size demonstrated the importance of the number of events per variable to develop prediction models in clustered data. J Clin Epidemiol. 2015;68(12):1406–1414.
Wood AM, White IR, Royston P. How should variable selection be performed with multiply imputed data? Stat Med. 2008;27(17):3227–3246.
Du J, Boss J, Han P, et al. Variable selection with multiply-imputed datasets: choosing between stacked and grouped methods. arXiv. 2020.
Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. Proceedings of the International Joint Conference on Artificial Intelligence. IJCAI (U S). 1995;14(2):1137–1143. https://www.ijcai.org/Proceedings/95-2/Papers/016.pdf
Schelldorfer J, Meier L, Bühlmann P. GLMMLasso: an algorithm for high-dimensional generalized linear mixed models using L1-penalization. J Comput Graph Stat. 2014;23(2):460–477.
Cleveland WS, Devlin SJ. Locally weighted regression: an approach to regression analysis by local fitting. J Am Stat Assoc. 1988;83(403):596–610.
Hastie T, Tibshirani R, Friedman JH. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed. Springer; 2017.
Supplement 2 PETAL Network Investigators and Collaborating Personnel
ALIGNE Clinical Center: Baystate Medical Center—Lori Kozikowski, Sherell Thornton-Thompson, Leslie de Souza, Sarah Romain, Cynthia Kardos; Brigham and Women's Hospital—Rebecca M Baron, Mayra Pinilla Vera, Antonio Arciniegas
BOSTON Clinical Center: Beth Israel Deaconess Medical Center—Valerie Banner-Goodspeed, Marie McGourty, Lauren Kelly, Krystal Capers, Melisa Joseph; Massachusetts General Hospital—Kathryn Hibbert, Kelsey Brait, Natalie Pulido, Layla Rahimi, Anna Nicole Dartley; University of Mississippi Medical Center—Alan E. Jones, Rebekah Peacock, Utsav Nandi, James Galbraith, Kelsey Kunk Morgan; Hennepin County Medical Center—Paige DeVries, Brian Driver, Audrey Hendrickson, Michael Puskarich, Jamie Stang
CALIFORNIA Clinical Center: UCSF San Francisco—Kimberly Yee, Brian Daniel; UCLA—Nida Qadir, Steven Chang, Gregory Hendey, George Lim, Andrea Tam, Rebecca Beutler, Trisha Agarwal, Julia Vargas; Stanford University Hospital—Jonasel Roque, Rosemary Vojnik; UC Davis—Skyler J. Pearson; UCSF Fresno—Alyssa Hughes; University of Texas—Elizabeth Vidales
COLORADO Clinical Center: University of Colorado Hospital—Marc Moss*, Adit Ginde*, Carrie Higgins, Jeffrey McKeehan, Lani Finck, Michelle Howell, Jennifer Friedel; Denver Health Medical Center—Jason S. Haukoos, Ivor Douglas, Carolynn Lyle, Stephanie Gravitz, Terra Hiller, Judy Oakes, Alicia Cupelo; National Jewish Health I Saint Joseph's Hospital—James Finigan, Christine Griesmer
MICHIGAN Clinical Center: University of Michigan Medical Center—Robert C. Hyzy, Pauline K. Park, Kristine Nelson, Norman Olbrich, Kelli McDonough, Stephen Kay, Andrew Admon, Theodore J. Iwashyna; Henry Ford—Namita Jayaprakash, Emanuel P. Rivers, Jasreen Kaur Gill, Anja Kathrina Jaehne, Aaron Cook, Jennifer Swiderek, Jacqueline Day; Wayne State University—Robert Sherwin, James Wooden, Thomas Mazzocco, Jeffrey Harrison, Theodore Falcon
MONTEFIORE-SINAI Clinical Center: Montefiore Moses—Michelle Ng Gong*, Rahul Nair, Omowunmi Amosu, Hiwet Tzehaie, Ayesha Asghar, Aluko A. Hope, Jen-Ting (Tina) Chen; Montefiore Weiler—Brenda Lopez, Caroline Boyle, Tori Aspir, Alexandra Gordon, Bryan Musmacker; University of Arizona—Jarrod Mosier, Cameron Hypes, Elizabeth Salvaggio, Boris Gilson, Jonathan Blohm; Mt. Sinai Hospital—Lynne Richardson, Kusum Mathews, Maxime Centeno, Sam Acquah, Patrick Maher, Neha Goel, George Loo
OHIO Clinical Center: Cleveland Clinic Foundation—Abhijit Duggal, Omar Mehkri, Alexander King, Stuart Houltham; University of Cincinnati Medical Center—Kristin M. Hudock, Robert Duncan Hite*, Nicole Hummel, Jessica Anderson, Tammy Roads
PACIFIC NORTHWEST Clinical Center: Harborview Medical Center—Nicholas J Johnson, Bryce RH Robinson*, Anna Ungar, Sarah Katsandres, Stephanie Gundel; University of Washington—T. Eoin West, Natalie L Cobb, Lara Lovelace-Macon, Denisse Bazan Dow, Navya Garimelle, Ellen Caldwell, Engi Attia; Oregon Health and Science University—Catherine L Hough*, Akram Khan, Olivia Krol, Stephanie Nonas, Nicole Fontanese, Kelly Vranas; Swedish Hospital—Shane O'Mahony, Julie Wallick; Cedars Sinai Medical Center—Tanzira Zaman, Giuliana Cerro Chiang, June Choe, Lisa Herrera, Niree Hindoyan
PITTSBURGH Clinical Center: UPMC—Derek C. Angus, Donald M. Yealy, Elizabeth Gimbel, Denise Scholl, Sara DiFiore, Hunter Skroczky, Amy Magoun, Alexandra Weissman, David T. Huang; Temple University—George Souiarov, Hannah Reimer, Lillian Finlaw, Sarah Loughran
SOUTHEAST Clinical Center: Wake Forest Baptist Health—D. Clark Files, Chadwick Miller, Lauren Koehler; Virginia Commonwealth University Medical Center—Marjolein de Wit, Jessica Mason; Medical University of South Carolina—Andrew J. Goodwin, Abbey Grady, Katie Kirchoff; University of Virginia—Jeff Sturek, Mary Marshall
UTAH: Intermountain Medical Center—Samuel Brown*, Ithan Peltan, Julia Bryan, Jason Jacobs, Brent Armbruster, Joseph Bledsoe*, Rebecca Roper, Michael Lanspa; University of Utah Hospital—Estelle S. Harris, Lisa J. Weaver, Macy A.G. Barrios, Lindsey J. Waddoups, Ann M. Lyons, Robert Paine III, Benjamin A. Haaland
VANDERBILT Clinical Center: Duke University Medical Center—John Eppensteiner, Grace Hall, Andrew Bouffler, Lauren McGowan, Sam Francis, Bria Hall; Louisiana State University Health Sciences Center—Bennet deBoisblanc, Paula Lauto, Connie Romaine, Marie Sandi; Vanderbilt University Medical Center—Todd W. Rice*, Wesley H. Self*, Margaret Hays, Megan Roth, Jakea Johnson
Clinical Coordinating Center: Massachusetts General Hospital Biostatistics Center (CCC)—David A. Schoenfeld*, B. Taylor Thompson*, Douglas L. Hayden, Nancy Ringwood, Cathryn Oldmixon, Christine Ulysse, Richard Morse, Ariela Muzikansky, Laura Fitzgerald, Samuel Whitaker, Alexander Nagrebetsky
Johns Hopkins Medicine: Roy G. Brower, Sarina Sahetya
National Heart, Lung, and Blood Institute: Lora Reineck, Neil Aggarwal, Karen Bienstock, Michelle Freemer, Myron Maclawiw, Gail Weinmann
Protocol Review Committee: Laurie J. Morrison, Mark N. Gillespie, Richard J. Kryscio, Wojciech Zareba, Anne Rompalo, Michael Boeckh
Data and Safety Monitoring Board: Polly Parsons, Jason D. Christie, Jesse R. Hall, Nicholas J. Horton, Laurie S. Zoloth, Neal Dickert Jr, Deborah Diercks
*Clinical Center or CCC Principal Investigator.
The authors thank Dr Angela Presson for input regarding statistical analyses and Alison Pollock, Xiaoqi Bao, and Dr Bo Zhao for assistance creating data visualizations. See the list of PETAL Network contributors in Supplement 2 (available online only).
Am J Crit Care. 2022;31(2):146-157. © 2022 American Association of Critical-Care Nurses