Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States
Author
Cramer, Estee YRay, Evan L
Lopez, Velma K
Bracher, Johannes
Brennen, Andrea
Castro Rivadeneira, Alvaro J
Gerding, Aaron
Gneiting, Tilmann
House, Katie H
Huang, Yuxin
Jayawardena, Dasuni
Kanji, Abdul H
Khandelwal, Ayush
Le, Khoa
Mühlemann, Anja
Niemi, Jarad
Shah, Apurv
Stark, Ariane
Wang, Yijin
Wattanachit, Nutcha
Zorn, Martha W
Gu, Youyang
Jain, Sansiddh
Bannur, Nayana
Deva, Ayush
Kulkarni, Mihir
Merugu, Srujana
Raval, Alpan
Shingi, Siddhant
Tiwari, Avtansh
White, Jerome
Abernethy, Neil F
Woody, Spencer
Dahan, Maytal
Fox, Spencer
Gaither, Kelly
Lachmann, Michael
Meyers, Lauren Ancel
Scott, James G
Tec, Mauricio
Srivastava, Ajitesh
George, Glover E
Cegan, Jeffrey C
Dettwiller, Ian D
England, William P
Farthing, Matthew W
Hunter, Robert H
Lafferty, Brandon
Linkov, Igor
Mayo, Michael L
Parno, Matthew D
Rowland, Michael A
Trump, Benjamin D
Zhang-James, Yanli
Chen, Samuel
Faraone, Stephen V
Hess, Jonathan
Morley, Christopher P
Salekin, Asif
Wang, Dongliang
Corsetti, Sabrina M
Baer, Thomas M
Eisenberg, Marisa C
Falb, Karl
Huang, Yitao
Martin, Emily T
McCauley, Ella
Myers, Robert L
Schwarz, Tom
Sheldon, Daniel
Gibson, Graham Casey
Yu, Rose
Gao, Liyao
Ma, Yian
Wu, Dongxia
Yan, Xifeng
Jin, Xiaoyong
Wang, Yu-Xiang
Chen, YangQuan
Guo, Lihong
Zhao, Yanting
Gu, Quanquan
Chen, Jinghui
Wang, Lingxiao
Xu, Pan
Zhang, Weitong
Zou, Difan
Biegel, Hannah
Lega, Joceline
McConnell, Steve
Nagraj, V P
Guertin, Stephanie L
Hulme-Lowe, Christopher
Turner, Stephen D
Shi, Yunfeng
Ban, Xuegang
Walraven, Robert
Hong, Qi-Jun
Kong, Stanley
van de Walle, Axel
Turtle, James A
Ben-Nun, Michal
Riley, Steven
Riley, Pete
Koyluoglu, Ugur
DesRoches, David
Forli, Pedro
Hamory, Bruce
Kyriakides, Christina
Leis, Helen
Milliken, John
Moloney, Michael
Morgan, James
Nirgudkar, Ninad
Ozcan, Gokce
Piwonka, Noah
Ravi, Matt
Schrader, Chris
Shakhnovich, Elizabeth
Siegel, Daniel
Spatz, Ryan
Stiefeling, Chris
Wilkinson, Barrie
Wong, Alexander
Cavany, Sean
España, Guido
Moore, Sean
Oidtman, Rachel
Perkins, Alex
Kraus, David
Kraus, Andrea
Gao, Zhifeng
Bian, Jiang
Cao, Wei
Lavista Ferres, Juan
Li, Chaozhuo
Liu, Tie-Yan
Xie, Xing
Zhang, Shun
Zheng, Shun
Vespignani, Alessandro
Chinazzi, Matteo
Davis, Jessica T
Mu, Kunpeng
Pastore Y Piontti, Ana
Xiong, Xinyue
Zheng, Andrew
Baek, Jackie
Farias, Vivek
Georgescu, Andreea
Levi, Retsef
Sinha, Deeksha
Wilde, Joshua
Perakis, Georgia
Bennouna, Mohammed Amine
Nze-Ndong, David
Singhvi, Divya
Spantidakis, Ioannis
Thayaparan, Leann
Tsiourvas, Asterios
Sarker, Arnab
Jadbabaie, Ali
Shah, Devavrat
Della Penna, Nicolas
Celi, Leo A
Sundar, Saketh
Wolfinger, Russ
Osthus, Dave
Castro, Lauren
Fairchild, Geoffrey
Michaud, Isaac
Karlen, Dean
Kinsey, Matt
Mullany, Luke C
Rainwater-Lovett, Kaitlin
Shin, Lauren
Tallaksen, Katharine
Wilson, Shelby
Lee, Elizabeth C
Dent, Juan
Grantz, Kyra H
Hill, Alison L
Kaminsky, Joshua
Kaminsky, Kathryn
Keegan, Lindsay T
Lauer, Stephen A
Lemaitre, Joseph C
Lessler, Justin
Meredith, Hannah R
Perez-Saez, Javier
Shah, Sam
Smith, Claire P
Truelove, Shaun A
Wills, Josh
Marshall, Maximilian
Gardner, Lauren
Nixon, Kristen
Burant, John C
Wang, Lily
Gao, Lei
Gu, Zhiling
Kim, Myungjin
Li, Xinyi
Wang, Guannan
Wang, Yueying
Yu, Shan
Reiner, Robert C
Barber, Ryan
Gakidou, Emmanuela
Hay, Simon I
Lim, Steve
Murray, Chris
Pigott, David
Gurung, Heidi L
Baccam, Prasith
Stage, Steven A
Suchoski, Bradley T
Prakash, B Aditya
Adhikari, Bijaya
Cui, Jiaming
Rodríguez, Alexander
Tabassum, Anika
Xie, Jiajia
Keskinocak, Pinar
Asplund, John
Baxter, Arden
Oruc, Buse Eylul
Serban, Nicoleta
Arik, Sercan O
Dusenberry, Mike
Epshteyn, Arkady
Kanal, Elli
Le, Long T
Li, Chun-Liang
Pfister, Tomas
Sava, Dario
Sinha, Rajarishi
Tsai, Thomas
Yoder, Nate
Yoon, Jinsung
Zhang, Leyou
Abbott, Sam
Bosse, Nikos I
Funk, Sebastian
Hellewell, Joel
Meakin, Sophie R
Sherratt, Katharine
Zhou, Mingyuan
Kalantari, Rahi
Yamana, Teresa K
Pei, Sen
Shaman, Jeffrey
Li, Michael L
Bertsimas, Dimitris
Skali Lami, Omar
Soni, Saksham
Tazi Bouardi, Hamza
Ayer, Turgay
Adee, Madeline
Chhatwal, Jagpreet
Dalgic, Ozden O
Ladd, Mary A
Linas, Benjamin P
Mueller, Peter
Xiao, Jade
Wang, Yuanjia
Wang, Qinxia
Xie, Shanghong
Zeng, Donglin
Green, Alden
Bien, Jacob
Brooks, Logan
Hu, Addison J
Jahja, Maria
McDonald, Daniel
Narasimhan, Balasubramanian
Politsch, Collin
Rajanala, Samyak
Rumack, Aaron
Simon, Noah
Tibshirani, Ryan J
Tibshirani, Rob
Ventura, Valerie
Wasserman, Larry
O'Dea, Eamon B
Drake, John M
Pagano, Robert
Tran, Quoc T
Ho, Lam Si Tung
Huynh, Huong
Walker, Jo W
Slayton, Rachel B
Johansson, Michael A
Biggerstaff, Matthew
Reich, Nicholas G
Affiliation
Department of Mathematics, University of ArizonaIssue Date
2022-04-08
Metadata
Show full item recordPublisher
National Academy of SciencesCitation
Cramer, E. Y., Ray, E. L., Lopez, V. K., Bracher, J., Brennen, A., Castro Rivadeneira, A. J., Gerding, A., Gneiting, T., House, K. H., Huang, Y., Jayawardena, D., Kanji, A. H., Khandelwal, A., Le, K., Mühlemann, A., Niemi, J., Shah, A., Stark, A., Wang, Y., … Reich, N. G. (2022). Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States. Proceedings of the National Academy of Sciences of the United States of America.Rights
Copyright © 2022 the Author(s). Published by PNAS. This open access article is distributed under Creative Commons Attribution License 4.0 (CC BY).Collection Information
This item from the UA Faculty Publications collection is made available by the University of Arizona with support from the University of Arizona Libraries. If you have questions, please contact us at repository@u.library.arizona.edu.Abstract
Significance: This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the United States. Results show high variation in accuracy between and within stand-alone models and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public-health action.Note
Open access articleEISSN
1091-6490PubMed ID
35394862Version
Final published versionae974a485f413a2113503eed53cd6c53
10.1073/pnas.2113561119
Scopus Count
Collections
Except where otherwise noted, this item's license is described as Copyright © 2022 the Author(s). Published by PNAS. This open access article is distributed under Creative Commons Attribution License 4.0 (CC BY).
Related articles
- Comparative assessment of methods for short-term forecasts of COVID-19 hospital admissions in England at the local level.
- Authors: Meakin S, Abbott S, Bosse N, Munday J, Gruson H, Hellewell J, Sherratt K, CMMID COVID-19 Working Group., Funk S
- Issue date: 2022 Feb 21
- How New Mexico Leveraged a COVID-19 Case Forecasting Model to Preemptively Address the Health Care Needs of the State: Quantitative Analysis.
- Authors: Castro LA, Shelley CD, Osthus D, Michaud I, Mitchell J, Manore CA, Del Valle SY
- Issue date: 2021 Jun 9
- Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S.
- Authors: Reich NG, McGowan CJ, Yamana TK, Tushar A, Ray EL, Osthus D, Kandula S, Brooks LC, Crawford-Crudell W, Gibson GC, Moore E, Silva R, Biggerstaff M, Johansson MA, Rosenfeld R, Shaman J
- Issue date: 2019 Nov
- Interval forecasts of weekly incident and cumulative COVID-19 mortality in the United States: A comparison of combining methods.
- Authors: Taylor KS, Taylor JW
- Issue date: 2022
- Combining Probabilistic Forecasts of COVID-19 Mortality in the United States.
- Authors: Taylor JW, Taylor KS
- Issue date: 2021 Jun 28
Related items
Showing items related by title, author, creator and subject.
-
LSST: From Science Drivers to Reference Design and Anticipated Data ProductsIvezić, Željko; Kahn, Steven M.; Tyson, J. Anthony; Abel, Bob; Acosta, Emily; Allsman, Robyn; Alonso, David; AlSayyad, Yusra; Anderson, Scott F.; Andrew, John; et al. (IOP PUBLISHING LTD, 2019-03-11)We describe here the most ambitious survey currently planned in the optical, the Large Synoptic Survey Telescope (LSST). The LSST design is driven by four main science themes: probing dark energy and dark matter, taking an inventory of the solar system, exploring the transient optical sky, and mapping the Milky Way. LSST will be a large, wide-field ground-based system designed to obtain repeated images covering the sky visible from Cerro Pachon in northern Chile. The telescope will have an 8.4 m (6.5 m effective) primary mirror, a 9.6 deg(2) field of view, a 3.2-gigapixel camera, and six filters (ugrizy) covering the wavelength range 320-1050 nm. The project is in the construction phase and will begin regular survey operations by 2022. About 90% of the observing time will be devoted to a deep-wide-fast survey mode that will uniformly observe a 18,000 deg(2) region about 800 times (summed over all six bands) during the anticipated 10 yr of operations and will yield a co-added map to r similar to 27.5. These data will result in databases including about 32 trillion observations of 20 billion galaxies and a similar number of stars, and they will serve the majority of the primary science programs. The remaining 10% of the observing time will be allocated to special projects such as Very Deep and Very Fast time domain surveys, whose details are currently under discussion. We illustrate how the LSST science drivers led to these choices of system parameters, and we describe the expected data products and their characteristics.
-
Event Horizon Telescope imaging of the archetypal blazar 3C 279 at an extreme 20 microarcsecond resolutionKim, Jae-Young; Krichbaum, Thomas P.; Broderick, Avery E.; Wielgus, Maciek; Blackburn, Lindy; Gomez, Jose L.; Johnson, Michael D.; Bouman, Katherine L.; Chael, Andrew; Akiyama, Kazunori; et al. (EDP SCIENCES S A, 2020-08)3C 279 is an archetypal blazar with a prominent radio jet that show broadband flux density variability across the entire electromagnetic spectrum. We use an ultra-high angular resolution technique - global Very Long Baseline Interferometry (VLBI) at 1.3 mm (230 GHz) - to resolve the innermost jet of 3C 279 in order to study its fine-scale morphology close to the jet base where highly variable gamma -ray emission is thought to originate, according to various models. The source was observed during four days in April 2017 with the Event Horizon Telescope at 230 GHz, including the phased Atacama Large Millimeter/submillimeter Array (ALMA), at an angular resolution of similar to 20 mu as (at a redshift of z=0.536 this corresponds to similar to 0.13 pc similar to 1700 Schwarzschild radii with a black hole mass M-BH=8x10(8) M-circle dot). Imaging and model-fitting techniques were applied to the data to parameterize the fine-scale source structure and its variation. We find a multicomponent inner jet morphology with the northernmost component elongated perpendicular to the direction of the jet, as imaged at longer wavelengths. The elongated nuclear structure is consistent on all four observing days and across different imaging methods and model-fitting techniques, and therefore appears robust. Owing to its compactness and brightness, we associate the northern nuclear structure as the VLBI "core". This morphology can be interpreted as either a broad resolved jet base or a spatially bent jet. We also find significant day-to-day variations in the closure phases, which appear most pronounced on the triangles with the longest baselines. Our analysis shows that this variation is related to a systematic change of the source structure. Two inner jet components move non-radially at apparent speeds of similar to 15 c and similar to 20 c (similar to 1.3 and similar to 1.7 mu as day(-1), respectively), which more strongly supports the scenario of traveling shocks or instabilities in a bent, possibly rotating jet. The observed apparent speeds are also coincident with the 3C 279 large-scale jet kinematics observed at longer (cm) wavelengths, suggesting no significant jet acceleration between the 1.3 mm core and the outer jet. The intrinsic brightness temperature of the jet components are less than or similar to 10(10) K, a magnitude or more lower than typical values seen at >= 7 mm wavelengths. The low brightness temperature and morphological complexity suggest that the core region of 3C 279 becomes optically thin at short (mm) wavelengths.
-
Genetic determinants of risk in pulmonary arterial hypertension: international genome-wide association studies and meta-analysisRhodes, Christopher J; Batai, Ken; Bleda, Marta; Haimel, Matthias; Southgate, Laura; Germain, Marine; Pauciulo, Michael W; Hadinnapola, Charaka; Aman, Jurjan; Girerd, Barbara; et al. (ELSEVIER SCI LTD, 2019-03-01)Rare genetic variants cause pulmonary arterial hypertension, but the contribution of common genetic variation to disease risk and natural history is poorly characterised. We tested for genome-wide association for pulmonary arterial hypertension in large international cohorts and assessed the contribution of associated regions to outcomes. We did two separate genome-wide association studies (GWAS) and a meta-analysis of pulmonary arterial hypertension. These GWAS used data from four international case-control studies across 11 744 individuals with European ancestry (including 2085 patients). One GWAS used genotypes from 5895 whole-genome sequences and the other GWAS used genotyping array data from an additional 5849 individuals. Cross-validation of loci reaching genome-wide significance was sought by meta-analysis. Conditional analysis corrected for the most significant variants at each locus was used to resolve signals for multiple associations. We functionally annotated associated variants and tested associations with duration of survival. All-cause mortality was the primary endpoint in survival analyses. A locus near SOX17 (rs10103692, odds ratio 1·80 [95% CI 1·55-2·08], p=5·13 × 10 This is the first study to report that common genetic variation at loci in an enhancer near SOX17 and in HLA-DPA1/DPB1 is associated with pulmonary arterial hypertension. Impairment of SOX17 function might be more common in pulmonary arterial hypertension than suggested by rare mutations in SOX17. Further studies are needed to confirm the association between HLA typing or rs2856830 genotyping and survival, and to determine whether HLA typing or rs2856830 genotyping improves risk stratification in clinical practice or trials.