Numerical Weather Prediction (NWP) is the cornerstone of modern environmental forecasting. The United States uses a large variety of NWPs for its operational weather forecasting activities at the National Oceanic and Atmospheric Administration (NOAA), ranging from large synoptic-scale models such as the Global Forecast System (GFS), and short-range convection-allowing models such as the High-Resolution Rapid Refresh (HRRR). Such a wide range of modeling systems and products have been a challenge to streamline community innovations and advance operational NWPs. Former NOAA Acting Administrator Neil Jacobs spearheaded a vision to address existing challenges, by unifying weather models in an open-source framework that is overhauling modern forecasting capabilities leveraging the Unified Forecast System (UFS).
Jacobs recently recounted the history of his efforts in establishing the Earth Prediction Innovation Center (EPIC) in support of the UFS in his American Meteorological Society paper titled ‘Open Innovation and the Case for Community Model Development’. The paper describes the ideas that led to the NOAA and National Center for Atmospheric Research (NCAR) memorandum, along with the Weather Research and Forecasting Innovation Act of 2017, which set in motion “the most significant restructuring of coordination efforts among federal agencies since ”. Jacobs’ paper also discusses the challenges that will be faced in restructuring the models and the need for close integration of the government with the broader weather modeling community, including universities, research centers, and private industry.
The baseline challenge of modern government NWP, as the paper describes, is how the models were created. The GFS was designed using specific code to work only under NOAA’s high-performance computing (HPC) system. Due to this constraint, NOAA models and operations are isolated from the private sector and academia. Government policies, such as requiring a security clearance to work on the models, only exacerbated the problem. This has created a disfunction in the weather modeling world, where industry and academic modeling efforts and NOAA’s projects would compete for funding from the federal government. The solution described in the paper originated from an unprecedented collaboration between NOAA and the community, opening government modeling systems and data to open-source development and collaboration, better allowing the private and academic sectors to collaborate with the Federal government.
Jacobs’ paper states the ultimate goal of open-sourced unified forecasting where all stakeholders are able to contribute and benefit. The ability to parallel-test a needed feature in the cloud, and eventually be able to implement it into NOAA’s operational source code allows all parties involved to achieve several benefits from the unified model. This allows for a wide range of stakeholders to diversify features in the unified model, which allows even more users to access desired features and accelerate the rate of scientific innovations into operational forecast systems and products.
Open-source software and the proposed open-development approach allow for greater community outreach and collaboration. As pointed out in the paper, UFS community members from high school students to professionals using cloud HPC compiled and ran the UFS model, and gave feedback directly to Jacobs, leading him to realize that open-source development is not perfect and a process is needed to allow for efficient and stable modeling, that is where the Earth Prediction Innovation Center (EPIC) comes in. EPIC is a program that will support the development of the UFS bringing the community together using cloud resources. Jacobs expects that EPIC will accelerate the flow of community innovations to operational NOAA models, through a process that can be roughly illustrated as a funnel where new additions to the UFS are vetted in a series of tests from the UFS community, and gradually transition into higher operational readiness levels before becoming operational.
The vision described in Neil Jacobs’ paper, built through open collaboration with community scientists and NOAA, lays the groundwork to strengthen the UFS and leverage the awesome capabilities of the American Earth sciences community, the largest in the world. This unique open-development modeling vision will tool the UFS to be suited to any government, private sector, and academic needs. When realized, the shared vision of the UFS leveraged by EPIC, championed by Neil Jacobs, will change the modeling landscape forever.
Read more in Neil Jacobs’ original paper.
Neil Jacobs is the former acting NOAA administrator and early pioneer in EPIC and Unified modeling. He holds a BS in Mathematics and Physics from the University of South Carolina and a PhD. in atmospheric science from North Carolina State University and has worked in both the public and private sectors for the past 15 years, most notably for Panasonic. He served as the chair of forecasting improvement for the American Meteorological Society, and was affiliated with the World Meteorological Association.