Mission

The Model Diagnostics Task Force is constituted to develop, coordinate, and implement process-based model evaluation metrics and a metrics framework in National modeling center metrics packages, leveraging ongoing efforts at the modeling centers toward advancing model evaluation and development capabilities.

This Task Force is constituted of researchers whose projects were successfully evaluated as part of the FY15 competition held by NOAA’s Modeling, Analysis, Predictions, and Projections program on the topic of Process Oriented Metrics. This competition was motivated by a number of factors, including community interest in moving beyond performanceoriented metrics toward processoriented metrics, ongoing efforts to develop next-generation climate and Earth system models, evolving plans for CMIP, and a need to link model development and evaluation efforts across modeling centers. Nine projects were selected as a result of the competitive process, including eight focused on developing metrics and one that will develop metrics as well as define a direction and framework for the overall effort.

Task Force Function and Implementation

This Task Force will initiate its activities in October 2015 and will have a duration of three years. It is expected that researchers who were selected through the NOAA competitive process will participate actively in this Task Force helping to build an integrative processoriented metrics framework serving NOAA and other modeling centers. Task Force membership may also include affiliates, elected by Task Force leadership, who are not supported by MAPP funding but whose expertise and work are relevant to achieving the goals of this task force. The Task Force will coordinate with other relevant ongoing diagnostic efforts including emerging community processoriented metrics efforts such as the WCRP/WGCM Metrics Panel, PCMDI’s UVCDAT effort, and the EMBRACE ESMValTool project.

The Task Force will connect all of the funded investigators and other invitees. Beyond enhancing communication between investigators, expectations of this group include contribution by the individual projects to the development and implementation of the collective effort with a focus on implementing a cohesive metrics framework, open documentation of the efforts, and a focus on maximizing the community utility of metrics and the metrics framework. It is expected the main group will have monthly teleconferences and consider inperson meetings, as appropriate.

 

As of Spring 2017, the Task Force has developed a functional Application Program Interface (API) for the metrics package, which is documented here. The linked document also describes the overall status of the effort as of April 2017. The API is python based and designed to be flexible such that metrics coded in different non-proprietary software packages can be integrated into the API.

MAPP Task Force Concept and Terms of Reference

Leadership

Lead: Eric Maloney, Colorado State University
Co-Lead: Yi Ming, NOAA Geophysical Fluid Dynamics Laboratory
Co-Lead: Andrew Gettelman, National Center for Atmospheric Research
Co-Lead: Aiguo Dai, University at Albany

Participants

Relevant MAPP Program PIs and selected additional invitees.
To view the full Participants list, please visit the Participants page.

Projects

To view the full list of Projects, please click here.

News & Events

NOAA Research leads to a new milestone in improving operational predictions from weeks to seasons

  • 27 September 2017
  • Number of views: 1203

Participants at the NMME/SubX Science Meeting held September 13-15 at the National Center for Weather and Climate Prediction.  

 

Typically, the weather and climate have been known as separate communities -- weather being the short-term state of the atmosphere (from minutes out to two weeks) and climate being the long-term pattern of atmospheric conditions (from a season and beyond). This leaves a weather-climate prediction gap (from two weeks to a season) that scientists call the subseasonal to seasonal (S2S) timescale.

Bridging this gap has remained challenging for scientists, but public demand and promising research has focused NOAA’s attention on this prediction problem. As an important milestone for NOAA’s ongoing efforts, researchers from universities, NOAA OAR research laboratories and the National Weather Service (NWS) recently met to discuss efforts to improve S2S predictions.

The meeting focused on the use of real-time forecasts and forecasts of past dates (called hindcasts) from two robust databases: the North American Multi-Model Ensemble (NMME), a state-of-the-art seasonal prediction system combining forecasts from leading climate models; and the Subseasonal Experiment (SubX), an ongoing two-year project testing predictions 3-4 weeks in advance from individual and combinations of multiple global models.

"The SubX project bridges the gap between the NMME forecast and day-to-day weather forecasts," said Ben Kirtman, lead of the SubX project team and University of Miami Rosenstiel School atmospheric scientist. "This is particularly important for hurricanes - we can use the NMME forecast to make seasonal hurricane outlooks and then update these outlooks within the season using the SubX data. Essentially we intend to predict the chances of enhanced (or reduced) hurricane potential 3-4 weeks in advance."

More skillful on average than other seasonal forecast systems, the NMME system transitioned to operations last year and provides a 30-year set of hindcasts that have been used extensively for ongoing research. The SubX project’s data was publicly released this August and the system will be rigorously evaluated over the next year to determine whether the ensemble or any individual models should become operational. Both systems have been developed through OAR research projects as part of the Climate Program Office, Modeling, Analysis, Predictions and Projections Program and the NOAA Climate Test Bed in partnership with the NWS and U.S. other agencies.

Emily Becker, one of the meeting organizers and a research scientist at NOAA’s NWS Climate Prediction Center, saw this meeting as an opportunity to help make connections between different projects. “A lot of people are doing complimentary things,” said Becker. “We wanted to try to build some of those into collaborations going forward.”

Investigators included Paul Dirmeyer, who is studying interactions between soil moisture and the atmosphere at George Mason University and co-leads the NOAA S2S Prediction Task Force, which addressing understanding and modeling of S2S predictability sources. He recognized the value of bringing the research teams together.

“There are a lot of different aspects of the Earth system that are addressable on these subseasonal timescales, and different people are working on different parts,” said Dirmeyer. “We all want to improve the skill of the forecasts.” 

The meeting also importantly brought together operational forecasters and representatives from the public who regularly use these forecasts to make decisions, such as private companies like The Weather Company and Accuweather. This fostered a direct trade of information, bringing new research findings to the forecasters and decision-maker/private sector needs to the scientists who can address them.

“It’s feeding back. What they see and what we see from the inside are often very different,” said Becker regarding the collaboration between the users and scientists. “Knowing that they are using [the forecasts] and hearing how they are and what is useful is helpful to continue developing from the inside.” 

Improving subseasonal to seasonal predictions could substantially help NOAA better prepare decision makers for hazards like heat waves, cold spells, and heavy rain. This meeting represents a milestone towards that mission.

“If the research we are doing leads to changes in operational forecasting that improve the skill of the forecasts, then we’ve done our job,” said Dermeyer.

The NMME/SubX Science Meeting was supported by the OAR CPO’s Modeling, Analysis, Predictions, and Projections Program and was held September 13-15 at the National Center for Weather and Climate Prediction in College Park, Maryland.

For more information, go to: http://cola.gmu.edu/kpegion/nmmeworkshop2017/

Print

Name:
Email:
Subject:
Message:
x

CONTACT US

Climate Program Office
1315 East-West Hwy, Suite 1100
Silver Spring, MD 20910

CPO.webmaster@noaa.gov

ABOUT OUR ORGANIZATION

Americans’ health, security and economic wellbeing are tied to climate and weather. Every day, we see communities grappling with environmental challenges due to unusual or extreme events related to climate and weather. In 2017, the United States experienced a record-tying 16 climate- and weather-related disasters where overall costs reached or exceeded $1 billion. Combined, these events claimed 362 lives, and had significant economic effects on the areas impacted, costing more than $306 billion. Businesses, policy leaders, resource managers and citizens are increasingly asking for information to help them address such challenges.