Stanford scientists develop a method for predicting unprecedented events

A black swan event is a highly unlikely but massively consequential incident, such as the 2008 global recession and the loss of one-third of the world's saiga antelope in a matter of days in 2015. Challenging the quintessentially unpredictable nature of black swan events, bioengineers at Stanford University are suggesting a method for forecasting these supposedly unforeseeable fluctuations.

"By analyzing long-term data from three ecosystems, we were able to show that fluctuations that happen in different biological species are statistically the same across different ecosystems," said Samuel Bray, a research assistant in the lab of Bo Wang, assistant professor of bioengineering at Stanford. "That suggests there are certain underlying universal processes that we can take advantage of in order to forecast this kind of extreme behavior."

The forecasting method the researchers have developed, which was detailed recently in PLOS Computational Biology, is based on natural systems and could find use in health care and environmental research. It also has potential applications in disciplines outside ecology that have their own black swan events, such as economics and politics.

"This work is exciting because it's a chance to take the knowledge and the computational tools that we're building in the lab and use those to better understand - even predict or forecast - what happens in the world surrounding us," said Wang, who is the senior author of the paper. "It connects us to the bigger world."

From microbes to avalanches 

{module INSIDE STORY}

Over years of studying microbial communities, Bray noticed several instances where one species would undergo an unanticipated population boom, overtaking its neighbors. Discussing these events with Wang, they wondered whether this phenomenon occurred outside the lab as well and, if so, whether it could be predicted.

In order to address this question, the researchers had to find other biological systems that experience black swan events. The researchers needed details, not only about the black swan events themselves but also the context in which they occurred. So, they specifically sought ecosystems that scientists have been closely monitoring for many years.

"These data have to capture long periods of time and that's hard to collect," said Bray, who is lead author of the paper. "It's much more than a PhD-worth of information. But that's the only way you can see the spectra of these fluctuations at large scales."

Bray settled on three eclectic datasets: an eight-year study of plankton from the Baltic Sea with species levels measured twice weekly; net carbon measurements from a deciduous broadleaf forest at Harvard University, gathered every 30 minutes since 1991; and measurements of barnacles, algae, and mussels on the coast of New Zealand, taken monthly for over 20 years.

The researchers then analyzed these three datasets using theory about avalanches - physical fluctuations that, like black swan events, exhibit short-term, sudden, extreme behavior. At its core, this theory attempts to explain the physics of systems like avalanches, earthquakes, fire embers, or even crumpling candy wrappers, which all respond to external forces with discrete events of various magnitudes or sizes - a phenomenon scientists call "crackling noise."

Built on the analysis, the researchers developed a method for predicting black swan events, one that is designed to be flexible across species and timespans, and able to work with data that are far less detailed and more complex than those used to develop it.

"Existing methods rely on what we have seen to predict what might happen in the future, and that's why they tend to miss black swan events," said Wang. "But Sam's method is different in that it assumes we are only seeing part of the world. It extrapolates a little about what we're missing, and it turns out that helps tremendously in terms of prediction."

Forecasting in the real world

The researchers tested their method using the three ecosystem datasets on which it was built. Using only fragments of each dataset - specifically fragments which contained the smallest fluctuations in the variable of interest - they were able to accurately predict extreme events that occurred in those systems.

They would like to expand the application of their method to other systems in which black swan events are also present, such as in economics, epidemiology, politics, and physics. At present, the researchers are hoping to collaborate with field scientists and ecologists to apply their method to real-world situations where they could make a positive difference in the lives of other people and the planet.

AI model to forecast complicated large-scale tropical instability waves in Pacific Ocean

Large-scale oceanic phenomena are complicated and often involve many natural processes. Tropical instability wave (TIW) is one of these phenomena.

Pacific TIW, a prominent prevailing oceanic event in the eastern equatorial Pacific Ocean, is featured with cusp-shaped waves propagating westward at both flanks of the tropical Pacific cold tongue.

The forecast of TIW has long been dependent on physical equation-based numerical models or statistical models. However, many natural processes need to be considered for understanding such complicated phenomena.

Recently, a research team led by Prof. LI Xiaofeng from the Institute of Oceanology of the Chinese Academy of Sciences (IOCAS) studied this type of complex oceanic phenomena through artificial intelligence (AI) technologies.

The team member includes ZHENG Gang from the Second Institute of Oceanology of Ministry of Natural Resources, ZHANG Ronghua from IOCAS, and LIU Bin from Shanghai Ocean University.

They used satellite data-driven deep learning model to forecast the complicated thousand-kilometer scale TIW for the first time in the world. Their study was published in Science Advances on July 15.

Basic rules governing the complicated oceanic phenomena are usually profoundly hidden in the fast-increasing satellite remote sensing big data itself. They need to be dug up by powerful information mining techniques such as deep learning in the AI field.

"AI technology may lead to a promising alternative for modeling complicated oceanic phenomena and circumventing the difficulties faced by traditional numerical models," said Prof. LI.

In this work, the researchers developed a deep learning model for forecasting sea surface temperature (SST) field associated with TIW based on current and previous satellite-derived SST data.

The long-term test of nine-year SST data showed that the model efficiently and accurately forecasted SST evolution and captured TIW propagation's spatial and temporal variation.

The study demonstrates that a purely data-driven and AI-based information mining paradigm can be a robust and promising way to model and forecast complicated oceanic phenomena in the satellite remote sensing Big Data Era. 

"AI-based models, statistical models, and traditional numerical models can complement each other and provide a novel perspective for studying complicated oceanic features," said Prof. LI.

A review article by Prof. LI's group was published in National Science Review on March 19, which systematically reviewed deep-learning-based information mining from ocean remote-sensing imagery.

Big data helps farmers adapt to climate variability

A new Michigan State University study shines a light on how big data and digital technologies can help farmers better adapt to threats -- both present and future -- from a changing climate.

The study, published in an academic journal, is the first to precisely quantify soil and landscape features and spatial and temporal yield variations in response to climate variability. It is also the first to use big data to identify areas within individual fields where yield is unstable.

Between 2007 and 2016, the U.S. economy took an estimated $536 million economic hit because of yield variation in unstable farmland caused by climate variability across the Midwest. More than one-quarter of corn and soybean cropland in the region is unstable. Yields fluctuate between over-performing and underperforming on an annual basis. The study shines a light on how big data and digital technologies can help farmers better adapt to threats -- both present and future -- from a changing climate.{module INSIDE STORY}

Bruno Basso, MSU Foundation professor of earth and environmental sciences, and his postdoctoral research fellow, Rafael Martinez-Feria, set out to address the key pillars of the National Institute for Food and Agriculture's Coordinated Agricultural Project that Basso has led since 2015.

"First, we wanted to know why -- and where -- crop yields varied from year to year in the corn and soybean belt of the U.S.," Basso said. "Next, we wanted to find out if it was possible to use big data to develop and deploy climate-smart agriculture solutions to help farmers reduce cost, increase yields and limit environmental impact."

Basso and Martinez-Feria first examined soil and discovered that alone, it could not sufficiently explain such drastic yield variations.

"The same soil would have low yield one year and high yield the next," Basso said. "So, what is causing this temporal instability?"

Using an enormous amount of data obtained from satellites, research aircraft, drones, and remote sensors, and from farmers via advanced geospatial sensor suites present in many modern combine harvesters, Basso and Martinez-Feria wove big data and digital expertise together.

What they found is that the interaction between topography, weather, and soil has an immense impact on how crop fields respond to extreme weather in unstable areas. Terrain variations, such as depressions, summits, and slopes, create localized areas where water stands or runs off. Roughly two-thirds of unstable zones occur in these summits and depressions and the terrain controls water stress experienced by crops.

With comprehensive data and the technology, the team quantified the percentage of every single corn or soybean field in the Midwest that is prone to water excess or water deficit. Yields in water-deficient areas can be 23 to 33% below the field average for seasons with low rainfall but are comparable to the average in very wet years. Areas prone to water excess experienced yields 26 to 33% below field average during wet years.

Basso believes their work will help determine the future of climate-smart agriculture technologies.

"We are primarily concerned with helping farmers see their fields in a new manner, helping them make better decisions to improve yield, reduce cost and improve environmental impact," Basso said. "Knowing that you have an area shown to be water deficient, you will plan your fertilizer applications differently. The amount of fertilizer for this area should be significantly lower than what you would apply in areas of the same field with more water available to the plants."