Tyler O'Neal, Staff Editor SYSTEMS

Machine learning models for diagnosing COVID-19 are not suitable for clinical use due to using Frankenstein datasets

Researchers have found that out of the more than 300 COVID-19 machine learning models described in scientific papers in 2020, none of them is suitable for detecting or diagnosing COVID-19 from standard medical imaging, due to biases, methodological flaws, lack of reproducibility, and 'Frankenstein datasets.'

The team of researchers, led by the University of Cambridge, carried out a systematic review of scientific manuscripts - published between 1 January and 3 October 2020 - describing machine learning models that claimed to be able to diagnose or prognosticate for COVID-19 from chest radiographs (CXR) and computed tomography (CT) images. Some of these papers had undergone the process of peer-review, while the majority had not.

Their search identified 2,212 studies, of which 415 were included after initial screening and, after the quality screening, 62 studies were included in the systematic review. None of the 62 models was of potential clinical use, which is a major weakness, given the urgency with which validated COVID-19 models are needed. The results are reported in an academic journal.

Machine learning is a promising and potentially powerful technique for the detection and prognosis of the disease. Machine learning methods, including where imaging and other data streams are combined with large electronic health databases, could enable a personalized approach to medicine through improved diagnosis and prediction of individual responses to therapies.

"However, any machine learning algorithm is only as good as the data it's trained on," said first author Dr Michael Roberts from Cambridge's Department of Applied Mathematics and Theoretical Physics. "Especially for a brand-new disease like COVID-19, it's vital that the training data is as diverse as possible because, as we've seen throughout this pandemic, there are many different factors that affect what the disease looks like and how it behaves."

"The international machine learning community went to enormous efforts to tackle the COVID-19 pandemic using machine learning," said joint senior author Dr. James Rudd, from Cambridge's Department of Medicine. "These early studies show promise, but they suffer from a high prevalence of deficiencies in methodology and reporting, with none of the literature we reviewed reaching the threshold of robustness and reproducibility essential to support use in clinical practice."

Many of the studies were hampered by issues with poor quality data, poor application of machine learning methodology, poor reproducibility, and biases in study design. For example, several training datasets used images from children for their 'non-COVID-19' data and images from adults for their COVID-19 data. "However, since children are far less likely to get COVID-19 than adults, all the machine learning model could usefully do was to tell the difference between children and adults, since including images from children made the model highly biased," said Roberts.

Many of the machine learning models were trained on sample datasets that were too small to be effective. "In the early days of the pandemic, there was such a hunger for information, and some publications were no doubt rushed," said Rudd. "But if you're basing your model on data from a single hospital, it might not work on data from a hospital in the next town over: the data needs to be diverse and ideally international, or else you're setting your machine learning model up to fail when it's tested more widely."

In many cases, the studies did not specify where their data had come from, or the models were trained and tested on the same data, or they were based on publicly available 'Frankenstein datasets' that had evolved and merged over time, making it impossible to reproduce the initial results.

Another widespread flaw in many of the studies was a lack of involvement from radiologists and clinicians. "Whether you're using machine learning to predict the weather or how a disease might progress, it's so important to make sure that different specialists are working together and speaking the same language, so the right problems can be focused on," said Roberts.

Despite the flaws they found in the COVID-19 models, the researchers say that with some key modifications, machine learning can be a powerful tool in combatting the pandemic. For example, they caution against the naive use of public datasets, which can lead to significant risks of bias. In addition, datasets should be diverse and of the appropriate size to make the model useful for different demographic groups and independent external datasets should be curated.

In addition to higher quality datasets, manuscripts with sufficient documentation to be reproducible and external validation are required to increase the likelihood of models being taken forward and integrated into future clinical trials to establish independent technical and clinical validation as well as cost-effectiveness.

UMaine's Edalatpour wins NSF CAREER award to study thermal radiation in quantum materials

Tyler O'Neal, Staff Editor SYSTEMS

Components the size of a few atoms, known as quantum materials, can enhance how technology functions and manages its heat. However, little is known about how heat is emitted and exchanged in quantum materials in contrast with their more common counterparts, three-dimensional bulk materials.

Sheila Edalatpour, an assistant professor of mechanical engineering at the University of Maine, is studying how the emission of heat changes when the materials involved are quantum-sized, or when they are separated by a gap of the same size as one or multiple atoms. The proposal earned her an intended amount of $526,858 from a National Science Foundation CAREER Award, the organization's most prestigious award for early-career faculty. The funding is jointly allocated by NSF's Thermal Transport Processes Program and the Established Program to Stimulate Competitive Research (EPSCoR). Shelia Edalatpour news feature 9028e

Optical and electronic properties can differ between bulk and quantum materials, and therefore, so can how they transfer radiated heat, according to Edalatpour. Determining how material size affects thermal radiation, the energy emitted from heated surfaces and transferred from one component to another in the form of electromagnetic waves, can help engineers design new materials to build more efficient, powerful, and reliable devices for energy, supercomputing, health care, and other purposes.

"Quantum size effects provide an excellent opportunity for engineering materials with novel thermal properties suitable for energy conservation and conversion technologies such as thermophotovoltaics, solar cells, and smart windows," Edalatpour says. "However, we know very little about how thermal radiation from materials is affected as the material size approaches the quantum scale. We plan to elucidate the quantum effects on thermal radiation via a theoretical-experimental study."

The study will involve creating a theoretical framework for how quantum materials radiate and exchange thermal energy, researching the thermal radiation of different types of quantum materials, and demonstrating how reducing the material size to atomic scales affects the magnitude and spectrum of thermal radiation. Tests will be conducted on zero-dimensional, dot-shaped quantum materials; one-dimensional, line-shaped materials; and two-dimensional, thin film-shaped, materials. Edalatpour will be the first to quantify in exact measurements the magnitude, spectrum, spatial coherence, and polarization of thermal radiation from atomic-scale materials.

By expanding scientists' understanding of thermal radiation at the quantum level, her research may help them create new materials with thermal radiative properties that can more effectively transfer heat in nano-scale devices. These materials could fuel "technological breakthroughs in thermophotovoltaic waste heat recovery, electronic devices, and thermal diodes," all of which can help reduce fossil fuel consumption, according to the UMaine researcher.

Edalatpour also hopes to elucidate how electron tunneling affects thermal radiation. Electron tunneling, when an electron moves through a barrier it cannot typically pass, can occur between two materials separated by a gap of the same size as a few atoms. How the process affects thermal radiation is "not fully understood," according to the UMaine assistant professor.

"Radiative heat transfer at the atomic length scale can play a significant role in thermal management of nanoscale and quantum-scale devices such as transistors, ultra-compact circuits, quantum computers, solar cells, and medical imagers," Edalatpour says.

Edalatpour will recruit female students and students with disabilities, two groups underrepresented in mechanical engineering, at the high school, undergraduate, and graduate levels to assist with the study. She will also seek supplemental financial support to provide research experience for high school teachers from rural Maine. Her work will help create a new course about radiative heat transfer with lectures and lab components.

"By involving high school girls and teachers, we hope to contribute to increasing the number of female students choosing mechanical engineering as their career," Edalatpour says.

Sweden's quantum supercomputer project shifts up a gear

Tyler O'Neal, Staff Editor SYSTEMS

Knut and Alice Wallenberg Foundation is almost doubling the annual budget of the research initiative Wallenberg Centre for Quantum Technology, WACQT, based at Chalmers University of Technology, Sweden. This will allow the center to shift up a gear and set even higher goals - especially in its development of a quantum supercomputer.

"Quantum technology has enormous potential and it is important that Sweden has the necessary skills in the area. During the short time since the center was founded, WACQT has built up a qualified research environment, established collaborations with the Swedish industry, and succeeded in developing qubits with proven problem-solving ability. We can look ahead with great confidence at what they will go on to achieve," says Peter Wallenberg Jr, Chair Knut, and Alice Wallenberg Foundation.

Since 2018, Chalmers University of Technology has been managing a large, forward-thinking research initiative - the Wallenberg Centre for Quantum Technology, WACQT - setting Sweden on course to global prominence in quantum technology. The main project is to develop and build a quantum supercomputer, offering far greater computing power than today's best supercomputers.

During the first three years, the quantum supercomputer researchers within WACQT have focused first on making the basic building blocks of the quantum supercomputer - the qubits - work as well as possible, at a small scale. A milestone was reached last year when they managed to solve a small part of a real-world optimization problem with their well-functioning two-qubit quantum supercomputer. The cryostat cooling Chalmers University's quantum processor to close to abolute zero. CREDIT Johan Bodell/Chalmers University of Technology

Increases the quality of the hundred qubits

Now comes the time to significantly scale up the number of qubits, and increase the efforts on developing software and algorithms. At the same time, the entire research initiative is being scaled up, with Knut and Alice Wallenberg Foundation (KAW) deciding to almost double WACQT's annual budget, from SEK 45 to 80 million per year for the coming four years. The investment has previously also been extended from its original ten years to twelve and has now a total funding of at least SEK 1.3 billion including contributions from industry and the participating universities.

"It is very encouraging that KAW shows such great confidence in us. It strengthens WACQT's research program and gives us the opportunity to build an even better quantum computer. In terms of the number of qubits, the goal is still one hundred, but now we are aiming at one hundred really high-performance qubits," says Per Delsing, director of WACQT and Professor at Chalmers.

Calculations have shown that the performance of the final quantum computer will benefit more from increasing the quality of the individual qubits, rather than the total number of qubits. The better their quality, the more useful the final quantum computer.

With the increased funding, WACQT will, among other things, invest in improving the materials in the superconducting chips that constitute the qubits. Quantum states are extremely sensitive, and the slightest disturbance in the materials can impair performance. The qubits manufactured at Chalmers are already among the best in the world, so improving them entails moving the entire research field into new territory.

"These disturbances are extremely small. It requires research just to understand what they are and which are most common. We need to study the entire manufacturing process in detail and explore new ways to eliminate disturbances in the material," Delsing explains. Per Delsing, Professor in Physics and Director of Wallenberg Centre for Quantum Technology, Chalmers University of Technology. CREDIT Johan Bodell/Chalmers University of Technology

Will employ another 40 researchers

With the increased funding, the number of researchers working on the quantum supercomputer project can now be significantly increased. For example, a new team will be formed to study nanophotonic devices that can enable the interconnection of several smaller quantum processors into a large quantum supercomputer. Within the next two years, the research force will be expanded by 40 people, almost double the current amount. In the first step, fifteen new postdocs will be recruited.

"This is ambitious recruitment in a highly competitive niche area. But our hopes are high - through previous recruitments, we have attracted top talents both from Sweden and internationally. We have a unique interaction with the industry, extensive experience of superconducting circuits, and an amazing cleanroom facility," says Delsing.

"These are very exciting times in quantum computing. New steps are being taken all the time and the competition is rapidly increasing, with many countries making major investments. This investment will ensure that Sweden and Chalmers remain at the global forefront," Delsing says.