Tyler O'Neal, Staff Editor

Chinese researchers use deep learning on 'selfies' to detect heart disease

New research uses artificial intelligence to analyse facial photos

Sending a "selfie" to the doctor could be a cheap and simple way of detecting heart disease, according to the authors of a new study published today (Friday) in the European Heart Journal.

The study is the first to show that it's possible to use a deep learning supercomputer algorithm to detect coronary artery disease (CAD) by analysing four photographs of a person's face.

Although the algorithm needs to be developed further and tested in larger groups of people from different ethnic backgrounds, the researchers say it has the potential to be used as a screening tool that could identify possible heart disease in people in the general population or in high-risk groups, who could be referred for further clinical investigations.

"To our knowledge, this is the first work demonstrating that artificial intelligence can be used to analyse faces to detect heart disease. It is a step towards the development of a deep learning-based tool that could be used to assess the risk of heart disease, either in outpatient clinics or by means of patients taking 'selfies' to perform their own screening. This could guide further diagnostic testing or a clinical visit," said Professor Zhe Zheng, who led the research and is vice director of the National Center for Cardiovascular Diseases and vice president of Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, People's Republic of China.

He continued: "Our ultimate goal is to develop a self-reported application for high risk communities to assess heart disease risk in advance of visiting a clinic. This could be a cheap, simple and effective of identifying patients who need further investigation. However, the algorithm requires further refinement and external validation in other populations and ethnicities."

It is known already that certain facial features are associated with an increased risk of heart disease. These include thinning or grey hair, wrinkles, ear lobe crease, xanthelasmata (small, yellow deposits of cholesterol underneath the skin, usually around the eyelids) and arcus corneae (fat and cholesterol deposits that appear as a hazy white, grey or blue opaque ring in the outer edges of the cornea). However, they are difficult for humans to use successfully to predict and quantify heart disease risk.

Prof. Zheng, Professor Xiang-Yang Ji, who is director of the Brain and Cognition Institute in the Department of Automation at Tsinghua University, Beijing, and other colleagues enrolled 5,796 patients from eight hospitals in China to the study between July 2017 and March 2019. The patients were undergoing imaging procedures to investigate their blood vessels, such as coronary angiography or coronary computed tomography angiography (CCTA). They were divided randomly into training (5,216 patients, 90%) or validation (580, 10%) groups. CAPTION Take-home figure from research paper CREDIT European Heart Journal

Trained research nurses took four facial photos with digital cameras: one frontal, two profiles and one view of the top of the head. They also interviewed the patients to collect data on socioeconomic status, lifestyle and medical history. Radiologists reviewed the patients' angiograms and assessed the degree of heart disease depending on how many blood vessels were narrowed by 50% or more (? 50% stenosis), and their location. This information was used to create, train and validate the deep learning algorithm.

The researchers then tested the algorithm on a further 1,013 patients from nine hospitals in China, enrolled between April 2019 and July 2019. The majority of patients in all the groups were of Han Chinese ethnicity.

They found that the algorithm out-performed existing methods of predicting heart disease risk (Diamond-Forrester model and the CAD consortium clinical score). In the validation group of patients, the algorithm correctly detected heart disease in 80% of cases (the true positive rate or 'sensitivity') and correctly detected heart disease was not present in 61% of cases (the true negative rate or 'specificity'). In the test group, the sensitivity was 80% and specificity was 54%.{module INSIDE STORY}

Prof. Ji said: "The algorithm had a moderate performance, and additional clinical information did not improve its performance, which means it could be used easily to predict potential heart disease based on facial photos alone. The cheek, forehead and nose contributed more information to the algorithm than other facial areas. However, we need to improve the specificity as a false positive rate of as much as 46% may cause anxiety and inconvenience to patients, as well as potentially overloading clinics with patients requiring unnecessary tests."

As well as requiring testing in other ethnic groups, limitations of the study include the fact that only one centre in the test group was different to those centres which provided patients for developing the algorithm, which may further limit its generalisabilty to other populations.

In an accompanying editorial, Charalambos Antoniades, Professor of Cardiovascular Medicine at the University of Oxford, UK, and Dr Christos Kotanidis, a DPhil student working under Prof. Antoniades at Oxford, write: "Overall, the study by Lin et al. highlights a new potential in medical diagnostics......The robustness of the approach of Lin et al. lies in the fact that their deep learning algorithm requires simply a facial image as the sole data input, rendering it highly and easily applicable at large scale."

They continue: "Using selfies as a screening method can enable a simple yet efficient way to filter the general population towards more comprehensive clinical evaluation. Such an approach can also be highly relevant to regions of the globe that are underfunded and have weak screening programmes for cardiovascular disease. A selection process that can be done as easily as taking a selfie will allow for a stratified flow of people that are fed into healthcare systems for first-line diagnostic testing with CCTA. Indeed, the 'high risk' individuals could have a CCTA, which would allow reliable risk stratification with the use of the new, AI-powered methodologies for CCTA image analysis."

They highlight some of the limitations that Prof. Zheng and Prof. Ji also include in their paper. These include the low specificity of the test, that the test needs to be improved and validated in larger populations, and that it raises ethical questions about "misuse of information for discriminatory purposes. Unwanted dissemination of sensitive health record data, that can easily be extracted from a facial photo, renders technologies such as that discussed here a significant threat to personal data protection, potentially affecting insurance options. Such fears have already been expressed over misuse of genetic data, and should be extensively revisited regarding the use of AI in medicine".

The authors of the research paper agree on this point. Prof. Zheng said: "Ethical issues in developing and applying these novel technologies is of key importance. We believe that future research on clinical tools should pay attention to the privacy, insurance and other social implications to ensure that the tool is used only for medical purposes."

Prof. Antoniades and Dr. Kotanidis also write in their editorial that defining CAD as 50% stenosis in one major coronary artery "may be a simplistic and rather crude classification as it pools in the non-CAD group individuals that are truly healthy, but also people who have already developed the disease but are still at early stages (which might explain the low specificity observed)."

Artificial intelligence recognizes deteriorating photoreceptors

Tyler O'Neal, Staff Editor

Study of the universities of Bonn, Stanford and Utah on atrophic AMD

A software based on artificial intelligence (AI), which was developed by researchers at the Eye Clinic of the University Hospital Bonn, Stanford University and University of Utah, enables the precise assessment of the progression of geographic atrophy (GA), a disease of the light sensitive retina caused by age-related macular degeneration (AMD). This innovative approach permits the fully automated measurement of the main atrophic lesions using data from optical coherence tomography, which provides three-dimensional visualization of the structure of the retina. In addition, the research team can precisely determine the integrity of light sensitive cells of the entire central retina and also detect progressive degenerative changes of the so-called photoreceptors beyond the main lesions. The findings will be used to assess the effectiveness of new innovative therapeutic approaches. The study has now been published in the journal "JAMA Ophthalmology".

There is no effective treatment for geographic atrophy, one of the most common causes of blindness in industrialized nations. The disease damages cells of the retina and causes them to die. The main lesions, areas of degenerated retina, also known as "geographic atrophy", expand as the disease progresses and result in blind spots in the affected person's visual field. A major challenge for evaluating therapies is that these lesions progress slowly, which means that intervention studies require a long follow-up period. "When evaluating therapeutic approaches, we have so far concentrated primarily on the main lesions of the disease. However, in addition to central visual field loss, patients also suffer from symptoms such as a reduced light sensitivity in the surrounding retina," explains Prof. Dr. Frank G. Holz, Director of the Eye Clinic at the University Hospital Bonn. "Preserving the microstructure of the retina outside the main lesions would therefore already be an important achievement, which could be used to verify the effectiveness of future therapeutic approaches." CAPTION Image of the central retina in a patient with geographic atrophy - serves as a reference for optical coherence tomography (OCT). CREDIT © Universitäts-Augenklinik Bonn{module INSIDE STORY}

Integrity of light sensitive cells predicts disease progression

The researchers were furthermore able to show that the integrity of light sensitive cells outside areas of geographic atrophy is a predictor of the future progression of the disease. "It may therefore be possible to slow down the progression of the main atrophic lesions by using therapeutic approaches that protect the surrounding light sensitive cells," says Prof. Monika Fleckenstein of the Moran Eye Center at the University of Utah in the USA, initiator of the Bonn-based natural history study on geographic atrophy, on which the current publication is based.

"Research in ophthalmology is increasingly data-driven. The fully automated, precise analysis of the finest, microstructural changes in optical coherence tomography data using AI represents an important step towards personalized medicine for patients with age-related macular degeneration," explains lead author Dr. Maximilian Pfau from the Eye Clinic at the University Hospital Bonn, who is currently working as a fellow of the German Research Foundation (DFG) and postdoctoral fellow at Stanford University in the Department of Biomedical Data Science. "It would also be useful to re-evaluate older treatment studies with the new methods in order to assess possible effects on photoreceptor integrity."

Skoltech supercomputer helps scientists reveal most influential parameters for crop

Tyler O'Neal, Staff Editor

Nowadays, agriculture is going to become AI-native: Skoltech researchers have used the Zhores supercomputer to perform a very precise sensitivity analysis to reveal crucial parameters for different crop yields in the chernozem region. Their paper was published in the proceedings of the International Conference on Computational Science 2020.

Farmers all over the world use digital crop models to predict crop yields; these models describe soil processes, climate, and crop properties and require environmental and agricultural management input data to calibrate them and improve the forecasts. In some countries, however, agrochemical data is not freely available for users of these models, and this calibration can become expensive and time-consuming. CAPTION A heatmap of the impact of key soil parameters on yield CREDIT Pavel Odinev / Skoltech{module INSIDE STORY}

A Skoltech team led by full professor Ivan Oseledets and assistant professor Maria Pukalchik used one of the popular open-source process-based model called MONICA and figured out a way to reveal only the most important parameters for crop yield based on historical data and process modeling. Moreover, they sped up computational efficiency from one simulation per day to half a million model simulations per hour using Zhores, the flagship Skoltech supercomputer.

This stunning amount of simulations is necessary to perform high-quality sensitivity analysis that helps determine how the changes in certain input factors (such as soil parameters or fertilizer) influenced the output crop yield prediction.

The research team used field data from an experiment in the Russian chernozem region, with seasonal crop-rotation of sugar beet (Beta vulgaris), spring barley (Hordeum vulgare), and soybean (Glycine max) observed from 2011 to 2017. They picked six main soil parameters for sensitivity analysis and performed what's called Sobol sensitivity analysis (named after Ilya Sobol, a Russian mathematician who proposed it in 2001).

"Soil is a very complicated issue in this country. Unfortunately, the data about soil properties and crop yield are not published. We have found an opportunity to overcome this barrier and set up the Zhores supercomputer to solve this issue. Now we can simulate all possible variants and reveal the most crucial parameters without time-consuming and costly work. We hope that our achievements will help farmers digitalize their crop growth," said Maria Pukalchik.