Foto: Bax Lindhardt

Fair treatment with AI

Tuesday 01 Dec 20


Aasa Feragen
DTU Compute
+45 26 22 04 98


Jakob Eyvind Bardram
Head of Sections, Professor
DTU Health Tech
+45 45 25 53 11
Prejudices and stereotypes can be reflected in algorithms used in the public health service. Researchers now want to develop fair algorithms.

Today, artificial intelligence (AI) and machine learning are increasingly being used in our healthcare system. Doctors and radiologists use, for example, algorithms to support the decisions they make when diagnosing the patient.

The only problem is that algorithms can be just as biased and prejudiced as people because they are based on data from previous observations.

For example, if an algorithm has seen more examples of lung diseases in men than in women, it will be better trained to detect lung diseases in men.

Health data often contains a bias, a misrepresentation of different demographic population groups that ends up influencing the decisions that a health algorithm is able to make. This can lead to underdiagnosis of some population groups because there is a predominance of data from a specific group.

“If the algorithm is trained by data that reflects social prejudices and stereotypes, there will also be imbalances or biases in the artificial intelligence that reproduces the data—and that’s not necessarily fair,” says Aasa Feragen, who is professor at DTU Compute, and continues:

"The idea that artificial intelligence will make important decisions about my state of health is frightening, considering that AI can be just as discriminatory as the most diehard racists or sexists,” she says.

How to ensure fair treatment?

Aasa Feragen is the manager of a research project which will investigate bias and fairness in artificial intelligence in medical applications over the next three years. The aim is to develop fair algorithms to help provide fair treatment to everyone in the public health service. Researchers from the University of Copenhagen, Rigshospitalet, and the Swiss university for science and technology ETH participate in the project.

In the project, the researchers will, for example, analyse the demographic data of all Danes diagnosed with depression in recent years. The researchers will test a hypothesis that the data contains imbalances—for example in how often Danes are diagnosed and the type of treatment they receive, based on gender, age, geography, and income. The researchers will then examine—together with ethicist and Associate Professor Sune Hannibal Holm from UCPH—how to develop a fair health algorithm.

"AI can be just as discriminatory as the most diehard racists or sexists."
Professor Aasa Feragen, DTU Compute

The issues to be discussed include questions like: ‘When is a decision discriminatory?’, ‘Can you develop methods for detection and removal of discriminatory biases in algorithms before they are taken into use?', and ‘How can you define which decisions are fair in medicine?’

“One could argue that because the biased predictions of the AI algorithm are simply a result of the biased decisions being made in our current healthcare system, the use of AI in the public health service will not cause new problems,” says Aasa Feragen.

But—according to Aasa Feragen—it is about exploiting that AI has the potential to detect these biases before the tool has made a single decision. This will make it possible to take the best elements from AI and find new ways to minimize bias through transparent and secure solutions.

The research project ‘Bias and fairness in medicine’ receives funding from Independent Research Fund Denmark.

Future-proofing the public health service

Artificial intelligence and big data are building blocks in development of personal healthcare technologies.

The increasing longevity of the Danes is accompanied by an increase in the number of people living with chronic diseases, such as type 2 diabetes and dementia. This has resulted in an ever-increasing demand for—and development of—healthcare solutions for mobile phones, smartwatches, and biosensors.

The next generation of smart and personal healthcare technologies requires artificial intelligence, the Internet of Things, big data, and better system integration. By integrating data from hospitals, doctors, the Health Platform, etc., it is possible not only to monitor patients’ condition, but also to predict the need for, for example, additional medication. This means fewer hospital admissions, medical visits, and sick days, and it also improves the patients’ quality of life, while saving Danish society billions of kroner.