According to experts, the system may be detecting signs of melanin, the pigment that gives skin its colour, which is unknown to science as of now
An international team of health researchers from the US, Canada, and Taiwan has found that deep learning models based on artificial intelligence (AI) can identify a person’s race from their X-rays.
The team trained their AI using hundreds of thousands of existing X-ray images labelled with details of the patient’s race and then tested their system on X-ray images that the computer software had not seen before and had no additional information on.
AI could predict the reported racial identity of the person/patient on the images with surprising accuracy. Even when the scans were taken from people of the same age and sex, the AI could predict it accurately.
According to the published paper in The Lancet Digital Health, the team aimed to show that, when trained, the standard AI deep learning models can predict race from medical images with high performance across multiple imaging modalities.
The scientists are unaware of why the AI system accurately identifies race from images that do not contain such information. Even in the case of the limited information provided, for instance, removing clues on bone density or focussing on a small part of the body, the AI models still performed surprisingly well at predicting race.
According to experts, the system may detect signs of melanin, the pigment that gives skin its colour, which is unknown to science. Researchers across the medical field say that the research acts as yet another evidence that AI systems can lead to biases and prejudices in human beings, sparking racism, sexism, or other forms of discrimination.
