New Technique Overcomes Spurious Correlations Problem in AI

AI models often rely on “spurious correlations,” making decisions based on unimportant and potentially misleading information. Researchers have now discovered these learned spurious correlations can be traced to a very small subset of the training data and have demonstrated a technique that overcomes the problem.

“This technique is novel in that it can be used even when you have no idea what spurious correlations the AI is relying on,” says Jung-Eun Kim, corresponding author of a paper on the work and an assistant professor of computer science at North Carolina State University. “If you already have a good idea of what the spurious features are, our technique is an efficient and effective way to address the problem. However, even if you are simply having performance issues, but don’t understand why, you could still use our technique to determine whether a spurious correlation exists and resolve that issue.”

Spurious correlations are generally caused by simplicity bias during AI training. Practitioners use data sets to train AI models to perform specific tasks. For example, an AI model could be trained to identify photographs of dogs. The training data set would include pictures of dogs where the AI is told a dog is in the photo. During the training process, the AI will begin identifying specific features that it can use to identify dogs. However, if many of the dogs in the photos are wearing collars, and because collars are generally less complex features of a dog than ears or fur, the AI may use collars as a simple way to identify dogs. This is how simplicity bias can cause spurious correlations.

“And if the AI uses collars as the factor it uses to identify dogs, the AI may identify cats wearing collars as dogs,” Kim says.

Conventional techniques for addressing problems caused by spurious correlations rely on practitioners being able to identify the spurious features that are causing the problem. They can then address this by modifying the data sets used to train the AI model. For example, practitioners might increase the weight given to photos in the data set that include dogs that are not wearing collars.

However, in their new work, the researchers demonstrate that it is not always possible to identify the spurious features that are causing problems – making conventional techniques for addressing spurious correlations ineffective.

“Our goal with this work was to develop a technique that allows us to sever spurious correlations even when we know nothing about those spurious features,” Kim says.

The new technique relies on removing a small portion of the data used to train the AI model.

“There can be significant variation in the data samples included in training data sets,” Kim says. “Some of the samples can be very simple, while others may be very complex. And we can measure how ‘difficult’ each sample is based on how the model behaved during training.

“Our hypothesis was that the most difficult samples in the data set can be noisy and ambiguous, and are most likely to force a network to rely on irrelevant information that hurt a model’s performance,” Kim explains. “By eliminating a small sliver of the training data that is difficult to understand, you are also eliminating the hard data samples that contain spurious features. This elimination overcomes the spurious correlations problem, without causing significant adverse effects.”

The researchers demonstrated that the new technique achieves state-of-the-art results – improving performance even when compared to previous work on models where the spurious features were identifiable.

The peer-reviewed paper, “Severing Spurious Correlations with Data Pruning,” will be presented at the International Conference on Learning Representations (ICLR), being held in Singapore from April 24-28. First author of the paper is Varun Mulchandani, a Ph.D. student at NC State.

-shipman-

Note to Editors: The study abstract follows.

“Severing Spurious Correlations with Data Pruning”

Authors: Varun Mulchandani and Jung-Eun Kim, North Carolina State University

Presented: April 24-28, ICLR 2025

Abstract: Deep neural networks have been shown to learn and rely on spurious correlations present in the data that they are trained on. Reliance on such correlations can cause these networks to malfunction when deployed in the real world, where these correlations may no longer hold. To overcome the learning of and reliance on such correlations, recent studies propose approaches that yield promising results. These works, however, study settings where the strength of the spurious signal is significantly greater than that of the core, invariant signal, making it easier to detect the presence of spurious features in individual training samples and allow for further processing. In this paper, we identify new settings where the strength of the spurious signal is relatively weaker, making it difficult to detect any spurious information while continuing to have catastrophic consequences. We also discover that spurious correlations are learned primarily due to only a handful of all the samples containing the spurious feature and develop a novel data pruning technique that identifies and prunes small subsets of the training data that contain these samples. Our proposed technique does not require inferred domain knowledge, information regarding the sample-wise presence or nature of spurious information, or human intervention. Finally, we show that such data pruning attains state-of-the-art performance on previously studied settings where spurious information is identifiable.

This post was originally published in NC State News.