Why do we value two AI assistants differently when they do exactly the same job?
While we were discussing bias in AI in class, a classmate mentioned a study that caught my attention. It raised a question about gender bias in AI assistants that I could not stop thinking about, so I decided to look into it a little further.
The Study: Johan and Johanna
Researchers from the University of Limerick, the University of Zurich and SKEMA Business School conducted a study involving 189 participants. The participants completed work-related tasks with AI assistants in a virtual reality environment. Afterwards, they were asked to divide real money between themselves and the AI assistant they had worked with.
Two of the assistants in the study were Johan and Johanna.
One was presented as male, the other as female. But the underlying technology and capabilities were the same.
Despite this, Johanna, the female-presenting assistant, received 10.25% less money than Johan. This pattern appeared among both male and female participants.
What People Said vs What People Did
What makes this even more interesting is that many of the participants interviewed as part of the study said the AI’s gender did not matter to them.
But their behaviour told a different story.
Where Does the Bias Actually Appear?
I think this is where we need to pay attention to where the bias is actually appearing.
In this experiment, the AI did not make the payment decision. People did.
So rather than showing an AI model making discriminatory decisions, the study makes us think about how people’s behaviour towards AI assistants can be shaped by the way those assistants are presented.
When we discuss bias in AI models, shouldn’t we also be asking whether we bring our own cultural and social biases into the way we interact with AI?
When we give an AI a human name, voice, face or gender, assumptions we are not even aware of may begin to influence our decisions.
This study alone does not tell us exactly what caused the difference. But the fact that systems with the same underlying capabilities were evaluated differently is worth thinking about.
Responsible AI Starts With Us Too
It also reminded me of something we discussed in class: bias is not something we should look for only in training data. We also need to consider how systems are designed, used and evaluated.
Maybe Responsible AI isn’t only about how we train models. Understanding the biases we bring into our interactions with AI may be part of it too.
I’m particularly curious to hear how women interpret this finding.
So I’ll leave you with the question that started all of this: does this result tell us more about AI, or more about ourselves?
You can read more about the research here:
