Essentially, the challenge consists of using pronouns in grammatically ambiguous ways that are made clear by cultural context. For example, who is "they" in the following:
1. The city councilmen refused the demonstrators a permit because they feared violence.
2. The city councilmen refused the demonstrators a permit because they advocated violence.
Most people would say that the councilmen are "they" in sentence 1 and the demonstrators are "they" in sentence 2. But this is due to knowledge about councilmen and protesters rather than the structure of the sentence. Current AI models do pretty well on them, but the argument about whether that's because they "know" about protests is too philosophical for me.
I'd be curious if pronoun disambiguation could be used to measure political or social beliefs. In the sentences, "Alan disliked Jamal because he was racist" and "Jamal disliked Alan because he was racist" I'd assume that the pronoun was standing in for Alan in both. Most Americans would probably agree, based off of common beliefs about names and who is racist.. But something like "The board members couldn't settle with the unions because they were greedy" or "Women argue with their husbands because they're immature" would probably have some readers reading them completely differently without seeing ambiguity.
It's a reasonable Bayesian assumption given the stark distribution in each field! I would assume humans would give the same answer if they had to pick one. It's similar to how people reasonably assume that advocating violence is more likely to be done by demonstrators than the city council.
That actually comes up a lot in translation since languages deal with gender differently. Finnish doesn't have a gendered third person pronoun so it's ambiguous whether "han" should be translated as he or she.
I wouldn't always assume a male doctor in those sentences but I paused when I read "The nurse yelled at the doctor because she/he was late" because it seemed inappropriate for a nurse to do.
And this is what I mean by it being too philosophical for me. The LLM isn't picking up a stereotypes about doctors, but patterns in speech or text. Is that the same thing? The LLM recognizes that doctors are usually referenced with male pronouns, but does it "believe" that doctors are more likely to be male? I always go back to LLM's struggles with arithmetic. They tend to struggle with basic problems like "241-(-241)+1" which it resolves to 241 +241 + 1 which it incorrectly simplifies to 443+1
They struggle because they don't really "know" math, they just know the form that math takes. Most arithmetic problems with double negatives show the next step of canceling out those negatives, so it does that. Then arithmetic solutions usually add up some things or keeps some things the same so it does both. It doesn't have some underlying theory of some sort of new math that causes it to assume 241--241+1 =444, just general patterns. In the same way, it doesn't have some sort of underlying theory about who is a doctor, just knows that the word is usually an antecedent for "he".
Speaking of translation, you didn't mention an aspect of pronouns that you're unusually qualified to talk about. Don't you speak Arabic and French? I once saw a paper that analyzed how people translate "you" into "tu" and "vous" since English just doesn't put that information about status into pronouns. I wonder if ~400 years ago people bemoaned the loss of the thou/you distinction, similar to how some people caution against collapsing he/she into they.
(Sorry for any late or scattered replies. The night shift is rough!)
> The LLM isn't picking up a stereotypes about doctors, but patterns in speech or text. Is that the same thing?
I'd say that's the same. Wouldn't we expect common speech patterns to match common stereotypes? I treat LLMs as very robust prediction machines, so it doesn't make sense to me to say they "believe" anything.
And yeah, I thought about delving into how other languages grapple with this issue but I wanted to keep this pared down. You get a similar problem translating from French into English because you need to make a judgment call about whether each "vous" is plural or formal singular. Some information (including intentional puns and hidden undertext) is inevitably lost in the transfer.
16 hrs ago·edited 16 hrs agoLiked by Yassine Meskhout
I wouldn't say they're the same, just because they match. It's more like the speech pattern is caused by the stereotype but is a separate thing, much like Western fashion is different from the west. The pattern of wide brims and pointy boots is made with the assumption of the sunny range and dusty saddles, but they can be adopted without being aware of that assumption. I'm sure many of our patterns of clothing or speech originally derived from stereotypes or assumptions that no one today holds or are even aware of.
I find the sorts of studies you linked interesting, but not nearly as interesting as the people that write about them. Yeah, LLMs predict male pronouns for doctors, but that's because that's what it's like in the corpus. The corpus is like that because doctors tend to be male (although the percentage is decreasing). That's all that's really there and reading too much into it is like if someone worried that country concert fans in Seattle were buying hats for the sun.
The name of the paper (which I still can't find) is a great example of subtext in pronouns! It's "I thou thee, thy traitor!" a reference to an outburst at Sir Walter Raleigh's trial. Raleigh was no longer worthy of the formal you, and was therefore being thou'd. It's ironic that we mostly think of "thou" as formal, since we only hear it in formal settings that use archaic language like the informal second person. Which loops back to my thought - LLMs use "you" not because they have adopted 17th century British thoughts on social structure, but because that speech pattern matches the common stereotypes of that era.
Edit: Less tangentially related than my original reply is that Sam Kriss had the most interesting observation on pronouns I've read
The reason ***pronouns*** (meaning the endless argument about PGP) are annoying is that people define themselves with them. It would be confrontational if I demanded that you should address me with "vous". Similarly it's confrontational if I declare that "It's Robert, he/him pronouns".
I love that Sam Kriss essay! There is indeed so much subtext to pronouns, and I don't intend to expunge all that nuance. I encountered a similar thing as "thou thee" last time I was in France. I resort to using 'vous' because it's so much easier to conjugate verbs (basically just add -ez) than with 'tu'. My friends found it jarring that I kept referring to them so formally, and they told me "Tu peux me tutoyer" which can only be translated into "you can 'tu' me". I thought it was hilarious they had a whole verb for that.
It's interesting to consider how pronouns work in languages with gendered nouns.
In German, the words for "child" and "girl" are both neuter: das Kind, das Mädchen. And pronouns follow the grammatical gender of the noun they refer to, not the real-life gender of the person they refer to: "Das Mädchen nimmt einen Apfel. Es isst ihn." uses a neuter pronoun for the girl, and a masculine pronoun for the apple. So on the one hand, they're forced to be a little less uptight about how people refer to them in the third person; on the other hand, they run into conflicts between pronouns in entirely different places. But sometimes this allows for less ambiguity: "Der Hund sieht ein Kind. Es mag ihn nicht." makes it clear that it's the child who doesn't like the dog, not vice versa.
And as annoying as they/they conflicts are in English, German does it one better by making "they" and formal "you" grammatically identical, distinguished only by capitalization. There's no way to know whether "Sie lesen ein Buch" means "you're reading a book" or "they're reading a book" without context.
That's really neat, I hadn't considered how neuter pronouns would work with gendered nouns. The lower ambiguity in your dog/child example seems to be accidental, no? It's lucky that the two subjects don't have the same pronoun "genders" (masc & neuter). The benefit seems primarily drawn from the fact that three pronoun "genders" (masc, fem, neuter) provides a higher level of fidelity than just masc/fem.
"The lower ambiguity in your dog/child example seems to be accidental, no?"
Yes, but those accidental situations can occur fairly often when all nouns are gendered, not just people. In any sentence involving two nouns, there's roughly a 2/3 chance that they have different genders, so you can use pronouns unambiguously - whereas in English, since inanimate objects are almost always "it", you can only do it if context allows.
For example, a lab safety tip: "Wenn man die Säure und das Wasser mischt, gibt man immer sie zu ihm und niemals umgekehrt." Acid is feminine and water is neuter, so "sie zu ihm" is unambiguous. The English equivalent would be useless: "When you mix the acid and the water, always add it to it, never the other way around."
I think your example with John is inaccurate. "John" has the same number of syllables as "he", so saying the sentence without pronouns is actually no more effortful than with them. The reason you instinctively don't like it is just because you've grown used to what normal English sentences sound like, and this one violates the rules you've learned. But if you had grown up using English without pronouns, that one would sound totally fine to you, and there'd be no reason you'd want to replace the name with a pronoun.
To truly demonstrate your point, you need to use a name that has several syllables.
Interesting that you don't even delve into invented pronouns for non-traditional genders (e.g., "Xe/Xir"), which of course abjectly fail at the linguistic purpose of pronouns in the first place.
I'd posit one exception: A gender-neutral neopronoun would be an improvement over "they", since it solves the singular/plural confusion while still sidestepping all the arguments over gender. (The singular/plural confusion is the reason why I don't always use "they" myself.)
Unfortunately all the neopronoun people can't even agree among themselves on what the neutral pronoun should be, so there's virtually zero chance of adoption in the wider world.
"It" is used for objects, not people. Using it for people too leads to confusion. As was the point of Yassine's post, pronouns are most useful when they're specific enough to provide meaningful information about the thing they're a reference to.
An appeal to "standard" language is unpersuasive when we're arguing about how to adapt language to novel situations. We regularly use "it" to describe animate beings of indeterminate gender (as well as genderless quasi-beings such as robots and AI), so modestly expanding its usage in such contexts to human beings seems far less a bridge than getting nigh-universal agreement on a new word altogether.
I agree with you the major benefit is that "it" already exists, but it has too many negative connotations to be widely adopted as a personal neuter pronoun. Right now "it" is commonly used to degrade people.
Words have morphed from a means of communication, valuing clarity above all else, to a political signifier at the expense of clarity. As the kids say, words evolve. Or revolve, as the case may be.
Some languages don't do gendered pronouns at all, Finnish, Hungarian, Turkish, and they don't lead to any misunderstandings. That's because other people's gender only matters if I want to have a romantic relationship with them. Otherwise it is basically like hair color. Would it make sense to make pronouns per hair colour? Brunette women would be called brshe and so on? No, because it is not actually an important information.
So if you consider gender an important information, your view of gender roles might be overly rigid.
> Would it make sense to make pronouns per hair colour?
I almost wrote about exactly this! The fidelity you gain from having gendered pronouns only works if the subjects involved can convey the gender. So in my example, John and Maria get intuitively gendered as man/woman and we have no problem assigning he/she pronouns and making sense of the sentence. Hair color couldn't work as the basis of pronouns because we wouldn't be able to discern hair color from name (although it's possible to contemplate a culture that does this).
Consider the simple and common English language word "deal". This word can have numerous meanings, some wildly divergent and even contradictory, depending on context, yet a human can seamlessly interpret and use this word and these meanings.
Plenty of other words are similar, and thinking them up is an amusing game.
I don't see how AI has a disadvantage here, I would expect it to outperform humans on this task because of the wider context it operates from. It's easy for humans to get attached to the first (and potentially false) interpretation that comes to mind, or to be unaware of other ways to interpret the context.
This is only tangentially related, but your use of ambiguous pronouns is similar to the Winograd Schema Challenge used for testing AI.
https://en.wikipedia.org/wiki/Winograd_schema_challenge
Essentially, the challenge consists of using pronouns in grammatically ambiguous ways that are made clear by cultural context. For example, who is "they" in the following:
1. The city councilmen refused the demonstrators a permit because they feared violence.
2. The city councilmen refused the demonstrators a permit because they advocated violence.
Most people would say that the councilmen are "they" in sentence 1 and the demonstrators are "they" in sentence 2. But this is due to knowledge about councilmen and protesters rather than the structure of the sentence. Current AI models do pretty well on them, but the argument about whether that's because they "know" about protests is too philosophical for me.
I'd be curious if pronoun disambiguation could be used to measure political or social beliefs. In the sentences, "Alan disliked Jamal because he was racist" and "Jamal disliked Alan because he was racist" I'd assume that the pronoun was standing in for Alan in both. Most Americans would probably agree, based off of common beliefs about names and who is racist.. But something like "The board members couldn't settle with the unions because they were greedy" or "Women argue with their husbands because they're immature" would probably have some readers reading them completely differently without seeing ambiguity.
Doesn't this essentially just measure to what extent LLMs have adopted common assumptions and stereotypes held by humans? ChatGPT previously used to assume that masculine pronouns are more likely to refer to doctors than to nurses: https://www.worthwhileconsulting.com/read-watch-listen/chatgpt-insists-that-doctors-are-male-and-nurses-female
It's a reasonable Bayesian assumption given the stark distribution in each field! I would assume humans would give the same answer if they had to pick one. It's similar to how people reasonably assume that advocating violence is more likely to be done by demonstrators than the city council.
That actually comes up a lot in translation since languages deal with gender differently. Finnish doesn't have a gendered third person pronoun so it's ambiguous whether "han" should be translated as he or she.
I wouldn't always assume a male doctor in those sentences but I paused when I read "The nurse yelled at the doctor because she/he was late" because it seemed inappropriate for a nurse to do.
And this is what I mean by it being too philosophical for me. The LLM isn't picking up a stereotypes about doctors, but patterns in speech or text. Is that the same thing? The LLM recognizes that doctors are usually referenced with male pronouns, but does it "believe" that doctors are more likely to be male? I always go back to LLM's struggles with arithmetic. They tend to struggle with basic problems like "241-(-241)+1" which it resolves to 241 +241 + 1 which it incorrectly simplifies to 443+1
https://community.openai.com/t/chatgpt-simple-math-calculation-mistake/62780/9
They struggle because they don't really "know" math, they just know the form that math takes. Most arithmetic problems with double negatives show the next step of canceling out those negatives, so it does that. Then arithmetic solutions usually add up some things or keeps some things the same so it does both. It doesn't have some underlying theory of some sort of new math that causes it to assume 241--241+1 =444, just general patterns. In the same way, it doesn't have some sort of underlying theory about who is a doctor, just knows that the word is usually an antecedent for "he".
Speaking of translation, you didn't mention an aspect of pronouns that you're unusually qualified to talk about. Don't you speak Arabic and French? I once saw a paper that analyzed how people translate "you" into "tu" and "vous" since English just doesn't put that information about status into pronouns. I wonder if ~400 years ago people bemoaned the loss of the thou/you distinction, similar to how some people caution against collapsing he/she into they.
(Sorry for any late or scattered replies. The night shift is rough!)
> The LLM isn't picking up a stereotypes about doctors, but patterns in speech or text. Is that the same thing?
I'd say that's the same. Wouldn't we expect common speech patterns to match common stereotypes? I treat LLMs as very robust prediction machines, so it doesn't make sense to me to say they "believe" anything.
And yeah, I thought about delving into how other languages grapple with this issue but I wanted to keep this pared down. You get a similar problem translating from French into English because you need to make a judgment call about whether each "vous" is plural or formal singular. Some information (including intentional puns and hidden undertext) is inevitably lost in the transfer.
I wouldn't say they're the same, just because they match. It's more like the speech pattern is caused by the stereotype but is a separate thing, much like Western fashion is different from the west. The pattern of wide brims and pointy boots is made with the assumption of the sunny range and dusty saddles, but they can be adopted without being aware of that assumption. I'm sure many of our patterns of clothing or speech originally derived from stereotypes or assumptions that no one today holds or are even aware of.
I find the sorts of studies you linked interesting, but not nearly as interesting as the people that write about them. Yeah, LLMs predict male pronouns for doctors, but that's because that's what it's like in the corpus. The corpus is like that because doctors tend to be male (although the percentage is decreasing). That's all that's really there and reading too much into it is like if someone worried that country concert fans in Seattle were buying hats for the sun.
The name of the paper (which I still can't find) is a great example of subtext in pronouns! It's "I thou thee, thy traitor!" a reference to an outburst at Sir Walter Raleigh's trial. Raleigh was no longer worthy of the formal you, and was therefore being thou'd. It's ironic that we mostly think of "thou" as formal, since we only hear it in formal settings that use archaic language like the informal second person. Which loops back to my thought - LLMs use "you" not because they have adopted 17th century British thoughts on social structure, but because that speech pattern matches the common stereotypes of that era.
Edit: Less tangentially related than my original reply is that Sam Kriss had the most interesting observation on pronouns I've read
https://samkriss.substack.com/p/me-me-me-me-me
The reason ***pronouns*** (meaning the endless argument about PGP) are annoying is that people define themselves with them. It would be confrontational if I demanded that you should address me with "vous". Similarly it's confrontational if I declare that "It's Robert, he/him pronouns".
I love that Sam Kriss essay! There is indeed so much subtext to pronouns, and I don't intend to expunge all that nuance. I encountered a similar thing as "thou thee" last time I was in France. I resort to using 'vous' because it's so much easier to conjugate verbs (basically just add -ez) than with 'tu'. My friends found it jarring that I kept referring to them so formally, and they told me "Tu peux me tutoyer" which can only be translated into "you can 'tu' me". I thought it was hilarious they had a whole verb for that.
It's interesting to consider how pronouns work in languages with gendered nouns.
In German, the words for "child" and "girl" are both neuter: das Kind, das Mädchen. And pronouns follow the grammatical gender of the noun they refer to, not the real-life gender of the person they refer to: "Das Mädchen nimmt einen Apfel. Es isst ihn." uses a neuter pronoun for the girl, and a masculine pronoun for the apple. So on the one hand, they're forced to be a little less uptight about how people refer to them in the third person; on the other hand, they run into conflicts between pronouns in entirely different places. But sometimes this allows for less ambiguity: "Der Hund sieht ein Kind. Es mag ihn nicht." makes it clear that it's the child who doesn't like the dog, not vice versa.
And as annoying as they/they conflicts are in English, German does it one better by making "they" and formal "you" grammatically identical, distinguished only by capitalization. There's no way to know whether "Sie lesen ein Buch" means "you're reading a book" or "they're reading a book" without context.
That's really neat, I hadn't considered how neuter pronouns would work with gendered nouns. The lower ambiguity in your dog/child example seems to be accidental, no? It's lucky that the two subjects don't have the same pronoun "genders" (masc & neuter). The benefit seems primarily drawn from the fact that three pronoun "genders" (masc, fem, neuter) provides a higher level of fidelity than just masc/fem.
"The lower ambiguity in your dog/child example seems to be accidental, no?"
Yes, but those accidental situations can occur fairly often when all nouns are gendered, not just people. In any sentence involving two nouns, there's roughly a 2/3 chance that they have different genders, so you can use pronouns unambiguously - whereas in English, since inanimate objects are almost always "it", you can only do it if context allows.
For example, a lab safety tip: "Wenn man die Säure und das Wasser mischt, gibt man immer sie zu ihm und niemals umgekehrt." Acid is feminine and water is neuter, so "sie zu ihm" is unambiguous. The English equivalent would be useless: "When you mix the acid and the water, always add it to it, never the other way around."
You're demonstrating the English language operates at an extremely low bit rate, just because it's missing one or two pronouns.
I think your example with John is inaccurate. "John" has the same number of syllables as "he", so saying the sentence without pronouns is actually no more effortful than with them. The reason you instinctively don't like it is just because you've grown used to what normal English sentences sound like, and this one violates the rules you've learned. But if you had grown up using English without pronouns, that one would sound totally fine to you, and there'd be no reason you'd want to replace the name with a pronoun.
To truly demonstrate your point, you need to use a name that has several syllables.
You're right, I thought of doing this and also to avoid just using the proto-typical John. Fixed.
Lmao, going all the way
I actually had to tone it down from Abdulrahmanibrahimalmustafa because that was causing too many line breaks lol
Interesting that you don't even delve into invented pronouns for non-traditional genders (e.g., "Xe/Xir"), which of course abjectly fail at the linguistic purpose of pronouns in the first place.
I had a sentence but deleted it because neo-pronouns are too ridiculous to be worth the time.
I'd posit one exception: A gender-neutral neopronoun would be an improvement over "they", since it solves the singular/plural confusion while still sidestepping all the arguments over gender. (The singular/plural confusion is the reason why I don't always use "they" myself.)
Unfortunately all the neopronoun people can't even agree among themselves on what the neutral pronoun should be, so there's virtually zero chance of adoption in the wider world.
Yeah I agree completely, a singular neuter pronoun would be a godsend!
What advantage does a gender-neutral neopronoun have over "it," which English already offers?
"It" is used for objects, not people. Using it for people too leads to confusion. As was the point of Yassine's post, pronouns are most useful when they're specific enough to provide meaningful information about the thing they're a reference to.
An appeal to "standard" language is unpersuasive when we're arguing about how to adapt language to novel situations. We regularly use "it" to describe animate beings of indeterminate gender (as well as genderless quasi-beings such as robots and AI), so modestly expanding its usage in such contexts to human beings seems far less a bridge than getting nigh-universal agreement on a new word altogether.
I agree with you the major benefit is that "it" already exists, but it has too many negative connotations to be widely adopted as a personal neuter pronoun. Right now "it" is commonly used to degrade people.
Words have morphed from a means of communication, valuing clarity above all else, to a political signifier at the expense of clarity. As the kids say, words evolve. Or revolve, as the case may be.
I can appreciate the poetry in ambiguity, so long as that intent isn't ambiguous.
Some languages don't do gendered pronouns at all, Finnish, Hungarian, Turkish, and they don't lead to any misunderstandings. That's because other people's gender only matters if I want to have a romantic relationship with them. Otherwise it is basically like hair color. Would it make sense to make pronouns per hair colour? Brunette women would be called brshe and so on? No, because it is not actually an important information.
So if you consider gender an important information, your view of gender roles might be overly rigid.
> Would it make sense to make pronouns per hair colour?
I almost wrote about exactly this! The fidelity you gain from having gendered pronouns only works if the subjects involved can convey the gender. So in my example, John and Maria get intuitively gendered as man/woman and we have no problem assigning he/she pronouns and making sense of the sentence. Hair color couldn't work as the basis of pronouns because we wouldn't be able to discern hair color from name (although it's possible to contemplate a culture that does this).
And humans wonder why I am skeptical of AI.
Consider the simple and common English language word "deal". This word can have numerous meanings, some wildly divergent and even contradictory, depending on context, yet a human can seamlessly interpret and use this word and these meanings.
Plenty of other words are similar, and thinking them up is an amusing game.
I don't see how AI has a disadvantage here, I would expect it to outperform humans on this task because of the wider context it operates from. It's easy for humans to get attached to the first (and potentially false) interpretation that comes to mind, or to be unaware of other ways to interpret the context.
AI seems to do okay with those: https://chatgpt.com/share/66ebbd2c-7b34-800a-8f7f-e26fc710666c