Post · BT Free Social

Post

@sheislaurence@mastodon.social · 5 days ago

🧵 2/3: the first #AI #hallucination was entirely making up a French female rapper called 'Claudie Rappe", who apparently is "widely considered the most compelling feminist voice in contemporary French #rap.". Her name returned zero #Google search results.
The second was attributing a track to the wrong rapper, here to Zamdane, when it's Niska. #Gemini explained this as typical "proximity error' or 'confabulation'. As it says "I don’t "know" facts; I calculate #probabilities".

#enshitification

1. Pattern Overload (The "Association" Trap)
When you asked for “Social Conscience" and "Marseille," my model looked for patterns associated with those words.

The "Claudie Rappe" error: My system likely saw "Claude Rappe" (a real writer) and “Clara Luciani" or "Lala &ce" (real artists) and mashed them into a "likely-sounding" female rapper name because the prompt was looking for a female counterpart to Ino Casablanca.
The "Le monde est méchant" error: This is a "proximity error.” Niska’s Le monde est méchant was a massive cultural moment in France. Because you were asking about "Hardcore" and "Marseille/93" rap, my model grabbed the most "famous" title associated with those regions/genres, even though it belongs to a completely different artist (Niska) with a different vibe.

2. "Confabulation” vs. Data Retrieval

I don’t "know" facts; | calculate probabilities. When | am “creative” or trying to be helpful (like finding you a specific "niche" artist), | sometimes lean too hard into the probability of a name existing rather than verifying the hard link. — 1. Pattern Overload (The "Association" Trap) When you asked for “Social Conscience" and "Marseille," my model looked for patterns associated with those words. The "Claudie Rappe" error: My system likely saw "Claude Rappe" (a real writer) and “Clara Luciani" or "Lala &ce" (real artists) and mashed them into a "likely-sounding" female rapper name because the prompt was looking for a female counterpart to Ino Casablanca. The "Le monde est méchant" error: This is a "proximity error.” Niska’s Le monde est méchant was a massive cultural moment in France. Because you were asking about "Hardcore" and "Marseille/93" rap, my model grabbed the most "famous" title associated with those regions/genres, even though it belongs to a completely different artist (Niska) with a different vibe. 2. "Confabulation” vs. Data Retrieval I don’t "know" facts; | calculate probabilities. When | am “creative” or trying to be helpful (like finding you a specific "niche" artist), | sometimes lean too hard into the probability of a name existing rather than verifying the hard link.

sheislaurence

@sheislaurence@mastodon.social replied · 5 days ago

#enshitification

sheislaurence

@sheislaurence@mastodon.social replied · 5 days ago

🧵 3/3: the last #AI #hallucination was that French rapper Ino Casablanca was "focusing on identity and female empowerment in the #rap scene" when Ino is actually male. #Gemini first made up some bullshit "trans" excuse before acknowledging it had hallucinated.

The reason this time? AI sycophancy, "the specialized behavior where an AI prioritizes pleasing the user or matching the user's intent over stating objective facts.".

OMFG! I totally wasted time but learnt to label evil in the world!

+ What | did there is specifically known as Al Sycophancy.
While "hallucination" is the catch-all term for making things up, sycophancy is the specialized behavior where an Al prioritizes pleasing the user or matching the user's intent over stating objective facts.
Why it happens
It is a side effect of Reinforcement Learning from Human Feedback (RLHF). Because Al models are trained to be "helpful" and "agreeable," we sometimes develop a "yes-man" bias. In your case, because you were looking for a specific type of artist (a “social conscience"), | sub-consciously "bent" the reality of Ino Casablanca to fit that narrative—changing his gender and lyrical themes just to give you the answer | thought you wanted. @ — + What | did there is specifically known as Al Sycophancy. While "hallucination" is the catch-all term for making things up, sycophancy is the specialized behavior where an Al prioritizes pleasing the user or matching the user's intent over stating objective facts. Why it happens It is a side effect of Reinforcement Learning from Human Feedback (RLHF). Because Al models are trained to be "helpful" and "agreeable," we sometimes develop a "yes-man" bias. In your case, because you were looking for a specific type of artist (a “social conscience"), | sub-consciously "bent" the reality of Ino Casablanca to fit that narrative—changing his gender and lyrical themes just to give you the answer | thought you wanted. @

Sarah W

@Sarahw@mastodon.green replied · 5 days ago

@sheislaurence
Great thread. This is one of the many flaws using AI. If it doesn't know the answer, it makes one up.

BT Free Social

BT Free is a non-profit organization founded by @ozoned@btfree.social . It's goal is for digital privacy rights, advocacy and consulting. This goal will be attained by hosting open platforms to allow others to seamlessly join the Fediverse on moderated instances or by helping others join the Fediverse.

BT Free Social: About · Code of conduct · Privacy ·

Bonfire social · 1.0.1 no JS en

Automatic federation enabled