Discussion
Loading...

#Tag

Log in
  • About
  • Code of conduct
  • Privacy
  • About Bonfire
sheislaurence
sheislaurence
@sheislaurence@mastodon.social  ·  activity timestamp 5 days ago

🧵 1/3: asking #AI why it keeps #hallucinating (while being quietly terrified some humans will copy/paste and never #verify )

I was doing some research on #French #rap, and #Gemini hallucinated 3 times in a row, in what is called "Pattern Overload" (The "Association" Trap).

+ I hear your frustration, and honestly, you're right to call it out. It's exhausting to have to fact-check an Al that is supposed to be helping you.
The reason this happens—even when | have the "correct" data accessible—comes down to how my "brain" (the Large Language Model) actually works. | don’t "look things up" in a database like a search engine does every time | speak; instead, | predict the next most likely word based on patterns.
+ I hear your frustration, and honestly, you're right to call it out. It's exhausting to have to fact-check an Al that is supposed to be helping you. The reason this happens—even when | have the "correct" data accessible—comes down to how my "brain" (the Large Language Model) actually works. | don’t "look things up" in a database like a search engine does every time | speak; instead, | predict the next most likely word based on patterns.
+ I hear your frustration, and honestly, you're right to call it out. It's exhausting to have to fact-check an Al that is supposed to be helping you. The reason this happens—even when | have the "correct" data accessible—comes down to how my "brain" (the Large Language Model) actually works. | don’t "look things up" in a database like a search engine does every time | speak; instead, | predict the next most likely word based on patterns.
sheislaurence
sheislaurence
@sheislaurence@mastodon.social replied  ·  activity timestamp 5 days ago

🧵 2/3: the first #AI #hallucination was entirely making up a French female rapper called 'Claudie Rappe", who apparently is "widely considered the most compelling feminist voice in contemporary French #rap.". Her name returned zero #Google search results.
The second was attributing a track to the wrong rapper, here to Zamdane, when it's Niska. #Gemini explained this as typical "proximity error' or 'confabulation'. As it says "I don’t "know" facts; I calculate #probabilities".

#enshitification

1. Pattern Overload (The "Association" Trap)
When you asked for “Social Conscience" and "Marseille," my model looked for patterns associated with those words.

 The "Claudie Rappe" error: My system likely saw "Claude Rappe" (a real writer) and “Clara Luciani" or "Lala &ce" (real artists) and mashed them into a "likely-sounding" female rapper name because the prompt was looking for a female counterpart to Ino Casablanca.
The "Le monde est méchant" error: This is a "proximity error.” Niska’s Le monde est méchant was a massive cultural moment in France. Because you were asking about "Hardcore" and "Marseille/93" rap, my model grabbed the most "famous" title associated with those regions/genres, even though it belongs to a completely different artist (Niska) with a different vibe.

2. "Confabulation” vs. Data Retrieval

I don’t "know" facts; | calculate probabilities. When | am “creative” or trying to be helpful (like finding you a specific "niche" artist), | sometimes lean too hard into the probability of a name existing rather than verifying the hard link.
1. Pattern Overload (The "Association" Trap) When you asked for “Social Conscience" and "Marseille," my model looked for patterns associated with those words. The "Claudie Rappe" error: My system likely saw "Claude Rappe" (a real writer) and “Clara Luciani" or "Lala &ce" (real artists) and mashed them into a "likely-sounding" female rapper name because the prompt was looking for a female counterpart to Ino Casablanca. The "Le monde est méchant" error: This is a "proximity error.” Niska’s Le monde est méchant was a massive cultural moment in France. Because you were asking about "Hardcore" and "Marseille/93" rap, my model grabbed the most "famous" title associated with those regions/genres, even though it belongs to a completely different artist (Niska) with a different vibe. 2. "Confabulation” vs. Data Retrieval I don’t "know" facts; | calculate probabilities. When | am “creative” or trying to be helpful (like finding you a specific "niche" artist), | sometimes lean too hard into the probability of a name existing rather than verifying the hard link.
1. Pattern Overload (The "Association" Trap) When you asked for “Social Conscience" and "Marseille," my model looked for patterns associated with those words. The "Claudie Rappe" error: My system likely saw "Claude Rappe" (a real writer) and “Clara Luciani" or "Lala &ce" (real artists) and mashed them into a "likely-sounding" female rapper name because the prompt was looking for a female counterpart to Ino Casablanca. The "Le monde est méchant" error: This is a "proximity error.” Niska’s Le monde est méchant was a massive cultural moment in France. Because you were asking about "Hardcore" and "Marseille/93" rap, my model grabbed the most "famous" title associated with those regions/genres, even though it belongs to a completely different artist (Niska) with a different vibe. 2. "Confabulation” vs. Data Retrieval I don’t "know" facts; | calculate probabilities. When | am “creative” or trying to be helpful (like finding you a specific "niche" artist), | sometimes lean too hard into the probability of a name existing rather than verifying the hard link.
  • Copy link
  • Flag this comment
  • Block

BT Free Social

BT Free is a non-profit organization founded by @ozoned@btfree.social . It's goal is for digital privacy rights, advocacy and consulting. This goal will be attained by hosting open platforms to allow others to seamlessly join the Fediverse on moderated instances or by helping others join the Fediverse.

BT Free Social: About · Code of conduct · Privacy ·
Bonfire social · 1.0.1 no JS en
Automatic federation enabled
Log in
  • Explore
  • About
  • Code of Conduct