@stefano I know you know this, just reiterating to further depress myself and others I guess: This is sadly not specific to BSD or any one piece of knowledge. AI models don't 'get' anything. They have no concept of knowing anything, and SEO pages generated by AI containing incorrect information are polluting the training sets of AI models, so nothing good can come of it.

The video linked from this thread eloquently makes that point, among many others.

https://akkoma.xn--pikabl-0xa.se/objects/34faab37-a455-44f7-abeb-d0b937cd479a

@stefano

Just thinking out loud (and might be completely wrong).

There are not enough (correct) texts about BSD for "AI" scrappers to "learn" enough, so it might "evaluate" similar texts to Linux based systems and produce "answer" from it.

@stefano It's possible newer versions fare better, but this https://nxdomain.no/~peter/chatgpt_writes_pf.conf.html piece from mid-2023 had ChatGPT failing to produce anything even close to valid for pf.conf.

The results are classic GIGO but with lots of extra hardware thrown at it for no good reason.

@stefano

By working backwards from a Tree of Usages, you force the AI (or the human author) to adhere to a strict Contract.

Input: "To start a service on illumos, use systemctl start..."

Working Backwards:
Target: systemctl.
Parent Tree: Linux/Systemd.
Declared Context: illumos.

Assertion Failure: systemctl does not exist in the illumos usage tree.

@stefano As somebody who was around back in the days when some BSD folks were arrogant "all Unixes are BSD, who cares about silly HP-SUX etc", there is a certain Schadenfreude here. (Joke.)

@stefano has anyone missed the point as hard as people who try to use AI when learning BSD?

@stefano Or maybe the info about BSDs is so complex, fragmented, and outdated that it cannot be automatically processed?

@stefano This is a good thing. While the mindset thinks using AI - as in LLM’s - is okay, they won’t understand the BSD’s. When they are ready - mature enough to understand the BSD mindset - they will reject the LMM’s first and from that everything else will follow.

@stefano

Is that a problem... Or an opportunity?

Hear me out: what if we all put everywhere "do not trust AI with [BSD | Illumos], use these sites instead"?

This would actually redirect people to valid sources of information and puncture the AI nonsense.

@stefano I think we can combat this with more and better blog posts and targeted documentation. The AI walks our docs, if we're better, it is better. I plan on starting doing this myself. I am working on getting my homelab in better shape, then I plan on posting about all of the stuff I do more.

@stefano that's why I will go back to FreeBSD to host my site and maybe the wife's and I will try with all my forces to document it all and make it public

@stefano

The problem I'm seeing is the decline of "traditional search".. while Google being Google for a long time now, it was still a search engine that somewhat provided a desired results, especially when tech-related topic were searched for. But now, people ask AI instead of searching..

The cherry on top - that not even Google's own AI can use the search engine correctly.. well, what does it say?

And now, is it even worth it to write tutorials, answers, etc.? When the whole traffic will be the AI crawler and no new people will find your blog/website/whatever? Then the chicken and egg returns to the beginning again - if nobody will see my blog, why should I write it - and yes, then the AI generated content will win even faster, because it is generated "for free", so nobody cares about the economics of it - AI generated, AI crawled, AI processed, used for a new content by AI.

@stefano similar issue, though more subtle, for Linux as well. Most "ai"-generated answers are wrong in ways not obvious to people who don't know much about the subject (which is clearly the intended audience for this junk).

I've gone so far as telling my guys to not use chatgpt & ilk to find answers. That was after a couple of near misses due to BS command line arguments.

@stefano I’ve noticed this. They show similar problems with macOS because they hallucinate command line options and flags that exist in the GNU/Linux versions of tools but not in the macOS versions which I assume have more BSD heritage.

@stefano when it's not full of linuxisms it hallucinates features from other BSDs because it doesn't see every BSD OS as different. We do things the old way on BSD. This is a good thing.

Similarly, even when using PyTorch, they often suggest deprecated features or options that may exist in another library but not in this one. I don't see AI being very useful for development in general other than having something to brainstorm with.