Post · BT Free Social

oh good, the “you’re just doing purity culture” thing is already taking hold over on bluesky

so the line is now supposed to be that local LLMs are good and moral and SaaS LLMs are bad, when local LLMs come from the same fucked system that’s also actively making it impossible to buy computing hardware powerful enough to run even a shitty local LLM? is that about right? I’m supposed to clap cause someone with money is running a plagiarism machine but slower and shittier on their desktop?

Da_Gut

@Da_Gut@dice.camp · 3 days ago

@zzt I thought that with local models you strictly controlled what information that they were drawing upon?

David Chisnall (*Now with 50% more sarcasm!*)

@david_chisnall@infosec.exchange · 3 days ago

@Da_Gut @zzt

An LLM is two things: the model and the weights. The model is basically a description of how different layers fit together. It’s usually not that complicated (you can create quite good ones in a few hundred lines of code with modern frameworks). But, by itself, the model is useless because each layer is something like ‘take an input and permute it using this operation with an NxM matrix as the other operand’. That other operand is not part of the model, it’s in the weights. The weights are large. They are the result of training. You process a lot of data to generate them.

In a classical neural network, the model defines the topology, but each neurone has an activation threshold. When you train it, you feed a bunch of data through it and this sets the threshold values. Eventually, you stop and now you have a trained model. Modern deep learning models work in a similar way, but with a huge pile of optimisations. The weights are the valuable thing because it takes vast amounts of compute and data to produce them. They’re also completely opaque. They’re just a massive blob of data, so trying to figure out the behaviour of a trained model by looking at the weights is almost impossible, as is working out what went into their training sets.

Very few ‘open’ LLMs have weights that were trained on known and reproducible data sets. Things like Meta’s LLaMa are ‘open’ in that you can recreate the model yourself (as llama.cpp did) and download their weights, but you have no visibility into what the weights were trained on, can’t reproduce the training (unless you have a data centre and a massive pile of lawyers who will be able to defend you against copyright infringement lawsuits). Oh, and the license says that you agree never to sue Meta for any IP infringement, so if @pluralistic is using one of the ‘open’ LLaMa weights, he has just given Meta a perpetual license to use all of his work for any purpose. I’m sure he considers that a great deal for a grammar checker with a 50% false positive rate.

This, by the way, is why I really like Mozilla’s translation models (which are much simpler than a general purpose LLM, though they use much of the same underlying technology). They are trained on curated open datasets designed for training machine-translation systems and they are specifically designed so that you can redo the training on a single (powerful, but affordable [at least, before the bubblers decided to buy everything]) machine. That made them things that people could experiment with, exploring different model structures to see how it affected speed and accuracy.

So, yes, a local model will not send data across the network when you use it (hopefully. Unfortunately, most are distributed as Python code and a load of the ones on Hugging Face also came with bundled malware. I hope they’ve managed to fix that now), but they’re not open in any meaningful way, they are still subject to the whims of massive corporations, and they are building a dependency on the exact companies that Doctrow criticises and handing them a load of control over your workflow.

Da_Gut

@Da_Gut@dice.camp · 3 days ago

@david_chisnall @zzt @pluralistic

Thank you. I was unaware of the specifics of how the things work for the local models.

Giacomo Tesio

@giacomo@snac.tesio.it · 3 days ago

@david_chisnall@infosec.exchange

I find much easier to explain this technology in terms of virtualized hardware and software executed on it.

The "model" is just a custom special-purpose machine architecture executing the "weights", that are the encoding of the software such machines can run.

The match between the topology of the model and the cardinality of the weigths' matrices is a hint about this simple relationship.

I call the "model" as "vector mapping (virtual) machine" or VMM.

Just like a CPU is built by composing transistors, the fundamental component of a VMM is a "vector reducer", that is a little function from a vector (an array of floats) to a scalar (a float, usually in a carefully crafted range).

Vector reducers have a few nice properties:

they compose well, so you can compute M of them in parallel over the same array of N floats, and you get a map from a N-dimensional vector space to a M-dimensional vector space (a layer, in ANN jergon)
they stack well, so you can take the M-dimensional output vector of a layer and feed it to another fleet of O vectors reducers to obtain another O-dimensional vector. Such pipeline effectively map the original M-dimensional vector to a O-dimensional one, appyling a complex non-linear trasform to it.
crucially, by carefully picking each layer's parametric function and recording each reducer parameters (usually, polinomial coefficients, aka "weights", bias and threasolds), you can back-propagate statistical errors against your intended output for any given input, iteratively programming the machine.

Once you realize we are using the data to iteratively compute a software designed to be executed by a very specific, special purpose machines, everything becomes clear about this technology.

You are not "training" an "artificial intelligence" with IP-infridging datasets. No machine is "learning".

You are just compiling such datasets into an executable, just like you would do with a compiler on C sources.

The datasets are the source code for such compilation process.
And if an x86 binary executed by a CPU retain the source's authors' copyright, the same applies to float executed by a GPU.

Also you stop thinking about such software as a subject: it's never #ChatGPT who is harming or defaming a person, but #OpenAI through ChatGPT.

Removing the antropomorphization restores the full chain of accountability.

We should all refuse to talk about "artificial intelligences" that "hallucinate" and speak in terms of defective statistically programmed software that was compiled from illegal and unknown sources and that was likely injected with undetectable backdoors.

We should refuse to call #MachineLearning what is just a compilation process and to use evocative terms like "latent space" what is just a pattern-preserving statistical heavy-loss compression.

Unfortunately we need to wait for the #AIbubble to blast, because even #academia is too dependent on #BigTech money and subsides to... think clearly about this stuff.

Let's just hope it busts before fascists find a way to leverage either the crash or the tech.

#AI #LLM #security

@Da_Gut@dice.camp @zzt@mas.to @pluralistic@mamot.fr

arXiv.org

Planting Undetectable Backdoors in Machine Learning Models

Given the computational cost and technical expertise required to train machine learning models, users may delegate the task of learning to a service provider. We show how a malicious learner can plant an undetectable backdoor into a classifier. On the surface, such a backdoored classifier behaves normally, but in reality, the learner maintains a mechanism for changing the classification of any input, with only a slight perturbation. Importantly, without the appropriate "backdoor key", the mechanism is hidden and cannot be detected by any computationally-bounded observer. We demonstrate two frameworks for planting undetectable backdoors, with incomparable guarantees. First, we show how to plant a backdoor in any model, using digital signature schemes. The construction guarantees that given black-box access to the original model and the backdoored version, it is computationally infeasible to find even a single input where they differ. This property implies that the backdoored model has generalization error comparable with the original model. Second, we demonstrate how to insert undetectable backdoors in models trained using the Random Fourier Features (RFF) learning paradigm or in Random ReLU networks. In this construction, undetectability holds against powerful white-box distinguishers: given a complete description of the network and the training data, no efficient distinguisher can guess whether the model is "clean" or contains a backdoor. Our construction of undetectable backdoors also sheds light on the related issue of robustness to adversarial examples. In particular, our construction can produce a classifier that is indistinguishable from an "adversarially robust" classifier, but where every input has an adversarial example! In summary, the existence of undetectable backdoors represent a significant theoretical roadblock to certifying adversarial robustness.

Orb 2069

@Orb2069@mastodon.online · 3 days ago

@Da_Gut @zzt
... Sure, in the sense that you picked a file, but that's like saying you control what's in SPAM because you picked THIS can and not THAT one.

[object Object]

@zzt@mas.to · 3 days ago

@Da_Gut that’s incorrect for all of the local, supposedly open source models I know of

all of the research I’ve read on this has easily extracted verbatim plagiarized text from the models, because all of them have their origins in the same sources — usually Facebook’s leaked llama model or deepseek (which itself took from previous models). it isn’t possible for LLM models to be trained by anything other than a billion dollar company or a state operating like one.

Da_Gut

@Da_Gut@dice.camp · 3 days ago

@zzt oh, I did not know that. So if you disconnect it from the Internet, it ceases to work?

Matthew Thomas

@mpt@mastodon.nz · 3 days ago

@Da_Gut @zzt It has nothing to do with whether you’re connected to the Internet. The plagiarism was baked into the model when it was trained weeks/months earlier.

[object Object]

@zzt@mas.to · 3 days ago

@Da_Gut the factors I’m talking about aren’t technical ones, they’re social and systemic. specifically:

- local LLMs are worse than cloud ones, and necessarily must always be. it isn’t possible for independent development of models to happen, and LLMs are already on an intentionally fast deprecation cycle. old models aren’t viewed as useful by anybody.
- it’s very easy for established companies to take action against local models as IP theft, and they’re already laid the groundwork for this

[object Object]

@zzt@mas.to · 3 days ago

it’s important to note that it isn’t always a big sack of cash. lately I keep seeing this pattern happen with engineers:

- “as an AI skeptic I finally have empirical proof that LLMs are good/useful/thinking/feeling <posts slop>”
- “uhhh are you ok? I checked the LLM output you posted and it doesn’t make any sense if you dig in at all and the citations are all fake”
- “this is empirical proof and you’re being emotional.”

this is engineer brain. Doctorow isn’t an engineer, so sack of cash it is.

aburka 🫣

@aburka@hachyderm.io · 3 days ago

@zzt I think AI boosterism is the first stage of AI psychosis

Jo - pièce de résistance

@JoBlakely@mastodon.social · 3 days ago

@aburka @zzt
I didn’t see any AI boosterism. I’m not sure I understand the issue in that article with the purity culture quote. It wasn’t referring to AI, AFAICT. It was referring just to being on bluesky.

[object Object]

@zzt@mas.to · 3 days ago

thx for telling me that everything I have hosted on the web getting repeatedly scraped to death by what would previously be considered a massive attack but is now being carried out by the largest corporations in the world is normal, actually. hope they give us good licensing terms on our data, uhhh no wait their IP, once they’re done killing and buying all the original data sources

[object Object]

@zzt@mas.to · 3 days ago

I am wasting my time of course, Cory is and always has been a stack of Wired magazines with a flesh-colored mic strapped to it. every talk Cory does is a ted talk.