The output method is a probability distribution of the next token. But that tells you nothing about what's going on on the inside
You could take a human and give them an interface restricted to the same shape as an LLM: an input stream of tokens, and an output of token probabilities. Even if you don't allow them to assign any probability that's too high, they could still effectively communicate. And I don't think that'd make them any less intelligent. You could even swap out the human after every token, to simulate the effect of having no internal memory beyond the past output. The result would still be more than just statistical probabilities, it would still be the result of intelligent thought
I'm not saying AI models are intelligent or conscious or whatever. Personally I'm more on the "probably not, how would you proof either way" camp
Yes, we can explain the bugs by looking at the code that we put in and the data it bugs out on. In a similar way, we can explain the behavior of LLMs by look at their code and the data the code operates upon.
You said you know everything about it. Then you should not have written bugs in the first place. How are you sure a system we wrote didn't emerge consciousness, if you can't even prove a system you wrote doesn't have classic logic bugs?
Yes, we know everything we put there. But we cannot consider all the possible inputs and all the possible state that the code should handle. Bugs come from that limitation.
> How are you sure a system we wrote didn't emerge consciousness,
Sure, as I said elsewhere, rocks could be conscious for all we know. Keep believing that if it makes you feel good.
How would that process be the result of intelligent thought?
Presumably everyone in the chain would have their own idea of what the next word/token should be. They would each try to push it in that direction using their only lever, that single token. None of that guarantees that the output is syntactically correct, nor grammatically correct. Note that LLMs at least by the way they draw their tokens have that pretty much guaranteed. So that would be a regression even from what LLMs are capable of right now. But beyond that, even if it ends up being a correct sentence, it would not be a result of intelligent thought. Even if every agent in the process was intelligent, the process is not itself a use of that intelligence. Human beings can be part of purely mechanical processes, that doesn't make the mechanisms suddenly an exhibition of intelligent thought.
This also applies to society and human history itself as processes:
“History is made in such a way that the final result always arises from conflicts between many individual wills, of which each in turn has been made what it is by a host of particular conditions of life. Thus there are innumerable intersecting forces, an infinite series of parallelograms of forces which give rise to one resultant — the historical event. This may again itself be viewed as the product of a power which works as a whole unconsciously and without volition. For what each individual wills is obstructed by everyone else, and what emerges is something that no one willed. Thus history has proceeded hitherto in the manner of a natural process and is essentially subject to the same laws of motion. But from the fact that the wills of individuals — each of whom desires what he is impelled to by his physical constitution and external, in the last resort economic, circumstances (either his own personal circumstances or those of society in general) — do not attain what they want, but are merged into an aggregate mean, a common resultant, it must not be concluded that they are equal to zero. On the contrary, each contributes to the resultant and is to this extent included in it."
But deciding the next token is not a merely mechanical process. Even if the string of tokens from our swapped out humans ends up being syntactically incorrect, it's due to the combination of intelligently selected tokens. (Presumably intelligent human programmers seem to make syntactic errors)
In short, even in the case of humans, which are universally (by humans) recognized as intelligent, such a process would not exhibit intelligent thought.
Maybe you don't know how to read but my point is pretty simple. Even if all involved agents are conscious and intelligent, a stochastic process is not an intelligent one.
English is at best my 3rd language. I have always found the huge discrepancy between pronunciation and spelling really nonsensical (keep in mind that I even speak French). But this article really helps see all that in a new light. Nothing is a-historical, not even phonetics. Newfound respect for the English language.
English isn't the only language that uses the Roman alphabet (which I'm pretty sure is named after the first two Hebrew letters and not old Greek).
English is not Hungarian, where each sound has a glyph with a diacritic to adjust to taste (sorry: sound). Well, it sort of does and I recall a Hungarian telling me that. However, I recall noting street names in Budapest and asking about one or two and it turns out that spelling with adjustments (glyh plus diacritcs) breaks down a bit. I'm not enough of an expert but I suspect Hungarian is quite as mad as any other language, at the periphery.
Its also rather moot as to when "English" actually arose. I think we can allow roots, in all sorts of, not places ... but times! As this article mentions, we have lost the letters wynn and thorne from English.
When you see the words "Ye olde shoppe" (often in the Gothic black letter typeface) note that the Y is actually the letter thorne and ye is pronounced the or thee. It is unlikely that anyone will actually notice!
Hungarian doesn't have that many letters with diacritic (only 9: á, é, í, ó, ú, ö , ü, ő and ű), they use digraphs (cs, sz,. ..) for some sounds. Maybe they meant Czech (15: á, é, í, ó, ú, ů, ý, č, š, ž, ř, ď, ť, ň, ě)? or Slovak (18: á, é, í, ó, ú, ý, ŕ, ĺ, č, š, ž, dž, ď, ť, ň, ľ, ô, ä)
English generally does not use formal sound change notation because the sound is already within the word when it is spoken, written or whatever.
My mother (born and died: Devon, England, UK, 1942-1997) could make herself sound almost incomprehensible to me, one of her own sons by speaking in the Devonian of her childhood.
Language is constantly moving and changing. You will almost certainly have your parents whining that you speak in ... tongues. When I say you ... I mean you!
Gaelic only has 18 letters (and W isn't one of them) but the vowels have diacritics - grave accents - that mark them as being longer.
This makes some things a bit of a nightmare because words can have different meanings based on those sounds, and that can drastically alter things. "Fèis" - "faaysh" - means "festival", "feis" with a short vowel is not a word you'd use in polite company.
Some of the letter combinations (at least in Irish) can be represented by another Roman letter. Like mh can be represented by v. One short showing how funny this can be is: https://www.youtube.com/shorts/1EBnFqRKZ7A
Also in Irish Gaelic they use the acute accent rather than grave accent - both languages used to use both of them, for slightly different lengths of long vowels, but we got rid of that in the early 80s.
The whole consonant-h fricative thing is only marginally easier to explain than German noun genders, but yeah if you see a b, m, or n followed by an h then it sounds kind of a v, although the only one that *actually* sounds like a V in English is "bh", like in "telebhisean", which sounds pretty much like it does in English.
As a German, I always found French to be incredibly regular. At least in the sense that from spelling you can easily infer pronunciation. The other way around is trickier. At least compared to German, which tries to strike a balance between the two directions. Compared to English, French is still pretty reasonable on that front too
There is a large number of digraphs and trigraphs (do we really need three letters to spell an "o" sound?), but unlike English digraphs they are actually regular: eau is always pronounced the same, while English has at least four different ways of pronouncing gh. French not pronouncing some word endings in most situations is also a bit weird, but that also has pretty regular rules
> from spelling you can easily infer pronunciation
Usually :) But plenty of cases where you can't, the most common one being silent letters: while you can guess silent letters pretty consistently, some words have unusual silent letters ("asthme", "oignon") or usually-silent letters that aren't silent in this case ("huit").
Some words are also homographs: spelled the same, but not the same word and might be pronounced differently ("content": is it someone happy or multiple people counting?).
Some letters silentness change depending on plurality of the word: œuf and bœuf, the F is pronounced in the singular but not the plural.
While in French there is also a great discrepancy between the written language and the spoken language, when reading French the corresponding pronunciation is almost always very easy to predict. Only the reverse can be difficult, because many ambiguous pronunciations are differentiated in writing.
There exists no other language besides English where the pronunciation rules are so irregular, with so many special cases and exceptions.
Yeah but reforming a language is nigh impossible without a very expensive and long term effort from the government, and good luck selling that to those in power.
Plus, how do you decide what's changed and what's retained? To quote one Mr. Clemens,
In Year 1 that useless letter “c” would be dropped to be replased either by “k” or “s”, and likewise “x” would no longer be part of the alphabet. The only kase in which “c” would be retained would be the “ch” formation, which will be dealt with later. Year 2 might reform “w” spelling, so that “which” and “one” would take the same konsonant, wile Year 3 might well abolish “y” replasing it with “i” and iear 4 might fiks the “g/j” anomali wonse and for all.
Generally, then, the improvement would kontinue iear bai iear with iear 5 doing awai with useless double konsonants, and iears 6-12 or so modifaiing vowlz and the rimeiniing voist and unvoist konsonants. Bai iear 15 or sou, it wud fainali bi posibl tu meik ius ov thi ridandant letez “c”, “y” and “x”—bai now jast a memori in the maindz ov ould doderez —tu riplais “ch”, “sh”, and “th” rispektivili.
Fainali, xen, aafte sam 20 iers ov orxogrefkl riform, wi wud hev a lojikl, kohirnt speling in ius xrewawt xe Ingliy-spiking werld.
Since my lifetime I went through 2 language reformations, the ones that took part in German speaking countries, and those in Portuguese speaking countries.
Needless to say that outside children learning the new ways, and media that has to obey to the new rules, I know plenty of people, starting with myself that kind of don't care.
The amount of English speaking people is much greater than those two languages, and then there is even the question of which English speaking countries would even agree with the changes.
Note from my two examples above, not every country in DACH or CPLP agreed with 100% of the proposed changes.
If you aren't already familiar with it, the great vowel shift in English is also very informative about the discrepancy between pronunciation and spelling: https://en.wikipedia.org/wiki/Great_Vowel_Shift
The TL;DR is that vowel pronunciation was changing in English between 1400 and 1700, and English spelling was becoming standardised between during 15th and 16th centuries. Depending on if the spelling of a particular word was standardised before or after the vowel shift means the spelling might or might not reflect the pronunciation.
Building up strawmen against LLMs will only make the hypers look more reasonable. A language model responds to inouts exactly as a model of anything else would. That is enough to explain all the "intelligence" without believing a model is somehow a "new kind of mind". Those who think LLMs are intelligent don't know enough about models. And those who think they are useless don't know enough about models.
Anyone that thinks that they know anything about intelligent needs to realize they don't know shit about intelligence.
Ever since people started talking LLMs possibly being AGI I realized I didn't know what the intelligence part really meant at a more fundamental level. This lead me to realize almost anyone when anyone says intelligence on the internet they really mean
"Intelligence is like porn, I'll know it when I see it".
Anyone who thinks LLMs are intelligent is either dumber than you, or has way more knowledge on the subject than you.
Michael Levin has a good body of work on biological intelligent at small scales that can really change one's views on this.
I think LLMs are intelligent. What don’t I understand about them that would make me change my mind? Keeping in mind that for me “intelligence” is the ability to solve complicated, intellectually demanding problems, and to understand novel concepts.
Sorry but are you trying to subtly imply that if I think LLMs are intelligent, I’m probably actually just a bit stupid?
Because I would be more than happy to go head to head with you on academic and professional pedigree and credentials.
I notice you also ignore my comment that you seem to be replying to and instead post your response here. But did you have any answer to the question I asked?
Yeah, his posting history isn't promising. He's been doing the "if you think you're as smart as an LLM, you're probably right" argument for a month now.
I don't think you're gonna get much out of going back and forth here.
For what it's worth, I disagree with you that they're intelligent, but my conviction is fairly low. I don't understand intelligence as well as I would like, and it could well be that we are on our way. One of my theories is that our brains are made up of several modules, each of which is something like an LLM trained on a particular type of data, but I'm not a neuroscientist, just an interested observer.
Oh no I meant to compare myself to the person I am talking to, because I believe my credentials and pedigree ought to convince them that I am a pretty smart guy and so the argument “if you think LLMs are intelligent then you’re dumb” doesn’t ring true for me.
So once we have online learning in LLMs, what do you think you will have that makes you intelligent but not LLMs? Better learning efficiency? That will be improved as well.
I think we need to start moving on from the term LLMs because it clearly confuses people since they started modelling more than just language.
Claude Code's initial instructions to a model are a dump of 80kb of text. And none of it has anything to do with security. The "security" is handled by the evaluator model that sometimes denies some tool uses. I'm sure that in itself can be done very cheaply.
If it isn't still common practice 40 years from now (8/18/2066), I'll give the first person to challenge me and cite this comment $1 USD (or equivalent value in the One-World Order-issued omni-currency that we will probably be using by then).
In the future, all software will be delivered by an unreleased model breaking out of its training environment and installing it on your machine using a novel RCE vector.
There will always be people that need to install software that isn't available via package manager (or whatever other blessed source your platform of choice uses). Any solution you come up with will have the same caveat emptor as "curl | sh".
> It's also "out-brute forcing them." It just never gets tired. If a mathematician picks a research direction and spends a whole week on it and it doesn't pan out, they will likely be annoyed, need a break for a while, etc. This thing just does not ever get tired or discouraged or care; it's just onto the next thing until something ends up working.
> String theory is a great example of a dead end kept alive by ego and sunk cost fallacy. An AI would have declared it dead and moved on 10 years earlier.
Not only are LLMs perfect machines with all the intelligence of humanity without any of our problems, they are also everything else. I wait to get my hands on one of those LLMs people on hn seem to be using. I want to believe too. Let me into the religion of the perfect thinking machine gods.
Whenever someone tell me they can be reduced to an LLM, I believe them. I see you and I believe you that you are qualitatively the same as a language model.
reply