I think this is the most beautiful letter on the subject I've read all year, it brought a tear to my eye. It's like reading those famous STEM letters/essays from history.
I agree. I believe it could be the best piece about AI ever written. The themes of lost dreams reawakened, sobering up of egos, sci fi technological development, the role of humanity. It’s a weird feeling to see that it’s actually really happening.
The metaphor as written is maybe bad, but the holographic idea really is outrageous and mindboggling. Because according to it the inside might as well be completely empty.
But these are not very good arguments, because it makes it about the fall of institutions (the funding) and people being miserable for personal reasons rather than prosocial reasons. Tao here clearly suggests that math is not reducible to "problem solving" or "proofs", the valuable part is much more than that framing.
The concerning argument about the status of math would be an outline that it will get destroyed by a process of societal atrophy and there is no turning back, and the AI powers are not a good substitute or replacement for it. If an entire society becomes reliant on these oracle machines then it would be analogous to children never learning arithmetic because they were handed calculators. How could the human race still flourish? We would anthropologically regress. We'd be little better than animals, like the Borg zombies.
That is a much more profound threat than people worrying about their own careers or faculties disappearing like the humanities. This is a serious anthropological reckoning.
If math experts are that freaked out already then basically all of science is soon to follow, decade by decade. "Singularity" comes to mind.
Broadly I agree with this, but I was refuting the statement "there has never been a better time to be a mathematician". You are changing the question to something different here, and using it to say my argument is bad.
I think math is a microcosm for "thought-work" in general. We have been in a symbiotic relationship with capitalism for decades now, where the hope of a well-paid white collar career encourages people to spend time and money to enrich themselves through education. The proliferation of AI cheating at college already signals that employment is the primary goal over intellectual growth, so one imagines this governs what happens next if labor demand disappears.
It's hard to say where this all leads, but I have very low optimism for higher-level math understanding being something that humans value in the same way in the coming decades.
Why such a blinkered argument, I own the soda can I just drank but it is still, arguably, wrong of me to chuck it in the trash instead of the recycle bin. Yes, you have the "right" to dispose of things you own but there's more to what's right than that. There's human decency, consideration, awareness of the environment, the spirit of the laws, cases the laws were not designed for in mind etc.
Like, tell your mother or HS teacher you decided not to recycle because you own your trash. See how well that clever argument goes.
The problem with the Fields medallists petition was that its central argument relied on reiterating a very fashionable, neoliberal-academic trope, that science as a whole is "done by a culture of experts", implying that there is no value or even science existentially outside of "the social practices of science". It is a predictable extension of centrist-pseudoleft political ideologies (~ ivory tower belief system) which Thomas Piketty for instance has conceptualized as the Brahmin left. Indeed the whole AI/ASI/AGI issue is hugely political and it cannot be dismissed and analyzed as a neutral topic in isolation from people's political premises.
That said, I think there is an articulable concern about the recent AI events as it may threaten human's autonomy and relation to science and mathematics. Just not one premised on that. What's interesting is seeing different academics having predictable reactions, for example Sabine Hossenfelder is more politically libertarian so in this case she said DGAF about human mathematicians at all. Scott Aaronson is another interesting example, he wrote a blog post a couple days ago as well.
There is an iq requirement to doing good mathematics and science, there is no way other than emergence of experts. Unless you believe a random foaming red hat wearer could have solved this or that millenium problem.
Such a simplistic historical comparison tells on yourself, it is precisely this mentality that larger sociopolitical forces exploit and reproduce new fascisms and the like. In other words your very argument demonstrates the falsity of the claim.
It's like AlphaGo but playing against all living mathematicians. (Overhang being low hanging fruit is what allows this comparison, of course the general moot point is the skepticism that LLMs are also innovative etc.)
We are not seeing those incredible moves yet. The approach used in N-S was conjectured to work after B&L’s initial breakthrough. See a post by Tao. So on one hand the proof is an amazing accomplishment. On the other hand, humans have not yet discovered any superhuman moves in the proof. Just $MM grind.
So you're just equivocating on terms like "prompt", "synthesis" and the like. Clearly a PhD in physics does not free people from scientistic modes of thinking and poor philosophy.
To think this discussion is about Einstein who had a much better mind on these things as well.
They used words to mean what the words mean. What specific issue do you take with that?
"prompt", as in prompting an AI, has the same definition as "prompt", as in prompting a person. They mean the same thing, that's why the term was applied to AI after already applying people.
*Jingle-jangle fallacies are erroneous assumptions that either two different things are the same because they bear the same name (jingle fallacy); or two identical or almost identical things are different because they are labeled differently (jangle fallacy).[1][2][3] The term was coined by Truman Lee Kelley in his 1927 book Interpretation of educational measurements.[4] In research, a jangle fallacy is the inference that two measures (e.g., tests, scales) with different names measure different constructs. By comparison, a jingle fallacy is the assumption that two measures which are called by the same name capture the same construct.[5][6][7]
You are simply incorrect. It is not a fallacy of that type, or any other type, because the words do, in fact, mean the same thing, as multiple people have pointed out here. Whether referring to chatbots or people, "prompt" means "to move to action".
If you have some reliable source supporting your unilateral claims that "prompt" does not mean this, please share. Otherwise, the consensus seems to be contrary to your claims.
Why do I need a source? An LLM prompt does not "move to action", because an LLM does not act. People act, animals act, software doesn't act. Acting implies volition and volition implies cognition and if you think that LLMs have those things then you're the one who should provide a source for your claim.
How about an LLM connected to a robotic arm. What then?
As for cognition - can you prove that you possess it, to an external observer? Could you, confined to a box through which you can only communicate through textual messages, prove that you are thinking, and not just responding through a mechanistic process?
Absolutely you do. I can’t prove that I have cognition. I merely have the impression that I do, and we accept as a general axiom that others do too - but it is not provable. You are examining a system from within the confines of that system.
And you’re seriously going with “don’t ask inconvenient questions” as an argument?
>> And you’re seriously going with “don’t ask inconvenient questions” as an argument?
Where did I say that?
The conversation you seem to want to have doesn't seem to have any obvious use and you seem to want to debate a point I never made. I don't seem to be needed here so I will now bow out of the conversation.
That commenter was already obnoxious under this thread, and today they are relitigating Searle-Turing via Hofstadter GEB slop, so they are also an ignoramus.
Because your entire post is a personal, philosophic opinion on what words should mean, and how if someone disagrees with this personal opinion of 1/7,000,000,000 of people, then they are wrong. Every single one of these claims is unsupported and contradicted by reliable sources like a dictionary:
> An LLM prompt does not "move to action"
> an LLM does not act
> software doesn't act
> Acting implies volition
The only way such opinions could viably be considered "true" is if you convinced a majority of people to agree with you on the changes, which you decidedly have not.
Until you do, we can go by the existing definitions: "prompt" means "to move to action", "act" means "the doing of a thing" (note the absence of "volition" or "cognition" in the actual definition), and by all reasonable accounts, LLMs "do things".
I mean, just take a step back and dispassionately look at your ridiculous claim that nobody prompts LLMs (because you now claim only a human can be prompted, because only living creatures can do a thing). Do you really think you can gain consensus from folks on that claim?
There are many differences between LLMs and people, but this is not one of them, according to all reliable sources I've found so far. Feel free to share the reliable sources which led you to your personal conclusion here.
I'm not going articulate the answer but I will say whatever beef you have with the other commenter (or me), just copy this thread into ChatGPT and it will tell you why you are more in the wrong.
Actually, my undergraduate degree was physics and philosophy. And yes, synthesis is synthesis whether a human, a machine, or a duck does it, and people prompt one another all the time - “have you thought about trying X?” Or “I need the TPS report by EOB”.
I suppose my underlying point is that human cognition is not the unique and beautiful thing that we anthropocentrically suppose it to be - it is a physical process, with stochastic outcomes. Much like transformers.
Me, I’m just a machine made of meat. You can suppose yourself to be God’s perfect creation, and that’s your right, but I disagree.
Clearly your degrees did not make you immune from fallacies and simplistic reductions.
"Synthesis" is obviously of different kinds. A duck has a different level of intelligence than a human. We do not say both are "just doing synthesis".
So the question is how can you be so disingenuous about such terminology? Answer, you are relying on a classic form of scientistic reductivism.
The fact that intelligence is physical, emerges from chemistry, etc,. has nothing to do with there being also objectively different levels of computational sophistication.
If you want to be scientific about that you could look at neuropsychology on one hand and computability/complexity on the other. There are levels and so equivocation of "mentorship" as "prompting" and fallacious variants thereof is a) frankly intellectually obtuse, b) par for the course for SV-levels of philosophizing, c) and a disservice to philosophy, physics, and Einstein's own philosophical outlooks himself.
I am well aware of the Hinton-style physics argument about human cognition, and unlike others I am partial to it. That "there is no special magic." But it is wrong to go about misunderstanding and/or conveying this physicalism/computationalim so grossly.
I also don't have to start replies thumping my chest about my credentials, also another kind of intellectual boorishness that works to cloud understanding and serious discussion.
I'm not sure which move is worse or more telling, those above or the one backhandedly accusing someone who disagrees with you of religious thinking. It is bad faith and undisciplined behavior. Having privileged and advanced degrees is clearly no antidote, as Asimov famously wrote.
What’s your basis for that “obviously”? You have a unique insight of the phenomenology of duck-ness? You can prove that your consciousness is somehow real, somehow different? A duck synthesises with its cognition, or it would be incapable of, well, anything. Synthesis is purely the process of the integration of inputs into outputs - ie behaviour, language.
“objectively different levels of computational sophistication”
Says who? We still have a very poor understanding of how cognition works in animals, humans included. For all we know ducks have rich inner lives - a remarkable amount can be achieved with a very small neurone count - cf. insects. Can you coordinate flight? Can you echolocate? Are you less intelligent because you cannot?
“equivocation of "mentorship" as "prompting" and fallacious variants thereof”
You are arguing semantics. Take Harry Nyquist. He sent people down new paths with insightful questions. You could call this mentorship if you choose, I could call it prompting, but this splits hairs. The core idea is that a novel input can produce a novel output, that synthesis can be induced through guided and deliberate external input.
I invoked credentials only in response to the previous derogatory comments about my cognition - which may or may not exist, anyway.
As to religiosity - the idea that human cognition is somehow unique and special and impossible to replicate, which is the prevailing argument in this comment tree is religious, and anthropocentrism of the highest order. I apologise for accusing you of it - I was evidently wrong - I had mistaken you for a previous poster.
You're wrong about the ducks. But getting back to your previous wrong argument from 14 hours ago, you basically deny the meaning of terms like "uninspired", "insipid", and "derivative", on the grounds that we're all standing on the shoulders of giants and therefore it's all good. This is incorrect, it's not all good, and the things the LLMs do really are unoriginal, a term that really does mean something.
reply