Why does a blog page need three loading spinners at the top to load a couple of simple text banners asynchronously?! ... And this from one of the biggest software giants. We have fallen so far.
I just tried ChatGPT on a simple find an error in the proof question today and it completely butchered it ignoring the main constraint that two variables a and b were natural numbers. It said, "a and b might not be natural numbers" and then went on to produce a very elaborate, completely wrong answer.
Thank you. The apps created like this are absolutely an embarrassment. But to me the bigger horror is if people are going to be convinced to develop the critical libraries and infrastructure in the same way. Then we'll have bloat and rot in the deeper levels and everything will become exponentially worse and unreliable. It makes me so sad that people can't see this.
When people talk about AI being able to handle everything I keep wondering, have these people built anything complex, novel or serious? Just because people can see a website or a simple app improved, does that mean that all code can be handled by LLMs? It's like people are totally forgetting a whole category of careful, well-thought out programming for the critical parts.
Yes, what you are seeing are amateur developers that barely understand the tools they are using either giving LLMs poor instructions, or totally taking whatever it says at face value, then not bothering to put in further effort.
If they just kept prompting it, or maybe used a different thinking level, it could have identified and solved this problem. Sometimes an engineer would look at a system and say "the current approach isn't delivering the desired engineering requirements. Maybe we need to rethink".
Either engineer or LLM could take that sentence and run with it. OP of the article clearly can't do either.
> the current approach isn't delivering the desired engineering requirements. Maybe we need to rethink
I'm reminded of the times I've tried to let Claude fix some well-documented bug in the background, and it ends up burning 4 million tokens and 20 self-review cycles re-writing the same set of code a dozen times with ever more complex unit test mocks / overcomplicated regexes / giant comments restating the same thing the code does, when the actual fix turned out to be "change three to ten lines to do something in a slightly different way that avoids the problem entirely".
You're missing the point where: in complex systems, sometimes optimizing code is both a high effort undertaking, and can totally not pay off. Having done hundreds of such exercises on our software over the years, it's liberating to have an idea of how to make something faster, being able to validate it without the fear of having to throw it all in the trash if it fails after days of work. What is still important is being able to provide proper guidance - we even built new tools to allow an AI agent to analyze memory usage in more depth, and instructions on how to benchmark in cloud environments where shared CPU usage and VM reallocation happen all the time and confuses the AI all the time with measurements
Well summarized, but I believe this is sadly why we're seeing a decline in software quality all over. And if people keep following this trend for libraries and core utilities... yikes things will get bad.
Yeah but keep in mind this tech only started to work last year? People still figuring it out, and I'm sure we will have more robust tooling, processes around it down the road.
The upside is that we are able to deploy more cognitive power into problems, security, edge cases, something that uses be to scarce. A lot of the people feeling insecure mainly because they benefited from that scarcity. But we will have more software deployed, I'd argue better, and larger, and this means the pie of software is getting bigger, the market is getting larger, deeper into the economy.
I wrote something original in a language learning grammar online. I coined a term to describe something about how a certain language with negative language functions.
I tried talking to an LLM about it and sure and enough, it had scraped it and learned to use that terminology and explanation that I made up. I asked the LLM, "Where did that concept/term come from, who made it up?" It did a bunch of searching and tried to site a whole bunch of other sources which did NOT contain the terms of the explanation I had written. It simply would not cite or mention my source. It kept parroting my material semi-correctly and hiding the source. By any other actor that would be an egregious act of sloppiness, dishonesty, and plagiarism.
Generative AI is speed-running a widespread corruption of truth. I do not believe any of the gains are worth this.
Your comment illustrates that AI companies are playing a kind of shell game. On the one hand there is theft of IP and violations of copyright, in the middle there is the obscuring of where things come from, and on the other hand there is the selling of it via tokens.
More countries should debatably follow what Japan is doing[1], where they are creating new laws to force AI companies to show their training data and how they collected it.
Your text may well have been in the training corpus, but not searchable with whatever terms and search engine that LLM used after you prompted it. It doesn’t have recall of sources of documents comprising its training corpus, unless the source is widely cited in other of its training documents. Sourcing and provenance aren’t currently an intentional part of LLM training.
That may already have been obvious, and central to the complaint, but I was just knee-jerking to the anthropomorphic language.
I would assume the parent understands that. The criticism is about the LLM behavior this results in. Being able to explain a behavior doesn’t necessarily excuse it. By some moral standards, you wouldn’t have trained an LLM that way, or wouldn’t have made it available, given the predictable outcome.
France, 1658 <-> World, 2026. AI can also do the reverse : for example very few in the world know that the term midwit was originally coined in french by Blaise Pascal (« demi habiles ») centuries ago.
Yes, that bell curve meme, he wrote about it. It’s not explained anywhere because of the language and culture separation between
french and english. But if you ask enough questions to AI, the AI will eventually find about Blaise Pascal and tell you about this unknown link. It’s magic.
There used to be a meme on Slashdot: "In Soviet Russia, X Ys you!" It came from a joke by Russo-American comedian Yakov Smirnoff: "What a country! In LA, everybody is looking for a party; in Russia, the Party comes looking for you!" But I long wondered if Smirnoff was influenced by his countryman Leon Trotsky who observed: "You may not be interested in the Dialectic, but the Dialectic is interested in you."
Well done Confucius, you've invented a new creditpocalypse. Its when llms hallucinate new authors for new words. It almost makes it worth starting a blogsurection!
A day ago blogsurection had zero Google search results, today it has 4 like: "The Rise and Fall of the Blog" and the search llm sniplet explains what the word means.
It's more than that though. I authored the original work. Then the Baron in the Castle collected the work and uses it without fully understanding it. The Baron gives it out but in a partly-true, distorted form. He misunderstand how it applies and teaches people false clearly facts with it, with an air of authority. But when people try to find the source argument to see if the Baron is really on track, they have no idea. The Baron is just confidently parroting a compelling sounding set of words.
For me the issue is more about the loss of truth than the loss of my IP.
That is pretty weak logic. I mean obviously it is the Baron who becomes wealthy. But what follows from this?
By the same logic if Baron bought her baroness masterful dress to enjoy her look in that dress, and you when walked past the Castle saw the baroness in that dress, should you compensate to the Baron or not?
Or like the difference between chemistry and alchemy? Understanding and reasoning with the building blocks as opposed to throwing random stuff together, trying different things and hoping it somehow produces gold.
I just pulled a PWA app off the app store and I am going to keep it a pure PWA. It's installable, offline-first. I know I feel like I'm swimming against the current but I would love people to try to push for awareness of PWA installs more.
I'd love to see people distribute their apks directly.
Itch.io is an option for people who don't want to set up a site or handle payments. While they focus on games, they're cool with distributing non-games:
I personally have a TrustedWebActivity. Most update doesn't need a review because I only need to update server-side. Behave like a PWA while being an Android App. (And you can implement Android Auto, other native stuff)
It allows the users to discover my web app through the Google Play Store.
My app was also available as a PWA but I mostly push the TWA currently.
Would I be wrong to assume that you are building end-user applications?
If people use AI for libraries, OSs, and mission critical software, the apparent productivity gains would have to be weighed against the reliability and performance hits that bubble up to the things that are built on them and rely on them.
In my experience - a robust testing harness will get you the safety you need. And most software you describe has such testing.
I think the Bun port is a great example where testing enabled a very successful implementation. (Both the original tests themselves and runtime comparisons to the previous implementation.)
Yes of course, but you also understand that as Dijkstra said "tests cannot show the absence of bugs."
Tests can show you problems, if you can find them, but they cannot show that there are no problems. Property based testing or fuzzing gets your more coverage, and is a good step, but it is still nothing compared to proving things or understanding how something is built and that it is solid. Testing works towards checking for reliability and robustness, but often it's only 10% (?) of the job.
Well, yes, I understand that testing does not create a provably correct solution. But I'd love to hear the source of your "extremely inadequate" or your "10%" claims. I mean - there is a reason why it's used extensively in software engineering - right? Or don't you see value in that, either?
I'm curious - is there any data on the Bun port error rate? I think that would be very indicative of how successful or not the testing is.
reply