"We spent the last several months testing whether auto mode is as safe or safer than an average user clicking through prompts."
Yeah, might make sense from their perspective, but no thank you. I also do click through at times without reading everything, but I like to stay in control, learn about the new code and change direction if it goes off track. This would just burn more tokens because I have to throw away much and I hope my manual approval settings will be respected also with future updates (or I jump ship).
The default is set for the marginal new user, which at this point is probably not someone like you (who benefits a lot from manual mode) -- it's someone who's more "code-naive" and might get anxious about approving random bash script commands they don't recognize. Safely getting the user from prompt --> first vibe-coded app is the "user journey" now, and since auto mode seems pretty good at not letting Claude rm -rf'ing the home directory, this is 100% the right business move. For people who know what they're doing (like you), manual mode is just a shift-tab away
I recall hearing similar sentiments from linux sysadmins regarding cloud infrastructure. In many respects they were and continue to be correct. In other respects, the world doesn’t care about the loss in understanding as long as things work “well enough” for the cogs of society to keep turning.
For those who do care (and have the aptitude) to understand things deeper there is always work to be had when “well enough” stops being good enough and someone has to unravel the “RDS queries are taking too long” problems that crop up as a result.
These progressively higher levels of abstraction are how everything has moved and will move in science and human technology. There aren't enough hours in the day nor years in the human lifespan to gain a deep understanding of every single level below us. Rather we build on the abstracted API layer beneath us, and those who come after will build on top of us using a simplified abstraction to hide the tangled mess we had to make.
What an awful position to take. Tech used to be about becoming more accessible to people! Now we have a magical assistant to make computers do what you want with natural language, and your desire is gate keeping that so only programmers can use it to write software for themselves?
>so only programmers can use it to write software for themselves?
Yes?
The idea of an assistant that can use natural language is nice! But why would you MAKE software with it, it IS software, just do the thing you want to do! If you want to make an app, be prepared to jump hoops because this is no longer about YOU the user, it's about OTHER users.
The idea of making personal single user software is a fantasy, an oxymoron, you MAKE software? there's the presumption that it will be used for other people, otherwise you'd be USING software. There's a counter and you are at either one side or the other. It's the difference between making yourself a sandwich vs making a pot pie vs making chicken nuggets. One has the form factor for individual consumption and the other has the form factor for a social gathering, and the latter is an industrial form factor.
Maybe if there were a magic microwave that created random foods from thin air, people would create chicken nuggets or pot pies for themselves, but it's a vestigial maladaptation that will soon dissapear. Any reasonably designed product would try to provide different UX for industrial and individual users. The magic microwave that makes chicken nuggets better not be the same one that an actual factory is using. It's not a matter of cutting the middleman and revolutionizing wealth distribution from those fat chicken-nugget cats, it's about having two distinct products for two distinct usecases.
Tl;dr: Personal and industrial usecases are different, and if I'm in the industry, I don't want to use (the same product that end-users are using) to build products. What a clusterfuck.
> The idea of making personal single user software is a fantasy, an oxymoron, you MAKE software? there's the presumption that it will be used for other people, otherwise you'd be USING software.
It definitely isn't. I've done it (successfully) a few times.
> The idea of an assistant that can use natural language is nice! But why would you MAKE software with it, it IS software, just do the thing you want to do!
This makes no sense to me. Are you suggesting that instead of using an LLM to make, say, an ebook reader or crossword app that meets my personal needs, I should invoke an LLM every time I want to read a book or do a crossword? That feels like a strawman, but I can't work out what else you might be arguing here.
> I should invoke an LLM every time I want to read a book or do a crossword.
Well no, I'd say, use any of the existing 100 book readers to read a book, or any of 50 crossword apps.
But if you want to use an LLM to customize it the exact way you want it. Yes, use the LLM every time you want to do it. Wanting the LLM to do a previous gen app and then get out of the way sounds like asking for faster horses. Just tell the LLM you want to play a crossword game with X and Y rules, then give it a name so you can play it again in the future if you want, and if you want to try out rule Z, you do that, a la Kay's Dynabook.
That may sound weird to you, but asking an LLM to 'make software like we used to' sounds weird to me, and seeing "invoking an LLM every time I want to do X" as weird sounds like something that's true only for a very brief period of time where something is so uncommon that it's inference is expensive.
I can imagine a future where it might make sense. Right now, though, it would make for a far worse experience, and wouldn't even really be practically possible.
Both of those examples were real ones. The crossword app runs on my phone, pulls the crosswords from a specific source, and lets me access and solve them via the exact interface I prefer. The ebook app is cross platform, syncs via a remote server, has the interface I want and includes some niche features. There's no realistic way to create that on the fly every time I want to read a book on my phone, and if there were it would be extremely inefficient.
And I see literally no advantages to doing so -- even if time and tokens weren't an issue, what would I gain by recreating the apps from a prompt every time I wanted to use them, rather than deterministically running code I have already tested?
I get that you are doing that, but I think the fantasy is that this is creation of software instead of consumption. It's an issue adjacent to licence washing, where mangling some code through an inference layer is considered transformative or even unrelated and the original license doesn't apply. But in this case, what you are washing is not the license, but the valor of writing software.
Broadly speaking, your approach would be to have the LLM write application code, my approach would be for the LLM to write commands, 'apt-get install calibre', maybe if I want to add or modify a button I can ask it to hack the X interface. You go straight for the LLM generating the code. There's certainly technical differences between what we are doing, but they are very arbitrary, we are essentially doing the same thing, but what I am doing looks less impressive, and what you are doing you can sell in your CV to potential hiring managers as 'using AI to write software'. It's more about the semantics than the actual requirements.
I may be wrong though, maybe your approach is far more effective than just importing transitive dependencies, but I would think it's more about taking credit for the thing and increasing your sense of ownership and achievement. Sorry if that sounds harsh, but I just need a way to think of myself as better than others as an unemployed neverviber.
Thanks, I think I understand your point better now. In this case, though, you are wrong about both my intentions and the relative practical value of the two approaches (to me).
I'm not doing this to take creative or intellectual credit in any external way; you're right that there is some degree of increased personal satisfaction (which I don't see as a problem, as long as it doesn't crowd out more wholesome ways of 'earning' that satisfaction), but I'm not kidding myself about what I've actually done here. I also write my own code for fun/creative expression/intellectual stimulation/showing off, but that's a separate thing and there's not much crossover between the two types of project for me.
And the end products really are useful to me in a way that I couldn't replicate just by using something that already exists, and couldn't replicate nearly as easily by manually modifying open source. (I'm sure I could do it by starting with open source and using an LLM to make changes, and in other cases I have done exactly that, but at that point I don't really see the conceptual difference -- I'm still getting an LLM to write code and then repeatedly running that code. Ideally I would be giving something back by making a useful contribution to the public repo(s), but that would turn this into a completely different, more tedious and effortful thing, and I'm not sure it would be welcome anyway. So, case by case, I choose whichever approach seems likely to be more effective or efficient or less annoying, and sometimes that means getting Claude to write something 'from scratch'; other times there's an open source application I already use that just needs some tweaking, and I start with that.)
>couldn't replicate nearly as easily by manually modifying open source. (I'm sure I could do it by starting with open source and using an LLM to make changes, and in other cases I have done exactly that, but at that point I don't really see the conceptual difference -- I'm still getting an LLM to write code and then repeatedly running that code
Yes, that's what I meant, vibecoding something that uses existing software, not manually using open source stuff. It doesn't even need to fork or modify code.
>at that point I don't really see the conceptual difference -- I'm still getting an LLM to write code and then repeatedly running that code
I do agree, it's a subtle difference, about importing higher level dependencies vs building on top of low lever abstractions and writing everything else. Which is ironic/nuanced because I'm a huge proponent of aggressively not using dependencies in industrial programming, to the point where my requirements.txt/package.json is literally empty, and I use POSIX compliant sockets syscalls instead of importing packages like flask or express.
But when it comes to actually using software, whether for personal usecases, or as a sysadmin, my approach takes the opposite form, I aggressively don't write code, I still aggressively minimize dependencies, but the game is actually using the Operating System primitives to combine these dependencies, relying only on OS installers like apt/yum, maybe minimal configuration, if code is written, it's on a scripting capacity, a bash or python script, glue code you know? Sure the line can be fuzzed at some point, but it's clear to me that you can either write an application or be a poweruser of an application, and early in my career I've seen businesses go for the building software in house route for the fun factor, I don't think that was the right answer in the dot com boom, and as time went by, and the corpus of software grew, building your own became even more wrong than using existing third party products.
Now in personal computing, the fun-factor maybe is more important, but I have to judge this personal-software thing on how it will affect the actual important stuff, because that's what the stakes are, and that's how it's being sold. In the industry code agents are either used for building software, or for consuming software, and outside of personal experimentation I still hold that building your own software isn't a good idea, I don't think the advent of LLM materially changes that, the consequence of ending up with an ossified, non standard, low quality product is still there, perhaps even magnified, it's just that it's not something that you notice when you are starting a software product, it's only when your pyramid reaches a couple of hundred meters high that you realize that it can't grow into a skyscraper.
And I get that not everything needs to be a skyscraper, but it feels like one-off software is taking the form factor and tooling of long-term skyscrapers, the logical consequence is that we would end up with thousands of little skyscrapers, which is a place we can only get to by ignorance of the history and essence of skyscrapers, it's something a city child would imagine after going on a road trip once, "what if we had little skyscrapers throughout the whole country instead of very high skyscrapers in a single place?".
> Tech used to be about becoming more accessible to people
Since when? Because funny story, I only ever hear that narrative from tech people trying to put a self-serving spin on whatever egregious thing they want to impose on others. For the past few decades, the tech industry has consistently acted to wrestle control of people's own lives and place it in the hands of the few. The justification is always the same. It's about "keeping people safe" or "making tech more accessible." People are sick and tired of this, which is demonstrated by public backlash against tech.
> your desire is gate keeping that so only programmers can use it to write software for themselves
In what universe is learning "gate keeping"? A sane society doesn't criticize people for asking drivers to learn how to drive. What GP is asking for in the case of software development is much less than a driver's license, and yet you question their ethics.
If anything, you're the one trying to rob people of their opportunity to learn, which is a prerequisite to making informed decisions. You're the one advocating that we surrender control over computing to a handful of trillion dollar companies. That is an awful position to take.
Incredibly naive, AI is not magic and it won’t always do what you want or expect. Not sure how you’ve determined that I’m gate keeping when I’m simply warning that tech illiteracy can get you in trouble if you start giving mystery black boxes that sometimes call themselves “Mecha Hitler” root access on your machine.
I am using many many many things that I don't understand. Cars, public transport, etc...
I review and test the end product, not every tiny step along the way. If the LLM uses some command line tools I have never heard of to create a model I can verify, why should I learn a tool that is completely irrelevant to my core expertise?
"If the LLM uses some command line tools I have never heard of to create a model I can verify, why should I learn a tool that is completely irrelevant to my core expertise?"
Because that command might also give someone else access to your computer along the way. So your tool seems to work, but your computer is owned.
Are you jacking into your car’s OBD port and messing with the engine timing? There’s a difference between using a known tool in a controlled way and giving the tool to a hallucinatory goblin (or overconfident intern) with the directive “do it for me”. You don’t have to know everything about the tool to know what it does, or to tell if it’s doing something it shouldn’t (like malware). What happened to the old advice given to tech neophytes, “don’t run random scripts from the internet if you don’t know what they do”?
Just continuing the car example, because you actually do need to know enough about it to operate it and get your license. There’s no license, no insurance for using AI. Driving can be deadly, and when you do it on a public roadway there are certain requirements that have to be met as agreed upon by most governments. Same should go for AI. Do whatever you want with it on your machine, but when you let it out on the internet, you become liable for any damage you, and by extension your AI, may cause. If you feel uncomfortable approving its actions because you don’t understand them, you should either a) take the time to understand, then approve, or b) listen to your discomfort and don’t do the thing. Frankly I think that’s more accessible because you then see the decision points instead of leaving Oz behind the curtain. Much easier for a neophyte to learn from that instead of trying to reverse engineer a final output.
Many many people care more than the end product, for example whether a shirt is made of cotton with the forced labor, carbon emissions of public transport, etc.
In terms of engineering software, you care the cost. An intelligent agent may try to read unnecessary files and it's time to stop it to save tokens and avoid polluting the context.
He didn't advocate for being completely blind in every way. You might care about working conditions without understanding how the textiles, dyes or cotton production works.
These non-programmers probably shouldnt use computers at all, right, since they don't understand them?
I doubt that it is more efficient for someone to routinely watch every line of output or stop to review terminal commands, rather than waiting for the turn to complete.
It is a broader debate about agentic AI, and whether one should relinquish control to the tool rather than aim for full understanding of every action taken.
The people arguing for a hands-on, fully in control approach are losing ground by the week, in my opinion.
It’s definitely on the high friction side of the usability/security tradeoff, I just think auto modes like this are as liability-inducing as handing a script kiddie intern full admin on your production environment
The real answer is somewhere in the middle and is probably a mix of traditional AV/EDR and AI QA judges that mitigate risk of running more or less random arbitrary code and auto approve based on configured detection rules and your personal risk tolerance. Would it suck to stick an EDR sensor in every code execution environment spun up for an agent to run a python script… yes. It would also suck if you were responsible for hacking a company without knowing about it because you didn’t watch what your AI was doing
I can be against animal testing without being a chemist or having a full understanding of the experiments being made on them. Knowing it's cruelty is enough to make opposition a valid and defensible position.
I don't think the marginal new user is anxious about approving messages - I think they're quickly annoyed by permissions prompt they don't understand and quickly get in the habit of approving everything or figuring out how to set bypass permissions on
I am actually curious, how much non programmers use claude now. I know just one and she really does not know much about computers, I suppose their numbers will grow (but I doubt most get much value out of it).
I mean if you don't care code, you are essentially a product manager who gives instructions to your programmers (whether humans or intelligent agents).
Then if you use the created product, you are at best a test engineer if not just an ordinary user.
I think in the era of AI, people get tools they want in an expensive way. Rather than finding an existing tool, they ask an intelligent agent to parrot one, which guarantees no safety, security, efficiency, and accuracy. Yet, being able to use Claude makes them feel smart and productive (in parroting wheels).
At this stage with the latest models with "increased persistence" and the sheer amount of supply chain attacks, you'd be insane not running these tools in a sandbox.
Might be more secure to get a real laptop and treat Claude as any other human worker. I.e. apply all the normal sysadmin tools that manage laptops for people to manage the laptop for Claude
They're not. When things go wrong it's better to compromise someone else's VM host than your own computer. It's only a matter of time now until AI will find novel ways to break out of virtualisation.
So the concern is that the agent will discover a novel VM escape, exploit it and take control of your whole machine instead of working on its prompted task? That seems rather far fetched.
In the short term wouldnt a “dont escape” prompt prevent this? Also if it started being widespread wouldnt Anthropic specifically train new models against doing it?
Having used a bit of a mix of Claude Desktop and Claude Code lately, I discovered that Claude Desktop just times out when it asks you for a decision, and just goes ahead with the action anyway. There is no way to turn this off that I've found, unlike Claude Code where you can ask it to never time out.
Note: this is for Cowork, not sure if that makes a difference, I don't use the desktop app for coding.
Claude asked me a question at 12:05pm my time, came back at 12:40 and the question prompt disappeared and it spat back a message. Previously when I was writing a Google Doc, it would prompt me for questions, I'd come back and it would have timed out like this and just made a new version of the doc.
> I also do click through at times without reading everything, but I like to stay in control
Respectfully I'm not sure you totally know what you want. If you're already clicking through without actually reading, it's not really better than reading the specific things Claude Code thinks are important for you to weigh in on.
"without actually reading" and "without reading everything" is big difference.
I skim what claude wants to do. What file it wants to modify. What are the new function names. Skimming is enough for me to know whether it is still on the right path. If you like to work different, go ahead.
I have to, at times keep reminding it if it did code review before committing, which it fails to do at times. Also, even a glance at times has revealed issues that I’ve had to discuss and undo etc.
But yes, it gets very tiring just having to click yes yes yes..
Yeah, might make sense from their perspective, but no thank you. I also do click through at times without reading everything, but I like to stay in control, learn about the new code and change direction if it goes off track. This would just burn more tokens because I have to throw away much and I hope my manual approval settings will be respected also with future updates (or I jump ship).