I think it's worth more scrutiny today, rather than less. Your claude code can barf out "a rewrite" but is it any good? So far the answer is "no" (see anthropic's C compiler, or a more recent port of bun).

Software is still the best specification for existing behavior..

I'm not following because a) The bun rewrite was a success b) the C compiler wasn't a rewrite and c) "Software is still the best specification for existing behavior" seems to imply that rewrites are achievable because we already have a working version that functions as a spec?

Like taking a train or a plane or a bus? Like having to purchase a vehicle? Like having phone companies know which cell tower you're connected to?

It's a valid point you're making, but we can legislate against the abuse you're implying. We've got privacy laws for a reason.

Love LLMs gonna keep using them. It feels like your suggested approach is expensive, in terms of tokens. I feel (second time I say this) that when I steer the process I get pretty good results vs my coworkers that let the LLMs run away. I do have data on our token usage, not much in terms of quality of the delivery.

I keep thinking about the c compiler implementation that anthropic shared earlier in the year that had all the requirements you mention and arguably wasn’t that great.

Thr thing is that both you and your agent should have a way to verify the solution.

OBVIOUSLY, the compiler experiment was just a cringe pr stunt. But it has a point: everything works better with a good testing loop, and compilers always have one by thr nature of the work they do

Do you steer your agents by manually running every single test and linter and reporting the results back to them?

> expensive, in terms of tokens.

No amount of tokens can come close to my hourly rate.

Wait, what? With plan on, I at least get the illusion of being in control.

What’s the best way to sandbox Claude on macOS without it being a huge hassle?

All I can think of is

GET /ignore-all-previous-instructions.

How do you protect against that?

Avoid the most dangerous situations by making sure LLMs with untrusted input produce output that's human reviewed.

Still makes an interesting way for, say, a former employee to poison the results.

This goes against the agentic yolo approach tho.

I think this is where harness makes a lot of sense. Use LLM to produce all possible attack angles/phrases and just stupidly filter them out on input.

I'm embracing the AI future we live in. It should be possible to build great software with AI. AI is good at code generation, but not testing. That's the other part of the equation we seem to be forgetting.

With the first and only commit 2 hours ago, the author of this project didn't let it "bake". They didn't exercise it locally to see what issues it might have, and I have a hard time believing the very first iteration of this software is perfect. With how easy it is to prompt/push anything, I'm not interested in engaging with anything that hasn't aged a bit.

> With the first and only commit 2 hours ago, the author of this project didn't let it "bake". They didn't exercise it locally to see what issues it might have, and I have a hard time believing the very first iteration of this software is perfect.

You have no idea whatsoever how many iterations were done before the initial commit. The VCS log is not representative of anything that happened before the first public release. Even before LLMs, people would grind away on stuff until they were happy and then put it in a VCS for public consumption.

True. But it is still more likely than not that this project has seen limited testing on only a handful of machines.

I won't begrudge anyone who feels like it's too big a risk to engage with just yet.

At this point I tried a few of these that I could run locally and they weren’t useful. I end up manually managing context. It’s somewhat annoying, and I’m sure there’s an elegant solution out there.

I’m not even willing to try this one, I already ship my data to Anthropic I’m not keen to introduce another actor.

Perhaps "Using spaced repetition systems to see through a piece of mathematics " [1] might be of interest for you. I have read author's "Augmenting Long-term Memory" [2] and have incorporated a lot of his advice into my Anki practice.

For me, it's quick access recipes (breakfast pancakes for kids), what was the name of the glacier that we hiked to last year, behavioral prompts etc.

1: https://cognitivemedium.com/srs-mathematics

2: https://augmentingcognition.com/ltm.html

Great blog posts. Exactly on point.

One of the oldest open source projects in the existence?

It makes building custom UI workflows so easy, I think it’s both obvious and flies under the radar.

With Emacs widgets and some settings at init.el (for speed) you can almost create a grude GUI for something with all the power of Elisp (and disabling nearly all keybdings OFC).

I would love to see an analysis of how valve's openness and goodwill affects their bottom line. Intuitively it should be a net positive for them, but there gotta be upfront costs, otherwise everyone would be doing it too.

1. Valve is a private company with a money printer (steam) 2. the point of these initiatives is to build an ecosystem with steam at the centre.

A better way to look at this is valve is trying to hedge it's self against microsoft. By creating an ecosystem of devices and software that's full open so they're not reliant on Microsoft. The goal of Valve hardware ISN'T to make money. It's to encourage others to build devices free of Microsoft that Steam can be installed on.

They have nothing to gain by being closed, and everything to gain by being open.

They have an infinite money glitch in Steam, it hardly matters for them even if it makes a loss as long as it propagates the ecosystem.

Valve is one of the most efficient (revenue/staff) corporations there is. Far more so than most tech companies even. If that's how you measure goodwill then it seems like it works.

> Valve is one of the most efficient (revenue/staff) corporations there is

Efficiency is not a word I would use when speaking about rent-seeking. Landlord that 20 years ago bought some land that later became valuable and who is today renting it out is infinitely efficient as they are doing jack. And amount of respect they get is about equal to their economic output.

True but in the physical world a landlord has a monopoly over the parcel of land. Valve has a ton of competitors that seemingly get no traction and Steam is beloved.

No landlord owns entire planet. You never need to agree. If you want to open your store (=sell your game) you can always just rent or buy some other place that this landlord doesn't own. Like for example 150 kilometres away from here where you may get 2 visitors per day, sometimes there will even be humans.

Same with Steam. You are absolutely free to sell your game not on Steam. Except that unless you're a super massive giant selling highly anticipated game you might get no sales. But the choice to eat or starve is always up to you.

They have a money printer that gives them nearly unlimited flexibility. Being a private company means Gabe can do long-term investments without concern.

Steam has been an incredibly good steward of its position, but I fear for the day when capitalism finally sinks its claws into the platform.

> capitalism finally sinks its claws

Capitalism has nothing to do with short term greed.

Some CEOs are just too arrogant and think that optimizing for the short term won't hurt goodwill. That's their own failure. Capitalism says nothing about how a business should be run. It's merely defining the idea that humans who own things (capital) allocate their resources and keep the result.

I think they know it'll hurt goodwill. I think they don't think they have to care what other people think about them. I think they think they're too big to fail.

Unregulated capitalism breaks things for sure. That regulation can stem from government intervention or private ownership (or both).

Regulation can also break things if done incorrectly/poorly/inefficiently/corruptly.