To be fair there's probably a considerable amount of engineering that went into evaluating those markdown files so the agent behaviour is statistically reliable. The markdown is the product, not the process
I do, I like to know how badly my agents' context is wasted and what unexpected side effects to watch for (like, "always start with ${clitool} --help" == always waste few hundred tokens when even touching the skill; or instructions asking it to do something that generalize into stupid thing in larger context).
If those skills were unreadable, however, that would imply proper engineering - like e.g. the skills themselves being an output of iterative RL over set of evals.
I don't think unreadable skills implies proper engineering at all. It's just as or more likely that they're the result of a blind iterative process with no clear improvement signal. (And whether iterative RL over a set of evals is actually proper engineering here is another question...)
> And whether iterative RL over a set of evals is actually proper engineering here is another question...
I'd put it like this: regardless of the merit of how they're applied, it would at least demonstrate possession of the advanced skills expected of experienced software engineers.
I had more success starting from scratch. Especially claude's skill-creator skill, it micromanages, which is in fact worse for 5+ models than just leaving the instructions out and crossing your fingers.
Start from scratch, do some test runs, find the bugs, add the minimal possible text to avoid the bug, iterate
You can get 95% of my impl workflow skill by just telling Claude "split the work into slices and use ephemeral subagents" and the other 5% takes like 10x as much text to achieve
Markdown can never guarantee deterministic agent operations. It is an influence on inference, not a deterministic code path. How "statistically reliable" is it?
Write a prompt, evaluate the prompt, understand that is succeeds 95% of the time.
Write a new prompt, evaluate, it now succeeds 99% of the time. Measure what changes between prompt #1 and prompt #2, understand what contributed to the performance jump.
Write a third prompt, this one succeeds 100% of the time. Increase the size of your evaluation set, find a 1/5000 error-class and a 1/10000 error-class, add some explicit code to correct for this cases.
Roll out to production, collecting usage metrics. You make some tweaks to your harness, your prompts. Eventually you have confidence that your system has fewer mistakes than 1 in 100k.
Now, multiply this iteration across all your different prompts and different ways that they might interact with one another.
There's a reason engineers are prissy about people coming along and saying "I write code, I'm an engineer" that people periodically try to sand-paper away.
Engineers don't just tie a sheet to a rock and throw it off a cliff and call themselves aerospace engineers.
They do full diligence on the theory, math, physics, material science, fluid dynamics, etc, and plan a controlled series of tests specifically designed to verify/challenge/disprove their concept and the theories behind it.
Sure, there's a team member ultimately responsible throwing half a dozen rocks off a cliff in the first test.
A technician.
The guy who throws the rock off the cliff is a technician.
The other glossed over part is that the above sounds like science.
Engineering often continues until the concepts and theories are developed into safe, practical methods. "If you stay within these parameters, you can confidently expect these results." The reliability can be codified and reproduced without going from first principles on every application of it.
It's not clear to me that the current AI fad is really developing such reproducible, safe methods. "If you stay within these parameters, you might get these results. Or a teapot. Or some subtly misleading fabrication."
You have to do full due diligence to validate every result. There is safe usage where the hard work was done up front so that day to day practice can skip to boring and reliable application.
To a software engineer, a (current) LLM is a stateless algorithm that performs an idempotent transformation on a large numeric input.
People who think it's a system that thinks and reasons have confused the agentic harness, perhaps forgotten(?) layer0[0] is a seed, the inference engine sets to a concrete value when the caller leaves it as 0.
They probably work on (current) AI software by repeatedly writing prompts like "DON'T READ THE FILES IN /tmp. SOME OF THE FILES IN /tmp ARE VERY LARGE. DUE TO THEIR SIZE, YOU ARE NOT TO READ THE FILES IN /tmp." and wondering why the model becomes obsessed with files in /tmp 100k tokens into every conversation.
"Most classical engineering fields deal with probabilistic system components all of the time. In fact I'd go as far as to say that inability to deal with probabilistic components is disqualifying from many engineering endeavors."
This is cope and fundamentally misrepresents engineering. Engineers deal with a problem space that is probabilistic (although they try to model it as best as they can), but design solutions in a deterministic space. With LLMs the solution space itself is non-deterministic.
> It's not engineering if you're just guessing as to what is degrading the performance and what might improve it.
Engineering is literally the art of making educated guesses and then testing/proving/disproving/improving upon them. Nothing is exact. Everything is approximate. Iterate until the result is good enough.
This is false. A bridge is not built with approximations, it is built with a deep understanding of structural physics. Yes there are some unknowns, no it's not "educated guesses".
If you can identify gradient (what direction your change will impact the ultimate goal), then just repeating the process (or reverse-process) can find local maximum.
Still it can be a software engineering if the gradient candidate / measuring gradient / repeat process can be done at scale.
Things like Voice to Text and biometric unlocks (fingerprint scanners, face ID) have worse success rates and they're used every day by billions of people.
The fundamental issue is, that "we" somehow decided it would be a good idea to throw all the fundamental ideas of computing (determinism, context, separation between data and execution,...) away and try to solve the issues by running a probabilistic/stochastic word generator on top of deterministic circuits instead at 10 magnitude worse efficiency.
It's not a fundamental issue. Determinism and "separation between data and execution" are artificial constructs, make-believe universe in which we design classical code, and a whole lot of hardware engineering goes into allowing us to briefly forget it's all fake.
Real world is probabilistic in practical / metrological, if not fundamental sense, and separation between data and execution does not exist. Our reality does not support such separation.
> a probabilistic/stochastic word generator on top of deterministic circuits instead at 10 magnitude worse efficiency
It's 10 magnitude better efficiency end-to-end, if you factor in design time you'd have to spend to get your "deterministic circuits" (which really aren't, we just paper over it) into shape so they deterministically solve a specific problem, for each problem you want to solve - where with the "stochastic word generator", you just need to change the text prompt.
It might not be apparent from the start what are the best demands to put inside a skill, you can only know by evals. There are whole papers dedicated to changing a few details in a coding harness. https://arxiv.org/abs/2609.20519
Engineering is the use of mathematics to turn science into technology. Statistics is mathematics, comp sci is science, and technology is the end product.
Citation needed. Have you read some of the skills slop Anthropic were pushing at some point? Here is "frontend design":
> Consider Chanel's advice: before leaving the house, take a look in the mirror and remove one accessory. Human creatives have memory and always try to do something new, so if you have a space to quickly jot down notes about what you've tried, it can help you in future passes.
How about "canvas design"?
> THE ESSENTIAL PRINCIPLE: The topic is a subtle, niche reference embedded within the art itself - not always literal, always sophisticated. Someone familiar with the subject should feel it intuitively, while others simply experience a masterful abstract composition. The design philosophy provides the aesthetic language. The deduced topic provides the soul - the quiet conceptual DNA woven invisibly into form, color, and composition.
This author seems a tad obsessed. 15 times they threw the word fascist around. DHH certainly has some controversial views that many (including myself) find distasteful, but the idea that he will use his funding to suppress and take funding from politically opposed founders and push fascism is nothing short of conspiracy theory.
Use Omarchy if you like the OS. Don't use it if you don't like it. A political analysis of the founders and leadership behind every product you use must be an exhausting way to live your life
He's on the record supporting Tommy Robinson, who is one of the most prominent fascists in England. He's on the record calling for "remigration" (which is a specific word with specific meaning and context, it's not just synonym for deportation, it's a fascist dog whistle).
I think that's plenty evidence, I'm not waiting for him to tatoo "I am a fascist" on his forehead.
Also, a minimum awareness of the politics of the people producing the stuff you use is actually pretty necessary when fascism is on the rise all around the world. Might be exhausting, sure, but it's better than the alternative
Robinson being fascist is dubious. If you look at the wikipedia on him it doesn't say he is https://en.wikipedia.org/wiki/Tommy_Robinson It has "Tommy Robinson, is a British far-right, anti-Islam activist", he was a member of the BNP briefly and called Islam fascist.
A bit disingenuous to say "he was a member of the BNP briefly" - that almost makes it sound like he left the BNP by choice!
In reality, he was kicked out because he insisted on prioritising violence over politics (contradicting the BNP's "political soldier" leadership, who said that both should have equal weight). It was this disagreement that led to him coming to prominence in the first place, and he's traded on his "too violent for the mainstream fascists" image ever since.
Accusations of fascism are rarely made in good faith either. As I pointed out in response to @undeveloped, Merrian-Webster's definition doesn't seem applicable to DHH.
or Fascist, plural Fascists : a person who advocates or works to further the aims of fascism : an adherent of or participant in a populist political philosophy, movement, or regime that exalts nation and often race above the individual, that is associated with a centralized autocratic government headed by a dictatorial leader, and that is characterized by severe economic and social regimentation and forcible suppression of opposition
As far as I can tell, the only thing DHH has that is consistent with this definition is racism/nationalism, but:
1. He hasn't spoused actual collectivist beliefs, like "nothing out of the state, everything within the state" kind of BS (YET), just intra-race and national solidarity, so "exalts nation and often race above the individual" is argueable;
2. I don't see him promoting a supreme leader style of leadership, so "that is associated with a centralized autocratic government headed by a dictatorial leader" is not applicable;
3. I also don't see him pushing for stronger state control over the economy. If anything, he probably wants a more developmentists/startup culture economy given point 2 and 3 in (https://world.hey.com/dhh/three-sacred-cows-that-must-die-so...), so "severe economic and social regimentation and forcible suppression of opposition" is out.
Overall, he just seems like a white supremacist atm. Not every white supremacist is fascist.
Central autocratic government, run by a cult of personality leader, seeking loyal disciples willing to subsume themselves into the project. Proud claims to heritidge. Obsession with a veneer of beauty...
I don't know why people are finding this so hard to pin down. Maybe not nationalistic facsism but it sure dresses the same, walks the same, barks the same.
>Europe’s first fascist leader, Benito Mussolini, took the name of his party from the Latin word fasces, which referred to a bundle of elm or birch rods (usually containing an ax) used as a symbol of penal authority in ancient Rome. Although fascist parties and movements differed significantly from one another, they had many characteristics in common, including extreme militaristic nationalism, contempt for electoral democracy and political and cultural liberalism, a belief in natural social hierarchy and the rule of elites, and the desire to create a Volksgemeinschaft (German: “people’s community”), in which individual interests would be subordinated to the good of the nation.
If you think him getting money from Jack Dorsey and Michael Dell has no political implications, I have a bridge in Brooklyn and a tower in Paris to sell you. This isn't conspiracy theory, this is following incentives.
We should be aware of the motivations of the software we use. This is why we've had licensing fights for decades.
>If you think him getting money from Jack Dorsey and Michael Dell has no political implications,
And the device you type this on is made by which company? Apple? Google? Microsoft? Lenovo? They all involved in politics and lobbying. And it's powered by which ASICS? Apple? Intel? AMD? Nvidia? All of them are involved in lobbying and politics you might disagree with. And yet you still use them and nobody is giving you shit about it.
You're a pot calling the kettle black here. If you wanna get online and do work, unless you're fabbing chips and laptop boards in your garage, you're gonna be directly or indirectly funding some tech business involved in politics you might not like.
So it's pretty hypocritical and somewhat mental of you to spend so many comments here trying to dunk on people willing to use Omarchy just because the owner has politics you disagree. Why can't you live your life and let others live their own in piece without turning discussions into your own political war (which BTW is also against HN rules)?
I am much more likely to get someone to stop using omarchy and actually learn something about computers doing this than I am to convince someone to stop using a Google product. If I be can active when something like this is fresh, I will. And if you think this is just about DHHs politics, think again, its also about having that maniac in any kind of position of power anywhere. I'd say the same if he were an MLM or Kim family fanatic, but those people dont really exist, where weirdo racists do, and in large numbers.
I really dont care about the guidelines on politics here. They seem to have been made by children who are afraid to have their monied friends political passion projects criticized by smart people.
Without knowing what makes consciousness possible, the paper cannot justify biology as necessary - it mistakes a lack of evidence for conscious AI for proof that conscious AI is impossible.
I do not believe current AI or LLMs are conscious, but there is no proof one way or another that they can or cannot be. The paper authors are making up their own definitions and building an argument from them
Your argument requires that there is some objective truth for what consciousness is. It will always hinge on what definition one accepts.
I, and apparently many others, don’t think it would be any useful to describe the mathematical properties of an AI as consciousness. To me it is inherently a way to describe the “experience” arising from physical processes in biological beings as ourselves.
That’s what the argument comes down to for me. Could an LLM “fall unconscious”?
I think you, and apparently many others, are hiding behind mathematical strictures to avoid the discussion. Your argument is actually "what requires an objective truth" Or else you could admit there's no real difference between zapping amino acids with electricity and zapping silicon with electricity.
I would make one further argument: as the top-of-the-food-chain species which originated the concept of consciousness, we get to define it however we want.
Consciousness is defined in the human context. We can just say “hey buddy that’s not biological enough to qualify.”
> the obvious astroturfing that occurs on this site (along with reddit, etc.) when it comes to Grok isn't helping. All I hear about Claude, GPT, Gemini, etc., are how terrible they are, yet any discussion of Grok seems to always revolve around sensible, but confident, assertions that it's actually a great product and every new release is the point where Grok finally catches up.
this is the exact opposite of my experiences on HN and Reddit. In my experience, Grok is typically reduced to hitlerbot and CSAM generator and rarely taken as a serious competitor. People let their hatred of Musk blind them to the tech of his companies
We have Google, Figure, Tesla, Boston Dynamics, and literally dozens of Chinese companies building humanoid robots and making huge improvements in the tech. Not there yet by any means as a general-purpose home assistant, but I don't think it will be too surprising if at least one of them succeeds
reply