Scares me. When the ruling class get robots servants they will have no need for even simple human interaction anymore and that will truly unleash them to do harm.
DeepSeek models have such good benchmark performance, amazing pricing, and the team over there seems to be widely considered impressive.
I just haven't found them to be very good? I've had a ton more success with the GLM models (since 5.2 anyway). Maybe I'm just holding it wrong, DS models seem to get stuck in loops or tell me nonsense. GLM feels like budget Claude.
My experience is complete opposite from yours. The V4 Flash was already quite good, but V4.1 is really very close to SOTA in my books. I've eval'd these models for weeks against Gemini 3.8, Kimi K3 and Opus 5, and DeepSeek absolutely wins these evals.
It's really crazy. We price per token our customers. If Kimi was about 30% of the price of Opus 5 for the same quality, DeepSeek is 1/10th of a price of Kimi K3. We've come down in price so much that I seriously cannot recommend other models before they reduce their pricing.
And what is really interesting is its programming ability. As I've said in my previous comments, I use agents a lot in my work. Since early Opus days until now I have 7-8 agents working in parallel for different tasks. Rust, design, GEPA, evals, analysis. For a long time Kimi K3 was the best model for this work, and before that GPT 5.5. But I still can't really believe how well DeepSeek works here. I really try to find faults from it, trying to see that it must be doing sloppy work and be worse than the others. But it does not. It finishes every task I give to it. And the cost per task is under a dollar, usually 15-30 cents.
In comparison the same task with Kimi would be 3-15 dollars; sometimes closing to 100. And before that with GPT 5.5 a 800 dollar task was not uncommon if I spent days evaluating models.
Now it's less than a dollar.
For me if the other providers will not drop their prices dramatically in the coming weeks I see no reason to use them. Even with a 200 dollar subscription, paying per token for DeepSeek is better value.
Unfortunately, it made a mistake and bricked it. I handed over to Astra but noticed that my watch was dead, it must have died immediately before. Too bad, unlucky me.
It’s funny, in oh-my-pi I’ve had much better luck with DS 4.1 Flash than GLM 5.3 flash. Glm seems to just constantly reason in circles before attempting a tool call
Are you using the API directly from Deepseek? Otherwise, performance will suffer greatly. If you're using OpenRouter make sure to pin Deepseek as the provider. If you're using OpenCode Go who knows who the provider is.
How are you holding it? You'll get wildly different real-world results from different inference providers, for one. For two there's the harness and what you do with it. In general you'll need to provide more direction to lightweight models, better guardrails, more planning, tighter goals.
Making Android apps was my introduction to publishing software. I remember in ~2011-2012 you could upload an APK and it would be published immediately and the Android Market had a "New" page which was just a chronological feed of recently published or updated apps. Just uploading your app update was an easy way to get ~500 free downloads which I found very exciting.
At the time I couldn't imagine paying $99 for an Apple Developer account and waiting a week to publish my app...
We're very happy with OpenRouter in production, although we use it in a pretty limited fashion. We have only ~10 providers allowlisted (based on their location and ZDR policies) and we use only a few models, often frontier models. Has been very reliable for us in production so far. It's generally been great for us to be able to have a single place to manage spend, policies, view logs, etc. Also really nice in code to have a single-line change for switching models. OpenRouter's console is so much better than what you get from the actual frontier labs, and it's nice not to have to translate numbers and concepts across providers.
Does anyone have any recommendations for something like Cursor but more open? Cursor's harness is so much faster than anything else I've used that I just can't give it up, using Claude Code feels like being on 0.25x speed. Also I do like the IDE-style UI, being able to highlight code in a file editor and then quickly add that to my chat (Cmd-L) is really nice and much more difficult in CLI based harnesses.
Right now I think GPT 5.6 Luna is by far the most cost effective model, so this change is effectively kicking me off of Cursor.
This, and the keep/undo flow, forking agent sessions (including renaming them)... The UX could be better at places but the core features of what Cursor has (for someone who came from VS Code) is an unbeatable mix. Really would like to find something else.
I did try Kiro which seems to have the same philosophy but it's just not there. Most of the features I like in Cursor are missing.
Some other commenters mentioned that VS Code + Copilot would be better these days. Dunno.
I am also wondering the same thing as OP. Somewhat tangential thought, but I recently realised that what I would really love in an IDE is a "first class" Markdown editing experience, just because most of my day-to-day is managing and writing docs rather than editing code. Cursor has Markdown rendering support of course but it seems to have two implementations, each sort of clunky in their own way. And I want something more WYSIWYG but with Latex / flowchart / etc. support built-in.
OpenKnowledge seems to maybe fill that niche somewhat (it also supports annotating Markdown with comments to feed into an agent), and has agent integration within the GUI, but one killer feature that seems to be missing is Remote SSH support (short of hosting its web server on the remote and having the IDE live in the browser).
There really has never been another product priced like AI is being priced right now. Each of these things has been done before, but all of them together is new.
1. Insanely discounted starter plans. Claude $200/mo plan is like $5k-$8k of API rate usage.
2. Very limited cost visibility, they make it hard to figure out where you spent money (unless you're on the enterprise plan which is for people with unlimited money).
3. Nobody, not even the model provider, knows what your request will cost before it returns. You're writing a blank check every time you hit enter.
4. When you run out you run out very suddenly and disruptively. It's very hard to tell a developer on the 28th of the month "sorry, code by hand until the 1st of next month" so you tend to grant exceptions.
5. The price is changing all the time. New models come in, old models come out, prices change, caching behavior changes, harnesses change, etc. The cost of doing a single task is not predictable even if the task does not change.
6. Basically no volume discounting. Anthropic offered us 2% off for committing to $1M+ per year at API rates.
I manage AI spend for my team at work and I try really hard to keep costs under control but it's absolutely herding cats. Much harder than any other spending I've ever had to manage at work.
Anthropic's run as the AI Good Guys lasted ... 6 months? Hope they had fun. Turns out there aren't any AI good guys. Every single one of these companies engages in all the same BS. At least with open weights models we (the users) get to keep something when the whole thing collapses.
This is a fight I deal with every day. We have folks in the technology group at work who use AI to write code and do so without issue. But now folks in supply chain or in the executive suite are using it to generate web pages that they want published or apps they want on the internet, and while Claude can generate an HTML file how that gets published, how authentication works, etc is just glossed completely over, and generates a ton of work for the IT team to build up around this stuff as it comes in
Apple has totally failed to deliver interesting AI experiences so far ... and I still think they're going to be the dominant provider of AI in 5 years. We're just one or two advances in chips / models / both away from being able to run very good local models for free on mid-tier Apple devices. The privacy, cost, and latency story there will be too much for OpenAI/Anthropic/Google to beat.
Just writing this down so I can be praised/mocked in 5 years.
I'm perfectly happy with Apple not becoming an "everything we do is AI-centric" business.
I'm fatigued by it all at this point. It's streamlining the interesting and fun parts out of my job (by practical necessity of use there), and if I used it half as much outside of work I'm sure it'd do the same there too.
I know public opinion polling supports that, but the parts of my social circle which are outside of tech seem to be, at worst, apathetic (and at best enthusiastic, though that's not a big fraction).
That said, I think it's a good thing that this sentiment is coming to the forefront.
I was actually surprised to hear my brother-in-law deride LLMs as being useless for areas he has expertise in when I visited for the 4th of July.
He was complaining that he would ask how to perform a certain repair on a car, and the LLMs he tried (ChatGPT & Grok) would give him a long involved process and he'd ask why not do it this simpler way and it would say, oh you're right! He just found it gave bad advice and realized (rightly) that in areas he has less expertise in he has no way to judge how good the outputs are.
This is from a guy who loves tech, historically worshipped Elon, loves his Tesla, and (rightfully again) didn't buy into SpaceX because he thought it was overvalued.
In the past when I visited for holidays he was liable to have a positive outlook on LLMs and their utility. Seems telling that he's starting to see the cracks.
This is easily the biggest problem with the current models. The models are just way too eager to please / say yes to the point that the models are happy to lie/make shit up if it means it can say yes.
Yep. In my areas of expertise I can easily catch it coming up with wrong information, bad calculations etc. So laypeople are probably being led astray quite often.
> It's streamlining the interesting and fun parts out of my job
Interesting. For me it's streamlining the tedious and attentionally taxing parts of my work tasks. I love solving problems, I don't particularly love shaving yaks.
Here’s the two main reasons why local inference won’t compete any time soon with the cloud:
1. Most useful LLM work is done in parallel. A Mac Mini can run one LLM inference thread at a time. The cloud can spool up dozens and spread that inference across efficiently batched operations over a fleet of hardware.
2. Faster inference hardware such as the chips from Cerebras and Groq cannot be run locally. But the advantages of running >5x the token throughput per thread can’t be overstated. Add in the multi-threading advantage and it’s a knock-out punch for local LLMs.
Local inference has a role: if you’re working with extremely private matters or you want an uncapped model that will talk dirty or generate NSFW photos, local is the only option. I think Apple and others will continue to also run a lot of useful workloads locally such as text editing suggestions, speech to text, text to speech, and image manipulation. As local hardware improves, these capabilities will get better too.
But, for most LLM work, the cloud will continue to dominate for a long time to come, if not forever.
There is so much happening in that scene, where tokens/sec double or 10x
So I could see the same hardware doing 20 tokens/sec on a large model suddenly doing 200 tokens/sec in the future, a better device in the future doing 500 tokens/sec, while having vision models baked in, audio models etc
Users wont consciously switch to local, they will just have it and use it
and a toyota camry can’t compete with a lamborghini but the camry vastly outsells them.
you’re far far far more likely to see a camry or equivalent in an americans driveway than you are a super car.
you’re also more likely to see an enthusiast with a corvette or equivalent muscle car spending way more than it’s worth to tinker on that car in their garage than you are a super car.
> A Mac Mini can run one LLM inference thread at a time.
That’s not accurate. With MLX, at least, parallel inference is both possible and useful. Model serving tools like LM Studio and oMLX support parallel generation with continuous batching, and the total throughput increases with it.
You are not wrong, but the practical reality of local hardware is to be batch-constrained in comparison with a multi-user inference cloud. You will never be able to compete cost effectively in your home lab with a cloud that has >100M end users streaming millions of inference requests per second across a gigantic fleet of machines.
Can I run a few inferences in parallel on my Mac Mini? Yes. But put 1,000 Mac Minis in a datacenter serving 1,000 copies of myself? That's going to be more efficient.
Oh certainly, a local machine can't parallelize like the cloud can. Does anyone think that, though? What it can do is provide good-enough inference to support common business needs, without creating unnecessary data risks.
Why on earth would you need to do that for most things? Not everyone wants to be the next Coreweave. It sounds like you're killing a fly with a cannon to me.
I guess what I'm doing is not considered that useful then? I usually only have zero, one, or occasionally two things actively doing inference at a time, be it claude code sessions or one of the chatgpt/claude web interfaces, and i bet that's true for like 95% of people using llms. And anyway i bet even the hardcore people using a bunch of parallel agents would appreciate having access to local, private inference for some things.
You're obviously right though that cloud inference isn't going away anytime soon
What about when claude code starts spawning sub-agents? it's pretty good about doing that now even if a user doesn't request it. Does this could as parallel inference?
I haven't seen it do it by itself that much so far. Even then, a lot of the time is spent waiting for some shell command to complete, which isn't inference. I guess being able to do parallel inference is like having multiple cores in a cpu, you can get by without it by just sharing that single core, and having it just gives you a performance boost under certain circumstances
Interesting. For me it seems to be much more common that even basic requests start spinning up sub-agents. But I also routinely ask it to do this so maybe it's stored this as a preference somewhere deep in its bowels.
And yes, I agree. I find the experience better less for speed and more for context management. But it's far from necessary.
100%. Orchestrating multiple sub agents is a key requirement for many workflows. If only because it minimizes context sprawl (amongst multiple other reasons).
There is probably a middle ground though. Something like a main thread agent (say a local model for Hermes) orchestrating cloud based sub agents.
You can always buy multiple Macs. I think Apple's great differentiator could be making frontier class models local. I you want more threads, buy more Macs.
I don't want to run any workflows on someone else's computers.
As far as I know, there isn’t an interface like nvlink that allows these macs to work in tandem; they would just send their data over ethernet, maybe thunderbolt/usb-c?
I don't mind the subtle ML integrations that they have put in the photos app: plant ID, recognizing faces, removing background, OCR text search (even for handwriting!), etc.
You’re listing every feature I’d like to remove; Meanwhile Android users have great holiday pictures at the bottom of the Pyramids because they can remove the people from their pictures.
I suspect we'll see a hybrid before an all or nothing. Local models for computer control or delegating, online models for things that need strong reasoning, planning, and knowledge access. Again, I'd be more than happy to be wrong. I just see models growing faster than the hardware can.
I'm also of this opinion, but also that it doesn't have to be Apple (but they are well positioned). What I've seen with running local models on my 48GB M4 MBP is really impressive - it's not the same level as hosted stuff, but it's better than what I was using a year or two ago.
You should not buy a fully loaded Mac Studio for AI unless you absolutely NEED macOS. You will be wasting so much electricity idling on prefill while your GPU pulls 150-250w from the wall.
Buy an Nvidia Spark, then whatever cheap Mac you want to use as a thin client. There's no reason to force Apple Silicon's round peg into a square hole like AI inference.
Outperforms doing what? Inference is not a homogeneous workload, memory bandwidth correlates to decode speed and layer swapping but not necessarily inference speed overall.
The other half of that equation is latency, predicated on prefill performance which needs a powerful GPU and ideally ALU-level optimization to build larger KV caches quickly. Even the M5 gets smoked in this department, the M5 Max has a 50% longer TTFT on Qwen's 27b dense model at only 16k of context, which is a pretty typical starting context to use for agentic editing in normal apps like OpenCode/Claude Code: https://raw.githubusercontent.com/Osmantic/MMBT-Messy-Model-...
For agentic, 50-256k token on-device coding sessions, the Spark will be faster and consume less power running larger models. Without an external GPU (which Apple doesn't support), Apple Silicon will always be bottlenecked during prefill. Apple's failure to address this with their GPU architecture is a big reason why Apple Silicon viewed as a waste of time and money for professional datacenter deployment.
For agentic work, you just cache the prefill kv cache of the relevant system prompts. TTFT is a little slow (10s of seconds, oh no!) the first time you boot up a new harness.
I keep hearing people make this claim that TTFT is a problem, and… it just isn’t, if you’re running oMLX.
Folks in my camp keep saying this, and folks in your camp keep beating a drum we tell you isn’t resonating. Not sure why I keep bothering to argue; you can’t buy a high RAM Mac Studio like mine anymore.
They're not usable for deployment. They're perfectly fine for "enthusiast" low-end usage with 10-30B models, but the same goes for almost every dGPU made in the last 10 years. Your Mac Studio cannot run frontier LLMs at an interactive speed, even Apple has given up on using it as an inference backbone.
Apple is doing something very different. Their AI experience for end users definitely has been a little behind.
Apple Silicon, however, has been quite unique for the last 4-6 years and it's increasing overlap with LLMS.
The model/chip optimizations are definitely improvements, the thing that is really standing out the past 2 years is how much the open source model community has been making possible, especially when you know a group of use cases.
There is common wisdom: during gold rush, sell shovels.
Apple's shovel (ahem, Mac mini) is the highest quality.with Companies burning money left, right and center, Apple can dispense with advertising altogether
Apple has failed to live up to the Steve Jobs era and initial iPhone hype. It's not just "AI experiences", it's computing in general. Maybe the consumer sector is just dead/dying. Maybe consumers are just running out of cash because filthy VCs are destroying communities and forcing the 99% into poverty.
The one thing that is marginally exciting: the Apple SoC or M series chips.
It's unfortunate they are locked behind crappy macOS and other proprietary apple crap.
Unsurprising. Apple seriously thought the iPad would replace computers and usher in a "post-PC" word during their "What is a computer?" ad campaign era. Now they are sticking phone chips in laptop chassis.
For the general consumer, they were basically right though. Most people don't use laptops except for work. The primary computing device is the phone, and phones have basically become become mini-ipads in form factor since that ad aired.
> Apple seriously thought the iPad would replace computers
...for some users. See their "Mac is a truck" analogy.
And it has. My parents haven't owned a Windows or Mac machine in six years, since they got rid of the one I gifted to them a decade ago. Its all iPad and iPhone.
reply