Hi HN. I'm an independent researcher and I've been working (completely self-funded) on my inference engine since January 2025. It's built from scratch and uses highly optimized fused kernels and I've been testing it against vLLM and SGLang. Thank you, Victor

MTurk existed since forever, and somehow flew under the radar for all these years. Very few developers I know didn't even know it existed.

Damn. I just bought a maxed out MacBook Pro M5 Max 128GB 8TB, still waiting for it to be delivered. I could get 256GB RAM M5 Ultra 1TB for roughly the same price, and it's double the memory bandwidth. Which one would you recommend? I do plan to run local LLMs.

M5 Ultra if the portable form factor is not a requirement. (I have a M5 Max 128GB 2TB for context.)

That being said: you might want to consider at least 2TB storage. Having 256GB RAM to fit models is less useful when you can only store a 3 or 4 large models on your disk. The M4 -> M5 transition doubled the drive bandwidth (at least for the MacBook Pro, I'd assume the Studio is no different) making them extra nice for loading large models. Then again, you can always add an external NVMe drive over thunderbolt, so the tradeoff is less straightforward than with a portable MBP.

Contact Apple care and see what they can do. They have been known to do upgrades if you buy around the same time. But with the ram situation that courtesy might be gone now.

Ultra definitely. M5 Max is too hot for a laptop from hearsay. I have a M3 Max in 14in chassis and it's a jet engine that gets really hot

You should be able to cancel it for free.

In EU you also have 14 days to return products bought online, for any reason.

M5 Ultra and toss in a MacBook Neo or older MacBook Air to SSH in

Both can be true at the same time.

With native support for sessions in an inference engine we can make use of idle GPUs... doing work!

I bought a server with 768GB RAM a couple of years ago, for cheap. Today just the RAM itself is worth more than I paid for the entire server. DDR4... Same thing with SSD, two years ago you could buy 4TB SSDs for like 250 USD, and today its more than $700. Madness!

Fastest tool calling on the planet!

Attended Dreamhack in the late 90s. Fun times! I was always AMAZED by the demoscene.