Memory used : 38GB, and I haven't even started a LLM nor podman, I always fight ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		Foobar8568 9 days ago \| parent \| context \| favorite \| on: Apple caught off guard by AI demand for Mac Mini a... Memory used : 38GB, and I haven't even started a LLM nor podman, I always fight with memory when using LLM on my mac with 48gb. And I don't remember to have been able to have pushed to 200k context Qwen 3.6. 3.8 is running on my RTX 5090.
		help

redox99 9 days ago [–]

Qwen 27B runs very comfortably on a 5090. You need to use Q4 quants and Q8 KV cache. Here's the math

https://news.ycombinator.com/item?id=49514141

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact