Part 1: The 24GB Wall — What Actually Broke
Part 2: The Hardware Shopping List — Where Do You Even Buy That Memory (this post)
Part 3: The API Break-Even Point — Was Buying Ever Worth It
Recap: Five Numbers From Part 1
Part 1 walked through what broke when I ran Hermes Agent on a MacBook Air M5 24GB — seven walls, six solved by configuration, one that only money could solve. It closed with a settled ledger:
| Settled in Part 1 | Value |
|---|---|
| macOS usable memory | listed capacity × 0.75 |
| 46B-class model at Q4 | needs 26–28GB usable |
| Hermes main-model context requirement | 64K or more |
| 8GB dedicated VRAM, measured | fast, capped at 7B |
| This post's question | Where — and for how much — do you buy that 26–28GB? |
I took those five lines straight to the market. Short version: 2026 turned out to be the worst possible year for this math. AI data centers were absorbing the world's memory supply, and every price tag was distorted as a result. A coefficient far crueler than Part 1's 0.75 was waiting.
Step 0: Cutting the Requirements First
Before pricing anything, I trimmed the scope again. I'd initially wanted local video generation too, and the moment that entered the picture, required specs doubled. Video models have a totally different memory profile from LLMs, and the quality gap against cloud services made running them locally pointless anyway. Video went to Kling, Runway, and Pika; the scope narrowed to LLM only.
That single decision cut the candidate list in half. Three conditions remained: 46B-class at Q4, 26–28GB+ usable memory, and — ideally — plug-and-play. That last one mattered a lot, because with a day job to worry about, time spent wrestling drivers had to count as a cost too. After four months of configuration debugging in Part 1, that condition was non-negotiable.
Candidate 1: Mac mini / Mac Studio — the Price Sheet Collapsed
The obvious path, since I was already on a Mac. Apple's unified memory architecture genuinely favors local LLM work — CPU and GPU share the same pool, so a large model can be loaded whole. Applying Part 1's 0.75 coefficient, hitting 26–28GB usable meant needing 36GB or more listed capacity.
The problem was the June 25, 2026 price increase. Apple's own statement: "We have now reached a point where we need to begin raising prices... We have never seen a component price increase this much, this quickly."
The increases were substantial. The M4 Mac mini's 24GB configuration jumped 43%, the M4 Pro model rose 14.3%, and the Mac Studio saw the largest flat-dollar increase at $1,300. Mac Studio starting prices landed at $2,499 for M4 Max and $5,299 for M3 Ultra. The M4 Pro Mac mini went from $1,399 to $1,599.
What hurt more was that capacity options themselves disappeared. Apple pulled the 512GB unified memory option from the M3 Ultra Mac Studio entirely; as of May 2026 you could only choose 36GB, 64GB, or 96GB. Money wasn't even enough — the option simply wasn't for sale. The top configuration climbed to $14,299 for 96GB, working out to roughly $149 per GB, a genuinely absurd unit price.
Shipping compounded it. Delivery estimates ran 6 to 10 weeks depending on configuration. And the Mac Studio hadn't been refreshed since March 2025, sitting near the end of its product cycle, with M5 Max/M5 Ultra models possibly arriving around October 2026. There was a real risk of paying peak price now and seeing a new generation two months later.
Candidate 2: Strix Halo Mini PCs — Cheap Capacity, Slow Bandwidth
Tired of Apple's price sheet, I looked at x86. The AMD Ryzen AI Max+ 395 (codename Strix Halo) mini PC emerged as the strongest candidate — 16 Zen 5 cores, Radeon 8060S integrated graphics, and up to 128GB of unified LPDDR5X. Up to 96GB of that 128GB can be allocated as VRAM, enough to run a 70B Q4 model (roughly 42GB) at 5–10 tokens/second.
This is where Part 1's RTX 5070 experience reappears, in reverse. Where that GPU was "fast but small," this chip hits 100 t/s on a 30B model while also running 120B models that don't fit on any single consumer GPU. The catch: memory bandwidth is 256GB/s, well below the RTX 5090's 1,792GB/s, so anything under 32B actually runs faster on a discrete GPU. Compared to a used RTX 3090, it's roughly 3.6x slower.
In other words, this machine buys capacity, not speed. Misunderstand that and you'll be disappointed after purchase. But the per-GB price is compelling: around $9 per GB of GPU-accessible memory, the cheapest large unified-memory pool on the x86 market right now. Compare that to Apple's roughly $149/GB — a 16x gap.
Options were plentiful too. More than 30 third-party mini PCs use the same chip, with 128GB configurations starting around $2,399; AMD's own first-party system runs $3,999. But prices climbed here as well — one reviewer noted watching a 128GB configuration go from $2,099 to $3,299 over six months.
One more trap worth flagging: the widely quoted $1,499 price point is the 64GB configuration, and neither a 235B-class MoE model nor a properly quantized 70B dense model fits in 64GB. Memory is soldered to the board, so you can't upgrade later — if you're buying this, only the 128GB configuration counts. Part 1 taught me not to trust listed capacity at face value; this taught me not to trust listed price either.
Candidate 3: eGPU — 2026 Brought a Reversal
The third path was leaving the Mac alone and attaching a GPU externally. This wasn't even on the table originally — conventional wisdom held that Apple Silicon uses unified memory with no separate VRAM pool to expand, and that macOS, unlike Linux, doesn't expose PCIe device tunneling over Thunderbolt. Then research turned that up wrong.
On March 31, 2026, Tiny Corp's TinyGPU driver extension received official Apple approval — the first time external NVIDIA or AMD GPUs could be used on Apple Silicon Macs without disabling System Integrity Protection. Requirements: macOS 12.1 or later, a Thunderbolt or USB4 port, and an AMD RDNA3+ or NVIDIA Ampere+ GPU.
Momentarily tempting — no need to buy a new $2,300 machine at all. But real-world measurement was the problem. Plugable's testing showed both a Radeon 9070 and an RTX 5070 delivering no meaningful improvement over baseline system performance, and NVIDIA additionally required a Docker Desktop install that could introduce its own bottleneck.
"Works" and "worth using" turned out to be different claims. Add enclosure cost and power requirements and the economics stop making sense. It failed the plug-and-play condition set in Step 0. Still, if the technology matures, the calculus could change — so it stays on the list as something to watch rather than something ruled out entirely.
Candidate 4: Self-Built Desktop + Discrete GPU
The last path was the direct one. Part 1 confirmed that an RTX 5070 8GB caps out at 7B. So what if I just bought a real card? This is where 2026's real absurdity showed up.
The RTX 5090, launched at $1,999 MSRP, has climbed to $4,288 — a 114.5% increase. Korea is worse: as of August 2026, distributor pricing has hit roughly 7.3 million won (about $5,112), up roughly 1.5 million won in a single month. The RTX 50 Super refresh has been delayed indefinitely, precisely because the product designed to fix the VRAM shortage needed the exact component that's shortest in supply.
It all traces to one cause: on some high-end cards, VRAM now accounts for more than 80% of total bill-of-materials cost. You're effectively buying memory, not a graphics card.
Alternatives didn't help much either. The RTX 5080's 16GB caps AI use around the 7B–13B range; a used RTX 4090's 24GB covers 30B-class models and lightly quantized 70B. Against Part 1's 26–28GB requirement, even the 4090's 24GB fell short. Two 5090s would run around $10,000 — the math simply didn't work.
Summary: It Was Never NVIDIA vs. Radeon
Having priced all four paths, the real axis became clear. This was never a brand fight — it was a fight over memory architecture.
| Unified memory | Dedicated VRAM | |
|---|---|---|
| Reachable capacity | 96–128GB | 32GB max, consumer-grade |
| Memory bandwidth | 256GB/s | 1,700GB/s+ |
| Cost per GB | ~$9 (x86 class) | Very high |
| Ecosystem | ROCm / Metal | CUDA (the de facto standard) |
| Power draw | ~55W TDP class | 400–500W |
| Upgradability | Soldered, none | Cards can be swapped |
Put simply: a bigger model, slowly, versus a smaller model, fast. Given Part 1's 46B-class target, the answer is unified memory. Choosing Radeon here isn't about performance — it's that nowhere else offers that capacity at that price. You give up the CUDA ecosystem in exchange for capacity.
Power cost isn't trivial either. A desktop RTX 4090 draws 400–500W under load, running roughly 4–5x the electricity cost of a Strix Halo box at the same six-hours-a-day usage. If the goal is an always-on inference server, that difference compounds annually.
Trouble Shooting
| Symptom | Cause | Fix |
|---|---|---|
| Got excited about NPU TOPS numbers, then disappointed | Ollama, llama.cpp, and LM Studio route LLM inference to the GPU, not the NPU; NPUs handle fixed-function tasks like video upscaling | Drop NPU TOPS from purchase criteria; evaluate GPU and memory only |
| Nearly bought based on the $1,499 sticker | That price is the 64GB configuration, which doesn't fit large models | Only the 128GB configuration counts; re-budget accordingly |
| ROCm broken on Strix Halo | Known bug in linux-firmware version 20251125 | Avoid that version; run Ubuntu 24.04 LTS headless with the RADV Vulkan path |
| Windows configuration was unstable | Driver stack not well optimized for inference workloads | Switch to headless Linux, removing GUI overhead too |
| Nearly changed plans over the eGPU news | Officially approved, but measured performance fell short | Removed from active candidates; kept on a watch list |
| Mac Studio quote kept changing between visits | Memory shortage driving repeated option removals and price hikes | Fix comparisons to a single snapshot in time; consider waiting it out |
Part 2 Ledger: What's Settled, What's Left
| Item | |
|---|---|
| Settled #6 | For a 46B-class target, unified memory is the only realistic path |
| Settled #7 | Viable candidate: a Strix Halo 128GB mini PC, roughly $2,200 (~₩3.12M) |
| Settled #8 | OS choice: Ubuntu 24.04 LTS headless + RADV Vulkan (avoiding linux-firmware 20251125) |
| Settled #9 | eGPU is officially sanctioned but measured performance falls short — watch list only |
| Settled #10 | 2026 is the worst possible buying window (30–114% price increases across candidates) |
| Open question | How many months of API usage does that $2,200 actually buy? |
Takeaways / What's Next
The biggest lesson: look at the bottleneck, not the spec sheet. NPU TOPS doesn't apply to LLMs, unified 24GB is really 18GB, and a 128GB mini PC is slow. Part 1's lesson repeated itself, almost exactly, in the hardware market. Skip the asterisk attached to any number, and the whole estimate falls apart.
Second lesson: timing can matter more than specs. The 2026 memory shortage is a variable no individual controls, and it pushed every candidate's price up 30–114%. If the same machine cost 60% less six months ago, waiting another six months is itself a legitimate strategy.
And with a quote in hand, a more fundamental question remained. Part 1 asked "how much do I need," Part 2 asked "how much does it cost" — but "is buying even worth it" hasn't been asked yet. Does $2,200 buy months of API access, or years? Part 3 gathers August 2026 token pricing to calculate that break-even point.
← Previous: Local LLM Upgrade Log, Part 1 / Next → Part 3: The API Break-Even Point
Comments
Post a Comment