Part 1: The 24GB Wall — What Actually Broke
Part 2: The Hardware Shopping List — Where Do You Even Buy That Memory
Part 3: The API Break-Even Point — Was Buying Ever Worth It (this post)
Recap: A $2,200 Quote in Hand
Part 1 diagnosed the limits of a MacBook Air M5 24GB and settled on one number: a 46B-class model needs 26–28GB of usable memory. Part 2 priced out five different ways to buy that memory, and landed on a Strix Halo 128GB mini PC at roughly $2,200 — the best available option in a year when every candidate had gotten 30–114% more expensive due to the 2026 memory shortage.
Quote in hand, a question I'd never actually asked came into focus. Part 1 asked "how much do I need," Part 2 asked "how much does it cost." "Is buying even worth it" was still open. How many months of API access does $2,200 actually buy? I pulled together August 2026 pricing to find out — and the answer favored API far more than I expected.
Step 1: Fixing the Decision Criteria First
Before picking a model, I locked in three conditions, in order.
First, preference for U.S.-origin providers. Once you're piping data through a personal automation pipeline, jurisdiction and data-handling policy matter. Slightly better performance elsewhere didn't override this.
Second, agentic and tool-calling compatibility. Part 1 established that for an agent, "calls functions correctly" beats "writes elegant prose" — that same ranking applied here. Hermes' core function is MCP-based tool calling; if function calls are inaccurate, nothing else matters.
Third, cost-to-performance ratio. This was the last filter, applied only after the first two had already narrowed the field.
Step 2: August 2026 Token Pricing
Rates per million tokens, input/output. Gemini 3.1 Pro runs $2/$12, Gemini 3.6 Flash $1.50/$7.50, Gemini 3.5 Flash-Lite $0.30/$2.50. At the flagship tier, Gemini 3.1 Pro sits at less than half of GPT-5.6 Sol ($5/$30) and Claude Opus 4.8 ($5/$25).
| Model | Input (per 1M) | Output (per 1M) | Note |
|---|---|---|---|
| GPT-5.6 Sol | $5.00 | $30.00 | Top tier |
| GPT-5.6 Terra | $2.00 | $12.00 | Mid tier |
| GPT-5.6 Luna | $0.20 | $1.20 | Lightweight |
| Gemini 3.1 Pro | $2.00 | $12.00 | $4/$18 above 200K |
| Gemini 3.6 Flash | $1.50 | $7.50 | Primary candidate |
| Gemini 3.5 Flash-Lite | $0.30 | $2.50 | High-volume work |
| Claude Opus 4.8 | $5.00 | $25.00 | Reference |
| Claude Sonnet 5 | $3.00 | $15.00 | Reference |
Two traps surfaced here. One: Pro-tier long-context surcharges — Gemini 3.1 Pro jumps to $4/$18 past 200K tokens. Two: reasoning ("thinking") tokens get counted and billed at output rates.
So the real bill always comes in above the rate card. This is the third time I've hit the same kind of trap in this series: Part 1's listed 24GB was really 18GB; Part 2's listed $1,499 was really a 64GB configuration. If there's a thread running through all three posts, it's that the number on the label and the number that actually lands are never quite the same.
Step 3: Finding the Discounts First
The rate card alone isn't the whole picture — you need to know what can be discounted before the math means anything. Three levers mattered.
Batch API is roughly half price. Gemini 3.1 Pro drops to $1.00/$6.00 with async batch processing (24-hour turnaround). Anything without a real-time requirement — automated blog posting, for instance — belongs here, cutting cost in half.
Prompt caching matters too. Cached input reads bill at just 10% of the base input price. This is where Part 1's architecture comes back around. Hermes' four-layer memory sends the same system prompt and tool definitions on every single turn — a burden on local memory, but on the API side, that repetition is exactly what caching is built to exploit. The same structure is a liability in one environment and an advantage in another. That symmetry was genuinely interesting to notice.
Free tiers help as well. The Gemini 3.x family offers up to 5,000 free Google Search grounding calls per month — plenty for a personal news-gathering pipeline.
One caution: the Gemini 2.0 line was retired June 1, 2026, and 2.5 is scheduled to retire October 16, 2026. Hardcoding a cheap, older model into a pipeline just sets up a migration bill a few months out.
Step 4: The Hidden Variable — Korean Token Efficiency
This one's specific to Korean-language users. Based on prior work, Korean consumes roughly 1.5 to 2.5 tokens per character. English runs closer to 0.75 tokens per word, so the gap is significant.
Concretely: a 2,000-character Korean document runs roughly 3,000–5,000 tokens. An English document carrying equivalent information typically finishes around 1,500–2,000 tokens. A Korean pipeline burns roughly double the tokens for the same task — which means cost estimates borrowed from English-language blog posts aren't reliable.
This changes batch-size design too. When packing documents against a token limit, Korean needs a wider safety margin — in practice, I capped batches at 70–80% of the token limit. Skip that margin and a batch gets truncated mid-run, forcing a full re-run.
Step 5: The Break-Even Calculation
Time to open Part 2's quote. Baseline hardware: a Strix Halo 128GB mini PC, roughly $2,200, or about ₩3.12 million at an exchange rate near 1,417 KRW/USD as of August 2026. Electricity is negligible — at a 55W TDP class and six hours a day, it adds roughly ₩2,000/month, not enough to move the calculation.
| Monthly API spend | KRW equivalent | Months to recoup ₩3.12M | Verdict |
|---|---|---|---|
| Light use ($10) | ~₩14,000 | ~223 months (18.6 yrs) | Buying doesn't make sense |
| Everyday automation ($50) | ~₩70,000 | ~45 months (3.7 yrs) | Borderline |
| Heavy agent use ($200) | ~₩280,000 | ~11 months | Buying makes sense |
| High-volume processing ($500) | ~₩710,000 | ~4.4 months | Buy immediately |
Looking at these numbers was a bit deflating. My actual usage sits closest to the second row — nearly four years to break even, and models and hardware both cycle through two or three generations in that span. Part 2 already noted the Mac Studio was nearing a generational refresh; factor in the mini PC's resale value four years out, and the real payback period stretches longer still.
Apply Step 3's discounts and API looks even better. Batch API cuts cost roughly in half, and prompt caching trims input costs further — a $50/month scenario becomes realistically $25–30, pushing the break-even point out to 6–7 years. On pure economics, buying local hardware wasn't the rational call.
Step 6: Why I Kept Local Anyway
Even with the math favoring API, four reasons kept local infrastructure in the picture.
First, data never leaves the building. Pipelines handling company documents were never API candidates to begin with — closed-network work makes the cost comparison moot from the start, and it's a condition that doesn't show up as a line in the table above.
Second, no metered anxiety. Part 1 involved running the same request hundreds of times while tuning prompts and configs, and not worrying about a running bill mattered more than it sounds. During experimentation, that psychological cost sometimes outweighs the real one.
Third, predictable latency. API response time varies with provider load and occasionally goes down entirely; local is slower but consistent.
Fourth, it works offline. Not a common need, but a nice one to have.
Step 7: Landing on a Routing Design
The conclusion wasn't binary — it became a routing design. Part 1 split roles between a main orchestrator and a Korean sub-agent; the same logic now applies between local and API.
| Task type | Routes to | Reason |
|---|---|---|
| Complex reasoning, precise tool calls | API (Gemini 3.x Pro / GPT-5.6 Terra) | Small local models aren't reliable enough |
| High-volume, simple classification/summarization | API lightweight tier + batch | Cost lands near local electricity rates |
| Company documents, closed-network work | Local (MacBook Ollama) | Cannot leave the network |
| Prompt experimentation, repeated testing | Local | No metered cost |
| Korean-specific processing | Local (EXAONE skill call) | The specialized role settled in Part 1 |
Trouble Shooting
| Symptom | Cause | Fix |
|---|---|---|
| Actual bill exceeded rate-card estimate | Reasoning tokens billed at output rates | Estimate reasoning-model costs generously, using output rates as the baseline |
| Korean-language work cost more than expected | Korean runs 1.5–2.5 tokens per character | Apply a 2x multiplier to English-derived estimates; cap batches at 70–80% of the limit |
| Batch truncated mid-run, forcing a full retry | Insufficient safety margin in token-limit sizing | Conservative batch sizing, with retry logic handled separately |
| Costs spiked on long documents | Pro-tier surcharge above 200K tokens | Chunk under 200K; combine with batch API |
| Pipeline broke on a retirement notice | Gemini 2.0 retired; 2.5 retiring October 2026 | Keep model names in config, not hardcoded, so swapping is trivial |
| Agent output contained a fabricated system error message | Prompt injection via external web content | Audit prior task logs; tighten write-action approval settings |
Part 3 Ledger: Series Conclusion
| Item | |
|---|---|
| Settled #11 | At ~$50/month usage, payback on a ₩3.12M rig is 3.7 years, or 6–7 with discounts applied |
| Settled #12 | Korean pipelines need roughly a 2x multiplier over English-derived cost estimates |
| Settled #13 | Batch (~50% off) plus caching (10% of input rate) outweighs rate-card differences between models |
| Final conclusion | Hold off on the hardware purchase; run a local/API routing split instead |
| Next task | Redesign write-action approval rules in response to the prompt injection incident |
Takeaways / Closing the Series
Across three posts, the question itself kept changing. Part 1 started with "why doesn't this work." Part 2 asked "how much does it cost." By the end, it had become "does this task actually need this much intelligence?" A surprising share of the work I'd assumed required a 46B model turned out to run fine on Flash-tier or smaller. I was the one who'd inflated the requirements.
Second lesson: the rate card is only half the cost story. Batch, caching, free tiers, and Korean token overhead all have to go into the real number. Trusting the sticker price alone would have led to a completely wrong conclusion — the exact same structural mistake as Part 1's "24GB is really 18GB" and Part 2's "$1,499 is really a 64GB box." Three posts, three versions of the same lesson, learned the hard way each time.
For now, hardware purchases are on hold. Better to wait out the memory shortage and run a hybrid API setup in the meantime — but with routing rules written into code this time, not decided case by case, since that makes cost tracking impossible.
And the genuinely urgent item is elsewhere. Rewriting write-action approval rules in response to the prompt injection incident. Handing an agent that reads external web content the ability to write files or send data turned out to mean something I hadn't fully appreciated until this happened. That comes before any more calculators.
Part 1 opened with a scene: waiting tens of seconds for a Telegram reply. That delay got fixed with three lines of environment variables; what was left was a money problem, and the answer to the money problem was "don't spend it." A slightly anticlimactic place to land after four months — but at least the reasoning behind it is written down in numbers now.
← Previous: Part 2, The Hardware Shopping List / ← Back to Part 1: The 24GB Wall
Comments
Post a Comment