Picking the economic news blog automation project back up, I finally did the one thing that had been left unverified — whether drafts were actually being saved to Blogger. Along the way I found the real reason a small local model kept ignoring an instruction, and caught two HTML-nesting bugs that had been hiding in the existing code. It all boils down to one lesson: don't explain to a small model — show it the skeleton.
Step 1: Status check and regression tests
I read through the three handoff documents and checked status. Both automated jobs were healthy, and all 54 regression tests passed. Though at the end of the tests, "Failed to start Ollama server" appeared and gave me a scare — Ollama was actually up and fine. It turned out a test had deliberately mocked the server status to False to exercise the failure path. It's a normal-but-alarming-looking output, so I noted it in the docs.
Step 2: Real verification of Blogger draft saving
This session's main task. The dedup logic had already been verified against the real API, but writing to Blogger itself was still unverified. I ran all three modes in order.
| What was checked | Result |
|---|---|
| 1st run → draft saved | 990 words, market-summary label |
| 2nd consecutive run → skipped | Finished in 1.6 seconds (no LLM or Blogger call made at all) |
| substitute mode | A concept-explainer post saved with the concept-explainer label |
| Automated trigger | Only 1 of 4 combinations skipped; first confirmed write to the Korean blog |
I also fully inspected the HTML of all six drafts. No tag nesting issues, no leftover markdown, no leaked length instructions.
Step 3: Three problems found during verification
Verification passed, but three things bothered me. First, the Korean meta description included "provides analysis to help with investment decisions" — the system prompt saying "not investment advice" was only being applied to the body's tone, not to the meta field. Since the meta description shows up in search results, this couldn't be waved off. Second, every post started straight at a subheading with no intro paragraph. Third, the original theme XML file had gone missing.
Step 4: The real reason the intro paragraph wasn't working
I added "write an intro paragraph before the first subheading" to the prompt and ran it — the model ignored it. Making the wording stronger didn't help either.
The cause was somewhere unexpected. The output-format example was written in the order subheading is h2, paragraph is p, so the model was reading it as "start with a heading." When a prose instruction conflicts with a format example, the model follows the format example. Once I deleted the explanation and put in an actual HTML skeleton instead, all 8 Korean/English test runs produced an intro paragraph.
Step 5: A hidden HTML-nesting bug
Once intro paragraphs started appearing, malformed HTML popped out. The parser only split off a heading at the very front of a block, so when an intro paragraph and a subheading arrived fused together, the paragraph tag ended up wrapping the entire heading too.
After fixing that, a different variant remained. It was an intermittent bug — about 1 in 6 runs — and running it four more times didn't reproduce it. Instead of chasing reproduction, I read the code and found a branch that let through "if it starts with a block-level tag, leave it as-is" with no actual check. I proved the flaw with a synthetic input and fixed it. Both bugs turned out to be shared bugs that also existed in the existing market-recap post path.
Step 6: Restoring the theme and correcting a mistake
I re-pulled the missing theme XML and checked it against the fix. That file is actually how I found out I'd said something wrong — I'd claimed "there's no intro paragraph, so there's nowhere for the drop cap to apply," but the actual CSS selector targets the first paragraph among its siblings, so the drop cap had actually been landing under a subheading in the middle of the post. The fix itself was correct, but my reasoning for it was wrong, so I corrected the documentation.
Trouble Shooting
| Symptom | Cause | Fix |
|---|---|---|
| "Failed to start Ollama" printed at the end of a test run | Server status was deliberately mocked to False to exercise the failure path | Normal behavior — noted explicitly in the docs |
| Model ignored the intro-paragraph instruction | The output-format example listed the heading first, so it read as "start with a heading" | Replaced the description with an HTML skeleton example → 8/8 succeeded |
| Malformed HTML where the paragraph tag wraps the whole heading | The parser only split off a heading at the very front of a block | Cut at a block-level tag mid-block and handle before/after separately |
| Even after the fix, nesting recurred 1 in 6 times, unreproducible | A "leave as-is if it starts with a block tag" branch let it through without checking | Proved it with synthetic input, then added a function to strip the outer tag |
| Meta description had a subheading and line breaks mixed in | The model dropped the meta line entirely, so it got regenerated from the body as a fallback | Strip headings and normalize whitespace during fallback generation |
| Test failed, value came out as "m" | The test helper was auto-appending the meta line itself | Changed it to call the parser directly |
| No progress visible for 12 minutes | Redirecting output to a file hit Python's output buffering | Documented the need to use the -u option |
Reflections / Next steps
The biggest takeaway: when an instruction isn't landing, don't write it more forcefully — suspect the format example instead. The actual answer was in the ordering of the output example.
I also handled the intermittent bug differently. Running something over and over hoping to reproduce a bug that shows up 1 in 6 times is a gamble. Reading the code and proving it with a synthetic input was much faster.
Regression tests grew from 54 to 65, and I moved the verification scripts into the repo so they're ready to use next time. Next session starts with cleaning up a repo that currently has just one commit.
Comments
Post a Comment