Our troubleshooting began with a report that the Ollama desktop app seemed frozen, eventually uncovering through process stack analysis that macOS’s file selection window was entirely obscured, holding up the entire app. Taking advantage of this insight, we outlined how to integrate vision capabilities into the Hermes agent and experimentally merged two local vision models.
Step 1: "The system has halted but won't power down.
Initially, I thought the entire Ollama system had crashed. Checking processes with ps aux revealed that ollama serve and llama-server, which were interacting with the model, were still running normally. Directly sending curl requests to the API returned a successful response within 5 seconds, indicating the backend was operational while the GUI shell had ceased functioning. Killing the GUI process with kill -9 from the terminal resolved the issue temporarily after restarting it.
Step 2: Re-upload Image from Attach Image Button
The next day, pressing the + button in the chat window for image input resulted in the same issue reappearing. This time, even the Quit option in the right-click menu was deactivated, leaving the application completely unresponsive, necessitating a forceful shutdown as the only solution. Checking the logs (~/.ollama/logs/app.log) revealed that an update had already been downloaded from version 0.32.7 to 0.32.14, prompting a full reinstallation of the application. Unfortunately, the problem persisted.
Step 3: The Cache Was Not the Culprit
Upon reviewing logs, it was confirmed that the app utilizes a combination of Go and native WKWebView instead of Electron, leading to suspicion around potential corruption in the WebKit repository located at ~/Library/WebKit/com.electron.ollama. A backup followed by a reset was performed, yet the symptoms persisted. At this point, it became evident that a superficial restart would not resolve the issue.
Step 4: Examined the process stack directly
There's a way to identify the cause without screen access permissions—by sampling the thread stack of a stopped process using sample <pid> 3 for 3 seconds. The results were clear: the main thread was stuck within -[NSSavePanel runModal]. Essentially, the app wasn't frozen; instead, a modal file selection window was displayed, awaiting events, though it was invisible on the screen. Bringing Ollama to the foreground and pressing Esc didn't resolve it, as re-sampling confirmed the process remained stuck at the same point.
Step 5: Completing the Bypass
Concluded that a compatibility bug existed between this macOS version (26.5.2, latest/beta lineage) and the native file dialog implementation of the app. Directly dragging files from Finder into the chat window resolved the issue seamlessly. This workaround bypassed the bug by circumventing the NSSavePanel path entirely.
Review of Hermes Agent Integration
Exploiting this opportunity, I investigated whether this local vision capability could be integrated into the Telegram-based Hermes agent as well. Examining Hermes' config.yaml revealed that the Telegram toolkit already had vision enabled, and the core model, qwen35-hermes, inherently possessed /api/tags-based vision capabilities. Notably, no separate model designation under auxiliary.vision was necessary; the default routing utilized the primary model directly. This suggests a high likelihood of seamless image processing via Telegram alone, confirmed at the code level without additional configuration.
Comparison of gemma4:12b vs qwen35-hermes in Real-World Conditions
I was curious whether the gemma4:12b specifically installed for vision tasks might perform better, so I ran identical prompts on both models side-by-side using the same screenshot for comparison.
| Item | gemma4:12b | qwen35-hermes |
|---|---|---|
| Required Time | 224.7 seconds | 207.4 seconds |
| Generation Token | 1435 | 787 |
| Accuracy | All screen elements accurately recognized | Capture including detailed icons as well |
The results surprisingly showed that qwen35-hermes performed faster and marginally better in detail, leading us to conclude that there was little practical benefit in separating the vision model. Consequently, gemma4:12b was removed from consideration.
Trouble Shooting
| Symptoms | Cause | Solution method |
|---|---|---|
| Ollama GUI unresponsive, right-click Quit disabled | GUI shell (similar to Electron process) itself gets stuck. | Leave backend processes (ollama serve, llama-server) unchanged, terminate only GUI processes with kill -9. |
| Even after updating to version 0.32.14 and resetting the cache, the issue persists in the + button. | The issue stemmed from a distinct underlying cause, not related to updates or cache problems. | Switch to directly inspecting thread stacks using sample <pid> 3 |
| Even bringing Ollama to the front and pressing Esc doesn't close the modal window. | Native file selection window (NSSavePanel) appears modally without rendering, blocking the key window (likely a macOS compatibility bug) | Drag files directly into the chat window from Finder instead of using buttons. |
Reflections / Next Steps
It was another day where, despite the GUI appearing merely "frozen," diving into thread stack traces using sample/spindump without screen access permissions confirmed the precise issue point once again. Attempts at superficial restarts, updates, and cache initializations merely masked the root cause, which was succinctly identified through a stack trace. This reinforced the notion that direct comparison of vision models through actual images, rather than relying solely on benchmark metrics, yields far more accurate insights. Moving forward, a real-world test will involve sending images via Hermes Telegram to evaluate if the vision tool functions seamlessly without configuration changes.
Comments
Post a Comment