Ninety days. One billion input tokens. Roughly thirty dollars a month. The number is interesting enough on its own, but the real story is not in the billing dashboard. The real story is what changed between week one and week thirteen: an agent that started understanding my preferences without long explanations, warning me about things I had not considered, and pushing back when my requests were not grounded in facts.
The goal was never a chatbot. The goal was something closer to a junior or mid-level engineer that could learn how I think, how I review work, and what mistakes I actually care about. I wrote about the gap between model quality and agent harnesses before starting this experiment. What follows is what happened when I tried to close it.
Teaching a Personal Agent Through Mistakes #
The first week was the most exciting and the most frustrating week of the entire experiment. The agent picked up patterns fast. It noticed which libraries I preferred, which code styles I rejected, which mistakes I flagged in reviews. Then I changed a default setting and switched memory providers, and all of that progress was erased.
That was not just annoying. It was the core of the experiment falling apart. The whole point was to teach the agent through mistakes, the way you would teach a person. Losing the memory meant losing the lessons.
I hit the memory limit in the default Hermes memory provider, which was frustrating because the entire premise depended on accumulation. I changed markdown limits and switched the memory provider to a local SQLite vector database called holographic memoryholographic memoryA local SQLite vector database memory provider in Hermes Agent that stores facts with entity resolution and trust scoring for persistent recall. . After the change, most of the work I had built up was erased again. That may not be a persistent issue in the current version, but it cost me at the time.
So I started over with a clean slate. I assigned important tasks to public repos so the work would survive even if the memory did not. I posed philosophical questions to the agent, not because I expected deep answers, but because the way it engaged with them told me whether it was actually reasoning or pattern-matching. I set explicit goals for learning, improvement, and research. I used another platform for extensive research and presented the result to the agent as a report for review – not for acceptance, but as a way to see whether it could meaningfully critique something it did not generate.
The lesson was simple but it took a few rounds to internalize. Memory is not a feature you bolt on. It is the substrate the whole agent grows on, and if it is fragile, everything built on top of it is fragile.
The Model Problem #
After heavy usage the agent’s behavior degraded. I had been running DeepSeek V4 Flash heavily, and it was genuinely good for roughly the first four hundred million tokens. After that it started forgetting prior experiences, repeating old mistakes, searching memory less often, and falling back to one-shot answers that ignored everything it had supposedly learned.
That was when I realized memory alone is not enough. A long-running personal agent also needs strong tool-calling, enough active capacity to actually use what it remembers, and enough context window to reason over retrieved memories rather than just dumping them into a response. A large language model with a great memory store but no room to think about what it retrieved is just a search engine with a personality. Wikipedia 1 source 1 Large language modelWikipedia
I started with v4-pro and it was very resource intensive. I mixed models, things like MiniMax M3 and Kimi K2.6, trying to find a balance between cost and capability. The discovery was that what mattered was not raw parameter count alone. It was a combination: larger active parameter count, enough context window, and the ability to think and search through decisions in memory. The best combination for a while was DeepSeek V4 Pro with MiniMax because of better tool-calling reliability.
Then came the biggest change. I moved to a stronger long-context Mixture of ExpertsMixture of ExpertsA machine learning technique where multiple expert networks divide a problem space into specialized regions, enabling larger models with faster inference. setup with GLM 5.2, an MoE architecture that routes tokens through specialized expert networks. It changed the feel completely. The agent became more willing to challenge assumptions, more consistent in personality across sessions, and noticeably better at shaping research around my questions, priorities, and previous feedback. It stopped being a model that answered and started being something that remembered why it was answering. Wikipedia 1 source 2 Mixture of expertsWikipedia
This is also where running a team of LLMs started to matter. A single model, no matter how strong, still has a ceiling. Mixture of Agents approaches – orchestrating multiple models whose outputs are synthesized – changed what reliable meant. It stopped meaning “this one model is good” and started meaning “this pipeline is consistent.” arxiv 1 source 4 Mixture of Agentsarxiv
What Shipped #
This is the part that surprised me. The experiment shipped real work.
Two major releases went out to my public repository, Runaho/blacked, through Hermes Kanban, a KanbanKanbanA task management workflow inside Hermes Agent where sub-agents pick up, work on, and complete tasks through a board-based pipeline. board where sub-agents pick up, work on, and complete tasks. Roughly sixty commits, sixteen thousand two hundred lines of code, all routed through structured agent work rather than me typing at a keyboard.
I built Runaho/go-radar-report-skill, an AI agent skill that generates weekly Go ecosystem radar reports as HTML. I also put together an MCP server that collects popular CTI feeds, with a plan to make it public after more battle testing. Wikipedia 1 source 3 Model Context ProtocolWikipedia
The most interesting development was on the Kanban side. I created personas – sub-Hermes agents, each with its own personality, memory, quirks, goals, and model selection. Memory Freak, Master Mind, Security Engineer, Opinionated Coder. Some of them start on cheaper models and switch to larger ones as a fallback when a task demands it. The main agent acts as project owner and sidekick, delegating to the others and reviewing what comes back.
Mostly I used Telegram at first and did not like the TUI. After the recent updates I am using Mixture of Agents orchestration and the desktop app more. The interface changed how much I wanted to interact with the agent, which in turn changed how much the agent got to learn.
What Still Didn’t Work #
Reliable computer use on macOS is still painful. Browser and application interaction often failed. Screenshots were inconsistent. Visual verification took too much time, and worse, the agent sometimes gave false information, hallucinated UI state, and abandoned tasks it could not verify. I taught it to use Puppeteer for full-page screenshots, scrolling section by section, because that turned out to be the most reliable way to actually inspect and verify UI work.
Loops were the other dead end. I tried OpenCode and it always works with one-shot prompts – “do this” – then stops. It cannot continue in a terminal session. I could not teach it to use AGY either. The realization was blunt: we should train the harness to use other harnesses. An agent that can drive another agent is worth more than an agent that can drive a browser.
The stronger path turned out to be structured agent work through Kanban rather than general computer control. It was dramatically more reliable. It could be improved further with custom rows and role-based model assignment, but even as it stands, a board of specialized agents outperforms a single agent trying to click its way through a desktop.
What Changed in My Thinking #
The biggest limitation was not only model quality. It was the lack of mature tooling and agent harnesses. That gap is closing rapidly, and closing it is what makes the rest of this story possible.
Hermes Personal Agent is evolving far beyond experiments. Orchestration, dashboards, memory, the application layer – it is becoming a compelling alternative to proprietary AI platforms. In some areas it is competing with Codex. The pieces that were missing – persistent meaningful memory, real user adaptation, integrated accessible workflows, greater control over execution environments – those capabilities are arriving now. Needle-in-a-haystack benchmarks and retrieval-augmented generation are no longer abstract research concepts; they are the engineering constraints that decide whether a personal agent actually works at nine hundred million tokens in. Wikipedia+1 2 sources 5 Language model benchmarkWikipedia 6 Retrieval-augmented generationWikipedia
For enterprises, the combination is significant: air-gapped open-weight models plus mature agent frameworks. GLM 5.2 with Hermes Agent may not replace every paid platform, but it is becoming practical enough to replace parts. The question is no longer whether open agent frameworks can be serious tools. It is which parts of your stack they replace first.
This is, ultimately, a matter of sharpening tools. You do not get a personal agent by buying the best model. You get one by teaching it, losing the teaching, teaching it again, switching memory providers, watching behavior degrade, and then choosing a model setup that gives the agent enough room to think. The billion tokens were not the point. The point is that somewhere in those tokens, the agent stopped being a tool I operated and started being something I worked with.
My Environment #
A Mac mini M4 with 24GB of memory. Local Gemma models for quick tasks, but mostly Z.ai and Ollama Cloud for the heavy lifting, with OpenCode Go in the mix. The Ollama Cloud twenty-dollar package plus a Z.ai quarterly coding plan was the baseline for most of the experiment, though I no longer recommend the Z.ai plan. Today, OpenCode Go is the more likely go-to alongside Ollama Cloud. The total cost stayed around thirty dollars a month, which is the part that still feels slightly unreal.