Agentic Workflows: Inside My Local AI Stack
The necessity to evolve
It's not new, and I'm not the first one to perceive it. Technologies are evolving fast, probably too fast as AI Tech giants recently stated. I grow in age as well. I'm not old, but I think I start feeling what others felt before: my way of doing things may disappear sooner than I thought. But you don't have to let go so easily ! I think that staying on top of things is part of what a person has to do to keep enjoying its work time. The core idea is that if you don't evolve, you risk becoming unhappy at work, forced into repetitive tasks that stop you from learning anything.
Companies are adopting AI, with or without you
That what the last 2 to 3 years taught me. Companies are way into AI. They invested so much in it that they won't just abandon it the next day. You can speculate all you want on the "bubble" being real or not (I have my opinion...), rest assured that companies are pushing to make their employees use AI. There have been studies that show either pros or cons of using AI every day at work but either way, better have my own idea on these tools, on how they could make me work better or faster.
I recently started a new job, with colleagues that used AI much more than me. They also have better "faith" on the quality of the tools, on the ways they are used by developers or their managers. Starting a new job always makes you discover new ways of working, new experiences along with new people. That's generally a good thing, but it is the first time I felt threatened. Let me explain why. Given that all colleagues are using AI extensively, at the point where they are able to produce code much faster than it was possible even 2 years ago and with a quality that is improving, what will become of you ?
I am not saying that all coding jobs are threatened, merely that they can't stay as they were. The chief AI officers of most engineering and tech companies started a transformation going much faster than the profession has seen before with the emergence of new languages or frameworks.
Oh ! And I did not talk about the companies providers for AI, because they vary. Some are paying millions to cloud providers. Some are trying a more private or secure approach by having their inference on-premise, often with smaller models and the loss in quality that could be expected. My company stands more on the latter.
Staying up to date for my projects
There is also the question of my personal projects. Of course, there is nothing of real importance there, just pet projects, learning projects. But in the end, they could also be improved using AI. By improved I mean, I could maybe have better overall quality, using AI to help me learn. I could also work on more projects, using AI to speed things up this time.
This is how things stared to move. I would consider myself a skeptic regarding AI in general and for many reasons, but I'm not a fool. I still think that the tools are great, it's just the people selling them to you that are more interested in themselves than they should. And so, without more delays, here is what I changed recently, and my experience about it.
What was workflow, what is has become
From Ollama, openwebUI and continue...
I have been using LLMs for a few years now and mentioned it in a previous post. I refused for a long time to use closed source models from cloud providers. I have no intention to be a part of this. However, I have nothing against the tools and I started to self-host my models in the early days of Ollama. It was at the time, the most popular and practical way of running small models on your own hardware. Of course, alternatives existed already, but they were not as spread out in the community as they are now.
After that, chatting in the terminal is fun but come on ! I first discovered Page Assist and OpenWebUI as chart interfaces. They have been there for year as well now, they have seen many developments from the community but in the case of Page Assist, it seems to slow down. OpenWebUI on the other hand is still in active development and is even in use at my work. My opinion on this tool is that it is struggling right now to keep up with the rise of CLI coding tools. The traditional chat interface is losing a part of its user base to newer tools that better answer their use case.
And this use case is writing and reviewing code. It's also mine in general. My use of AI as always been code centric.
The newly released AI summary or AI overview features of major search engines are not my thing. I vastly prefer searching myself,
visiting website in order to evaluate the quality and trust level of a source. It's exactly what these AI summaries are eluding.
For coding, I mainly used Continue as an IDE integration. It was perfectly compatible with my local Ollama models. I had a
specific setup with a relatively small model, a qwen-coder 9b that was producing code completion, and a larger gpt-oss 20b used in preparation
of larger code chunks and features. I was happy with this setup honestly. The code completion model was fast and a small
dedicated model was enough for the completion task. I selected the larger model to fit barely into my GPU. Of course, the inference
was slower but when you are using AI to answer more complex questions, fixing a bug, or trying to plan coding tasks, I was glad to wait
a bit to get better outputs.
But the reason I changed the setup is not that it was bad. It's just that new option rose and promised better results, even on consumer hardware.
... to llama.cpp and Pi
The coding cli tool or coding harness era started with Claude Code. Combined with new dedicated models, Anthropic created the first developer dedicated tool that proved to be a game changer for people. The adoption in companies was almost instantaneous. Some companies do supply their teams with Claude subscriptions but not all of them. It's not the case with mine. But it does not stop people that are using personal accounts at work, with all the potential privacy and security issues that it creates. If companies can't pay for subscriptions, and tools are that good that people prefer leaking data than work the good old way, what can they do ? My company is pushing to use on-premise models inside Claude Code, with the downside of using smaller models.
Open source alternative to clode code rapidly emerged. The first one I tested is OpenCode. It is an equivalent to the Claude Code cli and allows the user to connect cloud models or local models. It has a complete set of features. The only downside I found with it was the requirement for a context window larger than 64k token. In my local setup, where a balance has to be found on context size, model size and output tokens per second, this requirement was causing trouble.
AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI
Early 2026, I was looking for another Claude Code alternative, better suited for local hardware. That's when Pi came out. It's a lightwait coding harness with only basic features baked in and customization options. Of course, being that small meant I had to search for extensions and skills in order to adapt it to my needs. And it's where it shines ! You can have the coding harness you want. You don't have all the features of the Claude Code I am used to at work but most of them I never used. On my local setup, I am able to chose what I need and want in order to satisfy the volume and power constraints. I found it promising and after a few months of testing, I still think it is the best tool for me.
With Pi, you probably want to have a look at extensions. For example, the first extension I installed was pi-permission-system to add a permission system for the tool calls. I do not want my AI agent reading, editing, writing files all over my system without me knowing. Skills are also the new standard in 2026. On paper, they're simple instructions writen in Markdown that are passed to the LLM context when it identifies it needs to use it (or you ask it to). I experimented with a lot of skills for work, even some skills repositories made by software engineers with various options. Some skills help you refine ideas and features, some are to write specifications, some create series of tasks in preparation for the coding sessions. The most popular at the moment I'm writing this is the Claude Code plugin Superpowers, but it has many alternatives as skills repos like Matt Pocock's repo. Many powerful option can be found in skills libraries like Skills.sh.
The open agent skills tool - npx skills
In my local setup, I tend to not use too much of them. My local setup is not without power, but it could not handle loading dozens
of skills in context, or call them in loops to work autonomously. My local workflow is adapted to the models power limitations. I tend
to work on small tasks, clearly specified in each session. I think creating a simple codebase summary file in each repo,
with context on the project, main design choices, features, is very useful to avoid context overloads. It could be either
in the commonly used AGENTS.md or in any Markdown file really.
LLM inference in C/C++
Working with AI on consumer hardware comes with constraints. The tools major AI companies develop are not made for this. I would argue that they do not care, and they would probably prefer if you need their big models tu use their power hungry harness. It's also the one of the reasons why I went away from Ollama. I started seeing multiple reports of it being less efficient than it base llama.cpp. Earlier, llama.cpp was kind of harder to use for me. It was not packaged on my distro, you were not able to run multiple models like some of my workflow required. In shortn, it lacked convenience. That's not the case anymore. And because the tools I use in combination have changed, using llama.cpp is actually easy enough now. Also, Ollama started to have some noisy detractors for multiple reasons that I won't detail here but that you could read from Sleeping Robots.
Llama.cpp and Pi combination is also well documented. Mainly with the support of HuggingFace via their Pi extension pi-llama. Start a llama.cpp server with one command line, start a Pi session, it's as simple as two commands in a terminal.
Where do I see it going in the future ?
I have been pretty impressed by what my local stack helped me do. Of course, it's not at the level of enormous cloud models but close enough if you pay attention to the way you interact with the models, your harness, and the skills you use. The way I use Claude Code with on-premise models at work shares the same fundamental nature. Even if the models my company hosts are bigger than the local ones I use, they are not powerful enough to drive the use of the new agentic IDEs that are being pushed lately.
The "dev manager" sensation
The main feeling I get from the new stack is the distance it puts between you, the developer, and the code. I'll explain what I mean. The way these harness work, you type instructions in plain language prompts and the agent spits out code diffs that it will introduce into your code files. Obviously, you are not stopped from coding yourself but these tools allow to avoid it. You have a position equivalent to a manager or team leader, specifying tasks to a developer, refining ideas, iterating on results. AI is producing code, not you. You can resist the urge to ask even the simplests tasks but, turns out this is the ones it does best. Tackling bigger of more complex tasks could require you to be more put to work. But the purpose of the skills repos or plugins made by software engineers that are popular right now is exactly to try helping the AI do better without you.
If I already get the sensation of becoming a dev team manager, what will I feel when the agentic IDE built like kanban boards for agents become the standard way of writing code ?
Is the code good enough ?
It is crucial to note that the landscape is rapidly shifting due to the maturation of open-weight models. These models are quickly narrowing the performance gap with large, closed-source paid subscription services. Furthermore, a significant segment of the AI community has focused specifically on model distillation, creating lighter, more efficient versions tailored precisely to the computational constraints of end-consumer hardware.
This brings up a fundamental question about utility: is the output from these agents, especially on constrained consumer hardware, truly "good enough" to replace human effort? The answer is nuanced, residing in the difference between functional and flawless.
When we look at the quality, we must shift our definition. If "good enough" means a working piece of code that meets the immediate requirements, then yes, the modern stack is rapidly achieving this. An agent can produce a boilerplate service or a unit test faster than I can type it, which is an enormous gain in velocity.
However, "good enough" should not equate to "best practice." The biggest weakness in the current paradigm is not the model itself, but the context. Local models are often forced to operate with a limited view of the entire codebase. They are excellent at localized tasks—refactoring a specific module, completing a function signature, or generating a CRUD endpoint. They struggle with systemic issues: architectural drift, long-term maintainability, or subtle design flaws that only emerge after weeks of integration.
The quality, therefore, is inversely proportional to the scope. For small, focused problems, the output is excellent. For large, complex problems, the output is a highly competent draft, requiring human expertise to elevate it to production readiness.
Fight back or let go?
The anxiety surrounding AI is not merely a fear of being replaced it is a conflict between convenience and control. The tech industry excels at making things easier, and often, easier means less private, less transparent, and less controllable.
I am not a dogmatic skeptic. I acknowledge that these tools offer phenomenal utility. They are, without question, accelerators. They allow a developer to move from concept to functional code at speeds previously unimaginable. However, the temptation to trade control for convenience, to accept a powerful, closed-source subscription model merely because it yields marginally better output, is a trade-off I am personally unwilling to make.
The ultimate value of these agents is not their raw quality, but their adaptability. If I can build a self-contained, robust workflow—even if it operates on a smaller model—that perfectly aligns with my privacy requirements and my preferred development philosophy, that is where I will continue to invest. The goal is not to chase the absolute state-of-the-art, but to build the optimal stack for my specific needs.
This means the future of development is less about being a passive recipient of AI-generated code and more about being an active architect of the system. The developer’s role evolves into that of a highly discerning editor, a meticulous validator, and a workflow optimizer. The question isn't whether we must adopt AI, but how we adopt it—and on what terms.
