Month: July 2026

How a Model Eval Turned out Agent Contract Refinement

Small Models Are Prompt Linters: How a Model Eval Turned out as an Agent Contract Refinement Exercise

I ran a five-model comparison to find out which model could drive my planner agent of my Forge agentic pipeline. I got back a code review of my prompt.

Building an Agentic Dev Pipeline — From Ad-Hoc Prompting to a Repeatable Protocol

My /forge pipeline starts with a planner agent. It interviews me, sends researchers into the codebase, and writes a plan.md that the coder agent implements without re-exploring anything. It had run on a frontier model since I built it, and I wanted to know whether a local model could take that seat. The planner is the most expensive stage, and it runs first, so every token it burns is spent before I know the task was worth doing.

So: a one-shot simple prompt, 4 models adn the pipeline. A frontier reference (Sonnet 5) plus three open-weight models I can host (qwen3.5-122b-a10b, qwen3.6-35b-a3b, qwen3.6-27b), producing plans, scored on contract compliance, grounding, hazard coverage, executability and consistency.

I got the ranking I wanted. I also got something I hadn’t gone looking for: a defect list for my own agent definition.

Continue reading How a Model Eval Turned out Agent Contract Refinement

Make your harness Learn From Its Mistakes

Claude Code ships with a set of native building blocks for shaping how the assistant works. Instruction files (CLAUDE.md, AGENTS.md), shareable Skills, deterministic Hooks, and a self-managing Memory. Each one moves knowledge out of the chat window and into something persistent: conventions the model always honors, workflows anyone can invoke, guardrails the harness enforces, and facts that survive across sessions. The first four sections below cover those natives briefly, mostly to set the stage.

The real subject of this article is what sits at the end of that progression: Lessons: a small custom layer I built on top of Memory to turn Claude into something that actively learns from its own mistakes. Native Memory remembers what your project is; Lessons remembers what went wrong and how not to repeat it. Read the natives as context, then linger on Memory and Lessons, where the interesting part lives.

Continue reading Make your harness Learn From Its Mistakes