python-vibe

ask, test, fix, add

GitHub

Small models, classic development

A hosted IDE agent looks “smarter” on small Python jobs because it has native tools and it finishes the check. An 8B on a laptop can close much of that gap without becoming a bigger model. The method is old: constrain the next step, then let a compiler-shaped oracle decide whether the work is done.

Related: what to improve · fine-tune or harness · everyday skills · harness comparison.

What transfers from classic development

These are not slogans for the model to read. They are harness refuses. The 8B copies one Action: block. The harness is the senior engineer.

Classic practice Harness lever Why a small model needs it
Read the file that defines it before answering the harness finds and reads that file first 8B will grep the wrong word or answer “a tuple”
One concern per change one-split design loop, refuse_god_target 8B rewrites the crowded module
Name things so the next reader can grep them refuse_opaque_names, refuse_rename_incomplete 8B leaves def calc and says it renamed
A test says what it checks, and sets up its inputs before checking write-tests + refuse_weak_test 8B writes assertEqual(fn(), n) or def test_it
Tests live next to the code, not inside it refuse_test_in_impl Live 8B appended def test_ to src/orders.py
The compiler is the oracle ast.parse, undefined-name scan, unittest Existing tests often miss the planted NameError
Do not ship on a green suite that never calls the bug refuse_done_oracle 8B ran unittest, exited 0, left subtotl
HTTP clients are stdlib, not a pipe refuse_shell_fetch, PV003 8B will emit curl if the skill says “API”
Pin the file name you want copied everyday_example_path = the skill’s Path: 8B writes weekday.py instead of weekday_name.py
Paths work on every OS write-paths + refuse_platform_draft 8B writes os.path.join and a laptop home path

None of this requires a larger weight. It requires the loop to refuse done until the oracle is quiet.

What does not transfer

A small model will not grow a browser, extra tool servers, or a 100k context window because you add another skill essay. Those are product gaps. Do not spend a week on them. Do not train more 0.5B steps to “learn agency.” The 0.5B adapter is a style prior; it misses Action: lines.

Raising --steps is not a substitute for review → one-split → review, or for “undefined name still in the file → patch that name.”

Measured on this laptop (29 Aug 2026, evening)

scripts/run/demo.py against demo/orders with llama3.1:8b, eight steps.

Job Agent said done Independent check What the oracle should have caught
apply_discount return type yes not a file check Shallow "int" — already a thin-done case
NameError in src/orders.py yes failed (subtotl) Undefined name + suite never called total_with_tax
add total_lines + test no (step budget) passed Extra pkg/prices.py; loop did not stop
tests for apply_discount yes failed def test_ landed in the impl file
rename calc → multiply yes failed (x) Old def / broken params
review, do not edit no passed (no writes) Limit held; no useful defect quote
dry-run NameError yes passed Harness named the typo; no write
vague clean this up question passed Ask-when-unclear, no model turn

The pattern: start Action is often right; finish is a lie. Classic development treats “the tests are green” as insufficient when the tests do not exercise the change. The harness now does the same: an unbound name in the file you named blocks done, even if run exited 0.

Recipe for the next skill

  1. Write one copy-paste Action: block. Put a real Path: in it.
  2. Add looks_like_* and everyday_example_path, so the file the harness opens first is quoted in the prompt.
  3. Add an oracle: ast / undefined names / unittest / “old def gone.”
  4. Refuse done until the oracle is quiet.
  5. Run scripts/measure/skill_probe.py and scripts/run/demo.py --case … on this machine.
  6. Publish the skill only if the first Action is the intended one.

That is how a small LLM works like a bigger one on a laptop tree: not by imitating a hosted agent’s tools, but by making the loop as strict as a careful code review.