AARON SPINDLER

Giving Up The Keys To An Agent

I tried handing more of my development loop to Codex: tickets, branches, implementation, tests, CI repair, draft PRs, review follow-up, and the unglamorous repo maintenance that piles up when nobody is looking.

It worked better than I expected.

Then the usage report made the real interface visible. Not the chat box. Not the editor. Not the GitHub comment. The cost. The report showed where my system was disciplined, where it was vague, and where automation was quietly expanding to fill every gap I had left open.

The line I keep coming back to is simple: the bill is the interface. Here, that means the usage cost estimate.

The table spans November 2025 through July 2026 and totals 32.4 billion recorded tokens, including 30.8 billion cache reads, with an estimated usage cost of $24,002.36 USD. Most of that token volume is cached context, not newly generated output. The estimate describes usage value; it should not be read as an invoice for that amount.

Viability Was The Easy Question

The first question was whether Codex could be useful as a worker for real engineering tasks. My answer is yes, with qualifications. It is strongest on bounded glue work, repo maintenance, small code changes, test repairs, and cleanup tasks where CI provides a repeatable first validation gate.

The product shape is already there: Codex can run tasks in the background through Codex cloud, review pull requests through the GitHub integration, and be wired into repository automation through the Codex GitHub Action. That is real leverage. It also means the surrounding system matters more, not less.

My experiment stopped being about whether the model could edit code. It could. The harder question was whether my queue, labels, instructions, budgets, and review process were good enough to make that work economically sane.

That framing matches the broader measurement context. The 2025 DORA report treats AI as an amplifier of the surrounding organization, not a tool that magically fixes a weak system. METR's early-2025 developer productivity study and 2026 follow-up on experiment-design limits are reminders that developer productivity measurement is messy, task selection matters, and a single usage report should not be mistaken for universal proof.

The Ticket Is The Work Contract

The most important interface in this workflow is not the prompt. It is the ticket.

A good ticket does several jobs at once. It defines the task, bounds the search space, names the files or packages likely to matter, specifies validation, and says what should not change. It turns an agent from a general reasoning engine into a constrained executor.

There is a huge difference between this:

Look into the flaky tests.

And this:

Investigate tests/api/test_webhook_retries.py::test_retries_after_429. Run the single test first, then the webhook file. If the test is stale, open a draft PR with a test-only fix. If the evidence points to a production retry bug, stop and explain the evidence before changing semantics. If neither is supported, report what you checked and stop.

The second ticket is longer. It gives the agent fewer ways to waste the budget.

Some of this can be driven by native Codex entry points, like asking Codex to act from GitHub. The label-triggered automation I want is a custom control plane on top: labels decide whether to dispatch work, which prompt to use, how much budget to allow, and whether Codex may edit code or only leave a plan.

Cost Is A Design Signal

It is tempting to treat model spend as accounting. That is the wrong abstraction. Cost is a product signal.

Codex billing depends on how you access it. ChatGPT plans provide included usage, with eligible additional usage charged through credits or plan-specific billing. API-key workflows follow API pricing. The Codex pricing page and credit rate card map credit-based usage to input tokens, cached input tokens, and output tokens. The ChatGPT plan guidance makes clear that limits and credit options vary by plan, and the Business spend controls exist because this has become an operational surface.

A high bill is not automatically bad. Expensive work can still be worth doing. But unexplained spend is a smell.

When spend spikes, I now ask four questions:

• Was the task too broad?
• Was the repo context too noisy?
• Did the agent lack reusable instructions?
• Was the validation loop too weak?

Good, Maybe, Bad

The useful question is not "can Codex do this?" The useful question is "is this worth paying Codex to attempt?"

Good tasks are bounded, repeatable, and externally checkable: dependency bumps with clear changelogs, mechanical refactors, test repairs, small feature flags, docs updates tied to code changes, grep-able cleanup, and CI failures with narrow logs.

Maybe tasks can work, but need tighter supervision: multi-file feature work, unfamiliar code paths, subtle product behavior, database migrations, performance improvements, cross-service integration, UI changes, and code that depends on undocumented conventions.

Bad tasks for unattended runs are vague, high-impact, or hard to validate: broad architecture changes, security-sensitive rewrites, production incident response without tight human control, ambiguous product decisions, billing logic, data deletion flows, and anything where correctness is mostly social or organizational.

The categories are not moral judgments. They are dispatch rules. Good work can go straight to a constrained run. Maybe work should be split into inspect, plan, implement, validate, stop. Bad work should stay human-led until the shape is clear enough to delegate a smaller piece.

The Stack And Context Discipline Matter

A naive workflow gives an agent a repo and asks it to figure things out. That works until the repository gets large, the task gets ambiguous, or the cumulative usage cost becomes visible.

The better pattern is layered context.

At the bottom, the repo needs short durable instructions. An AGENTS.md file should not be a novel. It should tell the agent how to work in this codebase: commands, conventions, test strategy, common traps, and review expectations.

Above that, the agent needs efficient ways to inspect code. Symbol-level context is better than dumping entire files. Focused search is better than wandering through the tree. Compressed shell output is better than pasting pages of logs. The goal is not to starve the model of context. The goal is to feed it context with intent.

For my workflow, the durable shape is boring and useful:

• Tickets define the job.
• Labels define risk and budget.
• Repo instructions define local conventions.
• CI defines the first correctness gate.
• Draft PRs define the review boundary.
• Humans decide what merges.

Labels And Budgets Are Policy

Once automation becomes routine, labels stop being metadata and start becoming policy.

A label can answer questions that should not be improvised mid-run:

• Can Codex run on this?
• Should it edit code or only leave a plan?
• What is the maximum budget?
• How many test retries are allowed?
• Can it touch production code?
• Which validation commands are required?
• Should failure create a follow-up ticket?

That is where the workflow starts to look less like "AI coding" and more like a job system. That is the right direction. A human engineer should not supervise every keystroke, but the system should make it easy to constrain the work before it starts and audit the result after it finishes.

Labels only become policy when the automation enforces them. A budget written in a ticket is not a spending cap by itself; the dispatcher needs to track usage and enforce the chosen time, retry, and cost limits.

The ticket is the contract. Labels select the policy. The PR is the artifact. CI is the first judge. Review is the final one.

The Human Role Moves Upstream

A common mistake is to evaluate Codex by asking whether it can replace the human inside the loop.

That is not the useful version.

The human role moves upstream and downstream.

Upstream, the human defines better work: clearer tickets, smaller scope, sharper success criteria, explicit non-goals, better repo instructions, and better task classification.

Downstream, the human reviews better artifacts: focused diffs, summarized tradeoffs, test output, known risks, follow-up suggestions, and rejected alternatives.

The middle gets automated. That middle is not all of software engineering, but it is a lot of the repetitive work around software engineering.

The Failure Mode Is Unbounded Work

Some failures are easy to spot: CI fails, the diff looks wrong, or review catches a misunderstanding. Passing CI can still miss incorrect behavior, security problems, or the wrong product decision.

The more dangerous failure mode is unbounded work that looks productive. The agent reads too much, tries too many approaches, rewrites more than necessary, fixes adjacent issues, discovers unrelated problems, and produces a large PR with a plausible explanation. It spends real money doing all of that.

This is why "does it work?" is an incomplete question. The better questions are:

• Did it stay inside the requested scope?
• Did it stop when it should have stopped?
• Did it ask for clarification when the task became ambiguous?
• Did it preserve reviewability?
• Did the cost match the value of the work?
• Would I run this class of task again?

The useful improvement loop is not "rerun the prompt until it works." It is traces, evals, failure summaries, and human feedback changing the system. OpenAI's agent improvement loop points in that direction: make the evidence reusable, then use it to tighten the next run.

What I Would Build Next

The next version of my setup would make the control plane more explicit.

I would not start by giving Codex more freedom. I would start by giving it fewer excuses to wander.

1. A stricter ticket template. Every automation-ready ticket should include scope, expected files, validation commands, non-goals, and maximum budget.
2. Budget-aware labels. A small docs fix and a multi-package refactor should not run under the same policy.
3. Better failure summaries. Failed runs should classify what happened: missing command, unclear instruction, flaky test, bad ticket, or genuine code ambiguity.
4. Shorter reusable repo instructions. The best instructions are not comprehensive. They are sharp enough to remove predictable mistakes.
5. A stronger stop rule. If the agent cannot find the right path quickly, it should stop and report. Continuing is not always diligence. Sometimes it is just spend.

The Practical Conclusion

Codex automation is viable when it is treated as an engineering system instead of a demo.

That means the boring parts matter: tickets, labels, budgets, logs, tests, draft PRs, review rules, and failures.

The exciting version of AI coding is a model that can do anything. The useful version is a system that knows what it is allowed to do, how much it is allowed to spend, how to prove that it worked, and when to stop.

The goal is not autonomous coding. The goal is bounded delegation.

And the bill is how I learned where those bounds were missing.