AI Pro Tips

I’ve posted enough scattered AI tips on Slack that it’s time to put them in one place. These are operating principles, not commandments. Nothing applies everywhere; don’t blindly follow any of this.

Use an agent, not a chatbot

For any non-trivial task I use a CLI agent. The important distinction isn’t the UI — it’s that the model can inspect the source material, use tools, execute work, observe the result and continue. I’ve used the same pattern for code, 100K+ document extraction, electronics design and paperwork: provide the primary sources and ask for a concrete artefact, not an answer about how one might produce it. Prefer APIs and CLIs; use computer control only where no programmatic interface exists.

Spend most of the effort before code generation

Spec-driven planning and explicit clarification are non-negotiable. Define the problem, constraints, architecture, invariants and edge cases first. Break large plans down. Ambiguity at the start turns into confident, expensive nonsense later.

For agentic coding loops, make the acceptance criteria executable. I’ve had good results with a tiny, human-readable E2E language:

begin test 'source reconnects'
block source network
wait for log 'source_heartbeat_timeout' with timeout 10s
unblock source network
wait for log 'source_connected' with timeout 10s
ensure correct content for table 'xyz' end test

The agent can build however it likes; the loop only passes when the contract passes. Unit tests should similarly come from documented behaviour, not from copying the current implementation and enshrining its bugs.

Front-load relevant context

Provide as much relevant data as possible before an unattended workflow starts. Once an agent commits to a wrong hypothesis on insufficient data, it’s very bad at backtracking and “forgetting” its earlier decision. This does not mean dumping an unstructured landfill into the context window. Label the data, preserve its chronology and state what is authoritative.

Code is the source of truth for system behaviour. Let the model read it. Use docs to state contracts, intent and surprising constraints; use old discussions as pointers, not truth.

Give the run a budget and a finish line

Frontier agents no longer need elaborate Ralph loops just to keep working. A two-line prompt can now run for 12 hours. The problem has inverted: you need to tell the thing when to stop.

Set cost, complexity and time constraints alongside a measurable outcome. For example: “Get the core import flow working end to end by 10:30am. Keep the change under 500 lines, spend no more than $20, and stop for approval if that requires changing the schema.” Realistic constraints produce simpler, more creative solutions and prevent an agent from polishing the universe.

Close the loop with reality

Never ask an agent to “optimise” and accept prettier code as evidence. Give it a measurement loop: profile, change, rerun, compare. For ORM work, have it run EXPLAIN against a representative database and iterate on query cost. I watched an agent replace a disastrous .distinct() chain with the EXISTS subquery I would have written by hand — but only after it could see the query plan.

Do not outsource comprehension

Don’t advance past generated code until you can explain its purpose, trade-offs and interactions. Use agents to accelerate understanding, not bypass it. Personal comprehension and architectural ownership are what allow the next high-leverage decision; hidden code you don’t understand is merely debt with a faster creation rate.

Also, don’t get emotionally attached to generated work. If the result is suboptimal, throw it away, refine the prompt and retry. The marginal cost of a fresh attempt is now tiny; the maintenance cost of keeping bad code is not.

Make prompts part of the repository

Persistent project guidance belongs next to the domain it governs. Check review rules and engineering constraints into AGENTS.md, put package-specific guidance in subdirectories and allow local overrides where appropriate. Versioned prompts are reviewable; prompts living in everyone’s home directory become folklore.

Turn repeated workflows into small skills, but only after several real loops work. “Let’s throw AI at it” is not an architecture. Build the working cases, extract what they share, then standardise. Keep the skills concise and delete ones that stop earning their keep. Sometimes a simple prompt does 95% of the job; don’t reach for a complex AST checker when a cheap advisory check is all you need.

Assume review is the bottleneck

As code generation gets cheaper, review gets more expensive. Keep AI-authored PRs surgical, incremental and easy to inspect. Self-review in a clean context, run an adversarial review, rewrite the commit history, resolve findings and pass CI before asking a human to look.

LLMs are non-deterministic, so one clean report means little. Run the same review rules locally with clean-context reviewers, then use the PR bot as the last line of defence — not a substitute for your own quality check. A good red-team reviewer reconstructs intent first, identifies invariants and trust boundaries, then reports cause, failure mode, reproduction and the smallest safe fix, severity ordered.

Security boundaries must be structural

Turn on sandboxing. Use least privilege. Keep network and sensitive filesystem access off unless the task needs them. Never leave an unattended agent with broad access to production, customer data or your “god power” accounts. Agents still hallucinate, follow prompt injection and take erratic actions.

Prompts are not access controls. “Only read these tables” is not a security boundary. For browser agents, I issue a short-lived login URL that creates a session with only the required permissions, then require all mutations through the audited UI — no arbitrary POSTs. If the agent interacts with a human, disclose that it is an AI before the interaction starts.

Format data for the model, not the tokenizer

Hard no on removing repeated keys from large inputs merely to save tokens. LLMs still struggle with dense, unlabeled data such as tables. Repeat the keys and make relationships explicit. If compactness matters, compress the output: it’s usually the larger side of the bill, and APIs can enforce a precise output schema without the model having seen that format during training.

Make long-running agents observable and steerable

Treat an agent like any other long-running job. Notify yourself when it blocks for input — Codex supports notify = ["notify-codex"] in ~/.codex/config.toml — ask a side-chat for a status report, and interrupt only when the current path is wrong. In Codex CLI, Enter steers after the current step, Enter+Escape interrupts immediately, and Tab queues instructions for after the current turn. Queueing “run the tests, then review the diff” is considerably better than hovering over the terminal like a worried parent.

Force useful disagreement

Don’t ask a model for “the answer” to a disputed question. First ask it to steelman each position independently, including the premises, evidence, rebuttals and conditions under which each would be correct. For cross-domain problems, run separate security, performance, product and operations reviewers, then give their outputs to a final synthesis pass. Parallelise only independent work; correlated agents produce five copies of the same mistake.

When an explanation is too abstract, iteratively lower the assumed knowledge level until the model uses concrete examples. When a recommendation is too generic, state your must-haves, exclusions, acceptable exceptions and decision rules. Prompting is a control loop, not a magic sentence.

Make the agent learn you

One useful meta-prompt is:

Go through my conversation history — just the threads, not skills or global
prompts — and distil the coding, engineering and design principles I repeatedly
enforce. List them concisely and cite the conversations they came from.

Review the result, remove accidental preferences, then turn the durable parts into project guidance. Also add this small rule to any coding agent:

If you see a change you made suddenly change or disappear, assume the user
changed it. Do not put it back without asking.

That one sentence prevents a surprisingly common form of artificial stubbornness.