The Research Loop That Knows When Not to Run

Direct Answer

This is a scheduled research system that spends real money, and most of its engineering is about not spending it. Three agents run each week against my own site: a ranking check, a paid AI-visibility scan, and a content brief. The interesting part is not that they run. It is that the scan can refuse to start eight different ways, each with its own exit code, and that every refusal is reported to a person in plain language instead of being retried.

This is the operating system I use for my own business, not a client case study. I am describing it because the artifacts are inspectable: a versioned pricing file, a cost ledger, dated data snapshots, and weekly run reports.

Key Takeaways

• Autonomy is not permission to act. It is a schedule plus an explicit list of conditions under which the system stops.
• The budget gate runs before the first request, priced from a file with a date on it.
• Firing the same weekly job twice is refused, not charged twice.
• A partially finished measurement is reported as unusable rather than smoothed over.
• The same architecture is the first system on AI workflow automation.

What problem does this system actually own?

The work is research that has to happen on a rhythm: where the site ranks, whether AI answer engines cite it, and what to write next. Done by hand it is three or four hours a week, it slips whenever the week is busy, and the output quietly drifts from evidence to opinion — you end up writing about the topic that feels interesting rather than the one the data supports.

That is the same shape as the sales problem in the CRM follow-up teardown: a repeatable path, carried by one person’s memory, that degrades the moment attention moves. Different department, identical failure mode.

The difference is the risk profile. A missed follow-up costs an opportunity. A research loop with API access and a scheduler can cost money every time it misfires, and it can also produce a confident number that is wrong — which is worse, because you act on it.

The existing workflow I map first

Everything the system does now maps onto a row in that table. Nothing was automated because it was automatable.

How the loop runs

Three scheduled agents, all Europe/Athens, all defined as readable instruction files in the repository rather than as configuration hidden in a SaaS dashboard.

Monday — ranking check

Fetches fresh data from the Search Console API and writes a dated snapshot, so week-over-week comparison stops depending on someone remembering to export. It then verifies that the data window is no more than four days old before analysing anything. If a fetch fails it reports the failure and the age of the existing data. The rule in the file is blunt: do not analyse stale files silently.

Wednesday — AI visibility scan

The only step that costs money. It asks a fixed set of buyer queries across several AI platforms with web search enabled, and records who was cited. Midweek by design, so there is time to react before Friday.

Friday — content brief

Ranks opportunities on the rows the previous two days produced, and writes one brief. Not three. If no completed gap report exists it is instructed not to invent a topic — it ends by recommending that the gap audit be run instead.

When the system refuses to act

This is the part worth copying.

The Wednesday scan has eight outcomes, and only one of them means “a complete measurement was taken.”

Four mechanisms produce those refusals.

A budget gate that runs before the first request. The scan prices exactly what it is about to send, using a pricing file with a version, a pricing_date, and a note recording how each rate was verified. If the estimate exceeds the configured maximum per run, it stops and prints what it would have cost. Nothing is sent. Going over the cap requires a flag passed deliberately by a person, and the scheduled script is explicit that it must never be passed from a schedule.

An idempotency guard. The weekly job derives a stable run ID from the ISO week. Fire it twice in the same week and the second run is refused with exit 3 rather than charged again. Forcing a genuinely new run means choosing a new ID on purpose.

Resume that does not re-pay. If a scan dies halfway, re-running it with the same ID sends only the tasks that never succeeded. Completed work is not bought twice. A couple of transient network failures should not cost a full rescan to repair.

A plan-mismatch check. If the query list changes after a run ID has started, the system refuses to resume, because resuming would blend two different question sets into a single weekly figure. It tells you to start a new run ID instead.

Underneath all four is one principle: a re-run costs real money, so it is the owner’s decision, not the scheduler’s. The failure branches say so in text, and they end with an instruction not to launch another scan to compensate.

What stays under human control

• Publishing. The Friday agent produces a brief and stops. Drafting is the owner’s call.
• Spending above the cap. A manual flag, never a schedule.
• Re-running a failed paid scan. Reported, never automatic.
• Reporting an incomplete week. The script states plainly that an incomplete scan must not be reported as the weekly figure.

Anthropic’s note on building effective agents makes the same argument from the other direction: start with a simple, inspectable workflow and add autonomy only where it earns its place. The refusal codes are how I keep the workflow inspectable once it is running unattended.

How I measure the loop itself

Two habits do most of the work here, and both are about refusing to flatter the system.

Metrics that measure different things are never merged. Branded recognition and non-branded category discovery are reported separately. Combining them produces a single number that always looks better than reality, because your own name inflates it.

Matching rules get tightened when they are wrong. The brand matcher originally accepted the bare first and last name. Those matched unrelated people and inflated the mention rate with results that could not be audited, so they were removed and recorded as rejected aliases with the reason attached. The number went down. That was the correct outcome.

The same discipline applies to gaps: a platform with no API key is logged as unavailable, never estimated. And the reports carry a standing caveat — these are API results with web search enabled, which track direction reliably but are not a copy of what a person sees in a consumer app.

I am not publishing a performance claim for this system. The honest statement is narrower: the research now happens on a schedule, every paid run has a cost record, and the weeks where nothing was measured are visible as such. That is what I can show.

When this is the wrong build

• The work happens once a month and a person can just do it.
• Nobody will read the output. A scheduled report with no reader is a scheduled cost.
• The team wants a dashboard rather than a decision. This loop ends with one recommended action.
• There is no budget owner. Any system with an API key needs a person whose money it is.

If the constraint is the public website rather than the operations behind it, that is a different problem and belongs with SimplySites.

How this connects to the engagement

Two teardowns, two departments, one architecture: a trigger the company already owns, a model doing the preparation, a human on anything irreversible, and a measurement that is permitted to report nothing.

That pattern is what I build inside companies during an embedded AI engagement. Research operations tend to be the safer place to start, because the failure mode is a wasted run rather than a customer-facing mistake — which makes it a good first system for a team learning to supervise one.

If a repeated path in your business already depends on one person remembering to do it, describe the bottleneck and we scope the smallest useful loop.

Summary

I built a research loop that runs three times a week and is engineered mostly around the conditions under which it should stop: a budget gate before the first request, an idempotency guard that refuses to pay twice, resume semantics that never re-buy completed work, and eight exit codes that tell a person exactly what happened. Autonomy here means the system acts on a schedule without supervision and declines to act without permission. Join the AI automation newsletter for the next teardown.