How to run agent work on a schedule without building a queue

A scheduled agent needs five things a cron line does not provide: an idempotency key so a retry cannot send the same digest twice, a lease so two runs never overlap, one isolated run per tenant, a budget for each run, and a stored record of what the run did. Build those five before you build the schedule itself.

What a cron line leaves you holding

The first version is one line in a crontab, or a scheduled worker that calls an endpoint. It works for about a week. Then a provider returns 503 at 03:00, the wrapper retries, and every customer gets the same digest twice.

The failures arrive in a fairly predictable order:

• Tuesday's run takes 55 minutes instead of nine, and the next hourly run starts on top of it, reading the same rows.
• One tenant's expired API credential throws, the loop exits, and the 380 tenants after it in the list get nothing that morning.
• A run finishes clean and delivers an empty digest, because a query returned zero rows for a reason nobody notices for three weeks.
• A run that should have started never started, and nothing alarms on the absence of an event.
• Someone asks what last Thursday's digest said for one customer, and the only record is a log line that has aged out.

Each item on that list is ordinary distributed-systems work that arrives dressed as an AI feature. Scheduled work is one shape of the broader problem covered in agent orchestration, and the model call is rarely the part that breaks.

How do I run an AI agent on a schedule

Seven steps, in the order they tend to matter. Steps two and three are the ones teams skip and then rebuild after an incident.

1. Decide what belongs on a schedule

Scheduled work has three properties: nobody is sitting there waiting for it, its input is a time window rather than a user action, and skipping one occurrence is survivable. A daily digest, a weekly account review and hourly inbox triage all qualify. Anything a person is waiting on belongs on the request path, and anything that must happen within seconds of an external event belongs on a webhook.

One thing to separate out early is duration. Work that a single run cannot finish inside one wall-clock budget is a different design with different failure modes, covered in long-running agents.

2. Give every run an idempotency key

Derive the key from the work, not from the moment the run started. For a daily digest that is {tenant_id}:digest:{window_start_date}, which stays stable across a retry three minutes later and across a manual backfill six hours later.

Model output is not reproducible, so a retried run produces a different digest from the same input. That is exactly why the key has to describe the window rather than the content. Split the job into compute and deliver, and put the key on delivery: write a delivery row with a unique constraint on the key before you call the mail or Slack API, and treat a constraint violation as "already sent" rather than an error. A retry then costs you tokens and never costs the customer a duplicate.

3. Hold a lease, not a boolean

A running flag set by a process that then gets OOM-killed stays set forever, and the job is dead until a human clears it. Use a lease with a TTL instead, renewed while the run is alive:

{
  "lease_key": "daily-digest",
  "holder": "run_01J8Z3",
  "ttl_seconds": 1200,
  "renew_every_seconds": 120
}

Set the TTL above your slowest observed run and renew from inside the run loop, so a crash releases the lease within one TTL. Then pick the policy for the run that finds the lease held: skip, or queue exactly one. For digests, skipping is usually right, and queueing every blocked occurrence builds a backlog that never drains. Record the skip as a run outcome, because a silently skipped run and a run that never fired look identical afterward.

4. Fan out per tenant and isolate the failures

One parent run selects the tenants, and each tenant gets its own child run with its own key, budget and status. A child that throws marks itself failed and the parent keeps going. This is the difference between one customer's revoked token costing you one digest and costing you all of them.

Two details that bite later. Cap concurrency, because 400 children starting at once will hit provider rate limits and turn a tenant problem into a global one; a fixed pool of five to twenty in flight is usually enough. Rotate the tenant order between runs, so the same accounts are not always processed last and always the ones truncated when a budget runs out.

Timezones decide the schedule shape. "Daily at 08:00" means 08:00 where the customer is, so either run one schedule per timezone or run hourly and select the tenants whose local hour matches.

5. Budget every run

Nobody is watching a 03:00 run. An agent whose search tool returns an empty array can read that as a failed call and try again with a reworded query, twenty times, and the first sign of trouble is the invoice. Set four ceilings: wall-clock time per run, tool calls per turn, turns per run, and a cost limit for the run.

If the schedule spends against a customer's plan, the per-run ceiling and the per-customer ceiling have to agree, or a nightly job quietly consumes the allowance a customer expected to use during the day. That accounting is covered in usage limits per customer.

6. Record every run, and alert on the empty one

Store a row per run before the work starts and update it at the end: schedule id, tenant, window start and end, status, start and finish times, cost, model, tool call count, and the id of any side effect it produced. That row is what answers "what did last Thursday's digest say for this customer" without archaeology in a log aggregator.

Three alerts matter, and only one of them is the obvious one:

The second and third are where scheduled agents actually fail. A missing run produces no signal at all, so the check has to run on a separate timer and count expected occurrences against recorded ones.

7. Decide the failure policy before you need it

Sort failures into three buckets and give each one a fixed response. Transient provider errors (429, 5xx, timeouts) get bounded backoff, three attempts, same idempotency key. Deterministic errors (invalid credentials, a changed schema, a config the validator would have rejected) get no retry at all, because attempt three fails exactly like attempt one and only delays the alert to the account owner.

Partial failures get the narrowest response: retry the failed children, never the parent. Re-running the parent re-runs 380 successful tenants, which the idempotency key makes harmless for delivery but not for cost.

Then decide the staleness rule. A digest that is four hours late is often worse than no digest, so an occurrence that missed its window by more than some threshold should be marked skipped rather than run. Backfilling is a deliberate action with an explicit window parameter, not an automatic catch-up.

A worked design: a per-customer daily digest

Putting the seven steps together for a digest that summarizes each customer's last 24 hours:

job: daily-digest
schedule: '0 * * * *' # hourly; each run selects tenants whose local hour is 8
lease:
  key: daily-digest
  ttl: 20m
  renew_every: 2m
  on_conflict: skip
fan_out:
  select: tenants where digest_enabled and local_hour(tz) == 8
  max_concurrent: 10
  isolate_child_failures: true
  order: rotate_by_run
run:
  idempotency_key: '{tenant_id}:digest:{window_start_date}'
  window: previous 24h in the tenant timezone
  budget:
    wall_clock: 5m
    max_turns: 6
    max_tool_calls_per_turn: 10
deliver:
  only_if: digest.item_count > 0
  channel: email
  write_before_send: delivery(idempotency_key) unique
alert:
  - run_failed
  - fewer_than_expected_runs_in_26h
  - delivered_zero_for_tenant_with_activity

Two choices in there are worth stating plainly. The schedule is hourly rather than one entry per timezone, because a single cron expression plus a local-hour filter is far less to maintain than one schedule per offset. And delivery is gated on a non-empty digest while the run record is written either way, so "nothing to say today" and "the query broke" stay distinguishable.

The agent inside that run should reach your systems through registered tools rather than through data pasted into its prompt, so the same billing or usage lookup is callable, auditable and testable on its own. That pattern is exposing an internal workflow as an agent tool.

Where this gets easier

Most of the seven steps are platform features somewhere rather than code you should own. Runtype runs agents and flows on a schedule, triggers one on demand when an upstream system published late, and keeps run history, with a trace per run carrying each step's input and output, the tool calls with their arguments and results, and the cost of that run, so the durable record of what a run did is a byproduct of executing it rather than a table you maintain. The per-run ceilings are configuration: maxToolCalls per turn (default 10, maximum 100), loopConfig.maxTurns between 1 and 100, and a per-turn wall-clock budget of 30 minutes. Per-tenant fan-out inherits the tenancy strategy declared on the resource (internal, tenant-isolated or end-user-isolated), so a run executed for one customer is rejected before execution if its identity scope does not cover the data it asks for.