Ten Configuration Habits That Keep Automations Alive

An automation that runs unattended has two jobs: finish the work, and make it obvious when it did not. Most scheduled jobs die of the second failure first, quietly, while everyone assumes the first is fine. The ten habits below are configuration decisions, not architecture, and each one closes a specific way a job goes silent.

The ten habits, and the failure each one prevents

Habits 1, 2 and 9: keep configuration and dependencies out of the code

Anything that differs between environments — paths, table names, recipient addresses, model names, concurrency — belongs in a config file read at startup, not in a branch inside the script. The failure this prevents: every environment ends up running a slightly different version of the logic, and diffs stop being reviewable. Two details make it real: validate the config on startup and exit non-zero when a required key is missing, and keep the config in version control with a separate untracked file for machine-specific values.

One secret store. Pick a single place that holds credentials — a managed secrets service, an encrypted file in the repo, or the environment injected by your platform — and read from it and nowhere else. No copies in shell profiles, no tokens pasted into the workflow file, no silent fallback to a hardcoded default. The failure this prevents: rotating a key in the one place you remembered while the job quietly keeps using a stale copy.

Pinned dependency versions. Commit lockfiles, pin container images by digest, and avoid latest tags on anything scheduled. The failure this prevents: an unattended upgrade between two otherwise identical runs changes behaviour, and you spend an afternoon hunting a bug that is really a version bump. That problem shows up in model work too, which is why swapping a model in place is its own kind of change; see Swapping Models on a Single-GPU Box Without Downtime Drama.

# config/pipeline.yaml
schedule: "0 6 * * *"
concurrency: 4
source:
  url: "https://internal.example.com/export"
  timeout_seconds: 30
  retries: 3
output:
  table: "daily_export"
  mode: "upsert"

set -euo pipefail
CONFIG="${CONFIG_PATH:-./config/pipeline.yaml}"

# Fail fast: a missing key should stop the run, not default to something wrong.
python - "$CONFIG" <<'PY'
import sys, yaml
cfg = yaml.safe_load(open(sys.argv[1]))
required = {"source": ["url", "timeout_seconds"], "output": ["table", "mode"]}
missing = [f"{s}.{k}" for s, keys in required.items()
           for k in keys if k not in (cfg.get(s) or {})]
if missing:
    print("config error:", ", ".join(missing), file=sys.stderr)
    sys.exit(2)
PY

Habits 3, 4 and 8: make time and failure explicit

Every outbound call needs a deadline: --max-time on curl, a timeout= argument on the HTTP client, subprocess.run(..., timeout=...) on any child process, and a wall-clock limit on the whole job. A timeout is not a performance setting; it is the difference between a job that fails and a job that hangs. Set values from observed worst-case latency plus headroom, and keep them in config so you can raise one without touching code. The failure this prevents: a single stuck socket holds the worker, the queue grows behind it, and nothing in your logs says anything is wrong.

Use a small, fixed exit-code vocabulary and document it: 0 success, 1 unexpected error, 2 bad configuration, 75 (EX_TEMPFAIL) for something worth retrying. The failure this prevents: a scheduler that treats every non-zero code the same, either retrying a job that will never succeed or dropping one that needed a second attempt.

Alert on outcomes, not just errors. Two alerts matter: the job exited with a code that needs a human, and the job did not run at all. The second is the one people skip. Have the job ping a heartbeat on success and alert when the ping is missing, then route both alerts to one channel a person actually reads. The failure this prevents: a cron entry removed during a migration three weeks ago that nobody notices until a report comes up empty. That silent-failure class deserves its own habits, which I covered in Triage Habits for Automation That Fails While You Sleep.

Habits 5, 6, 7 and 10: make re-runs and logs boring

A dry-run flag: --dry-run prints every write, request and side effect it would perform, and performs none of them. The failure this prevents: the only way to check a change is to run it against live data and clean up afterwards.

Idempotent steps: writes are upserts keyed on something stable — a date, a source ID, a hash of the input — never blind inserts. The failure this prevents: a retry after a partial failure duplicates rows, and someone spends a morning deduplicating.

Log rotation: decide where output goes and cap it, with a RotatingFileHandler using maxBytes and backupCount, a logrotate rule, or plain stdout under a supervisor that already handles rotation. The failure this prevents: a verbose loop writes gigabytes, the disk fills, and the host stops running everything, not just this job.

A documented re-run command: one line in the README next to the schedule, with placeholders for the date or run ID. The failure this prevents: during an incident, nobody can reproduce the exact invocation from memory.

# Re-run the same job for a past date. Upserts make this safe to repeat.
./run.sh --date 2026-03-04 --dry-run   # inspect first
./run.sh --date 2026-03-04             # real run, safe to repeat

# Exit codes: 0 ok | 1 unexpected | 2 bad config | 75 retryable
timeout 900 ./run.sh --date 2026-03-04 || rc=$?
case "${rc:-0}" in
  0)  curl -fsS -m 10 "$HEARTBEAT_URL" >/dev/null ;;
  75) echo "retryable failure" >&2; exit 75 ;;
  *)  echo "needs a human, exit ${rc}" >&2; exit "${rc}" ;;
esac

# /etc/logrotate.d/pipeline
/var/log/pipeline/*.log {
  daily
  rotate 14
  compress
  missingok
  notifempty
  copytruncate
}

What to do next

1. Move every environment-dependent value out of your scripts into one config file, and add a startup check that exits 2 when a required key is missing.
2. Add deadlines to every outbound call in your longest-running job. If you cannot name the timeout on a call, it does not have one.
3. Write three lines into the README of your most important automation: the schedule, the exact re-run command, and what each exit code means.
4. Add a heartbeat ping on success and an alert when the ping is missing. The absent-run alert catches more failures than the error alert does.
5. If you are coordinating several agents rather than one cron job, the coordination problem grows: Cowork Pro ($59 one-time) is a dashboard for organising and orchestrating multiple AI agents on real projects, with task routing and run history.

Get Cowork Pro

Cowork Pro — $59, one-time payment, instant download. See the full breakdown on the review page.

Related reading

• The Ultimate Guide to AI Automation 2026
• How I Built a 10-Product Digital Business
• Building an AI Content Factory