Skip to content

Operator guide

This is the daily operating map for beampipe-core v2. Use it after installation to understand which process should run, what it owns, and where to look when work stalls.

Operating model

beampipe-core is one binary with multiple roles. All roles share PostgreSQL as the durable state store.

Run exactly one scheduler-enabled process per environment. Scale API processes for HTTP traffic and worker-only processes for queue throughput.

Role contract

Role Starts with Owns Scale rule
API beampipe serve --worker false HTTP API, auth, uploads, readiness, metrics Scale for request volume
Scheduler beampipe serve --worker true or scheduler-enabled beampipe worker Recurring discovery, execution, DALiuGE, and Slurm poll ticks Run exactly one
Worker BEAMPIPE_WORKER_SCHEDULER_ENABLED=false beampipe worker Queue claims, TAP calls, manifests, staging, translation, deployment, polling Scale horizontally
Database PostgreSQL Configs, source metadata, jobs, executions, provenance One logical primary

Operator workflow

Phase Action Command or API
Bootstrap Apply migrations beampipe migrate
Bootstrap Create first operator beampipe admin create-user --username admin --password ... --email ...
Config Validate survey YAML beampipe project validate -f config/wallaby_hires.v2.yaml
Config Upload survey YAML POST /api/v2/project-configs
Config Upload deployment profile POST /api/v2/deployment-profiles
Source load Register sources POST /api/v2/sources or POST /api/v2/sources/bulk
Discovery Trigger or schedule discovery POST /api/v2/sources/discover
Execution Create execution intent POST /api/v2/executions
Execution Queue execution POST /api/v2/executions/{id}/execute
Monitoring Check readiness GET /api/v2/ready
Monitoring Inspect provenance GET /api/v2/executions/{id}/events

For first-time setup, the supported path is:

beampipe init
beampipe setup
beampipe doctor
beampipe project add -f config/wallaby_hires.v2.yaml
beampipe start

Use beampipe console for the same durable system state without requiring a browser.

Use API workflow guide for concrete request examples.

Mock to real backend path

Start with mock backends to validate project config, discovery, manifest construction, and dry execution. Move to real backends only after the environment has credentials, TAP reachability, DALiuGE Translator Manager access, and a tested DALiuGE DIM or Slurm deployment profile.

export BEAMPIPE_USE_REAL_BACKENDS=true
export BEAMPIPE_ENV=production
export CASDA_USERNAME=...
export CASDA_PASSWORD_FILE=/run/secrets/casda_password
export SLURM_SSH_PRIVATE_KEY_FILE=/run/secrets/slurm_ssh_key
export SLURM_SSH_KNOWN_HOSTS_SOURCE=/run/slurm-ssh/known_hosts

Run beampipe security check before production startup and beampipe slurm ping --profile <name> before live Slurm submission.

What to watch

Signal Healthy shape Where
Readiness Database, queue, workers, and dependencies report ready GET /api/v2/ready
Queue depth Does not grow indefinitely after ticks readiness payload, metrics
Oldest queued job Stays near expected job time metrics
Discovery metadata Sources receive metadata and discovery flags source events
Execution ledger Runs move through stage, translate, submit, poll, terminal state execution events
Backend debug fields DALiuGE DIM or Slurm fields appear on execution responses execution response

First triage

Symptom First checks Next page
API cannot start DATABASE_URL, migrations, bind address, startup security gates Production runbook
Login fails Admin user exists, BEAMPIPE_JWT_SECRET is stable First run
Discovery stalls TAP health, source enabled state, queue depth, discovery caps Workers and scheduling
Execution remains pending Project config active, metadata ready, execution caps Workers and scheduling
Slurm run stalls Known hosts, SSH key, login node, poll events Deployment profiles
Redoc stale Export OpenAPI and copy it into docs assets OpenAPI export

Next: tune worker capacity in Workers and scheduling.