Architecture map

Beampipe is the durable control plane between archive metadata, project policy, workers, schedulers, and DALiuGE. It decides and records what should happen; external systems perform the scientific work.

Explore the boundary

INPUTCASDA + YAMLfacts and policy --> CONTROLAPI + ledgerintent and truth <--> CLAIMworkersleased effects --> EXTERNALSlurm + DALiuGEscience runtime

AUTHORITATIVE

PostgreSQL carries control-plane truth

The API records operator intent. The ledger, jobs, claims, artifacts, profile snapshots, and provenance make that intent recoverable across API and worker restarts.

Follow a concept

[01] Control-plane boundariesOwnership, durable records, and responsibilities. [02] Discovery and executionFrom source registration to verified outputs. [03] Execution state modelControl phase, external axes, and uncertainty. [04] Integration adaptersArchive, scheduler, and DALiuGE contracts. [05] Deployment profilesImmutable backend configuration per execution.

The invariant

One durable decision, many observed systems.
Every external action must be tied to persisted intent, deterministic identity, an immutable profile/config snapshot, and enough provenance to reconcile after interruption.