Attention
This project is still in an early phase of development.
The python API is not yet stable, and some aspects of the schema for the blueprint and workplan will likely evolve. Therefore whilst you are welcome to try out using the package, we cannot yet guarantee backwards compatibility. We expect to reach a more stable version in 2026.
Workplans#
Workplans define the contract for requesting the execution of one or more
Blueprints. A user-configured workplan informs C-Star which Blueprint
to execute, and in what order.
Workplan Schema#
Workplans are defined in cstar.orchestration.models.Workplan.
Workplan Attributes
|
The user-friendly name of the workplan. |
|
A user-friendly description of the workplan. |
|
The steps to be executed by the workplan. |
|
The current validation status of the workplan. |
|
A collection of key-value pairs specifying attributes for the target compute environment. |
|
A collection of user-defined variables that will be populated at runtime. |
State#
Note
This feature is currently in-development.
A workplan may be configured in a draft or validated state using state
a draft workplan can be freely edited and submitted for execution.
modifications to a validated workplan are restricted to ensure reproducibility.
Compute Environment#
Note
This feature is currently in-development.
The desired compute environment characteristic are specified using compute_environment
Runtime Variables#
Note
This feature is currently in-development.
C-Star performs some simple templating for user convenience. The variables identified in
runtime_vars are meant to be provided at runtime,
such that one workplan could be shared and used for a range of supported values without needing
to modify the yaml. The user API for specifying runtime variables is still in development.
Steps#
The heart of a workplan is the collection of steps found in steps.
Steps have a 1-to-1 relationship with blueprints - each step must specify the path to a blueprint file.
The step also specifies the application type to use for its execution.
Step Schema#
See cstar.orchestration.models.Step for complete details on configuring steps.
Step Attributes
|
The user-friendly name of the step. |
|
The user-friendly name of the application executed in the step. |
|
An optional list of external step names that must execute prior to this step. |
|
A collection of key-value pairs specifying overrides for blueprint attributes. |
|
A collection of key-value pairs specifying overrides for compute attributes. |
|
A collection of key-value pairs specifying overrides for workflow attributes. |
Workplan Examples#
The following example demonstrates the minimum possible workplan.
It contains a single step to be executed.
name: Simple Workplan
description: Run a simulation
state: draft
steps:
- name: job1
application: roms_marbl
blueprint: /home/x-seilerman/wp_testing/2node_1wk_new_a.yaml
The following example demonstrates a workplan with multiple steps. Note that each step can reference different blueprints.
Additionally, this example introduces a simple dependency with depends_on: job1. A
dependency must be specified if the steps of the workplan require specific ordering. Here,
job1 must complete successfully before job2 will start.
Important
A multi-step workplan without dependencies has no ordering guarantees.
Jobs are scheduled immediately and executed as the system launcher permits.
name: Multi-step Workplan Example
description: Run multiple ROMS-MARBL simulations
state: draft
steps:
- name: job1
application: roms_marbl
blueprint: /home/x-seilerman/wp_testing/2node_1wk_new_a.yaml
- name: job2
application: roms_marbl
blueprint: /home/x-seilerman/wp_testing/2node_1wk_new_b.yaml
depends_on:
- job1
- name: job3
application: roms_marbl
blueprint: /home/x-seilerman/wp_testing/2node_1wk_new_c.yaml
The following example demonstrates how to override configuration in a blueprint from the workplan. Overriding blueprints enables the same blueprint to be used with different inputs, data sources, etc.
Tip
Blueprint overrides are supplied as a dictionary with Blueprint schema
name: Workplan Overriding a Blueprint
description: Run a blueprint ensemble varying parameters with overrides
state: draft
steps:
- name: step 1
application: roms_marbl
blueprint: blueprint.yaml
blueprint_overrides:
- name: Run blueprint with development UCLA-ROMS branch
- code:
roms:
location: https://github.com/CWorthy-ocean/ucla-roms.git
branch: develop
- name: step 2
application: roms_marbl
blueprint: blueprint.yaml
blueprint_overrides:
- name: Run blueprint with custom UCLA-ROMS fork
- code:
roms:
location: https://github.com/github-user/ucla-roms.git
branch: main
Checking validity#
Workplans can be checked for errors using the CLI and in code.
Use the check command from the cstar CLI.
cstar workplan check my_workplan.yaml
Use the deserialize method to validate a YAML file in Python.
from cstar.orchestration.models import Workplan
from cstar.orchestration.serialization import deserialize
deserialize("my_workplan.yaml", Workplan)
Execution#
Tip
Review the configuration options available via environment variables before running.
Attention
An error will occur if the SLURM account and queue are not configured when running on a HPC.
Use the run command from the cstar CLI to execute the workplan.
cstar workplan run --run-id <my-unique-id> my_workplan.yaml
Use the dag_runner module to execute a Workplan in python.
from pathlib import Path
from cstar.orchestration.dag_runner import build_and_run_dag
path = Path("/path/to/my/workplan.yaml")
await run_workplan(path, run_id="my-unique-id")
Checking Workplan Status#
Use the status command from the cstar CLI to retrieve the current
status of steps in a running workplan.
cstar workplan status --run-id <my-unique-id>