Blueprints

Attention

This project is still in an early phase of development.

The python API is not yet stable, and some aspects of the schema for the blueprint and workplan will likely evolve. Therefore whilst you are welcome to try out using the package, we cannot yet guarantee backwards compatibility. We expect to reach a more stable version in 2026.

Blueprints#

Blueprints define the contract for communicating the available behaviors of an application. A user-configured blueprint informs C-Star which behaviors are desired.

Core Blueprint Schema#

The core attributes of a blueprint come from cstar.orchestration.models.Blueprint. See the blueprint YAML below for example usage fo the attributes.

Core Blueprint Attributes

name

A unique, user-friendly name for this blueprint.

description

A user-friendly description of the scenario to be executed by the blueprint.

application

The process type to be executed by the blueprint.

state

The current validation status of the blueprint.

cpus_needed

The number of CPUs needed to run this blueprint.

The core blueprint attributes do not contain enough information to be executed alone.

Customizing Blueprints#

Blueprint subclasses are created to define attributes for each supported application. These classes are responsible for exposing the set of configurable parameters that are user-facing.

Custom Blueprint Example: RomsMarblBlueprint#

cstar.applications.roms_marbl.app.RomsMarblBlueprint contains all information necessary to execute a coupled simulation using UCLA-ROMS with biogeochemistry handled by MARBL. It adds the following attributes:

RomsMarblBlueprint Attributes

valid_start_date

Beginning of the time range for the available data.

valid_end_date

End of the time range for the available data.

code

Code repositories used to build, configure, and execute the ROMS simulation.

initial_conditions

File containing the starting conditions of the simulation.

grid

File defining the grid geometry.

forcing

Forcing configuration.

partitioning

User-defined partitioning parameters.

model_params

User-defined model parameters.

runtime_params

User-defined runtime parameters.

cdr_forcing

Location of CDR input file for this run.

Explore the API reference of cstar.applications.roms_marbl.app.RomsMarblBlueprint for more detail on each item.

Preparing a Blueprint#

A blueprint can be prepared for execution in a few ways:

  1. If you are creating a brand new domain, consider using C-SON Forge to prepare your input files and blueprint all at once.

  2. For an existing set of inputs, you manually write a YAML file with the desired blueprint configuration.

  3. Write python code to define a RomsMarblBlueprint instance and export it to YAML.

RomsMarblBlueprint Example#

This example YAML demonstrates a configured RomsMarblBlueprint. Notice that:

  • ROMS code can be built from a fork, branch, or even a git commit hash, by specifying branch: or commit:

  • Remote or local resources can be used to build and execute a simulation, under compile_time:

  • C-Star handles both partitioned and unpartitioned data

  • Runtime and compile-time behaviors can be customized in the .opt and .in files

name: 2node_1wk_example
description: this is mainly to test infra like containers and workplans. it should run on 256 processors (2 nodes)
application: roms_marbl
schema_version: 2.0.0
working_dir: /anvil/scratch/x-seilerman/2node_1wk_job1/
state: draft
valid_start_date: 2000-01-15 0:00:00
valid_end_date: 2000-01-23 0:00:00
code:
  roms:
    location: https://github.com/CWorthy-ocean/ucla-roms.git
    branch: main
  marbl:
    location: https://github.com/marbl-ecosys/MARBL.git
    branch: marbl0.45.0

  run_time:
    location: /anvil/scratch/x-seilerman/2node_test_domain
    branch: "na"
    filter:
      files:
      - test_domain_1wk.in
      - marbl_in
      - marbl_tracer_output_list
      - marbl_diagnostic_output_list
  compile_time:
    location: /anvil/scratch/x-seilerman/2node_test_domain/compile
    branch: "na"
    filter:
      files:
      - bgc.opt
      - bulk_frc.opt
      - cppdefs.opt
      - diagnostics.opt
      - ocean_vars.opt
      - param.opt
      - tracers.opt
      - Makefile

grid:
  data:
    - location: /anvil/scratch/x-seilerman/2node_test_domain/input_files/partitioned_files/grid_64x64x5.000.nc
      partitioned: true
initial_conditions:
  data:
    - location: /anvil/scratch/x-seilerman/2node_test_domain/input_files/partitioned_files/init_condis_bgc.000.nc
      partitioned: true

forcing:
  tidal:
    data:
      - location: /anvil/scratch/x-seilerman/2node_test_domain/input_files/partitioned_files/tides_Jan1_2000.000.nc
        partitioned: true
  surface:
    data:
      - location: /anvil/scratch/x-seilerman/2node_test_domain/input_files/partitioned_files/surf_phys_filepath_200001.000.nc
        partitioned: true
      - location: /anvil/scratch/x-seilerman/2node_test_domain/input_files/partitioned_files/surf_frc_bgc_clim.000.nc
        partitioned: true
  boundary:
    data:
      - location: /anvil/scratch/x-seilerman/2node_test_domain/input_files/partitioned_files/boundary_force_phys_jan15_feb21_200001.000.nc
        partitioned: true
      - location: /anvil/scratch/x-seilerman/2node_test_domain/input_files/partitioned_files/boundary_force_bgc_jan15_feb21_clim.000.nc
        partitioned: true

partitioning:
  n_procs_x: 16
  n_procs_y: 16

model_params:
  time_step: 900

runtime_params:
  start_date: "2000-01-15 00:00:00"
  end_date: "2000-01-22 00:00:00"

Checking validity#

Blueprints can be checked for errors using the CLI and in code.

Use the check command from the cstar CLI.

cstar blueprint check my_blueprint.yaml

Use the deserialize method to validate a YAML file in Python.

from cstar.orchestration.models import RomsMarblBlueprint
from cstar.orchestration.serialization import deserialize

deserialize("my_blueprint.yaml", RomsMarblBlueprint)

Execution#

Tip

Review the configuration options available via environment variables before running.

Warning

The post-processing step joining partitioned data may consume all available cores of a login node and be terminated (and make the admins angry).

  • We strongly recommend setting CSTAR_NPROCS_POST to a small number (~2) when running a ROMS-MARBL blueprint directly on a HPC login node.

  • Consider making a single-step workplan to run a simulation entirely on the compute cluster.

CLI#

Use the run command from the cstar CLI to execute a blueprint.

Use the run command from the ``cstar` CLI.

cstar blueprint run my_blueprint.yaml

Use a RomsMarblRunner to execute the blueprint.

Executing a blueprint YAML file in python.#
  from cstar.entrypoint.config import JobConfig, ServiceConfiguration
  from cstar.applications.roms_marbl.app import RomsMarblBlueprint, RomsMarblRunner
  from cstar.applications.core import RunnerRequest

  account_id = "your-account-id"
  queue_id = "wholenode"

  request = RunnerRequest("my_blueprint.yaml", RomsMarblBlueprint)
  service_cfg = ServiceConfiguration()
  job_cfg = JobConfig(account_id=account_id, walltime="00:90:00", priority=queue_id)

  runner = RomsMarblRunner(request, service_cfg, job_cfg)
  await runner.execute()