Release notes

Attention

This project is still in an early phase of development.

The python API is not yet stable, and some aspects of the schema for the blueprint and workplan will likely evolve. Therefore whilst you are welcome to try out using the package, we cannot yet guarantee backwards compatibility. We expect to reach a more stable version in 2026.

Release notes#

v0.5.0#

Note

This release is currently in development

Breaking Changes#

  • Deprecated/removed CSTAR_ORCH_REQD_ENV environment variable

  • Modified parameters to DagStatus.__init__

  • Removed workplan-monitor CLI command

  • Removed dag_runner.attach_to_run (duplicates process_plan functionality)

  • The flag constant is removed (#428)

  • Modified the interpretation of Status.is_running to exclude Unsubmitted (#461)

  • Rename env var group name constants to omit leading underscore (#462)

  • The roms executable is now executed in the work subdirectory instead of output (#465)

  • The output location in the runtime settings (roms.in) file is now _always_ modified to place outputs in the output directory (since we no longer run ROMS in the output directory) (#465)

New features#

  • Enable user supplied variables for templating from cstar workplan run …

  • The orchestrator writes a sentinel file to disk to dynamically alter dependencies (#459)

  • Add trace log level to reduce log frequency during Workplan execution (#460)

  • Persist WorkplanRun record for every invocation of the orchestrator for run history and lookup (#461)

  • Add cstar workplan monitor –run-id <run-id> for reattaching to a running Workplan (#461)

  • Add nesting_info block to RomsMarblBlueprint (#465)

  • Create work-dir symlinks to predetermined paths for CDR and nesting info files (#465)

  • If no Makefile is provided in the compile_time additional code block, ROMS will copy and use the appropriate Makefile from the ROMS repo (#465)

Security Fixes#

  • N/A

Bug Fixes#

  • Fix bug in reload_dag_status where the correct launcher may not be loaded

  • Fix comparison-to-self bug for orchestrator open_set

  • Fix mutable default attribute value in WorkplanRun

  • Remove repeated check identifying remote workplan URI

  • Fix bug causing intermittent FileNotFound errors

  • Fix workplan template with inconsistent blueprint path

  • Fix bug causing failures when invoking blueprints via cstar blueprint run

  • Fix bug handling user-relative paths in JobFileSystemManager (#430)

  • Fix regression causing blueprint output directory overrides to not be applied (#434)

  • Fix failure to include run-id in output path directory tree (#434)

  • Fix defect where steps defined before dependencies were not mapped correctly after time-splitting (#434)

  • Mitigate failures in the CachedRemoteRepositoryRetriever.refresh by automatically re-cloning if a refresh fails (#438)

  • Fix posix-path conversion bug when passing blueprint URLs to cstar blueprint run (#440)

  • Catch FileNotFoundError in cstar.cli.blueprint.check (#443)

  • Catch FileNotFoundError in cstar.cli.workplan.check (#443)

  • Update tests to ignore default factory where appropriate (#444)

  • Fix a JSON serialization bug with pydantic models using field aliases (#451)

  • Fix case-sensitivity bug when configuring split frequency via CSTAR_ORCH_TRX_FREQ env variable (#452)

  • Fix incorrect initial-conditions file paths created in RomsMarblTimeSplitter (#453)

  • Fix incorrect working directories resulting from LiveStep.from_step when a parent is specified (#453)

  • Use step.fsm.output_dir as the source of reset files instead of step.fsm.joined_output_dir (#455)

  • Fix failure to terminate health check thread (#456)

  • Fix issue when parsing SLURM status of the form cancelled by Username (#459)

  • Fix cstar workplan status … triggering a re-build of the entire workplan (#461)

  • Fix incorrect env var discovery if env_ appears repeatedly in variable name (#462)

  • Fix unhandled exception caused by attempting to execute typer app (#462)

  • post_run() no longer tries to join _ext files (#466)

  • Pin version of tutorial data repo (#475)

  • Bump typer version to avoid list-default bug (#476)

  • Fix uninitialized attributes error in SlurmBatch

  • Mitigate a dependency issue with prefect and fakeredis (pin fakeredis<2.35)

  • Fix bug when merging dictionaries during application of blueprint overrides

  • Fix bug where SimulationRunner fails to display errors when parsing bad CLI arguments

Improvements#

  • Decouple launcher-specific precondition checks from the dag_runner

  • Use computed attributes in place of duplicating keys found in DagStatus.details

  • Re-use configure_environment where applicable

  • Refactor cli.workplan.run to make use of typer callbacks for input validation

  • Add –dry-run option to test workplan generation in cstar workplan run

  • Remove if is_feature_enabled(“CSTAR_FF_ORCH_TRX_OVERRIDE”) to always enable overrides. (#428)

  • Encapsulate creation of JobFileSystemManager in new LiveStep class (#434)

  • Perform input validation before executing workplan and blueprint in: (#443)

    • cstar.cli.blueprint.run

    • cstar.cli.workplan.run

  • Add additional tests for EnvItem functionality not tied to XDG home directories (#444)

  • Reduce module load times (#446)

  • Reduce package import time by lazy loading expensive modules (#447)

  • Extract nested/duplicate YAML representer functions for re-use (#451)

  • Add empty sections specified in release notes and _Code Review Checklist_ (#454)

  • Replace use of incorrect mp.Queue where queue.Queue is appropriate (#456)

  • Replace blocking time.sleep usage in health check thread (#456)

  • Remove default values provided in JobConfig class (#457)

  • Removed slow status retrieval loop for previously submitted jobs (#459)

  • Enable “path-less” CLI commands, such as cstar workplan status –run-id <id>. (#461)

  • Display detailed status information for tasks in a Workplan (#461)

  • Ensure env var display consistency in cstar env show output (#462)

  • The original runtime settings file and the one that is written and used by C-Star are now appended with _ORIGINAL and _PATCHED to reduce confusion. (#465)

  • Use computed attributes to in place of duplicating keys found in DagStatus.details (#467)

  • Mitigate un (#469)

  • Improve error message when failing to load a remote YAML file. (#473)

  • Loosen Pydantic type validation for override dictionaries to mitigate pydantic warnings (#474)

  • Update unit tests that fail under specific user conditions on Anvil (#483)

  • Fix unexpected linting warnings about missing required parameters to ServiceConfiguration

  • Improve naming of module containing shared CLI callbacks

  • Add callbacks for automatic export of pertinent environment variables from CLI parameters

  • Reduce unit test duration

  • Reduce usage of hardcoded strings in CLI and tests

Miscellaneous#

  • Default to having prefect analytics collection disabled

—

For more details, please refer to the `commit history`_.

  • Rename constant and move from cstar.orchestration.utils to cstar.base.env (#432)

  • Update all references to the variable (#432)

  • Remove unnecessary TypeVar (#451)

  • Fix inconsistent test naming convention (#451)

  • Fix typos in tutorial (#464)

v0.4.0#

Breaking Changes#

  • Rename env var group name constants to omit leading underscore

  • Modified the interpretation of Status.is_running to exclude Unsubmitted

New features#

  • Enable status retrieval for multiple SLURM jobs in a single sacct call

  • The orchestrator writes a sentinel file to disk to dynamically alter dependencies

  • Add trace log level to reduce log frequency during Workplan execution

  • Persist WorkplanRun record for every invocation of the orchestrator for run history and lookup

  • Add cstar workplan monitor –run-id <run-id> for reattaching to a running Workplan

Security Fixes#

  • N/A

Bug Fixes#

  • Fix failure to override output directories for orchestrated blueprints

  • Fix defect where steps defined before dependencies were not mapped correctly after time-splitting

  • Fix unhandled exceptions in cstar [blueprint|workplan] check with invalid paths

  • Fix posix-path conversion bug when passing blueprint URLs to cstar blueprint run

  • Fix case-sensitivity bug when configuring split frequency via CSTAR_ORCH_TRX_FREQ env variable

  • Fix a JSON serialization bug with pydantic models using field aliases

  • Fix incorrect working directories resulting from LiveStep.from_step when a parent is specified

  • Fix failure to terminate health check thread

  • Fix issue when parsing SLURM status of the form CANCELLED by <username>

  • Fix incorrect env var discovery if env_ appears repeatedly in variable name

  • Fix unhandled exception caused by attempting to execute typer app

  • Fix cstar workplan status … triggering a re-build of the entire workplan

Improvements#

  • Improve error handling when CachedRemoteRepositoryRetriever.refresh fails

  • Validate inputs prior to execution with cstar [blueprint|workplan] run

  • Improve module load times

  • Extract nested/duplicate YAML representer functions for re-use

  • Replace use of incorrect mp.Queue where queue.Queue is appropriate

  • Replace blocking time.sleep usage in health check thread

  • Removed slow status retrieval loop for previously submitted jobs

  • Ensure env var display consistency in cstar env show output

  • Enable “path-less” CLI commands, such as cstar workplan status –run-id <id>.

  • Display detailed status information for tasks in a Workplan

Miscellaneous#

  • N/A

—

For more details, please refer to the `commit history`_.

v0.3.0#

This is a bug-fix release focused on stabilizing the orchestration of blueprints.

Breaking Changes#

  • Environment variables CSTAR_HOME and CSTAR_OUTDIR are deprecated in favor of CSTAR_CONFIG_HOME and CSTAR_STATE_HOME

  • Relocated command conversion functions from cstar.orchestration.launch.slurm to cstar.orchestration.converter

  • Promote blueprint overrides to always-on, deprecate and remove feature flag CSTAR_FF_ORCH_TRX_OVERRIDE

New features#

  • Add remote repository caching for re-use across simulations

  • XDG base directory standard is implemented

Security Fixes#

  • N/A

Bug Fixes#

  • Fix path concatenation bug resulting in invalid source data paths

  • Fix indentation creating an unreachable code block in git utils

  • Add conda installation of rsync to environment.yml (fixes bug related to a flag used in newer ucla-roms makefiles that is not available in MacOS’s default rsync installation)

  • Reimplement CSTAR_CLOBBER_WORKING_DIR for single-blueprint cases

  • Fix bug handling user-relative paths in JobFileSystemManager

Improvements#

  • Avoid automatic failures by retrying failed git clone operations

  • Add informational logging when tasks fail and additional tasks are cancelled

  • Update default log format to include file name and line number

  • Add constants for the set of available feature flags

  • Implement CSTAR_LOG_LEVEL env-var

  • Automatically retry partitioning without coarse dims if it fails when they are included

  • Enable orchestrated simulation execution using the local runner

Miscellaneous#

  • Additional details added in docstrings

  • Silence OMP warning originating from numba

  • Modify a blueprint in the tests directory for a refactored layout in the remote data repository

  • Move env-var handling methods to separate module to avoid circular imports

  • Remove __init__-based import shortcuts

  • Adjust error handling and presentation in the worker/service layer

—

For more details, please refer to the `commit history`_.

v0.1.0#

New features#

  • Add support for river forcing

  • Add support for forcing corrections (used in legacy ROMS configurations)

Breaking Changes#

  • Merge Case and Component classes into a single class, Simulation, simplifying internal/blueprint structure. Remove old Case and Component modules.

  • Rename caseroot to directory

  • Rename BaseModel to ExternalCodeBase

  • Remove start_time and end_time parameters from InputDataset.get(), these are now obtained from the corresponding attributes

  • Rename ROMSComponent.namelists to ROMSSimulation.runtime_code and ROMSComponent.additional_source_code to ROMSSimulation.compile_time_code

Internal Changes#

  • Update calls to roms-tools to reflect latest changes in API

  • Update internal/test blueprints to reflect new structure

  • New backend data retrieval system cstar.io

  • Add ROMSRuntimeSettings class with ability to parse and create roms .in files

  • Save partitioned ROMSInputDatasets in the same directory as their un-partitioned versions, rather than a subdirectory “PARTITIONED”

Documentation#

  • Run most examples in documentation locally (unless HPC-specific), so HPC access is not required

  • Add ‘How-to Guides’ section

  • Remove tutorial page on using roms-tools

  • Replace all URLs to other doc pages with relative links

Bugfixes#

  • Complete missing unit test coverage from Case and Component in new Simulation modules

  • Add expanduser to Path instances to allow tildes to represent root in paths

  • Correct issue where AdditionalCode.modified_files list indices did not correspond to AdditionalCode.files

v0.0.8-alpha#

New features#

  • Add Case.persist() and Case.restore() methods to allow continuation of work in a new session

  • Add ExecutionHandler class to track tasks run locally (LocalProcess subclass) or submitted to a job scheduler (SchedulerJob subclass)

  • Improved tracking of local InputDataset and AdditionalCode files to prevent repeat fetching

  • Add ability to read blueprints from URL

  • Remote yaml files are now accessed via requests rather than Pooch, negating need for hash checks

Bugfixes#

  • git and DataSource information are now read-only attributes throughout

v0.0.3-alpha (9th Dec 2024)#

New features#

  • Add support for SDSC Expanse HPC

Bugfixes#

  • Fix bug where in certain circumstances environment variables were checked for before being set, prompting user to install already installed externals

v0.0.1-alpha (6th Dec 2024)#

The first release of C-Star!

This release provides basic functionality, including the ability to create, import, and export a “blueprint” for reproducible ocean model simulations (a C-Star “Case”) using supported ocean models (ROMS, optionally with MARBL biogeochemistry) and run those Cases locally or on supported HPC systems (via Slurm and PBS). There is support for using existing model input data in netCDF format, or creating new input data via integration with the roms-tools library.

Note that the python API is not yet stable, and some aspects of the schema for the blueprint will likely evolve. Therefore whilst you are welcome to try out using the package, we cannot yet guarantee backwards compatibility. We expect to reach a more stable version in 2026.