Testing and Continuous Integration

This page describes how Galacticus is tested, both locally and through the automated continuous-integration (CI) pipeline that runs on every pull request.

Test suites

Galacticus has two complementary sets of tests.

The model regression suite (testSuite/)

The testSuite/ directory contains the regression tests that exercise the compiled model — building small models, checking physics results against known answers, validating file formats, and so on. They are orchestrated by testSuite/test-all.py, which runs every testSuite/test-*.py script in turn and writes a log for each to testSuite/outputs/:

export GALACTICUS_EXEC_PATH=`pwd`
python3 testSuite/test-all.py

An individual test can be run directly, which is usually what you want while iterating on a specific area:

python3 testSuite/test-Python-interface.py

Most of these tests require a built Galacticus.exe and the run-time datasets (GALACTICUS_DATA_PATH); see Installation.

The Python unit tests (pytest)

The build system’s Python modules (under python/) and the documentation tooling (under scripts/doc/) have unit tests that are collected by pytest (the discovered paths are configured in pytest.ini):

export GALACTICUS_EXEC_PATH=`pwd`
python -m pytest -v

These run quickly and do not require a built executable. Note that a bare pytest does not run the testSuite/ model regression tests described above — those are a separate suite.

The CI pipeline

The CI workflows live in .github/workflows/. The two most relevant to contributors are described below; the others are scheduled (cron) or utility jobs (for example linkCheck.yml, bibliographyUpdater.yml, cloudyTableUpdate.yml, slocReport.yml, and notarizeMacOS.yml).

Pull-request checks (prChecks.yml)

These fast checks run on every pull request, mostly over the files changed in the PR:

  • Validate-YAML / Validate-XML / Validate-Python-Scripts — syntax and well-formedness of changed YAML, XML, and Python files.

  • Fortran-Static-Analysis — scripts/aux/staticAnalyzer.py checks for common Fortran issues (empty constructors/destructors, duplicate variables in constructorAssign directives, and similar).

  • Validate-Docstrings-RST — scripts/doc/convertDocstringsToRST.py --check ensures every embedded docstring is reStructuredText (no old-style LaTeX).

  • Spell-Check-RST — builds the documentation with sphinxcontrib-spelling and reports possible misspellings as an advisory PR comment; add legitimate technical terms to aux/words.dict.

Build, test, and benchmark (cicd.yml)

The main pipeline builds Galacticus on Linux (and macOS), runs the preprocessor and Python regression tests, and executes model and benchmark runs. It makes use of the reusable testCode.yml and testModel.yml workflows and compares outputs against references (hdf5Diff.yml). This pipeline is heavier and takes considerably longer than the pull-request checks.

Before opening a pull request

At a minimum, build Galacticus, run the quick test, and run the Python unit tests; if your change touches the model, run the relevant testSuite/test-*.py script(s) too. See the repository’s CONTRIBUTING.md for the full checklist.