Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 31 additions & 10 deletions docs/recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ A recipe is comprised of the following yaml files in a directory:
* `post-install`: _optional_ a script to run after Spack has been executed to build the stack.
* `pre-install`: _optional_ a script to run before any packages have been built.

[](){#ref-recipes-config}
## Configuration

```yaml title="config.yaml"
Expand All @@ -26,17 +27,18 @@ spack:
packages:
repo: https://github.com/spack/spack-packages.git
commit: develop
modules: true
description: "HPC development tools for building MPI applications with the GNU compiler toolchain"
default-view: develop
version: 2
```

* `name`: a plain text name for the environment
* `store`: the location where the environment will be mounted.
* `spack`: which spack and package repositories to use for installation.
* `modules`: (_deprecated_) _optional_ enable/disable module file generation.
* `description`: _optional_ a string that describes the environment (default empty).
* `default-view`: _default = null_ the name of a uenv view to load if no view is explicitly requested by the user. See the documentation for [default views][ref-recipes-default-view]. If no default view is specified, none will be set.
* `version`: _default = 1_ the version of the uenv recipe (see below)
* `modules`: (_deprecated_) _optional_ enable/disable module file generation.

!!! note "uenv recipe versions"
Stackinator 6 introduces breaking changes to the uenv recipe format, introduced to support Spack v1.0.
Expand Down Expand Up @@ -350,27 +352,34 @@ For example, the `views` description:
```yaml
cuda-env:
views:
default:
full:
no-python:
exclude:
- 'python'
```

will configure two views:

* `default`: a view of all the software in the environment using the default settings of Spack.
* `no-python`: everything in the default view, except any versions of `python`.
* `full`: a view of all the software in the environment using the full settings of Spack.
* `no-python`: everything in the `full` view, except any versions of `python`.

!!! warning "only name views default if they are loaded by default"
It is possible to name views `default`, and there are uenv images like `prgenv-gnu` that follow this practice for legacy reasons.

However the name default can be confusing for users because it implies that the view is loaded by default.
However, [default views][ref-recipes-default-view] must be explicitly set in [`config.yaml`][ref-recipes-config].

Stackinator provides some additional options that are not provided by Spack, to fine tune the view, that can be set in the `uenv:` field:

```yaml
cuda-env:
views:
uenv:
add_compilers: true
prefix_paths:
LD_LIBRARY_PATH: [lib, lib64]
env_vars:
my-view:
uenv:
add_compilers: true
prefix_paths:
LD_LIBRARY_PATH: [lib, lib64]
env_vars:
set:
- WOMBAT: null
- NOCOLOR: "1"
Expand All @@ -392,6 +401,18 @@ cuda-env:
!!! info
See the [interfaces documentation](interfaces.md#environment-views) for more information about how the environment views are provided.

[](){#ref-recipes-default-view}
#### Setting default views

!!! info "available in uenv 9.3 and later"
Default views are only loaded in recent versions of uenv.
Older uenv versions will ignore the default view - it is not an error to load a uenv that provides a default view using uenv 9.2 and older.

It is possible to specify a view that will be loaded automatically if no view is specified, by setting the `default-view` field in [`config.yaml` file][ref-recipes-config].

* a default view is only used if it is explicitly specified in `config.yaml`.
* if a uenv is started with a view specified, the default view is ignored.

#### Setting environment variables with `env_vars`

The `views:<view_name>:uenv:env_vars` field can be used to further fine-tune the environment variables that are set when the view is started.
Expand Down
1 change: 1 addition & 0 deletions stackinator/builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@ def environment_meta(self, recipe):
meta["name"] = conf["name"]
meta["description"] = conf["description"]
meta["views"] = recipe.environment_view_meta
meta["default-view"] = recipe.default_view
meta["mount"] = str(recipe.mount)
modules = None
if recipe.with_modules:
Expand Down
27 changes: 19 additions & 8 deletions stackinator/recipe.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,6 @@ def __init__(self, args):
self._logger.error(f"modules.yaml:{self.with_modules}")
raise RuntimeError("conflicting modules configuration detected")

# optional packages.yaml file
packages_path = self.path / "packages.yaml"
self._logger.debug(f"opening {packages_path}")
self.packages = None
if packages_path.is_file():
with packages_path.open() as fid:
self.packages = yaml.load(fid, Loader=yaml.Loader)

self._logger.debug("creating packages")

# load recipe/packages.yaml -> recipe_packages (if it exists)
Expand Down Expand Up @@ -169,6 +161,21 @@ def __init__(self, args):
schema.EnvironmentsValidator.validate(raw)
self.generate_environment_specs(raw)

# check that the default view exists (if one has been set)
self._default_view = self.config["default-view"]
if self._default_view is not None:
available_views = [view["name"] for env in self.environments.values() for view in env["views"]]
# add the modules and spack views to the list of available views
if self.with_modules:
available_views.append("modules")
available_views.append("spack")
if self._default_view not in available_views:
self._logger.error(
f"The default-view {self._default_view} is not the name of a view in the environments.yaml "
"definition (one of {[name for name in available_views]}"
)
raise RuntimeError("Ivalid default-view in the recipe.")

# optional mirror configurtion
mirrors_path = self.path / "mirrors.yaml"
if mirrors_path.is_file():
Expand Down Expand Up @@ -286,6 +293,10 @@ def with_modules(self) -> bool:
def find_spack_version(self, develop):
return "1.0"

@property
def default_view(self):
return self._default_view

@property
def environment_view_meta(self):
# generate the view meta data that is presented in the squashfs image meta data
Expand Down
25 changes: 7 additions & 18 deletions stackinator/schema/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -46,24 +46,6 @@
}
}
},
"mirror" : {
"type" : "object",
"additionalProperties": false,
"default": {"enable": true, "key": null},
"properties" : {
"enable" : {
"type": "boolean",
"default": true
},
"key" : {
"oneOf": [
{"type" : "string"},
{"type" : "null"}
],
"default": null
}
}
},
"modules" : {
"type": "boolean"
},
Expand All @@ -74,6 +56,13 @@
],
"default": null
},
"default-view" : {
"oneOf": [
{"type" : "string"},
{"type" : "null"}
],
"default": null
},
"version" : {
"type": "number",
"default": 1,
Expand Down
2 changes: 0 additions & 2 deletions unittests/recipes/base-nvgpu/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,4 @@ spack:
commit: 6408b51
packages:
repo: https://github.com/spack/spack-packages.git
mirror:
enable: false
version: 2
3 changes: 0 additions & 3 deletions unittests/recipes/cache/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,4 @@ spack:
commit: 6408b51
packages:
repo: https://github.com/spack/spack-packages.git
mirror:
key: /scratch/e1000/bcumming/secret/spack-key.gpg
enable: true
version: 2
4 changes: 0 additions & 4 deletions unittests/test_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@ def test_config_yaml(yaml_path):
assert raw["store"] == "/user-environment"
assert raw["spack"]["commit"] is None
assert raw["spack"]["packages"]["commit"] is None
assert raw["mirror"] == {"enable": True, "key": None}
assert raw["description"] is None

# no spack:commit
Expand All @@ -63,7 +62,6 @@ def test_config_yaml(yaml_path):
schema.ConfigValidator.validate(raw)
assert raw["spack"]["commit"] is None
assert raw["spack"]["packages"]["commit"] is not None
assert raw["mirror"] == {"enable": True, "key": None}
assert raw["description"] is None

# no spack:packages:commit
Expand All @@ -83,7 +81,6 @@ def test_config_yaml(yaml_path):
schema.ConfigValidator.validate(raw)
assert raw["spack"]["commit"] == "develop"
assert raw["spack"]["packages"]["commit"] is None
assert raw["mirror"] == {"enable": True, "key": None}
assert raw["description"] is None

# full config
Expand All @@ -94,7 +91,6 @@ def test_config_yaml(yaml_path):
assert raw["spack"]["commit"] == "6408b51"
assert raw["spack"]["packages"]["commit"] == "v2025.07.0"
assert raw["modules"] == False # noqa: E712
assert raw["mirror"] == {"enable": True, "key": "/home/bob/veryprivate.key"}
assert raw["description"] == "a really useful environment"

# unsupported old version
Expand Down
6 changes: 0 additions & 6 deletions unittests/yaml/config.defaults.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,5 @@ spack:
repo: https://github.com/spack/spack-packages.git
# default: None == no `git checkout` command
#commit: 6408b51
#mirror:
# default None
#key: None
# default True
#enable: True
# default True
#modules: True
version: 2
3 changes: 0 additions & 3 deletions unittests/yaml/config.full.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,6 @@ spack:
packages:
repo: https://github.com/spack/spack-packages.git
commit: v2025.07.0
mirror:
key: /home/bob/veryprivate.key
enable: True
modules: False
description: "a really useful environment"
version: 2
Loading