Skip to content

Commit

Permalink
expand the test documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
@wolfv
wolfv committed Jan 5, 2024
1 parent 5e1e749 commit 9a60165
Show file tree
Hide file tree
Showing 4 changed files with 238 additions and 148 deletions.
16 changes: 7 additions & 9 deletions docs/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -118,8 +118,8 @@ requirements:
run_constrained:
- xsimd >=8.0.3,<10

test:
commands:
tests:
- script:
- if: unix or emscripten
then:
- test -d ${PREFIX}/include/xtensor
Expand Down Expand Up @@ -181,13 +181,11 @@ requirements:
- python 3.10
- typing_extensions >=4.0.0,<5.0.0

test:
imports:
- rich
commands:
- pip check
requires:
- pip
tests:
- python:
imports:
- rich
pip_check: true

about:
homepage: https://github.com/Textualize/rich
Expand Down
245 changes: 156 additions & 89 deletions docs/recipe_file.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,11 @@ The reason for a new spec are:
- no full Jinja2 support: no conditional or `{% set ...` support, only string
interpolation. Variables can be set in the toplevel "context" which is valid
YAML
- Jinja string interpolation needs to be preceded by a dollar sign at the beginning of a string,
e.g. `- ${{ version }}` in order for it to be valid YAML
- Selectors use a YAML dictionary style (vs. comments in conda-build). Instead of `- somepkg #[osx]`
we use
- Jinja string interpolation needs to be preceded by a dollar sign at the
beginning of a string, e.g. `- ${{ version }}` in order for it to be valid
YAML
- Selectors use a YAML dictionary style (vs. comments in conda-build). Instead
of `- somepkg #[osx]` we use
```yaml
if: osx
then:
Expand All @@ -60,9 +61,10 @@ The recipe spec has the following parts:
Spec reference
--------------

The spec is also made available through a JSON Schema (which is used for validation).
The schema (and pydantic source file) can be found in this repository: https://github.com/prefix-dev/recipe-format.
To use with VSCode or other IDEs, start the document with the following line:
The spec is also made available through a JSON Schema (which is used for
validation). The schema (and pydantic source file) can be found in this
repository: https://github.com/prefix-dev/recipe-format. To use with VSCode or
other IDEs, start the document with the following line:

```yaml
# yaml-language-server: $schema=https://raw.githubusercontent.com/prefix-dev/recipe-format/main/schema.json
Expand Down Expand Up @@ -193,9 +195,11 @@ source:
lfs: true # defaults to false
```

Note: `git_rev` may not be available within commit depth range, consider avoiding use of both simultaneously.
Note: `git_rev` may not be available within commit depth range, consider
avoiding use of both simultaneously.

When you want to use git-lfs, you need to set `lfs: true`. This will also pull the lfs files from the repository.
When you want to use git-lfs, you need to set `lfs: true`. This will also pull
the lfs files from the repository.

#### Source from a local path

Expand All @@ -208,8 +212,9 @@ source is copied to the work directory before building.
use_gitignore: false # (defaults to true)
```

By default, all files in the local path that are ignored by git are also ignored by rattler-build.
You can disable this behavior by setting `use_gitignore` to `false`.
By default, all files in the local path that are ignored by git are also ignored
by rattler-build. You can disable this behavior by setting `use_gitignore` to
`false`.

#### Patches

Expand All @@ -227,10 +232,10 @@ Patches may optionally be applied to the source.
#### Destination path

Within boa's work directory, you may specify a particular folder to place source
into. `rattler-build` will always drop you into the same folder (build folder/work), but
it's up to you whether you want your source extracted into that folder, or
nested deeper. This feature is particularly useful when dealing with multiple
sources, but can apply to recipes with single sources as well.
into. `rattler-build` will always drop you into the same folder (build
folder/work), but it's up to you whether you want your source extracted into
that folder, or nested deeper. This feature is particularly useful when dealing
with multiple sources, but can apply to recipes with single sources as well.

```yaml
source:
Expand Down Expand Up @@ -498,118 +503,180 @@ This is the version bound consistent with CentOS 6. Software built against glibc
mamba tell the user that a given package can't be installed if their system
glibc version is too old.

Test section
------------
Tests section
-------------

If this section exists or if there is a `run_test.[py,pl,sh,bat]` file in the
recipe, the package is installed into a test environment after the build is
finished and the tests are run there.
Rattler-build supports 4 different types of tests. The "script" test installs
the package and runs a list of commands. The python test attempts to import a
list of python modules and runs `pip check`. The downstream test runs the tests
of a downstream package that reverse depends on the package being built.

<!--
### Test files
And lastly, the package content test checks if the built package contains the
mentioned items.

Test files that are copied from the _recipe_ into the temporary test
directory and are needed during testing.
The tests section is a list of these items:

```yaml
test:
files:
- test-data.txt
tests:
- script:
- echo "hello world"
requirements:
run:
- pytest
files:
source:
- test-data.txt
- python:
imports:
- bsdiff4
pip_check: true # this is the default
- downstream: numpy
```

### Source files
### Script test

The script test has 3 top-level keys: `script`, `files` and `requirements`. Only
the `script` key is required.

Test files that are copied from the _source work directory_ into the
temporary test directory and are needed during testing (note that the
source work directory is otherwise not available at all during testing).
#### Test commands

Commands that are run as part of the test.

```yaml
test:
source_files:
- test-data.txt
- some/directory
- some/directory/pattern*.sh
tests:
- script:
- echo "hello world"
- bsdiff4 -h
- bspatch4 -h
```

-->
#### Extra Test Files

### Test requirements
Test files that are copied from the source work directory into the temporary
test directory and are needed during testing (note that the source work
directory is otherwise not available at all during testing).

In addition to the runtime requirements, you can specify requirements needed
during testing. The runtime requirements that you specified in the "run" section
described above are automatically included during testing.
You can also include files that come from the `recipe` folder. They are copied
into the test directory as well.

At test execution time, the test directory is the current working directory.

```yaml
test:
requires:
- nose
tests:
- script:
- ls
files:
source:
- myfile.txt
- tests/
- some/directory/pattern*.sh
recipe:
- extra-file.txt
```

### Test commands
-->

Commands that are run as part of the test.
#### Test requirements

In addition to the runtime requirements, you can specify requirements needed
during testing. The runtime requirements that you specified in the "run" section
described above are automatically included during testing (because the built
package is installed like regular).

In the `build` section you can specify additional requirements that are only
needed on the build system for cross-compilation (e.g. emulators or compilers).

```yaml
test:
commands:
- bsdiff4 -h
- bspatch4 -h
tests:
- script:
- echo "hello world"
requirements:
build:
- myemulator
run:
- nose
```

### Python imports
### Python tests

For this test type you can list a set of Python modules that need to be
importable. The test will fail if any of the modules cannot be imported.

The test will also automatically run `pip check` to check for any broken
dependencies. This can be disabled by setting `pip_check: false` in the YAML.

List of Python modules or packages that will be imported in the test
environment.

```yaml
test:
imports:
- bsdiff4
tests:
- python:
imports:
- bsdiff4
- bspatch4
pip_check: true # can be left out because this is the default
```

This would be equivalent to having a `run_test.py` with the following:
Internally this will write a small Python script that imports the modules:

```python
import bsdiff4
import bspatch4
```

### Check for package-contents

Checks if the built package contains the mentioned items.
Checks if the built package contains the mentioned items. These checks are executed directly at
the end of the build process to make sure that all expected files are present in the package.

```yaml
test:
package-contents:
# checks for the existence of files inside $PREFIX or %PREFIX%
# or, checks that there is at least one file matching the specified `glob`
# pattern inside the prefix
files:
- etc/libmamba/test.txt
- etc/libmamba
- etc/libmamba/*.mamba.txt

# checks for the existence of `mamba/api/__init__.py` inside of the
# Python site-packages directory (note: also see Python import checks)
site_packages:
- mamba.api


# looks in $PREFIX/bin/mamba for unix and %PREFIX%\Library\bin\mamba.exe on Windows
# note: also check the `commands` and execute something like `mamba --help` to make
# sure things work fine
bins:
- mamba

# searches for `$PREFIX/lib/libmamba.so` or `$PREFIX/lib/libmamba.dylib` on Linux or macOS,
# on Windows for %PREFIX%\Library\lib\mamba.dll & %PREFIX%\Library\bin\mamba.bin
libs:
- mamba

# searches for `$PREFIX/include/libmamba/mamba.hpp` on unix, and
# on Windows for `%PREFIX%\Library\include\mamba.hpp`
includes:
- libmamba/mamba.hpp
tests:
- package-contents:
# checks for the existence of files inside $PREFIX or %PREFIX%
# or, checks that there is at least one file matching the specified `glob`
# pattern inside the prefix
files:
- etc/libmamba/test.txt
- etc/libmamba
- etc/libmamba/*.mamba.txt

# checks for the existence of `mamba/api/__init__.py` inside of the
# Python site-packages directory (note: also see Python import checks)
site_packages:
- mamba.api


# looks in $PREFIX/bin/mamba for unix and %PREFIX%\Library\bin\mamba.exe on Windows
# note: also check the `commands` and execute something like `mamba --help` to make
# sure things work fine
bin:
- mamba

# searches for `$PREFIX/lib/libmamba.so` or `$PREFIX/lib/libmamba.dylib` on Linux or macOS,
# on Windows for %PREFIX%\Library\lib\mamba.dll & %PREFIX%\Library\bin\mamba.bin
lib:
- mamba

# searches for `$PREFIX/include/libmamba/mamba.hpp` on unix, and
# on Windows for `%PREFIX%\Library\include\mamba.hpp`
includes:
- libmamba/mamba.hpp
```
### Downstream tests
!!! warning
Downstream tests are not yet implemented in `rattler-build`.

A downstream test can mention a single package that has a dependency on the package being built.
The test will install the package and run the tests of the downstream package with our current
package as a dependency.

Sometimes downstream packages do not resolve. In this case, the test is ignored.

```yaml
tests:
- downstream: numpy
```

Outputs section
---------------
Expand Down
Loading

0 comments on commit 9a60165

Please sign in to comment.