Testing your plugin#

HydroMT Core offers some functionalities that can help you test your plugins, as well as offering some examples on how to do it. Below are listed some tips and tricks to help test your plugin and to help your users test their code using your plugin.

Testing model components#

When you implement a ModelComponent we very strongly encourage you to write a test_equal method on your component that should test for deep equality. This means that it shouldn’t just test whether it’s literally the same python object, but should actually attempt to test whether the attributes and any data held by it are the same. This is not only important for your own testing, but also for your users, since they might want to use that functionality to test their own code.

The function should have the following signature:

def test_equal(self, other: "ModelComponent") -> tuple[bool, Dict[str, str]]:
    ...

The return signature is a boolean indicating whether the equality is true or not (as usual) and a dictionary. In this dictionary you can add any error messages of whatever keys you’d like. If possible, it is good for usability if you can report as many issues as possible in one go, so the user doesn’t have to run the tests over and over again.

As an example, suppose we have a RainfallComponent that must have a time dimension, and x dimension, and additionally the data inside it must have the correct crs. A function for that might look like this:

class RainfallComponent(ModelComponent):

    def test_equal(self, other: "ModelComponent") -> tuple[bool, Dict[str, str]]:
        errors: Dict[str, str] = {}
        if not isinstance(other, self.__class__):
            errors['type'] = """Other is not of type RainfallComponent and therfore
            cannot be equal"""
            return (False, errors)

        if not 'time' in other.dims().keys():
            errors['no_time_dim'] = """Component does not have the required time
            dimension"""
        else:
            if not self.data.sel(time=slice(None)).equals(other.data.sel(time=slice(None))):
                errors['time_dim_not_equal'] = "time dimension data is not equal"

        if not 'x' in other.dims().keys():
            errors['no_x_dim'] = """Component does not have the required x
            dimension"""
        else:
            if not self.data.sel(x=slice(None)).equals(other.data.sel(x=slice(None))):
                errors['x_dim_not_equal'] = "x dimension data is not equal"

        return len(errors) == 0, errors

Note that in the case the classes are not equal we return early since it probably doesn’t make sense to test for data equality on random classes. However, in the other cases we check both the time and x dimension at the same time. This gives users as much information about what is wrong as possible.

Testing models#

If all the components you use have a test_equal function defined, then testing for model equality should be relatively simple. The core base model also has a test_equal function defined that will test all the components against each other so if that is all you require you can simply use that function. If you wish to do additional checks you can override this method and simply call super().test_equal(other) and do whatever checks you’d like after that.

Testing your plugin as a whole#

Depending on how up to date with core developments you’d like to be it might be good to test against both the latest released version of hydromt core (which is presumably what your users will be using) as well as the latest version of Hydromt on the main branch (the development version as it were). This can help you anticipate if core might release features in the future that are incompatible for you and fix problems before they arise for your users. This also gives you the opportunity to file bug reports with core to fix things before they are released, which we highly encourage! If you use an environment/package manager that supports this such as pixi then you can do this by making a separate optional dependency-group for it, and simply run the test suite against the different environments in your CI.

Setting up a GitHub actions workflow#

To ensure your plugin is tested against multiple python versions, and both the latest released version of HydroMT core and the latest development version, you can set up a GitHub Actions workflow. This workflow will automatically run your test suite in different environments whenever you push changes to your repository.

Here’s a basic outline of how to set up a GitHub Actions workflow for your plugin:

1. In the pyproject.toml / pixi.toml of your plugin, ensure you have a test command defined that runs your tests. Additionally, you should define the features and environments for the various Python versions and dependency versions you want to test. A basic example:

[workspace]
authors = ["example-author <example@example.com>"]
channels = ["conda-forge"]
name = "my_plugin"
platforms = ["win-64"]
version = "0.1.0"

[tasks]
test = { cmd = "pytest tests" }

# Install your code in editable mode, dependencies here are installed in all environments
# Takes the dependencies from pyproject.toml, which will be overridden in the features below.
[pypi-dependencies]
my_plugin = { path = ".", editable = true }

# Define features for different Python versions and Hydromt versions
[feature.py310.dependencies]
python = "3.10.*"

[feature.py311.dependencies]
python = "3.11.*"

[feature.py312.dependencies]
python = "3.12.*"

[feature.py313.dependencies]
python = "3.13.*"

[feature.hydromt_dev.pypi-dependencies]
# Get the latest version from main
hydromt = { git = "https://github.com/Deltares/hydromt.git", branch = "main" }

[feature.hydromt_latest.pypi-dependencies]
# Replace with the actual stable version range you want to test against
hydromt = ">=1.0,<2.0"

# Define environments to combine features
[environments]
latest_310 = { features = ["py310", "hydromt_latest"], solve-group = "py310" }
latest_311 = { features = ["py311", "hydromt_latest"], solve-group = "py311" }
latest_312 = { features = ["py312", "hydromt_latest"], solve-group = "py312" }
latest_313 = { features = ["py313", "hydromt_latest"], solve-group = "py313" }

dev_310 = { features = ["py310", "hydromt_dev"], solve-group = "py310" }
dev_311 = { features = ["py311", "hydromt_dev"], solve-group = "py311" }
dev_312 = { features = ["py312", "hydromt_dev"], solve-group = "py312" }
dev_313 = { features = ["py313", "hydromt_dev"], solve-group = "py313" }

  1. Create a new file in your repository at .github/workflows/test.yml, add the following content and update it as needed:

name: Test Plugin

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, windows-latest]
        python-version: ['310', '311', '312', '313']
        hydromt-version: ['latest', 'dev']

    name: pytest ${{ matrix.hydromt-version }}-${{ matrix.python-version }} (${{ matrix.os }})
    runs-on: ${{ matrix.os }}

    concurrency:
      group: ${{ github.workflow }}-${{ matrix.os }}-${{ matrix.hydromt-version }}-${{ matrix.python-version }}-${{ github.ref }}
      cancel-in-progress: true

    steps:
      - uses: actions/checkout@v5
      - uses: prefix-dev/setup-pixi@v0.9.3
        with:
          pixi-version: "v0.59.0"
          environments: ${{ matrix.hydromt-version }}_${{ matrix.python-version }}

      - name: Run tests
        run: |
          pixi run --locked -e ${{ matrix.hydromt-version }}_${{ matrix.python-version }} test

  1. Commit and push the test.yml file to your repository.

This workflow will run your tests in different operating systems (Ubuntu and Windows), four different Python versions (3.10, 3.11, 3.12, and 3.13), and against both the latest released version of HydroMT and the latest development version. You can customize the workflow further based on your specific testing requirements.