Tip

For an interactive online version click here: Binder badge

Build a water quality model#

This notebook demonstrates how to prepare DELWAQ D-Water Quality model from scratch using the command line interace (CLI).

All lines in this notebook which start with ! are executed from the command line. Within the notebook environment the logging messages are shown after completion. You can also copy these lines and paste these in your shell to get more direct feedback.

HydroMT CLI build interface#

In HydroMT, you can interact with DELWAQ models either to be used by D-Emissions (demission) or D-Water Quality (delwaq).

Lets first check if the delwaq and demission models are recognized by hydromt

[1]:

# this should return "delwaq, demission, wflow, wflow_sediment"
!hydromt --models

model plugins:
 - wflow (hydromt_wflow 0.6.0)
 - wflow_sediment (hydromt_wflow 0.6.0)
 - delwaq (hydromt_delwaq 0.2.2.dev0)
 - demission (hydromt_delwaq 0.2.2.dev0)
generic models (hydromt 0.10.0):
 - grid_model
 - vector_model
 - mesh_model
 - network_model

Using the hydromt build API we can setup a complete model from scratch. Let’s get an overview of all the available options:

[2]:

!hydromt build --help

Usage: hydromt build [OPTIONS] MODEL MODEL_ROOT

  Build models from scratch.

  Example usage: --------------

  To build a wflow model for a subbasin using a point coordinates snapped to
  cells with upstream area >= 50 km2 hydromt build wflow /path/to/model_root
  -i /path/to/wflow_config.yml  -r "{'subbasin': [-7.24, 62.09], 'uparea':
  50}" -d deltares_data -d /path/to/data_catalog.yml -v

  To build a sfincs model based on a bbox hydromt build sfincs
  /path/to/model_root  -i /path/to/sfincs_config.yml  -r "{'bbox':
  [4.6891,52.9750,4.9576,53.1994]}"  -d /path/to/data_catalog.yml -v

Options:
  --opt TEXT               Method specific keyword arguments, see the method
                           documentation of the specific model for more
                           information about the arguments.
  -i, --config PATH        Path to hydroMT configuration file, for the model
                           specific implementation.
  -r, --region TEXT        Set the region for which to build the model, e.g.
                           {'subbasin': [-7.24, 62.09]}
  -d, --data TEXT          Path to local yaml data catalog file OR name of
                           predefined data catalog.
  --dd, --deltares-data    Flag: Shortcut to add the "deltares_data" catalog
  --fo, --force-overwrite  Flag: If provided overwrite existing model files
  --cache                  Flag: If provided cache tiled rasterdatasets
  -v, --verbose            Increase verbosity.
  -q, --quiet              Decrease verbosity.
  --help                   Show this message and exit.

Setup delwaq model base layers#

DELWAQ models are quite specific in the sense that they are usually added on top of an already existing hydrologic / hydraulic or hydrodynamic model. In HydroMT, for now, DELWAQ models (demission for D-Emissions and delwaq for D-Water Quality) can only be built on top of Wflow hydrologic models. You can find more information on this coupling in docs(coupling_wflow).

Here is how you can build a D-Water Quality model:

[3]:

# NOTE: copy this line (without !) to your shell for more direct feedback
!hydromt build delwaq "./WQ_test_base" -r "{'wflow': 'wflow_piave'}" --fo -vvv

2024-06-21 02:54:18,793 - build - log - DEBUG - Writing log messages to new file /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/WQ_test_base/hydromt.log.
2024-06-21 02:54:18,793 - build - log - INFO - HydroMT version: 0.10.0
2024-06-21 02:54:18,793 - build - main - INFO - Building instance of delwaq model at /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/WQ_test_base.
2024-06-21 02:54:18,793 - build - main - INFO - User settings:
2024-06-21 02:54:18,809 - build - model_api - INFO - Initializing delwaq model from hydromt_delwaq (v0.2.2.dev0).
2024-06-21 02:54:18,809 - build - model_api - INFO - setup_basemaps.region: {'wflow': 'wflow_piave'}
2024-06-21 02:54:18,809 - build - model_api - INFO - setup_basemaps.mask: basins
2024-06-21 02:54:18,809 - build - model_api - INFO - setup_basemaps.surface_water: sfw
2024-06-21 02:54:18,809 - build - model_api - INFO - setup_basemaps.boundaries: ['bd']
2024-06-21 02:54:18,809 - build - model_api - INFO - setup_basemaps.fluxes: ['sfw>sfw', 'bd>sfw']
2024-06-21 02:54:18,809 - build - model_api - INFO - setup_basemaps.maps: ['rivmsk', 'lndslp', 'strord', 'N']
2024-06-21 02:54:18,816 - build - log - DEBUG - Appending log messages to file /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/hydromt.log.
2024-06-21 02:54:18,816 - build - model_api - INFO - Initializing wflow model from hydromt_wflow (v0.6.0).
2024-06-21 02:54:18,816 - build - data_catalog - INFO - Parsing data catalog from /usr/share/miniconda/envs/hydromt-delwaq/lib/python3.11/site-packages/hydromt_wflow/data/parameters_data.yml
2024-06-21 02:54:18,825 - build - basin_mask - DEBUG - Parsed region (kind=model): {'mod': <hydromt_wflow.wflow.WflowModel object at 0x7f1db365fa10>}
2024-06-21 02:54:18,825 - build - delwaq - INFO - Preparing WQ basemaps from hydromodel.
2024-06-21 02:54:18,828 - build - model_api - DEBUG - Model config read from /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/wflow_sbm.toml
2024-06-21 02:54:18,828 - build - wflow - INFO - Read grid from /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/staticmaps.nc
2024-06-21 02:54:19,634 - build - delwaq - WARNING - nrofseg object in self.pointer should be an integer.
2024-06-21 02:54:19,634 - build - delwaq - WARNING - boundaries object in self.pointer should be a list of names.
2024-06-21 02:54:19,655 - build - model_api - ERROR - Default config file not found at /usr/share/miniconda/envs/hydromt-delwaq/lib/python3.11/site-packages/hydromt_delwaq/data/delwaq/delwaq.inp
2024-06-21 02:54:19,655 - build - delwaq - INFO - Write model data to /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/WQ_test_base
2024-06-21 02:54:19,665 - build - delwaq - INFO - Writing staticmap files.
2024-06-21 02:54:19,781 - build - model_api - DEBUG - No geoms data found, skip writing.
2024-06-21 02:54:19,781 - build - delwaq - INFO - Writing model config to file.
2024-06-21 02:54:19,781 - build - delwaq - INFO - Writing hydromap files.
2024-06-21 02:54:19,845 - build - delwaq - INFO - Writting pointer file in root/config
2024-06-21 02:54:19,858 - build - delwaq - DEBUG - No forcing data found, skip writing.

The example above means the following: run hydromt build with:

  • delwaq : i.e. build a delwaq model

  • ./WQ_test_base : output model folder

  • -r or --region "{'wflow': 'wflow_piave'}" : derive the model from the existing wflow model located in wflow_piave folder. All REGION options are described in the docs. For DELWAQ, the only possible options is from an existing wflow model.

  • -vv : give some extra verbosity (2 * v) to display feedback on screen. Now debug messages are provided.

NOTE: As we did not specify a model configuration, only the base maps (grid, segment IDs) have been setup using default parameters. To build a complete model we need the use a configuration file (yml).

Model setup configuration#

The configuration file (yml) contains the model setup configuration and determines which components are build and in which order and optionally sets non-default arguments for each component. This configuration is passed to hydromt using -i <path_to_configuration_file>. We have prepared several example files which are available in the model repository examples folder and from the docs(build_configuration).

Each header as shown between [...] (e.g. [setup_basemaps]) corresponds to a model component. All model components are explained in the docs(model_components).

Almost each DELWAQ model is unique depending on which substances and sources are included but also what kind of emission data is available. For these reasons, there is no default build configuration file for a delwaq model and still some manual steps are required to build and run a D-Water Quality model with HydroMT. You can learn more about these steps in the docs(hydromt_D-Water Quality).

We will load an example delwaq build configuration file for inspection:

[4]:

fn_ini = 'delwaq_build_WQ.yml'
with open(fn_ini, 'r') as f:
    txt = f.read()
print(txt)

global:
  data_libs:
    - local_sources.yml

setup_basemaps:
  mask: rivers
  surface_water: sfw
  boundaries:
    - land
    - soil
    - sett
  fluxes:
    - sfw>sfw
    - land>sfw
    - soil>sfw
    - sfw>sett
  maps:
    - rivmsk
    - lndslp
    - strord
    - N
    - SoilThickness
    - thetaS

setup_monitoring:
  mon_points: wflow_piave/staticgeoms/gauges.geojson
  mon_areas: subcatch

setup_hydrology_forcing:
  hydro_forcing_fn: wflow_output_wq
  starttime: "2010-02-03 00:00:00"
  endtime: "2010-02-10 00:00:00"
  timestepsecs: 86400
  add_volume_offset: True
  override:
    - sfw>sfw

write:

write_waqgeom:

Some explanations for the sections in the ini file above:

  • global: options true for every model components of delwaq. You can also add local data libaries in data_libs instead of in the command line here.

  • setup_basemaps: prepares the first delwaq model layers (grid, segment ID) and the pointer based on the provided list of fluxes to and from the surface water. It can also prepare some staticdata for delwaq using maps from the hydrological model.

  • setup_monitoring: prepares monitoring points using the gauges locations of the wflow model and the monitoring areas by subcatchments.

  • setup_hydrology_forcing: prepares hydrological data from wflow outputs. Information on the wflow_output source is available in the local_sources.yml.

For more information on all the options used (resampling method, NaN values handling…), please check the docs(model_components)

Data for model setup#

The <...>_fn arguments correspond to a data source from the DataCatalog which is based on a yml-file with references to the data paths/urls and how the data should be read. This file can be provided to hydromt using -d <path_to_yml_file>.

By default some example data for the Piave basin will be downloaded to ~/.hydromt_data/ which is also used for this example. An overview of the available example data is provided here This example data is a based on the data which are available from the Deltares p-drive. If you have acces to this drive, a pre-configured catalog file can be loaded using -d deltares_data.

More background how to write a data catalog yml file can be found in the hydromt core docs

Setup complete D-Water Quality model#

[5]:

# NOTE: copy this line (without !) to your shell for more direct feedback
!hydromt build delwaq "./WQ_test_full" -r "{'wflow': 'wflow_piave'}" -i delwaq_build_WQ.yml -d artifact_data --fo -vv

2024-06-21 02:54:22,169 - build - log - DEBUG - Writing log messages to new file /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/WQ_test_full/hydromt.log.
2024-06-21 02:54:22,169 - build - log - INFO - HydroMT version: 0.10.0
2024-06-21 02:54:22,169 - build - main - INFO - Building instance of delwaq model at /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/WQ_test_full.
2024-06-21 02:54:22,169 - build - main - INFO - User settings:
2024-06-21 02:54:22,187 - build - data_catalog - INFO - Parsing data catalog from /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/local_sources.yml
2024-06-21 02:54:22,204 - build - data_catalog - INFO - Reading data catalog artifact_data latest
2024-06-21 02:54:22,204 - build - data_catalog - INFO - Parsing data catalog from /home/runner/.hydromt_data/artifact_data/v0.0.9/data_catalog.yml
2024-06-21 02:54:22,791 - build - model_api - INFO - Initializing delwaq model from hydromt_delwaq (v0.2.2.dev0).
2024-06-21 02:54:22,791 - build - model_api - INFO - setup_basemaps.region: {'wflow': 'wflow_piave'}
2024-06-21 02:54:22,791 - build - model_api - INFO - setup_basemaps.mask: rivers
2024-06-21 02:54:22,791 - build - model_api - INFO - setup_basemaps.surface_water: sfw
2024-06-21 02:54:22,791 - build - model_api - INFO - setup_basemaps.boundaries: ['land', 'soil', 'sett']
2024-06-21 02:54:22,791 - build - model_api - INFO - setup_basemaps.fluxes: ['sfw>sfw', 'land>sfw', 'soil>sfw', 'sfw>sett']
2024-06-21 02:54:22,791 - build - model_api - INFO - setup_basemaps.maps: ['rivmsk', 'lndslp', 'strord', 'N', 'SoilThickness', 'thetaS']
2024-06-21 02:54:22,798 - build - log - DEBUG - Appending log messages to file /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/hydromt.log.
2024-06-21 02:54:22,798 - build - model_api - INFO - Initializing wflow model from hydromt_wflow (v0.6.0).
2024-06-21 02:54:22,798 - build - data_catalog - INFO - Parsing data catalog from /usr/share/miniconda/envs/hydromt-delwaq/lib/python3.11/site-packages/hydromt_wflow/data/parameters_data.yml
2024-06-21 02:54:22,807 - build - basin_mask - DEBUG - Parsed region (kind=model): {'mod': <hydromt_wflow.wflow.WflowModel object at 0x7f3457a42dd0>}
2024-06-21 02:54:22,807 - build - delwaq - INFO - Preparing WQ basemaps from hydromodel.
2024-06-21 02:54:22,810 - build - model_api - DEBUG - Model config read from /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/wflow_sbm.toml
2024-06-21 02:54:22,810 - build - wflow - INFO - Read grid from /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/staticmaps.nc
2024-06-21 02:54:23,641 - build - delwaq - WARNING - nrofseg object in self.pointer should be an integer.
2024-06-21 02:54:23,641 - build - delwaq - WARNING - boundaries object in self.pointer should be a list of names.
2024-06-21 02:54:23,663 - build - model_api - ERROR - Default config file not found at /usr/share/miniconda/envs/hydromt-delwaq/lib/python3.11/site-packages/hydromt_delwaq/data/delwaq/delwaq.inp
2024-06-21 02:54:23,664 - build - model_api - INFO - setup_monitoring.mon_points: /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/staticgeoms/gauges.geojson
2024-06-21 02:54:23,664 - build - model_api - INFO - setup_monitoring.mon_areas: subcatch
2024-06-21 02:54:23,664 - build - delwaq - INFO - Setting monitoring points and areas
2024-06-21 02:54:23,674 - build - geodataframe - INFO - Reading  vector data from /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/staticgeoms/gauges.geojson
2024-06-21 02:54:23,704 - build - geodataframe - DEBUG - Clip intersects [11.775, 45.808, 12.742, 46.692] (EPSG:4326)
2024-06-21 02:54:23,710 - build - delwaq - INFO - Gauges locations read from /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/staticgeoms/gauges.geojson
2024-06-21 02:54:23,721 - build - model_api - INFO - setup_hydrology_forcing.hydro_forcing_fn: wflow_output_wq
2024-06-21 02:54:23,721 - build - model_api - INFO - setup_hydrology_forcing.starttime: 2010-02-03 00:00:00
2024-06-21 02:54:23,721 - build - model_api - INFO - setup_hydrology_forcing.endtime: 2010-02-10 00:00:00
2024-06-21 02:54:23,721 - build - model_api - INFO - setup_hydrology_forcing.timestepsecs: 86400
2024-06-21 02:54:23,721 - build - model_api - INFO - setup_hydrology_forcing.add_volume_offset: True
2024-06-21 02:54:23,721 - build - model_api - INFO - setup_hydrology_forcing.min_volume: 0.1
2024-06-21 02:54:23,721 - build - model_api - INFO - setup_hydrology_forcing.override: ['sfw>sfw']
2024-06-21 02:54:23,721 - build - delwaq - INFO - Setting dynamic data from hydrology source wflow_output_wq.
2024-06-21 02:54:23,725 - build - rasterdataset - INFO - Reading wflow_output_wq netcdf data from /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/run_default/output.nc
2024-06-21 02:54:23,768 - build - rasterdataset - DEBUG - Clip to [11.775, 45.808, 12.742, 46.692] (epsg:4326))
2024-06-21 02:54:23,770 - build - rasterdataset - DEBUG - Convert units for 3 variables.
2024-06-21 02:54:23,821 - build - forcing - WARNING - Flux sfw>sett not found in hydro_forcing_fn. Using zeros.
2024-06-21 02:54:23,883 - build - delwaq - INFO - Write model data to /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/WQ_test_full
2024-06-21 02:54:23,897 - build - delwaq - INFO - Writing staticmap files.
2024-06-21 02:54:24,111 - build - model_api - DEBUG - Writing file geoms/basins.geojson
2024-06-21 02:54:24,124 - build - model_api - DEBUG - Writing file geoms/monpoints.geojson
2024-06-21 02:54:24,130 - build - model_api - DEBUG - Writing file geoms/monareas.geojson
2024-06-21 02:54:24,135 - build - delwaq - INFO - Writing model config to file.
2024-06-21 02:54:24,136 - build - delwaq - INFO - Writing hydromap files.
2024-06-21 02:54:24,193 - build - delwaq - INFO - Writting pointer file in root/config
2024-06-21 02:54:24,211 - build - delwaq - INFO - Writing dynamicmap files.
Writing dynamic data: 100%|███████████████████████| 9/9 [00:00<00:00, 42.17it/s]
2024-06-21 02:54:24,426 - build - delwaq - INFO - Writting waqgeom.nc file

With this example we build a complete delwaq model including hydrological data. Compared to the previous hydromt build we have added: * -i delwaq_build_WQ.yml : setup configuration file including all components to build and their arguments.

Next we check which files and folders have been created:

  • config: basic configuration setups for the Delwaq input file .inp. Names of the ASCII files start with the corresponding block in the .inp file.

  • dynamicdata: Delwaq time-dependant data (hydrological fluxes) in binary format.

  • fews: additional files for connection with Delft-FEWS (not implemented yet)

  • hydromodel: data extracted to connect the hydrological model (wflow) and Delwaq (tif format). These files are not mandatory for Delwaq but produced extra from HydroMT.

  • staticdata: static (spatial but non time-dependant) data for Delwaq (typically emission data) in binary format. A copy of the data in NetCDF format is also available for easier visualization.

  • geoms: related geometry files (geojson format). These files are not mandatory for Delwaq but produced extra from HydroMT.

[6]:

import os
root = 'WQ_test_full'
for path, _, files in os.walk(root):
    print(path)
    for name in files:
        if name.endswith('.xml'):
            continue
        print(f' - {name}')

WQ_test_full
 - hydromt.log
WQ_test_full/dynamicdata
 - volume.dat
 - flow.dat
WQ_test_full/staticdata
 - length.dat
 - river.dat
 - latitude_grid.dat
 - porosity.dat
 - width.dat
 - SoilThickness.dat
 - streamorder.dat
 - manning.dat
 - surface.dat
 - bankfull_volume.dat
 - slope.dat
 - staticdata.nc
WQ_test_full/hydromodel
 - basins.tif
 - river.tif
 - ptid.tif
 - modelmap.tif
 - resarea.tif
 - rivwth.tif
 - rivlen.tif
 - elevtn.tif
 - ldd.tif
 - ptiddown.tif
 - lakearea.tif
 - rivarea.tif
WQ_test_full/geoms
 - monareas.geojson
 - basins.geojson
 - monpoints.geojson
WQ_test_full/config
 - B2_timers_only.inc
 - B7_volume.inc
 - B7_flow.inc
 - B3_nrofseg.inc
 - B2_timers.inc
 - B3_waqgeom.nc
 - B4_pointer.inc
 - B2_monareas.inc
 - B5_boundlist.inc
 - B2_outputtimes.inc
 - B1_timestamp.inc
 - B4_pointer.poi
 - B4_nrofexch.inc
 - B2_sysclock.inc
 - B2_stations.inc
 - B2_nrofmon.inc
 - B2_stations-balance.inc
 - B3_attributes.inc
WQ_test_full/fews

You can see that we didn’t create any emission data here. In some cases, you may require some additional GIS data in order to run a D-Water Quality model, for example, when you do not need to run the D-Emissions model beforehand, but include point sources that emit pollutants into the surface water directly. When this is the case, you can still use the various [setup_emission_*] components of hydromt_delwaq.

Visualize and/or inspect model schematization#

  • The delwaq plot example notebook contains scripts to visualize your model (staticdata, dynamicdata, hydromodel and geoms).

In the meantime, feel free to have a look at some configuration files that were prepared by HydroMT. You can change the file name from the example code below.

[7]:

import os
model_path = './WQ_test_full'
fn_config = 'config/B3_nrofseg.inc'
with open(os.path.join(model_path,fn_config), 'r') as f:
    txt = f.read()
print(txt)

315 ; nr of segments

In the config folder, you can also see two pointer files:

  • B4_pointer.poi (Binary version)

  • B4_pointer.inc (ASCII version)

You can also see that a NetCDF file was created: B3_waqgeom.nc. This file can be used to produce NetCDF outputs directly when running D-Emissions or D-Water Quality but also to visualize the model in FEWS.

On this page
Edit on GitHub
Show Source