Tip

For an interactive online version click here: Binder badge

Build an emission model#

This notebook demonstrates how to prepare DELWAQ D-Emissions model from scratch using the command line interace (CLI).

All lines in this notebook which start with ! are executed from the command line. Within the notebook environment the logging messages are shown after completion. You can also copy these lines and paste these in your shell to get more direct feedback.

HydroMT CLI build interface#

In HydroMT, you can interact with DELWAQ models either to be used by D-Emissions (demission) or D-Water Quality (delwaq).

Lets first check if the delwaq and demission models are recognized by hydromt

[1]:

# this should return "delwaq, demission, wflow, wflow_sediment"
!hydromt --models

model plugins:
 - wflow (hydromt_wflow 0.6.0)
 - wflow_sediment (hydromt_wflow 0.6.0)
 - delwaq (hydromt_delwaq 0.2.2.dev0)
 - demission (hydromt_delwaq 0.2.2.dev0)
generic models (hydromt 0.10.0):
 - grid_model
 - vector_model
 - mesh_model
 - network_model

Using the hydromt build API we can setup a complete model from scratch. Let’s get an overview of all the available options:

[2]:

!hydromt build --help

Usage: hydromt build [OPTIONS] MODEL MODEL_ROOT

  Build models from scratch.

  Example usage: --------------

  To build a wflow model for a subbasin using a point coordinates snapped to
  cells with upstream area >= 50 km2 hydromt build wflow /path/to/model_root
  -i /path/to/wflow_config.yml  -r "{'subbasin': [-7.24, 62.09], 'uparea':
  50}" -d deltares_data -d /path/to/data_catalog.yml -v

  To build a sfincs model based on a bbox hydromt build sfincs
  /path/to/model_root  -i /path/to/sfincs_config.yml  -r "{'bbox':
  [4.6891,52.9750,4.9576,53.1994]}"  -d /path/to/data_catalog.yml -v

Options:
  --opt TEXT               Method specific keyword arguments, see the method
                           documentation of the specific model for more
                           information about the arguments.
  -i, --config PATH        Path to hydroMT configuration file, for the model
                           specific implementation.
  -r, --region TEXT        Set the region for which to build the model, e.g.
                           {'subbasin': [-7.24, 62.09]}
  -d, --data TEXT          Path to local yaml data catalog file OR name of
                           predefined data catalog.
  --dd, --deltares-data    Flag: Shortcut to add the "deltares_data" catalog
  --fo, --force-overwrite  Flag: If provided overwrite existing model files
  --cache                  Flag: If provided cache tiled rasterdatasets
  -v, --verbose            Increase verbosity.
  -q, --quiet              Decrease verbosity.
  --help                   Show this message and exit.

Setup demission model base layers#

DELWAQ models are quite specific in the sense that they are usually added on top of an already existing hydrologic / hydraulic or hydrodynamic model. In HydroMT, for now, DELWAQ models (demission for D-Emissions and delwaq for D-Water Quality) can only be built on top of Wflow hydrologic models. You can find more information on this coupling in docs(coupling_wflow).

Here is how you can build a D-Emissions model:

[3]:

# NOTE: copy this line (without !) to your shell for more direct feedback
!hydromt build demission "./EM_test_base" -r "{'wflow': 'wflow_piave'}" --fo -vvv

2024-06-21 02:54:04,684 - build - log - DEBUG - Writing log messages to new file /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_test_base/hydromt.log.
2024-06-21 02:54:04,684 - build - log - INFO - HydroMT version: 0.10.0
2024-06-21 02:54:04,684 - build - main - INFO - Building instance of demission model at /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_test_base.
2024-06-21 02:54:04,684 - build - main - INFO - User settings:
2024-06-21 02:54:04,700 - build - model_api - INFO - Initializing demission model from hydromt_delwaq (v0.2.2.dev0).
2024-06-21 02:54:04,700 - build - model_api - INFO - setup_basemaps.region: {'wflow': 'wflow_piave'}
2024-06-21 02:54:04,700 - build - model_api - INFO - setup_basemaps.maps: ['rivmsk', 'lndslp', 'strord']
2024-06-21 02:54:04,706 - build - log - DEBUG - Writing log messages to new file /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/hydromt.log.
2024-06-21 02:54:04,706 - build - model_api - INFO - Initializing wflow model from hydromt_wflow (v0.6.0).
2024-06-21 02:54:04,706 - build - data_catalog - INFO - Parsing data catalog from /usr/share/miniconda/envs/hydromt-delwaq/lib/python3.11/site-packages/hydromt_wflow/data/parameters_data.yml
2024-06-21 02:54:04,716 - build - basin_mask - DEBUG - Parsed region (kind=model): {'mod': <hydromt_wflow.wflow.WflowModel object at 0x7fa875e26fd0>}
2024-06-21 02:54:04,716 - build - demission - INFO - Preparing EM basemaps from hydromodel.
2024-06-21 02:54:04,718 - build - model_api - DEBUG - Model config read from /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/wflow_sbm.toml
2024-06-21 02:54:04,718 - build - wflow - INFO - Read grid from /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/staticmaps.nc
2024-06-21 02:54:05,537 - build - model_api - ERROR - Default config file not found at /usr/share/miniconda/envs/hydromt-delwaq/lib/python3.11/site-packages/hydromt_delwaq/data/demission/demission.inp
2024-06-21 02:54:05,537 - build - demission - INFO - Write model data to /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_test_base
2024-06-21 02:54:05,541 - build - delwaq - INFO - Writing staticmap files.
2024-06-21 02:54:05,654 - build - model_api - DEBUG - No geoms data found, skip writing.
2024-06-21 02:54:05,654 - build - delwaq - INFO - Writing model config to file.
2024-06-21 02:54:05,654 - build - delwaq - INFO - Writing hydromap files.
2024-06-21 02:54:05,719 - build - demission - INFO - Writting geometry file in root/staticdata
2024-06-21 02:54:05,723 - build - demission - DEBUG - No forcing data found, skip writing.

The example above means the following: run hydromt build with:

  • demission : i.e. build a demission model

  • ./EM_test_base : output model folder

  • -r or --region "{'wflow': 'wflow_piave'}" : derive the model from the existing wflow model located in wflow_piave folder. All REGION options are described in the docs. For DELWAQ, the only possible options is from an existing wflow model.

  • -vv : give some extra verbosity (2 * v) to display feedback on screen. Now debug messages are provided.

NOTE: As we did not specify a model configuration, only the base maps (grid, segment IDs) have been setup using default parameters. To build a complete model we need the use a configuration file (yml).

Model setup configuration#

The configuration file (yml) contains the model setup configuration and determines which components are build and in which order and optionally sets non-default arguments for each component. This configuration is passed to hydromt using -i <path_to_configuration_file>. We have prepared several example files which are available in the model repository examples folder and from the docs(build_configuration).

Each header as shown between [...] (e.g. [setup_basemaps]) corresponds to a model component. All model components are explained in the docs(model_components).

Almost each DELWAQ model is unique depending on which substances and sources are included but also what kind of emission data is available. For these reasons, there is no default build configuration file for a demission/delwaq model and still some manual steps are required to build and run a D-Emissions model with HydroMT. You can learn more about these steps in the docs(hydromt_D-Emissions).

We will load an example demission build configuration file for inspection:

[4]:

fn_ini = 'delwaq_build_EM.yml'
with open(fn_ini, 'r') as f:
    txt = f.read()
print(txt)

setup_basemaps:
  maps:
    - rivmsk
    - lndslp
    - strord
    - SoilThickness
    - thetaS

setup_monitoring:
  mon_areas: riverland

setup_emission_raster:
  emission_fn: ghs_pop_2015
  scale_method: average
  fillna_method: zero
  area_division: True

setup_emission_vector:
  emission_fn: gdp_world
  col2raster: gdp
  rasterize_method: value

Some explanations for the sections the ini file above:

  • setup_basemaps: prepares the first delwaq model layers (grid, segment ID) and for EM also the geometry.

  • setup_monitoring: prepares no monitoring points and monitoring areas by compartments (emissions only here).

  • setup_emission_raster: prepares emissions grid from a raster file. You can repeat this section several times for several rasters. Here we will prepare population data from the global ghs_pop_2015 dataset.

  • setup_emission_vector: prepares emissions grid from a vector file. You can repeat this section several times for several rasters. Here we will prepare GDP data from the global gdp_world data.

For more information on all the options used (resampling method, NaN values handling…), please check the docs(model_components)

Data for model setup#

The <...>_fn arguments correspond to a data source from the DataCatalog which is based on a yml-file with references to the data paths/urls and how the data should be read. This file can be provided to hydromt using -d <path_to_yml_file>.

By default some example data for the Piave basin will be downloaded to ~/.hydromt_data/ which is also used for this example. An overview of the available example data is provided here This example data is a based on the data which are available from the Deltares p-drive. If you have acces to this drive, a pre-configured catalog file can be loaded using -d deltares_data.

More background how to write a data catalog yml file can be found in the hydromt core docs

Setup complete D-Emissions model#

[5]:

# NOTE: copy this line (without !) to your shell for more direct feedback
!hydromt build demission "./EM_test_full" -r "{'wflow': 'wflow_piave'}" -i delwaq_build_EM.yml -d artifact_data --fo -vv

2024-06-21 02:54:08,047 - build - log - DEBUG - Writing log messages to new file /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_test_full/hydromt.log.
2024-06-21 02:54:08,047 - build - log - INFO - HydroMT version: 0.10.0
2024-06-21 02:54:08,047 - build - main - INFO - Building instance of demission model at /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_test_full.
2024-06-21 02:54:08,047 - build - main - INFO - User settings:
2024-06-21 02:54:08,076 - build - data_catalog - INFO - Reading data catalog artifact_data latest
2024-06-21 02:54:08,076 - build - data_catalog - INFO - Parsing data catalog from /home/runner/.hydromt_data/artifact_data/v0.0.9/data_catalog.yml
2024-06-21 02:54:08,693 - build - model_api - INFO - Initializing demission model from hydromt_delwaq (v0.2.2.dev0).
2024-06-21 02:54:08,693 - build - model_api - INFO - setup_basemaps.region: {'wflow': 'wflow_piave'}
2024-06-21 02:54:08,693 - build - model_api - INFO - setup_basemaps.maps: ['rivmsk', 'lndslp', 'strord', 'SoilThickness', 'thetaS']
2024-06-21 02:54:08,700 - build - log - DEBUG - Appending log messages to file /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/hydromt.log.
2024-06-21 02:54:08,700 - build - model_api - INFO - Initializing wflow model from hydromt_wflow (v0.6.0).
2024-06-21 02:54:08,700 - build - data_catalog - INFO - Parsing data catalog from /usr/share/miniconda/envs/hydromt-delwaq/lib/python3.11/site-packages/hydromt_wflow/data/parameters_data.yml
2024-06-21 02:54:08,709 - build - basin_mask - DEBUG - Parsed region (kind=model): {'mod': <hydromt_wflow.wflow.WflowModel object at 0x7f764ffe84d0>}
2024-06-21 02:54:08,709 - build - demission - INFO - Preparing EM basemaps from hydromodel.
2024-06-21 02:54:08,711 - build - model_api - DEBUG - Model config read from /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/wflow_sbm.toml
2024-06-21 02:54:08,711 - build - wflow - INFO - Read grid from /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/wflow_piave/staticmaps.nc
2024-06-21 02:54:09,531 - build - model_api - ERROR - Default config file not found at /usr/share/miniconda/envs/hydromt-delwaq/lib/python3.11/site-packages/hydromt_delwaq/data/demission/demission.inp
2024-06-21 02:54:09,531 - build - model_api - INFO - setup_monitoring.mon_points: None
2024-06-21 02:54:09,531 - build - model_api - INFO - setup_monitoring.mon_areas: riverland
2024-06-21 02:54:09,531 - build - delwaq - INFO - Setting monitoring points and areas
2024-06-21 02:54:09,531 - build - delwaq - INFO - No monitoring points set in the config file, skipping
2024-06-21 02:54:09,546 - build - model_api - INFO - setup_emission_raster.emission_fn: ghs_pop_2015
2024-06-21 02:54:09,546 - build - model_api - INFO - setup_emission_raster.scale_method: average
2024-06-21 02:54:09,546 - build - model_api - INFO - setup_emission_raster.fillna_method: zero
2024-06-21 02:54:09,546 - build - model_api - INFO - setup_emission_raster.fillna_value: 0.0
2024-06-21 02:54:09,546 - build - model_api - INFO - setup_emission_raster.area_division: True
2024-06-21 02:54:09,546 - build - delwaq - INFO - Preparing 'ghs_pop_2015' map.
2024-06-21 02:54:09,551 - build - rasterdataset - INFO - Reading ghs_pop_2015 raster data from /home/runner/.hydromt_data/artifact_data/v0.0.9/data.tar/ghs_pop_2015.tif
2024-06-21 02:54:09,580 - build - rasterdataset - DEBUG - Clip to [11.775, 45.808, 12.742, 46.692] (epsg:4326))
2024-06-21 02:54:09,701 - build - emissions - INFO - Deriving None using average resampling (nodata=-200.0).
2024-06-21 02:54:09,742 - build - model_api - INFO - setup_emission_vector.emission_fn: gdp_world
2024-06-21 02:54:09,742 - build - model_api - INFO - setup_emission_vector.col2raster: gdp
2024-06-21 02:54:09,742 - build - model_api - INFO - setup_emission_vector.rasterize_method: value
2024-06-21 02:54:09,742 - build - delwaq - INFO - Preparing 'gdp_world' map.
2024-06-21 02:54:09,750 - build - geodataframe - INFO - Reading gdp_world vector data from /home/runner/.hydromt_data/artifact_data/v0.0.9/data.tar/gdp_world.gpkg
2024-06-21 02:54:10,101 - build - geodataframe - DEBUG - Clip intersects [11.775, 45.808, 12.742, 46.692] (EPSG:4326)
2024-06-21 02:54:10,183 - build - demission - INFO - Write model data to /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_test_full
2024-06-21 02:54:10,194 - build - delwaq - INFO - Writing staticmap files.
2024-06-21 02:54:10,238 - build - model_api - DEBUG - Writing file geoms/monareas.geojson
2024-06-21 02:54:10,257 - build - model_api - DEBUG - Writing file geoms/basins.geojson
2024-06-21 02:54:10,263 - build - delwaq - INFO - Writing model config to file.
2024-06-21 02:54:10,263 - build - delwaq - INFO - Writing hydromap files.
2024-06-21 02:54:10,320 - build - demission - INFO - Writting geometry file in root/staticdata
2024-06-21 02:54:10,323 - build - demission - DEBUG - No forcing data found, skip writing.

With this example we build a complete demission model including emission data. Compared to the previous hydromt build we have added: * -i delwaq_build_EM.yml : setup configuration file including all components to build and their arguments.

Next we check which files and folders have been created:

  • config: basic configuration setups for the Delwaq input file .inp. Names of the ASCII files start with the corresponding block in the .inp file.

  • dynamicdata: Delwaq time-dependant data (hydrological fluxes) in binary format.

  • fews: additional files for connection with Delft-FEWS (not implemented yet)

  • hydromodel: data extracted to connect the hydrological model (wflow) and Delwaq (tif format). These files are not mandatory for DELWAQ but produced extra from HydroMT.

  • staticdata: static (spatial but non time-dependant) data for Delwaq (typically emission data) in binary format. A copy of the data in NetCDF format is also available for easier visualization.

  • geoms: related geometry files (geojson format). These files are not mandatory for DELWAQ but produced extra from HydroMT.

[6]:

import os
root = 'EM_test_full'
for path, _, files in os.walk(root):
    print(path)
    for name in files:
        if name.endswith('.xml'):
            continue
        print(f' - {name}')

EM_test_full
 - hydromt.log
EM_test_full/dynamicdata
EM_test_full/staticdata
 - ptiddown.dat
 - gdp_world.dat
 - river.dat
 - porosity.dat
 - ghs_pop_2015.dat
 - SoilThickness.dat
 - streamorder.dat
 - slope.dat
 - staticdata.nc
EM_test_full/hydromodel
 - basins.tif
 - river.tif
 - ptid.tif
 - modelmap.tif
 - resarea.tif
 - rivwth.tif
 - rivlen.tif
 - elevtn.tif
 - ldd.tif
 - ptiddown.tif
 - lakearea.tif
 - rivarea.tif
EM_test_full/geoms
 - monareas.geojson
 - basins.geojson
EM_test_full/config
 - B7_geometry.bin
 - B3_nrofseg.inc
 - B7_geometry.inc
 - B7_surf.inc
 - B2_monareas.inc
 - B5_boundlist.inc
 - B4_nrofexch.inc
 - B7_geometry-parameters.inc
 - B2_stations.inc
 - B2_nrofmon.inc
 - B3_attributes.inc
EM_test_full/fews

You can see that the dynamicdata folder (hydrological input) does not contain any file yet. We will add them in the update forcing example notebook.

Visualize and/or inspect model schematization#

  • The delwaq plot example notebook contains scripts to visualize your model (staticdata, dynamicdata, hydromodel and geoms).

In the meantime, feel free to have a look at some configuration files that were prepared by HydroMT. You can change the file name from the example code below.

[7]:

import os
model_path = './EM_test_full'
fn_config = 'config/B3_nrofseg.inc'
with open(os.path.join(model_path,fn_config), 'r') as f:
    txt = f.read()
print(txt)

1653 ; nr of segments

In the config folder, you can also see three geometry files:

  • B7_geometry-parameters.inc (Headers for the binary version)

  • B7_geometry.bin (Binary version)

  • B7_geometry.inc (ASCII version)

These files are specific for the D-Emissions plugin and specifies geometry properties of the D-Emissions segments. These properties are:

  • Segment area (m2)

  • Fraction of the segment area that is paved

  • Fraction of the segment area that is unpaved

  • Fraction of the segment area that is open water

[8]:

import os
model_path = './EM_test_full'
fn_config = 'config/B7_geometry-parameters.inc'
with open(os.path.join(model_path,fn_config), 'r') as f:
    txt = f.read()
print(txt)

PARAMETERS TotArea fPaved fUnpaved fOpenWater
On this page
Edit on GitHub
Show Source