Tip
Add (global) emission data#
Once you have a DELWAQ model, you may want to update your model in order to add new emission data, add sample locations, use different hydrological forcing data, create and run different scenarios etc.
With HydroMT, you can easily read your model and update one or several components of your model using the update function of the command line interface (CLI). Here are the steps and some examples on how to update and add global emission data to your model.
All lines in this notebook which starts with ! are executed from the command line. Within the notebook environment the logging messages are shown after completion. You can also copy these lines and paste them in your shell to get more feedback.
Import packages#
In this notebook, we will use some functions of HydroMT to prepare configuration and catalog files and plot the new emission data of the updated model. Here are the libraries to import to realize these steps.
[1]:
import numpy as np
[2]:
# for plotting
import matplotlib.pyplot as plt
import cartopy.io.img_tiles as cimgt
import cartopy.crs as ccrs
proj = ccrs.PlateCarree() # plot projection
[3]:
# import hydromt
import hydromt
from hydromt_delwaq import DelwaqModel, DemissionModel
[4]:
# setup logging
from hydromt.log import setuplog
logger = setuplog("update_model_emission", log_level=10)
2024-06-21 02:53:32,433 - update_model_emission - log - INFO - HydroMT version: 0.10.0
hydroMT CLI update interface#
Using the hydromt update API, we can update one or several components of an already existing DELWAQ model. Let’s get an overview of the available options:
[5]:
# Print the options available from the update command
! hydromt update --help
Usage: hydromt update [OPTIONS] MODEL MODEL_ROOT
Update a specific component of a model.
Set an output directory to copy the edited model to a new folder, otherwise
maps are overwritten.
Example usage: --------------
Update (overwrite!) landuse-landcover based maps in a Wflow model: hydromt
update wflow /path/to/model_root -c setup_lulcmaps --opt lulc_fn=vito -d
/path/to/data_catalog.yml -v
Update Wflow model components outlined in an .yml configuration file and
write the model to a directory: hydromt update wflow /path/to/model_root -o
/path/to/model_out -i /path/to/wflow_config.yml -d
/path/to/data_catalog.yml -v
Options:
-o, --model-out DIRECTORY Output model folder. Maps in MODEL_ROOT are
overwritten if left empty.
-i, --config PATH Path to hydroMT configuration file, for the model
specific implementation.
-c, --components TEXT Model methods from configuration file to run
--opt TEXT Method specific keyword arguments, see the method
documentation of the specific model for more
information about the arguments.
-d, --data TEXT Path to local yaml data catalog file OR name of
predefined data catalog.
--dd, --deltares-data Flag: Shortcut to add the "deltares_data" catalog
--fo, --force-overwrite Flag: If provided overwrite existing model files
--cache Flag: If provided cache tiled rasterdatasets
-q, --quiet Decrease verbosity.
-v, --verbose Increase verbosity.
--help Show this message and exit.
Adding global data to the model#
Using HydroMT, it is quite easy to add additional emission data from already available global data. Here we will see an example where we add a cattle map (glw_cattle) from the GLW 3 data.
You can see a list of all readily available data in HydroMT in the docs.
[6]:
#Let's see what are the attributes of the glw_cattle data source
data_catalog = hydromt.DataCatalog("artifact_data")
data_catalog["glw_cattle"]
Downloading file 'v0.0.9/data_catalog.yml' from 'https://raw.githubusercontent.com/Deltares/hydromt/main/data/catalogs/artifact_data/v0.0.9/data_catalog.yml' to '/home/runner/.hydromt_data/artifact_data'.
Downloading data from 'https://github.com/DirkEilander/hydromt-artifacts/releases/download/v0.0.9/data.tar.gz' to file '/home/runner/.hydromt_data/artifact_data/v0.0.9/data.tar.gz'.
SHA256 hash of downloaded file: 32de5b95c171628547f303d7f65d53cbb1b9da9af4834717c8efff93fe55aad4
Use this value as the 'known_hash' argument of 'pooch.retrieve' to ensure that the file hasn't changed if it is downloaded again in the future.
Untarring contents of '/home/runner/.hydromt_data/artifact_data/v0.0.9/data.tar.gz' to '/home/runner/.hydromt_data/artifact_data/v0.0.9/data.tar'
[6]:
crs: 4326
data_type: RasterDataset
driver: raster
meta:
category: socio economic
paper_doi: 10.7910/DVN/GIVQ75
paper_ref: Gilbert at al (2018)
source_author: glw (Gridded Livestock of World 3 Dataverse)
source_license: CC 4.0
source_url: https://dataverse.harvard.edu/dataverse/glw_3
source_version: GLW 3, last downloaded 2020-06-11
path: /home/runner/.hydromt_data/artifact_data/v0.0.9/data.tar/glw_cattle.tif
Here we can see that our cattle grid is a raster .tif file. The important properties for HydroMT are:
path: path to where the data is stored.
data_type:
HydroMT DataCatalog typeeither RasterDataset (gridded data), GeoDataFrame (vector data) or GeoDataset (point timeseries). Our file is a gridded tif file so RasterDataset.driver: driver used to open the data. Either raster (GDAL compliant raster file), netcdf (NetCDF file), zarr (zarr file) or vector (GDAL compliant vector file). Our file is a tif raster so raster.
The rest are just additional meta information on the data source. You can notice that the crs property is not mentionned. This is because this data source has the same crs as the default hydroMT one EPS4326.
Preparing the configuration file#
As the glw_cattle data is a raster data, we will add it to our model using the [setup_emission_raster] component of HydroMT Delwaq.
Let’s prepare a HydroMT configuration file with our options for preparing the cattle emission raster. All available options are available in the docs(setup_emission_raster).
[7]:
# Dictionnary with all the components and options we want to update
emission_raster_options = {
'setup_emission_raster': {
'emission_fn': 'glw_cattle',
'scale_method': 'average',
'fillna_method': 'zero',
'area_division': True
},
}
# Save it to a hydroMT ini file
fn_yml = "delwaq_update_emission_cattle.yml"
hydromt.config.configwrite(fn_yml, emission_raster_options)
# Open the file and visualize the content
with open(fn_yml, 'r') as f:
txt = f.read()
print(txt)
setup_emission_raster:
emission_fn: glw_cattle
scale_method: average
fillna_method: zero
area_division: true
Some explanations about the option we chose here:
emission_fn: name of the cattle data source in hydroMT Data Catalog.
scale_method: method for resampling from the data resolution (10km x 10km for GLW) to the model resolution (5km5km, same as the wflow_piave model resolution). Available methods are nearest, average, mode* (for classification).
fillna_method: method to fill NaN values in the data sources either nearest, zero or value to fill with a user defined value.
area_division: if needed do the resampling in head/m2 (True) instead of head (False). The GLW cattle grid presents the number of cows per grid cell. To resample to a different grid resolution, we need to resample cow densities instead. After resampling, the cattle grid will be again in cows per model grid cell.
Updating the model with the cattle grid#
[8]:
# NOTE: copy this line (without !) to your shell for more direct feedback
! hydromt update demission EM_piave -o ./EM_piave_extended -i delwaq_update_emission_cattle.yml -d artifact_data --fo -vv
2024-06-21 02:53:39,426 - update - log - DEBUG - Writing log messages to new file /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_piave_extended/hydromt.log.
2024-06-21 02:53:39,426 - update - log - INFO - HydroMT version: 0.10.0
2024-06-21 02:53:39,426 - update - main - INFO - Updating demission model at /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_piave (r).
2024-06-21 02:53:39,426 - update - main - INFO - Output dir: /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_piave_extended
2024-06-21 02:53:39,426 - update - main - INFO - User settings:
2024-06-21 02:53:39,573 - update - data_catalog - INFO - Reading data catalog artifact_data latest
2024-06-21 02:53:39,573 - update - data_catalog - INFO - Parsing data catalog from /home/runner/.hydromt_data/artifact_data/v0.0.9/data_catalog.yml
2024-06-21 02:53:40,162 - update - log - DEBUG - Writing log messages to new file /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_piave/hydromt.log.
2024-06-21 02:53:40,162 - update - model_api - INFO - Initializing demission model from hydromt_delwaq (v0.2.2.dev0).
2024-06-21 02:53:40,162 - update - model_api - ERROR - Model config file not found at /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_piave/demission.inp
2024-06-21 02:53:40,162 - update - model_api - DEBUG - Reading model file monareas.
2024-06-21 02:53:40,191 - update - model_api - DEBUG - Reading model file staticdata.
2024-06-21 02:53:40,236 - update - model_api - DEBUG - Reading model file basins.
nodata value missing for /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_piave/hydromodel/rivarea.tif
2024-06-21 02:53:40,364 - update - model_api - DEBUG - Reading model file staticdata.
2024-06-21 02:53:40,372 - update - model_grid - WARNING - Replacing grid map: river
2024-06-21 02:53:40,372 - update - model_grid - WARNING - Replacing grid map: slope
2024-06-21 02:53:40,373 - update - model_grid - WARNING - Replacing grid map: streamorder
2024-06-21 02:53:40,374 - update - model_grid - WARNING - Replacing grid map: SoilThickness
2024-06-21 02:53:40,375 - update - model_grid - WARNING - Replacing grid map: porosity
2024-06-21 02:53:40,376 - update - model_grid - WARNING - Replacing grid map: ptiddown
2024-06-21 02:53:40,376 - update - model_grid - WARNING - Replacing grid map: monareas
2024-06-21 02:53:40,377 - update - model_grid - WARNING - Replacing grid map: ghs_pop_2015
2024-06-21 02:53:40,378 - update - model_grid - WARNING - Replacing grid map: gdp_world
2024-06-21 02:53:40,378 - update - demission - INFO - Model read
2024-06-21 02:53:40,379 - update - log - DEBUG - Appending log messages to file /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_piave_extended/hydromt.log.
2024-06-21 02:53:40,386 - update - model_api - INFO - setup_emission_raster.emission_fn: glw_cattle
2024-06-21 02:53:40,386 - update - model_api - INFO - setup_emission_raster.scale_method: average
2024-06-21 02:53:40,386 - update - model_api - INFO - setup_emission_raster.fillna_method: zero
2024-06-21 02:53:40,386 - update - model_api - INFO - setup_emission_raster.fillna_value: 0.0
2024-06-21 02:53:40,386 - update - model_api - INFO - setup_emission_raster.area_division: True
2024-06-21 02:53:40,386 - update - delwaq - INFO - Preparing 'glw_cattle' map.
2024-06-21 02:53:40,390 - update - rasterdataset - INFO - Reading glw_cattle raster data from /home/runner/.hydromt_data/artifact_data/v0.0.9/data.tar/glw_cattle.tif
2024-06-21 02:53:40,406 - update - rasterdataset - DEBUG - Clip to [11.775, 45.808, 12.742, 46.692] (epsg:4326))
2024-06-21 02:53:40,526 - update - emissions - INFO - Deriving None using average resampling (nodata=-1.7e+308).
2024-06-21 02:53:40,566 - update - demission - INFO - Write model data to /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_piave_extended
2024-06-21 02:53:40,593 - update - delwaq - INFO - Writing staticmap files.
2024-06-21 02:53:40,656 - update - model_api - DEBUG - Writing file geoms/monareas.geojson
2024-06-21 02:53:40,667 - update - model_api - DEBUG - Writing file geoms/basins.geojson
2024-06-21 02:53:40,673 - update - delwaq - INFO - Writing model config to file.
2024-06-21 02:53:40,673 - update - delwaq - INFO - Writing hydromap files.
nodata value missing for /home/runner/work/hydromt_delwaq/hydromt_delwaq/docs/_examples/EM_piave_extended/hydromodel/rivarea.tif
2024-06-21 02:53:40,791 - update - demission - DEBUG - No forcing data found, skip writing.
The example above means the following: run hydromt with:
update delwaq: i.e. update a delwaq modelEM_piave: original model folder-o ./EM_piave_extended: output updated model folder-i delwaq_update_emission_cattle.yml: hydroMT configuration file containing the components and options to update-d artifact_data: predefined or local data catalog to use. Here the openly available predefined artifact_data catalog-v: give some extra verbosity (2 * v) to display feedback on screen. Now debug messages are provided.
Visualization of the new emission map#
We can now plot our newly created cattle grid.
[9]:
# Load the original and updated model with hydromt
mod = DemissionModel(root='EM_piave_extended', mode='r')
[10]:
# Edit the lines below to change the emission map and its colormap
emissionmap = 'glw_cattle'
colormap = 'Reds'
#Load the emission map
da = mod.grid[emissionmap].raster.mask_nodata()
da.attrs.update(long_name=emissionmap, units='-')
#Plot
# we assume the model maps are in the geographic CRS EPSG:4326
proj = ccrs.PlateCarree()
# adjust zoomlevel and figure size to your basis size & aspect
zoom_level = 10
figsize=(10, 8)
# initialize image with geoaxes
fig = plt.figure(figsize=figsize)
ax = fig.add_subplot(projection=proj)
extent = np.array(da.raster.box.buffer(0.02).total_bounds)[[0, 2, 1, 3]]
ax.set_extent(extent, crs=proj)
# add sat background image
ax.add_image(cimgt.QuadtreeTiles(), zoom_level, alpha=0.5)
## plot emission map
cmap = plt.cm.get_cmap(colormap)
kwargs = dict(cmap=cmap)
# plot 'normal' elevation
da.plot(transform=proj, ax=ax, zorder=1, cbar_kwargs=dict(aspect=30, shrink=.8), **kwargs)
ax.xaxis.set_visible(True)
ax.yaxis.set_visible(True)
ax.set_ylabel(f"latitude [degree north]")
ax.set_xlabel(f"longitude [degree east]")
_ = ax.set_title(f"Delwaq emission map")
