{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9a73590b",
   "metadata": {},
   "source": [
    "## Build a Delft3D FM 1D model from scratch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee510c6e",
   "metadata": {},
   "source": [
    "This notebook demonstrates how to prepare a **1D Delft3D FM** model from scratch using the command line interace (CLI)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b96aa15",
   "metadata": {},
   "source": [
    "All lines in this notebook which start with `!` are executed from the command line. Within the notebook environment the logging messages are shown after completion. You can also copy these lines and paste these in your shell to get more direct feedback."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "451c8168",
   "metadata": {},
   "source": [
    "### HydroMT CLI build interface"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c70853e",
   "metadata": {},
   "source": [
    "Lets first check if the Delft3D FM model is recognized by HydroMT:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "40470302",
   "metadata": {},
   "outputs": [],
   "source": [
    "# this should return \"dflowfm\" to build DFlowFM components of Delft3D FM\n",
    "# as well as the generic HydroMT models: \"model\" and \"example_model\"\n",
    "!hydromt --models"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d9b5a20",
   "metadata": {},
   "source": [
    "Using the **HydroMT build** API we can setup a complete model from scratch. Let's get an overview of all the available options:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d39e4a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt build --help"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7584dcae",
   "metadata": {},
   "source": [
    "### Data for model setup"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dad35651",
   "metadata": {},
   "source": [
    "Most of the time, the `<...>_fn` arguments correspond to a data source from the `DataCatalog` which is based on a yml-file with references to the data paths/urls and how the data should be read. This file can be provided to HydroMT using `-d <path_to_yml_file>`. \n",
    "\n",
    "By default, the pre-defined `artifact_data` catalog containing some example data for the Piave basin will be downloaded to `~/.hydromt_data/` which is also used for this example. An overview of the available example data is provided [here](https://deltares.github.io/hydromt/latest/user_guide/data_existing_cat.html).\n",
    "This example data is a based on the data which is available from the Deltares internal storage location: the p-drive. \n",
    "If you have acces to this drive, a pre-configured catalog file can be loaded using the `-d deltares_data` pre-defined catalog. \n",
    "\n",
    "More information on how to write a data catalog yml file can be found in the [HydroMT core docs](https://deltares.github.io/hydromt/latest/user_guide/data_prepare_cat.html) \n",
    "\n",
    "> **NOTE**: In HydroMT-Delft3D FM, an additional data catalog with all default parameters values is always used [parameters_data.yml](https://github.com/Deltares/hydromt_delft3dfm/tree/main/hydromt_delft3dfm/data/parameters_data.yml). It contains default values for 1D network elements and structures as well as default 2D landuse parameter mapping tables."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebd31242",
   "metadata": {},
   "source": [
    "### Model setup configuration"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb8282c9",
   "metadata": {},
   "source": [
    "The HydroMT configuration file contains the model setup configuration and determines which methods are used to prepare the different components of a Delft3D FM model and in which order and optionally sets non-default arguments for each method. This configuration is passed to HydroMT using `-i <path_to_configuration_file>`. We have prepared several example yml-files which are available in the model repository [examples folder](https://github.com/Deltares/hydromt_delft3dfm/tree/main/examples) and from the [docs (building a model)](https://deltares.github.io/hydromt_delft3dfm/latest/user_guide/dflowfm_build). \n",
    "\n",
    "Each section, before indent, (e.g. setup_rivers_from_dem) corresponds to a model method. All model methods are explained in the [docs (model components)](https://deltares.github.io/hydromt_delft3dfm/latest/user_guide/dflowfm_model_setup.html). The `global` section contains direct model initialisation properties."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc3fd44d",
   "metadata": {},
   "source": [
    "We will load the default dflowfm build 1D yaml file for inspection:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e88d4aed",
   "metadata": {},
   "outputs": [],
   "source": [
    "fn_yml = \"dflowfm_build1d.yml\"\n",
    "with open(fn_yml, \"r\") as f:\n",
    "    txt = f.read()\n",
    "print(txt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9607a239",
   "metadata": {},
   "source": [
    "Looking at this file, you see that we will prepare the following information for our model:\n",
    "\n",
    "- [global](https://deltares.github.io/hydromt_delft3dfm/latest/_generated/hydromt_delft3dfm.DFlowFMModel.html#hydromt_delft3dfm.DFlowFMModel): Our model will be defined in the projected CRS WGS84 EPSG 3857. TO ensure the 1D network is connected, we will allow snapping of 25 meters using argument network_snap_offset. The 1D computational grid for the open water system will be created at 40 meters distance using argument openwater_computation_node_distance. \n",
    "\n",
    "Then the model building ``steps`` are:\n",
    "\n",
    "- [setup_rivers_from_dem](https://deltares.github.io/hydromt_delft3dfm/latest/_generated/hydromt_delft3dfm.DFlowFMModel.setup_rivers_from_dem.html): We will derive the 1D rivers lines based on the *MERIT Hydro DEM* and that intersects with the region bounding box [12.4331, 46.4661, 12.5212, 46.5369]. River roughness will be a Manning constant of 25.0 and we will use data from *rivers_lin2019* database to prepare rectangular cross-sections. \n",
    "- [setup_pipes](https://deltares.github.io/hydromt_delft3dfm/latest/_generated/hydromt_delft3dfm.DFlowFMModel.setup_pipes.html): We will add pipes by assuming that they are located under the roads, with a standard length of 50 meters. The roads are defined in the *grip_roads* data and the standard length are defined in the argument spacing. Roughness will be a WhiteColeBrook constant of 0.003 and we will use a circular cross-sections of 0.5m. The pipes invert levels will be derived from the *MERIT Hydro DEM* assuming a constant depth of the pipe underground of 2 meters. \n",
    "- [setup_manholes](https://deltares.github.io/hydromt_delft3dfm/latest/_generated/hydromt_delft3dfm.DFlowFMModel.setup_manholes.html): For a pipe network, manholes are present at pipe connections. Here we do not have specific input data locations for our pipes, so manholes will be generated based on a set of standards specified in manholes_defaults_fn.\n",
    "- [setup_1dboundary](https://deltares.github.io/hydromt_delft3dfm/latest/_generated/hydromt_delft3dfm.DFlowFMModel.setup_1dboundary.html): Finally as a default, we will apply both upstream and downstream boundaries of our rivers with a constant waterlevel of -2.0 meter a.s.l\n",
    "\n",
    "Feel free to use the links of each components to know more about how each function works and what are all the available settings."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0242689",
   "metadata": {},
   "source": [
    "> **NOTE**: \n",
    "  In this example, we apply region argument in both setup_rivers_from_dem and setup_pipes. They are identical in this example but one can alter them to have different region for rivers and for pipes. This allows the user to customize their model as much as possible, as the extent of the 1D river can differ from the extent of the 1D urban network or the 2D inundation grid. The region bbox are in WGS 84 as required, but the user should specify the dflowfm model destination CRS in the `global` section in order to allow to use grid definition data (rivers, pipes) from different data sources or CRS. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12860ed2",
   "metadata": {},
   "source": [
    "### Building a 1D Delft3D FM model "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "hundred-trailer",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# NOTE: copy this line (without !) to your shell for more direct feedback - should take about 1 minute or 2 to run\n",
    "!hydromt build dflowfm \"./build/dflowfm_1d\" -i dflowfm_build1d.yml -d artifact_data --fo -v"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b31e92d2",
   "metadata": {},
   "source": [
    "The example above means the following: run **hydromt build** with:\n",
    "\n",
    "* `dflowfm` : i.e. build a FM model\n",
    "* `./build/dflowfm_1d` : output model folder\n",
    "* `-i dflowfm_build1d.yml`: model build configuration file\n",
    "* `-d artifact_data`: data catalog to use. Here the default artifact data catalog with global data for Northern Italy.\n",
    "* `--fo`: force overwritting in case a model already exists in the output folder.\n",
    "* `-v` : give some extra verbosity (1 * v) to display feedback on screen. Now info messages are provided rather than only warnings and errors.\n",
    "\n",
    "> NOTE: You can use as many data catalogs as you need within the same command line, by repeating the `-d` options and name of the catalog to use several times."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df54f1ca",
   "metadata": {},
   "source": [
    "Next we check the model files that have been created. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6b81208",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "root = \"./build/dflowfm_1d/\"\n",
    "for path, _, files in os.walk(root):\n",
    "    print(path)\n",
    "    for name in files:\n",
    "        print(f\" - {name}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2eed76e4",
   "metadata": {},
   "source": [
    "The model root should contain three files and four folders:\n",
    "\n",
    "- dimr_config.xml: Configuration file to execute Delft3D FM in Deltares Integrated Model Runner (DIMR).\n",
    "- hydromt.log: log saved by HydroMT on model building steps.\n",
    "- hydromt_data.yml: information saved by HydroMT on used datasets for reproducibility.\n",
    "- **dflowfm** folder: dflowfm files, here for our 1D model. Contains all necessary information to run a 1D model including the mesh1d, roughness, cross-sections, 1D boundary forcing.   The configuration MDU file is also created as well as a *.gui* file to visualize the 1D network in the Delft3D FM GUI.\n",
    "- **geoms** folder: folder containing a copy of the dflowfm data in an easily readable GIS format rather than the ini or UGRID netcdf file.\n",
    "- **maps** folder: folder containing regular gridded data that Delft3D FM can interpolate to the 2D mesh grid. Here empty as we have a 1D model only."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adef7fa1",
   "metadata": {},
   "source": [
    "### Visualize and/or inspect model schematization"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20ce9df3",
   "metadata": {},
   "source": [
    "* We can now load the **dimr_config.xml** into **Delft3D FM GUI**.\n",
    "* The **dflowfm plot** [example notebook](https://deltares.github.io/hydromt_delft3dfm/latest/_examples/plot_dflowfm_mesh.html) contains scripts to visualize your model in python: "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.20"
  },
  "vscode": {
   "interpreter": {
    "hash": "5fda536963eae36dbaef69f7df44483d2713515b4e134e53fb983ec0eb9438d0"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}