Tip

For an interactive online version click here:

Example Aggregation Zones

In spatial analysis and urban planning the division of objects into spatial zones is a pivotal tool to facilitate analysis and/or visualization. This jupyter notebook demonstrates how to create a spatial joint of the FIAT model data with single or multiple aggregation zones in order to link an the zone and objects of interest.

The base for this notebook is the DELFT FIAT toolbox.

Note: The FIAT model was already initialized for this notebook. Unless the user does not wish to use own data, there is no need to create a new model and the user can simply continue with the example data found in the ``/ “example” / “data”/ “aggregation_zone”`` - directory.

Install Delft-FIAT

HydroMT-FIAT is available on pypi and can be installed via:

pip install hydromt_fiat

If you want to know more about the installation process or if you are interested to install HydroMT-FIAT as a developer, please check out the Installation Guide.

Step 0: Import required packages

Now we can import the neccessary python packages to build the FIAT model.

# Import packages
import geopandas as gpd
from pathlib import Path
import os
from hydromt_fiat.fiat import FiatModel
from hydromt.log import setuplog
import pandas as pd
import yaml
import json
import shutil
from hydromt.config import configread

Before we start: Study area and aggregation zones

In this notebook we will spatially join three different aggregation zones to the FIAT exposure objects.

The aggregation zones handled in this notebook are: - Base_zones - Land_use - Accommodation_type

The data was obtained from https://gis.charleston-sc.gov/

Let’s have a look at the study area and one of the aggregation zones that we would like to join!

The map demonstrates the Base_zones.gpkg aggregation layer in Charelston, United States, which devides the county into base district zones. One of the three layers, that will be spatially joined with the objects in the exposure data (yellow markers) in this notebook. This can be achieved for multiple aggregation zone layers.

#Load aggregation zones as GeoDataFrames
exposure=gpd.read_file(Path(os.path.abspath("")) / "data" / "aggregation_zones" / "fiat_model" / "exposure" / "buildings.gpkg")
base_zone=gpd.read_file(Path(os.path.abspath("")) / "data" / "aggregation_zones" / "aggregation_zones" / "base_zones.gpkg")

# Create interactive map
m = base_zone.explore(column = 'ZONE_BASE')
m = exposure.explore(m=m, color = '#FFFACD')

# Display map
m

Make this Notebook Trusted to load map: File -> Trust Notebook

Step 1: Configure and initialize the FIAT model

In this notebook, the FIAT model has already been initialized and solely the exposure data will be updated by the add-on of the the aggregation zones. The required data is stored in the "examples"/ "data" / "aggregation_zones / "aggregation_zones" - directory.

Note: To configure and initialize your own FIAT model refer to global_OSM_JRC.ipynb. If you do so, don’t forget to update the folder dependencies and variables throughout this notebook pointing to your data.

Step 2: Define aggregation parameters in the configuration file

Two test cases are created in this jupyter notebook. For each test case we need a seperate configuration file (configuration.yaml/ configuration_2.yaml) with the required model input information:

1. Spatial joint of a single aggregation zone to the exposure data

2. Spatial joint of multiple aggregation zones to the exposure data

The configuration is saved in a yaml file, in which the following parameters must be specified by the user:

• aggregation_area_fn: File path to the aggregation file (vector file).

• attribute_names: Name of the column that specifies the aggregation zone in the aggregation vector file (case-sensitive).

• label_names: New aggregation label name, specified by the user.

Below an example of the input for the attribute_names. The column "ZONE_BASE"categorizes the aggregation zones in the vector file.

Step 3: Load the configuration file

This step is not neccessarry to run the model. It is only for visualizing the model input. The configuration files are located in the "example"/ "data" / - folder. The information in the file are converted into a python dictionary.

Note: Instead of loading a yaml file it is possible to create a python dictionary directly, built upon the structure seen below.

# Let's read the yaml file with the required information

# Test Case 1
with open(Path(os.path.abspath("")) / "data" / "aggregation_zones" / "configuration.yaml", 'r') as file:
    config_aggregation = yaml.safe_load(file)

print(f"\"Test Case 1\" {json.dumps(config_aggregation, indent=4, sort_keys=False)}")

# Test Case 2
with open(Path(os.path.abspath("")) / "data" / "aggregation_zones" / "configuration_2.yaml", 'r') as file:
    config_aggregation_2 = yaml.safe_load(file)

print(f"\"Test Case 2\" {json.dumps(config_aggregation_2, indent=4, sort_keys=False)}")

"Test Case 1" {
    "setup_additional_attributes": {
        "aggregation_area_fn": "data/aggregation_zones/aggregation_zones/base_zones.gpkg",
        "attribute_names": "ZONE_BASE",
        "label_names": "Base_zones"
    }
}
"Test Case 2" {
    "setup_additional_attributes": {
        "aggregation_area_fn": [
            "data/aggregation_zones/aggregation_zones/base_zones.gpkg",
            "data/aggregation_zones/aggregation_zones/land_use.gpkg",
            "data/aggregation_zones/aggregation_zones/accomodation_type.gpkg"
        ],
        "attribute_names": [
            "ZONE_BASE",
            "LAND_USE",
            "ACCOM"
        ],
        "label_names": [
            "Base_zones",
            "Land_use",
            "Accommodation_type"
        ]
    }
}

Step 4: Define variables for the FIAT model

Set up the root path to the FIAT model, the path where the output should be stored and the logger settings to re-build your model:

• root: Directory path from where the neccessary data for the FIAT model is stored.

• new_root: Directory path where the FIAT model output is stored.

• logger: The logger variable defines the frequencies of log-entries during the initialisation of the model.

# Set up the root for the Fiat Model
root = Path(os.path.abspath("")) / "data" / "aggregation_zones" / "fiat_model"

# Set up the file path for the FIAT model output
new_root =  Path(os.path.abspath("")) / "data" / "aggregation_zones" / "aggregation_zones_test1"

# Set up the logger
logger = setuplog("hydromt_fiat", log_level=10)

# Read the model configuration
opt = configread(Path(os.path.abspath("")) / "data" / "aggregation_zones"  /  "configuration.yaml")

2024-11-20 16:34:07,330 - hydromt_fiat - log - INFO - HydroMT version: 0.10.0

Test case 1: Run the FIAT model

It’s time to run the model with the new configuration of the aggregation zones. The output will be stored in the prior defined "new_root" - directory. The output will be a new exposure.csv, in which each object is assigned spatially to an aggregation zone.

# Read the existing FIAT model
fm = FiatModel(root=root, mode="r", logger=logger)
fm.read()

# If case exist
if new_root.exists():
        shutil.rmtree(new_root)

# Build the new model
fm.build(write=False, opt=opt)
fm.set_root(new_root)
fm.write()

2024-11-20 16:34:07,346 - hydromt_fiat - data_catalog - INFO - Parsing data catalog from /home/runner/miniconda3/envs/hydromt-fiat-docs/lib/python3.10/site-packages/hydromt_fiat/data/hydromt_fiat_catalog_global.yml
2024-11-20 16:34:07,351 - hydromt_fiat - log - DEBUG - Writing log messages to new file /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/fiat_model/hydromt.log.
2024-11-20 16:34:07,352 - hydromt_fiat - model_api - INFO - Initializing fiat model from hydromt_fiat (v0.4.2.dev0).
2024-11-20 16:34:07,353 - hydromt_fiat - fiat - INFO - Reading model data from /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/fiat_model
2024-11-20 16:34:07,375 - hydromt_fiat - model_api - DEBUG - User defined config read from /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/fiat_model/settings.toml
2024-11-20 16:34:07,376 - hydromt_fiat - fiat - INFO - Reading model table files.
2024-11-20 16:34:07,377 - hydromt_fiat - fiat - DEBUG - Reading vulnerability table /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/fiat_model/vulnerability/vulnerability_curves.csv
2024-11-20 16:34:07,415 - hydromt_fiat - fiat - DEBUG - Reading exposure table /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/fiat_model/exposure/exposure.csv
2024-11-20 16:34:07,422 - hydromt_fiat - fiat - INFO - Reading exposure geometries.
2024-11-20 16:34:07,422 - hydromt_fiat - exposure_vector - INFO - Setting geometry name to buildings...
2024-11-20 16:34:07,435 - hydromt_fiat - exposure_vector - INFO - Setting exposure geometries...
2024-11-20 16:34:07,436 - hydromt_fiat - model_api - INFO - setup_additional_attributes.aggregation_area_fn: data/aggregation_zones/aggregation_zones/base_zones.gpkg
2024-11-20 16:34:07,437 - hydromt_fiat - model_api - INFO - setup_additional_attributes.attribute_names: ZONE_BASE
2024-11-20 16:34:07,437 - hydromt_fiat - model_api - INFO - setup_additional_attributes.label_names: Base_zones
2024-11-20 16:34:07,438 - hydromt_fiat - model_api - INFO - setup_additional_attributes.new_composite_area: False
2024-11-20 16:34:07,554 - hydromt_fiat - log - DEBUG - Writing log messages to new file /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/aggregation_zones_test1/hydromt.log.
2024-11-20 16:34:07,555 - hydromt_fiat - fiat - INFO - Updating all data objects...
2024-11-20 16:34:07,556 - hydromt_fiat - fiat - INFO - Writing model data to /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/aggregation_zones_test1
2024-11-20 16:34:07,743 - hydromt_fiat - model_api - DEBUG - Writing file geoms/additional_attributes/base_zones.geojson
2024-11-20 16:34:07,821 - hydromt_fiat - fiat - INFO - Writing model exposure table file to exposure/exposure.csv.
2024-11-20 16:34:07,841 - hydromt_fiat - fiat - INFO - Writing model vulnerability_curves table file to vulnerability/vulnerability_curves.csv.
2024-11-20 16:34:07,843 - hydromt_fiat - model_api - INFO - Writing model config to /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/aggregation_zones_test1/settings.toml

Test case 2: Run the FIAT model

To run the test case 2, select it the configuration dictionary when re-running the model. Hence, the model will select the parameters of the second test case instead of the first and create spatial joints to multiple aggregation zones.

To run the second model, the model variables must be updated.

# Set up the root for the Fiat Model
root = Path(os.path.abspath("")) / "data" / "aggregation_zones" / "fiat_model"

# Set up the file path for the FIAT model output
new_root =  Path(os.path.abspath("")) / "data" / "aggregation_zones" / "aggregation_zones_test2"

# Set up the logger
logger = setuplog("hydromt_fiat", log_level=10)

# Read the model configuration
opt = configread(Path(os.path.abspath("")) / "data" / "aggregation_zones"  /  "configuration_2.yaml")

2024-11-20 16:34:07,849 - hydromt_fiat - log - INFO - HydroMT version: 0.10.0

# Read the existing FIAT model
fm = FiatModel(root=root, mode="r", logger=logger)
fm.read()

#If case exist
if new_root.exists():
        shutil.rmtree(new_root)

# Build the new model
fm.build(write=False, opt=opt)
fm.set_root(new_root)
fm.write()

2024-11-20 16:34:07,856 - hydromt_fiat - data_catalog - INFO - Parsing data catalog from /home/runner/miniconda3/envs/hydromt-fiat-docs/lib/python3.10/site-packages/hydromt_fiat/data/hydromt_fiat_catalog_global.yml
2024-11-20 16:34:07,861 - hydromt_fiat - log - DEBUG - Appending log messages to file /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/fiat_model/hydromt.log.
2024-11-20 16:34:07,861 - hydromt_fiat - model_api - INFO - Initializing fiat model from hydromt_fiat (v0.4.2.dev0).
2024-11-20 16:34:07,863 - hydromt_fiat - fiat - INFO - Reading model data from /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/fiat_model
2024-11-20 16:34:07,864 - hydromt_fiat - model_api - DEBUG - User defined config read from /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/fiat_model/settings.toml
2024-11-20 16:34:07,865 - hydromt_fiat - fiat - INFO - Reading model table files.
2024-11-20 16:34:07,865 - hydromt_fiat - fiat - DEBUG - Reading vulnerability table /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/fiat_model/vulnerability/vulnerability_curves.csv
2024-11-20 16:34:07,904 - hydromt_fiat - fiat - DEBUG - Reading exposure table /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/fiat_model/exposure/exposure.csv
2024-11-20 16:34:07,910 - hydromt_fiat - fiat - INFO - Reading exposure geometries.
2024-11-20 16:34:07,910 - hydromt_fiat - exposure_vector - INFO - Setting geometry name to buildings...
2024-11-20 16:34:07,923 - hydromt_fiat - exposure_vector - INFO - Setting exposure geometries...
2024-11-20 16:34:07,924 - hydromt_fiat - model_api - INFO - setup_additional_attributes.aggregation_area_fn: ['data/aggregation_zones/aggregation_zones/base_zones.gpkg', 'data/aggregation_zones/aggregation_zones/land_use.gpkg', 'data/aggregation_zones/aggregation_zones/accomodation_type.gpkg']
2024-11-20 16:34:07,924 - hydromt_fiat - model_api - INFO - setup_additional_attributes.attribute_names: ['ZONE_BASE', 'LAND_USE', 'ACCOM']
2024-11-20 16:34:07,925 - hydromt_fiat - model_api - INFO - setup_additional_attributes.label_names: ['Base_zones', 'Land_use', 'Accommodation_type']
2024-11-20 16:34:07,925 - hydromt_fiat - model_api - INFO - setup_additional_attributes.new_composite_area: False
2024-11-20 16:34:08,435 - hydromt_fiat - log - DEBUG - Writing log messages to new file /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/aggregation_zones_test2/hydromt.log.
2024-11-20 16:34:08,436 - hydromt_fiat - fiat - INFO - Updating all data objects...
2024-11-20 16:34:08,437 - hydromt_fiat - fiat - INFO - Writing model data to /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/aggregation_zones_test2
2024-11-20 16:34:08,602 - hydromt_fiat - model_api - DEBUG - Writing file geoms/additional_attributes/base_zones.geojson
2024-11-20 16:34:08,679 - hydromt_fiat - model_api - DEBUG - Writing file geoms/additional_attributes/land_use.geojson
2024-11-20 16:34:08,982 - hydromt_fiat - model_api - DEBUG - Writing file geoms/additional_attributes/accomodation_type.geojson
2024-11-20 16:34:08,994 - hydromt_fiat - fiat - INFO - Writing model exposure table file to exposure/exposure.csv.
2024-11-20 16:34:09,015 - hydromt_fiat - fiat - INFO - Writing model vulnerability_curves table file to vulnerability/vulnerability_curves.csv.
2024-11-20 16:34:09,016 - hydromt_fiat - model_api - INFO - Writing model config to /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/aggregation_zones/aggregation_zones_test2/settings.toml

Done!

Your FIAT model created a spatial joint with your original objects and the aggregation zones.

Let’s have a look at the output!

Result: Test Case 1

Load the newly created exposure.csv file from the / “output” / “aggregation_zones_test1” / - directory and the original exposure geopackage from the “exposure” / “buildings.gpkg” . Before the results can be displayed, we must merge the exposure.csv with the buildings.gpkg via common “Object ID”.

#Load *.csv into dataframe
df_single_aggregation = pd.read_csv(Path(os.path.abspath("")) / "data" / "aggregation_zones" / "aggregation_zones_test1" / "exposure" / "exposure.csv")

#Load original exposure geopackage into GeoDataFrame
new_exposure=gpd.read_file(Path(os.path.abspath("")) / "data" / "aggregation_zones" / "aggregation_zones_test1" / "exposure" / "buildings.gpkg")

#Merge dataframe with GeoDataFrame
merged_gdf = new_exposure.merge(df_single_aggregation, left_on='Object ID', right_on='Object ID', how='inner')

#Display base_zones aggregation zone
base_zones_map = merged_gdf.explore(column = 'Base_zones')
base_zones_map

Make this Notebook Trusted to load map: File -> Trust Notebook

Result: Test Case 2

Load the newly created exposure.csv file from the / “output” / “aggregation_zones_test2” / - directory and the original exposure geopackage from the “exposure” / “buildings.gpkg” . Before the results can be displayed, we must merge the exposure.csv with the buildings.gpkg via common “Object ID”.

# Load *.csv into dataframe
df_multiple_aggregation = pd.read_csv(Path(os.path.abspath("")) / "data" / "aggregation_zones" / "aggregation_zones_test2" / "exposure" / "exposure.csv")

# Load original exposure geopackage into GeoDataFrame
new_exposure=gpd.read_file(Path(os.path.abspath("")) / "data" / "aggregation_zones" / "aggregation_zones_test2" / "exposure" / "buildings.gpkg")

# Merge dataframe with GeoDataFrame
merged_gdf_multiple = new_exposure.merge(df_multiple_aggregation, left_on='Object ID', right_on='Object ID', how='inner')

Base Zones

# Display base_zones aggregation zone
base_zones_map = merged_gdf.explore(column = 'Base_zones')
base_zones_map

Make this Notebook Trusted to load map: File -> Trust Notebook

Land use

# Display land_use aggregation zone
land_use_map = merged_gdf_multiple.explore(column = 'Land_use')
land_use_map

Make this Notebook Trusted to load map: File -> Trust Notebook

Accomodation type

# Display accomodation_type aggregation zone
Accom_type_map = merged_gdf_multiple.explore(column = 'Accommodation_type')
Accom_type_map

Make this Notebook Trusted to load map: File -> Trust Notebook