Build a Delft-FIAT model anywhere in the world with OSM and JRC data#
This notebook demonstrates how to set up a Delft-FIAT model anywhere in the world. As an example we will create a model for the city of Kingston upon Hull, UK, in the Humber delta. The relevant data will be taken from OpenStreetMap (OSM) and the global flood vulnerability datasets from JRC. Both the data sources and the region of interest (the model domain) can be changed according to the user’s wishes.
Disclaimer: The outcomes of this model are not validated
Step 0: Import required packages#
First we need to import the necessary python packages.
[1]:
# import required packages
import os
from hydromt_fiat.fiat import FiatModel
from hydromt.log import setuplog
from pathlib import Path
import geopandas as gpd
import matplotlib.pyplot as plt
import shutil
%matplotlib inline
Step 1a: Configure - initialization#
The first step is to set up the configuration needed to initialize the FIAT model. Begin by specifying where to save the model with the following parameters: - model_root
: the path to a folder where the model should be saved in - model_name
: the name of the model
[2]:
model_root = "FIAT_database" # where to save the FIAT model
model_name = "example" # name of the case study
We need some additional utilities, such as a data catalog telling the model where to find the data and a logger keeping track of changes to the model. Specifying these, the parameters below don’t need to be altered. - model_folder
: the full path to the to be created model folder (model_root
+ model_name
) - data_catalog
: the relative path to the HydroMT-FIAT global data catalog. Make sure the root specified in the catalog is correct - logger_name
: the log-file name. A logger
object is also initialized below
[3]:
model_folder = Path(model_root) / model_name # full path to model folder
data_catalog = (
Path(os.path.abspath("")) / "data" / "hydromt_fiat_catalog_global.yml"
) # path to data catalog relative to this notebook
logger_name = "hydromt_fiat" # name of the logger
logger = setuplog(logger_name, log_level=10) # setup logger
2024-12-18 15:30:29,214 - hydromt_fiat - log - INFO - HydroMT version: 0.10.0
Step 1b: Configure - model setup#
Next we need to set up the sptial boundaries, or domain, of the model.
A useful tool for this is http://geojson.io. Here you can draw a polygon on the map and export it to a geojson file, or copy the GeoJSON description directly into the area_of_interest
parameter. In this example, we load in a GeoJSON file with our model domain.
[4]:
domain_fn = Path("./data/KingstonUponHull_domain.geojson")
region = gpd.read_file(domain_fn)
[5]:
# # The code block below can be used to define a custom region of interest
# # by assigning the GeoJSON description to the area_of_interest variable
# area_of_interest = {
# "type": "FeatureCollection",
# "features": [
# {
# "type": "Feature",
# "properties": {},
# "geometry": {
# "coordinates": [
# [
# [32.54354450016967, 0.3260186123942219],
# [32.54354450016967, 0.28330087448348706],
# [32.59681776425123, 0.28330087448348706],
# [32.59681776425123, 0.3260186123942219],
# [32.54354450016967, 0.3260186123942219],
# ]
# ],
# "type": "Polygon",
# },
# }
# ],
# }
# region = gpd.GeoDataFrame.from_features(area_of_interest, crs="EPSG:4326")
To assign the relevant maximum potential damages and vulnerability curves, HydroMT-FIAT must know in which country and continent the domain is located. This will be automated in a later version, but for now we specify this manually: - continent
: the name of the continent where your area of interest is located. Please choose one of the following: Europe, North America, Central America, South America, Asia, Africa, Oceania, or Global - country
: the name of the country where your area of
interest is located. Note that not all countries are included in the JRC database.
[6]:
continent = "Europe"
country = "United Kingdom"
Next, we need to specify properties of the various dataset. For the vulnerability and exposure data HydroMT-FIAT needs the names and unit of the datasets. The names should correspond to the names (keys) in the data catalog except from the OSM data, which is accessed with the OSMnx Python package. Should you wish to use different data sources, make sure to update the data catalog. For the output data, we need to specify the output file names and where to store them. The parameter names below are parsed by the model builder, so they should not be changed.
Vulnerability - vulnerability_fn
: the source name of the vulnerability curve dataset as defined in the HydroMT-FIAT global data catalog. In this example, we use the JRC global damage curves. - vulnerability_identifiers_and_linking_fn
: the source name of the occupancy type-vulnerability curves linking table as defined in the HydroMT-FIAT data catalog. In this example, we use the default table for linking the OSM land use classese to the JRC damage curves (i.e., the assets classified
as residential links to the residential vulnerability curves of JRC). - unit
: the unit of the vulnerability curves. The JRC curves are in meters.
Exposure - asset_locations
: the source name of the location and (optionally) geometry data of the assests for which damages will be calculated. In this example, the building footprints from OSM are used. - occupancy_type
: the source name of the occupancy type data to classify the assets. In this example, the land use data from OSM is used. - max_potential_damage
: the source name of the maximum potential damage values data. In this example, the JRC maximum damage values are used.
- ground_floor_height
: the height of the ground floor of all assets, in the same unit
- unit
: the unit of the exposure data
Output - output_dir
: the name of the output directory - output_csv_name
: the name of the output CSV - output_vector_name
: the name of the vector output file(s)
[7]:
### Setup vulnerability parameters ###
vulnerability_fn = "jrc_vulnerability_curves"
vulnerability_identifiers_and_linking_fn = "jrc_vulnerability_curves_linking"
unit = "meters"
### Setup exposure parameters ###
asset_locations = "OSM"
occupancy_type = "OSM"
max_potential_damage = "jrc_damage_values"
ground_floor_height = 0
damage_types = ["total"]
unit = "meters"
### Setup output parameters ###
output_dir = "output"
output_csv_name = "output.csv"
output_vector_name = "spatial.gpkg"
The above parameters are collected in a dictonairy to be passed to HydroMT-FIAT.
[8]:
configuration = {
"setup_output": {
"output_dir": output_dir,
"output_csv_name": output_csv_name,
"output_vector_name": output_vector_name,
},
"setup_vulnerability": {
"vulnerability_fn": vulnerability_fn,
"vulnerability_identifiers_and_linking_fn": vulnerability_identifiers_and_linking_fn,
"continent": continent,
"unit": unit,
},
"setup_exposure_buildings": {
"asset_locations": asset_locations,
"occupancy_type": occupancy_type,
"max_potential_damage": max_potential_damage,
"ground_floor_height": ground_floor_height,
"unit": unit,
"damage_types": damage_types,
"country": country,
},
}
Step 2: Initialize#
In this step we initialize HydroMT-FIAT with the model_folder
, data_catalog
, and logger
that we defined above.
Note that if the ``model_folder`` already exists, it is deleted before initializing HydroMT-FIAT.
[9]:
if model_folder.exists():
shutil.rmtree(model_folder)
fiat_model = FiatModel(
root=model_folder, mode="w", data_libs=[data_catalog], logger=logger
)
2024-12-18 15:30:29,289 - hydromt_fiat - data_catalog - INFO - Parsing data catalog from /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/data/hydromt_fiat_catalog_global.yml
2024-12-18 15:30:29,293 - hydromt_fiat - data_catalog - INFO - Parsing data catalog from /home/runner/miniconda3/envs/hydromt-fiat-docs/lib/python3.10/site-packages/hydromt_fiat/data/hydromt_fiat_catalog_global.yml
2024-12-18 15:30:29,297 - hydromt_fiat - log - DEBUG - Writing log messages to new file /home/runner/work/hydromt_fiat/hydromt_fiat/docs/_examples/FIAT_database/example/hydromt.log.
2024-12-18 15:30:29,298 - hydromt_fiat - model_api - INFO - Initializing fiat model from hydromt_fiat (v0.5.0.dev0).
Step 3: Build#
In this step we build the Delft-FIAT model. Depending on the extent of the model it can take some time to download the building footprints and land use data from OSM. During the building of the model, log messages display what is happening.
Note that the model will not yet be written because of setting the write argument to False.
[10]:
fiat_model.build(region={"geom": region}, opt=configuration, write=False)
2024-12-18 15:30:29,307 - hydromt_fiat - model_api - INFO - setup_region.region: {'geom': geometry
0 POLYGON ((-0.36004 53.72973, -0.33988 53.73519...}
2024-12-18 15:30:29,308 - hydromt_fiat - basin_mask - DEBUG - Parsed region (kind=geom): {'geom': 'GeoDataFrame [-0.39383064 53.72450849 -0.25232426 53.77820804] (crs = EPSG:4326)'}
2024-12-18 15:30:29,314 - hydromt_fiat - model_api - INFO - setup_output.output_dir: output
2024-12-18 15:30:29,315 - hydromt_fiat - model_api - INFO - setup_output.output_csv_name: output.csv
2024-12-18 15:30:29,315 - hydromt_fiat - model_api - INFO - setup_output.output_vector_name: spatial.gpkg
2024-12-18 15:30:29,316 - hydromt_fiat - model_api - ERROR - Default config file not found at /home/runner/miniconda3/envs/hydromt-fiat-docs/lib/python3.10/site-packages/hydromt_fiat/data/fiat/settings.toml
2024-12-18 15:30:29,316 - hydromt_fiat - model_api - INFO - setup_vulnerability.vulnerability_fn: jrc_vulnerability_curves
2024-12-18 15:30:29,317 - hydromt_fiat - model_api - INFO - setup_vulnerability.vulnerability_identifiers_and_linking_fn: jrc_vulnerability_curves_linking
2024-12-18 15:30:29,317 - hydromt_fiat - model_api - INFO - setup_vulnerability.unit: meters
2024-12-18 15:30:29,318 - hydromt_fiat - model_api - INFO - setup_vulnerability.functions_mean: default
2024-12-18 15:30:29,318 - hydromt_fiat - model_api - INFO - setup_vulnerability.functions_max: None
2024-12-18 15:30:29,319 - hydromt_fiat - model_api - INFO - setup_vulnerability.step_size: None
2024-12-18 15:30:29,319 - hydromt_fiat - model_api - INFO - setup_vulnerability.continent: Europe
2024-12-18 15:30:29,324 - hydromt_fiat - dataframe - INFO - Reading jrc_vulnerability_curves csv data from /home/runner/miniconda3/envs/hydromt-fiat-docs/lib/python3.10/site-packages/hydromt_fiat/data/damage_functions/flooding/JRC_damage_functions.csv
2024-12-18 15:30:29,339 - hydromt_fiat - dataframe - INFO - Reading jrc_vulnerability_curves_linking csv data from /home/runner/miniconda3/envs/hydromt-fiat-docs/lib/python3.10/site-packages/hydromt_fiat/data/vulnerability_linking/JRC_damage_functions_linking.csv
2024-12-18 15:30:29,352 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.asset_locations: OSM
2024-12-18 15:30:29,352 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.occupancy_type: OSM
2024-12-18 15:30:29,353 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.max_potential_damage: jrc_damage_values
2024-12-18 15:30:29,353 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.ground_floor_height: 0
2024-12-18 15:30:29,354 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.unit: meters
2024-12-18 15:30:29,354 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.gfh_unit: None
2024-12-18 15:30:29,354 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.gfh_attribute_name: None
2024-12-18 15:30:29,355 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.occupancy_attr: None
2024-12-18 15:30:29,355 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.occupancy_object_type: None
2024-12-18 15:30:29,356 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.extraction_method: centroid
2024-12-18 15:30:29,356 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.damage_types: ['total']
2024-12-18 15:30:29,357 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.damage_unit: $
2024-12-18 15:30:29,357 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.country: United Kingdom
2024-12-18 15:30:29,357 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.ground_elevation: None
2024-12-18 15:30:29,358 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.grnd_elev_unit: None
2024-12-18 15:30:29,358 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.bf_conversion: False
2024-12-18 15:30:29,359 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.keep_unclassified: True
2024-12-18 15:30:29,360 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.dst_crs: None
2024-12-18 15:30:29,360 - hydromt_fiat - model_api - INFO - setup_exposure_buildings.damage_translation_fn: None
2024-12-18 15:30:29,361 - hydromt_fiat - exposure_vector - INFO - Setting up exposure data from multiple sources...
2024-12-18 15:30:29,361 - hydromt_fiat - exposure_vector - INFO - Setting up asset locations...
2024-12-18 15:30:34,609 - hydromt_fiat.data_apis.open_street_maps - open_street_maps - INFO - Total number of buildings found from OSM: 26761
2024-12-18 15:30:34,683 - hydromt_fiat - exposure_vector - INFO - Setting exposure geometries...
2024-12-18 15:30:34,684 - hydromt_fiat - exposure_vector - INFO - Setting geometry name to buildings...
2024-12-18 15:30:34,684 - hydromt_fiat - exposure_vector - INFO - Setting up occupancy type from OSM...
2024-12-18 15:30:36,064 - hydromt_fiat.data_apis.open_street_maps - open_street_maps - INFO - Total number of land use polygons found from OSM: 474
2024-12-18 15:30:39,509 - hydromt_fiat - exposure_vector - INFO - The following unique landuse are found in the OSM data: ['retail', 'cemetery', 'allotments', 'recreation_ground', 'residential', 'industrial', 'commercial', 'grass', 'construction', 'railway', 'brownfield', 'orchard', 'military', 'plant_nursery', 'forest', 'street', 'flowerbed', 'meadow']
INFO:hydromt_fiat:The following unique landuse are found in the OSM data: ['retail', 'cemetery', 'allotments', 'recreation_ground', 'residential', 'industrial', 'commercial', 'grass', 'construction', 'railway', 'brownfield', 'orchard', 'military', 'plant_nursery', 'forest', 'street', 'flowerbed', 'meadow']
2024-12-18 15:30:39,513 - hydromt_fiat - exposure_vector - INFO - The following unique building are found in the OSM data: ['store', 'block', 'public', 'yes', 'warehouse', 'church', 'office', 'apartments', 'school', 'retail', 'residential', 'university', 'college', 'train_station', 'commercial', 'roof', 'industrial', 'house', 'hospital', 'sports_centre', 'corridor', 'storage_tank', 'stadium', 'temple', 'pub', 'dormitory', 'garages', 'terrace', 'semidetached_house', 'hotel', 'hut', 'no', 'garage', 'parking', 'conservatory', 'stable', 'construction', 'caf', 'service', 'detached', 'shed', 'ruins', 'signal_box']
INFO:hydromt_fiat:The following unique building are found in the OSM data: ['store', 'block', 'public', 'yes', 'warehouse', 'church', 'office', 'apartments', 'school', 'retail', 'residential', 'university', 'college', 'train_station', 'commercial', 'roof', 'industrial', 'house', 'hospital', 'sports_centre', 'corridor', 'storage_tank', 'stadium', 'temple', 'pub', 'dormitory', 'garages', 'terrace', 'semidetached_house', 'hotel', 'hut', 'no', 'garage', 'parking', 'conservatory', 'stable', 'construction', 'caf', 'service', 'detached', 'shed', 'ruins', 'signal_box']
2024-12-18 15:30:39,514 - hydromt_fiat - exposure_vector - INFO - The following unique amenity are found in the OSM data: ['parking', 'cinema', 'restaurant', 'place_of_worship', 'school', 'university', 'college', 'fountain', 'library', 'prison', 'theatre', 'pub', 'arts_centre', 'bicycle_parking', 'courthouse', 'crematorium', 'fast_food', 'bar', 'fuel', 'cafe', 'veterinary', 'bank', 'dentist', 'townhall', 'casino', 'doctors', 'post_depot', 'gambling', 'community_centre', 'hospital', 'recycling', 'fire_station', 'police', 'car_wash', 'childcare', 'pharmacy', 'social_facility', 'clinic', 'nightclub', 'toilets', 'car_rental', 'post_office', 'parking_space', 'grave_yard', 'shelter', 'stripclub', 'marketplace', 'events_venue', 'studio', 'ice_cream']
INFO:hydromt_fiat:The following unique amenity are found in the OSM data: ['parking', 'cinema', 'restaurant', 'place_of_worship', 'school', 'university', 'college', 'fountain', 'library', 'prison', 'theatre', 'pub', 'arts_centre', 'bicycle_parking', 'courthouse', 'crematorium', 'fast_food', 'bar', 'fuel', 'cafe', 'veterinary', 'bank', 'dentist', 'townhall', 'casino', 'doctors', 'post_depot', 'gambling', 'community_centre', 'hospital', 'recycling', 'fire_station', 'police', 'car_wash', 'childcare', 'pharmacy', 'social_facility', 'clinic', 'nightclub', 'toilets', 'car_rental', 'post_office', 'parking_space', 'grave_yard', 'shelter', 'stripclub', 'marketplace', 'events_venue', 'studio', 'ice_cream']
2024-12-18 15:30:39,515 - hydromt_fiat - dataframe - INFO - Reading jrc_osm_mapping csv data from /home/runner/miniconda3/envs/hydromt-fiat-docs/lib/python3.10/site-packages/hydromt_fiat/data/damage_functions/flooding/JRC_OSM_mapping.csv
INFO:hydromt_fiat:Reading jrc_osm_mapping csv data from /home/runner/miniconda3/envs/hydromt-fiat-docs/lib/python3.10/site-packages/hydromt_fiat/data/damage_functions/flooding/JRC_OSM_mapping.csv
2024-12-18 15:30:40,801 - hydromt_fiat - exposure_vector - WARNING - 4 objects were not overlapping with the land use data and will be classified as residential buildings.
WARNING:hydromt_fiat:4 objects were not overlapping with the land use data and will be classified as residential buildings.
2024-12-18 15:30:40,818 - hydromt_fiat - dataframe - INFO - Reading jrc_damage_values csv data from /home/runner/miniconda3/envs/hydromt-fiat-docs/lib/python3.10/site-packages/hydromt_fiat/data/max_potential_damages/JRC_base_damage_values.csv
INFO:hydromt_fiat:Reading jrc_damage_values csv data from /home/runner/miniconda3/envs/hydromt-fiat-docs/lib/python3.10/site-packages/hydromt_fiat/data/max_potential_damages/JRC_base_damage_values.csv
2024-12-18 15:30:42,383 - hydromt_fiat - exposure_vector - WARNING - ground_elevtn is not recognized by the setup_ground_elevation function
WARNING:hydromt_fiat:ground_elevtn is not recognized by the setup_ground_elevation function
2024-12-18 15:30:42,385 - hydromt_fiat - exposure_vector - WARNING - ground_elevtn will be set to 0
WARNING:hydromt_fiat:ground_elevtn will be set to 0
2024-12-18 15:30:42,390 - hydromt_fiat - exposure_vector - INFO - The primary_object_type was used to link the exposure data to the vulnerability curves for total damages.
INFO:hydromt_fiat:The primary_object_type was used to link the exposure data to the vulnerability curves for total damages.
Required variable column fn_damage_{} not found in exposure data.
Required variable column max_damage_{} not found in exposure data.
Step 4a: Inspect model#
We now inspect the resulting exposure data and vulnerability curves that will be saved in the fiat_model
.
Exposure data#
In the map below, the region and secondary_object_type of the exposure objects are plotted. You can zoom in and see whether the data makes sense, perhaps using Google maps and/or streetview to validate the occupancy types.
Note: In Delft-FIAT, exposure is defined with object footprints, lines or centroids. In this example we are only using the buildings extracted from the OSM data. This means we are not going to look into, e.g., farmland, roads, etc. but that is possible with Delft-FIAT.
[11]:
# Get the geodataframe with exposure data
gdf = fiat_model.exposure.get_full_gdf(fiat_model.exposure.exposure_db)
# Plot the region and the secondary_object_types of the exposure data
m = region.explore(name="Region", style_kwds={"color": "black", "fill": False})
m = gdf.explore(m=m, column="secondary_object_type", name="Exposure types")
m
[11]: