Notebook

Run it live: Open in Binder

Download this notebook: 02_elbow_deterministic_forecast.ipynb

02 - Precipitation Deterministic Forecasts#

Learning goals of this module#

  • Learn how to do a basic analysis, comparing observed precipitation with forecast precipitation from GDPS

  • Learn how to calculate and visualize simple error metrics

Assumptions#

  • We assume you are familiar with the concept of a forecast, and a lead_time.

Reference to data products#

Run imports and set-up logging#

[ ]:
import logging
import sys
import warnings
from pathlib import Path

from dotenv import load_dotenv

from veriflow import run_pipeline
from veriflow.constants import VERSION

# add project root (parent of notebook folder) to path
sys.path.append(str(Path("..").resolve()))


from tree_plots import (
    plot_score_vs_lead_time,
    forecast_timeseries_plot,
    get_pair_dataset,
    reanalysis_timeseries_plot
)

# Reload automatically
%load_ext autoreload
%autoreload 2

load_dotenv(dotenv_path="tutorial.env", override=True)

base_config = Path("config")
base_config.exists()

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s",
    handlers=[logging.StreamHandler()],
)
logging.info(f"Running Veriflow version {VERSION}")
2026-11-04 14:58:06,428 - INFO - Running Veriflow version 0.4.1

Inspecting the veriflow pipeline configuration#

  1. Open the config file in the “config” directory. The name of the file is identical to the name of the notebook.

  2. Inspect each of the sections to gain an understanding of what this configuration is about.

Running the veriflow pipeline#

[3]:
dt = run_pipeline((base_config / "02_elbow_deterministic_forecast.yaml", "yaml"))
2026-11-04 14:58:12,945 - INFO - Successfully initialized the configuration.
         verification_period_start = 2026-05-15 00:00:00
         verification_period_end = 2026-06-01 00:00:00
2026-11-04 14:58:12,948 - INFO - Starting dataset fetch (source_id=observed) from FewsWebservice.
2026-11-04 14:58:13,413 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.obs&moduleInstanceIds=ImportMeteoStations&startTime=2026-05-15T00%3A00%3A00Z&endTime=2026-06-01T00%3A00%3A00Z&timeSeriesType=EXTERNAL_HISTORICAL&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:13,416 - INFO - Starting dataset fetch (source_id=observed) from FewsNetCDF.
2026-11-04 14:58:14,753 - INFO - Dataset (source_id=observed) successfully loaded and validated.
2026-11-04 14:58:14,753 - INFO - Starting dataset fetch (source_id=simulated) from FewsWebservice.
2026-11-04 14:58:15,489 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-08T00%3A00%3A00Z&endForecastTime=2026-05-31T00%3A00%3A00Z&forecastCount=1000000&documentFormat=PI_JSON&onlyHeaders=True&omitEmptyTimeSeries=True
2026-11-04 14:58:15,494 - INFO - Configured verification period (2026-05-08 00:00:00 to 2026-05-31 00:00:00) exceeds what source 'simulated' has available (2026-05-15 00:00:00 to 2026-05-31 00:00:00); clamping to (2026-05-15 00:00:00 to 2026-05-31 00:00:00).
2026-11-04 14:58:15,578 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-15T00%3A00%3A00Z&endForecastTime=2026-05-31T00%3A00%3A00Z&forecastCount=1000000&documentFormat=PI_JSON&onlyHeaders=True&omitEmptyTimeSeries=True
2026-11-04 14:58:15,668 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-19T00%3A00%3A00Z&endForecastTime=2026-05-19T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:15,745 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-30T00%3A00%3A00Z&endForecastTime=2026-05-30T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:15,808 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-20T00%3A00%3A00Z&endForecastTime=2026-05-20T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:15,868 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-16T00%3A00%3A00Z&endForecastTime=2026-05-16T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:15,923 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-17T00%3A00%3A00Z&endForecastTime=2026-05-17T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:15,991 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-21T00%3A00%3A00Z&endForecastTime=2026-05-21T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:15,995 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-29T00%3A00%3A00Z&endForecastTime=2026-05-29T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:16,061 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-27T00%3A00%3A00Z&endForecastTime=2026-05-27T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:16,066 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-28T00%3A00%3A00Z&endForecastTime=2026-05-28T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:16,133 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-23T00%3A00%3A00Z&endForecastTime=2026-05-23T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:16,136 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-26T00%3A00%3A00Z&endForecastTime=2026-05-26T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:16,199 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-18T00%3A00%3A00Z&endForecastTime=2026-05-18T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:16,200 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-31T00%3A00%3A00Z&endForecastTime=2026-05-31T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:16,260 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-22T00%3A00%3A00Z&endForecastTime=2026-05-22T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:16,267 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-25T00%3A00%3A00Z&endForecastTime=2026-05-25T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:16,325 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-24T00%3A00%3A00Z&endForecastTime=2026-05-24T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:16,325 - INFO - Download successful from URL: https://veriflow-open.fews.deltares.nl/FewsWebServices/rest/fewspiservice/v1/timeseries?locationIds=3031092&locationIds=3050778&locationIds=MSC-005&locationIds=05BL813&locationIds=05BJ804&locationIds=05BL809&locationIds=FIRES-B4&locationIds=05BJ805&locationIds=05BL812&locationIds=FIRES-B5&locationIds=05BJ806&locationIds=05BH803&locationIds=05BF825&locationIds=05BH802&locationIds=05BL810&locationIds=05BF827&parameterIds=PC.nwp&moduleInstanceIds=PreprocessGDPS&startForecastTime=2026-05-15T00%3A00%3A00Z&endForecastTime=2026-05-15T00%3A00%3A00Z&documentFormat=PI_NETCDF&omitEmptyTimeSeries=True
2026-11-04 14:58:16,332 - INFO - Starting dataset fetch (source_id=simulated) from FewsNetCDF.
2026-11-04 14:58:34,325 - INFO - Opened dataset for source 'simulated' from [WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260515_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260515_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260516_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260516_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260517_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260517_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260518_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260518_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260519_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260519_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260520_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260520_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260521_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260521_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260522_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260522_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260523_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260523_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260524_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260524_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260525_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260525_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260526_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260526_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260527_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260527_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260528_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260528_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260529_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260529_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260530_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260530_000000_PC.nwp_externalforecasting_day.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260531_000000_PC.nwp_externalforecasting_3hours.nc'), WindowsPath('C:/Users/beunk/AppData/Local/Temp/tmpq1uly768/20260531_000000_PC.nwp_externalforecasting_day.nc')] (took 17.99 seconds)
2026-11-04 14:58:34,388 - INFO - Dataset (source_id=simulated) successfully loaded and validated.
2026-11-04 14:58:34,391 - INFO - Successfully loaded all data from sources.
2026-11-04 14:58:34,427 - INFO - Successfully computed ContinuousScores for verification pair PC.
2026-11-04 14:58:34,428 - INFO - Verification pipeline completed successfully.

Evaluating the results in the veriflow output DataTree#

Verification metrics and results can contain a level of abstraction. Although these abstractions can reveal important information about forecast quality, a basic “eyeball verification” is often the best and intuitive way to start your verification exercise. You’ll likely find strengths and weaknesses in your forecasts early on, without directly diving into levels of abstraction. In addition, a solid visual inspection may help you later on in understanding or explaining the more abstract results.

1 - Visual inspection of observed and forecast data#

A good starting point for “eyeball” verification is simple: just looking at your observations and forecasts in a visual way. Use the interactive elements in the plots below to zoom, pan and compare the results of our 3 NWP products.

[8]:
ds = dt.veriflow.get_aligned_input(dt.veriflow.verification_pairs[0])["observations"].to_dataset()
ds
[8]:
<xarray.Dataset> Size: 13kB
Dimensions:                  (station: 16, forecast_reference_time: 17,
                              lead_time: 7)
Coordinates:
  * station                  (station) <U8 512B '3031092' ... '05BF827'
    station_name             (station) |S255 4kB b'Calgary International' ......
    lat                      (station) float64 128B 51.12 51.08 ... 50.65 50.87
    lon                      (station) float64 128B -114.0 -115.1 ... -114.3
    y                        (station) float64 128B 51.12 51.08 ... 50.65 50.87
    x                        (station) float64 128B -114.0 -115.1 ... -114.3
    z                        (station) float64 128B 1.099e+03 1.321e+03 ... nan
  * forecast_reference_time  (forecast_reference_time) datetime64[ns] 136B 20...
  * lead_time                (lead_time) timedelta64[ns] 56B 1 days ... 7 days
Data variables:
    PC                       (station, forecast_reference_time, lead_time) float32 8kB ...
[10]:
stations = ds.coords["station"].values
lead_times = ds.coords["lead_time"].values

lead_times_hours = [str(lt.astype("timedelta64[h]")) for lt in lead_times]
print(f"Stations: {stations}")
print(f"GDPS Lead times (hours): {lead_times_hours}")
Stations: ['3031092' '3050778' 'MSC-005' '05BL813' '05BJ804' '05BL809' 'FIRES-B4'
 '05BJ805' '05BL812' 'FIRES-B5' '05BJ806' '05BH803' '05BF825' '05BH802'
 '05BL810' '05BF827']
GDPS Lead times (hours): ['24 hours', '48 hours', '72 hours', '96 hours', '120 hours', '144 hours', '168 hours']
[5]:
forecast_timeseries_plot(dt, station=stations[1])

2 - Looking into the Mean Error per lead time#

[7]:
# Plot the first 5 stations to avoid overcrowding the plot
plot_score_vs_lead_time(dt, score_name="continuous_scores", score_var="mean_error", stations=stations[:5])