What’s new#

UNRELEASED#

Fix#

  • robustly retrieving bbox in dfmt.meshkernel_get_bbox() in #1292

  • simpler parsing of uhslc geojson after pyogrio update in #1294

  • update for dfmt.meshkernel_get_illegalcells() to align with new meshkernel version in #909

  • only include illegalcells up to 6 edges in dfmt.meshkernel_get_illegalcells() in #1296

0.42.0 (2025-11-03)#

Fix#

  • removed references to soon to be discontinued CMEMS multiyear-interim datasets in #1279

  • allow for polyline names starting with numeric characters in #1281

  • avoid writing of mdu keywords that are obsolete in Delft3D FM 2026.01 in #1283

Deprecated#

  • removed poorly written and hycom-specific dfmt.download_OPeNDAP() in #1289

0.40.0 (2025-10-08)#

Feat#

  • support for list of files in dfmt.cmems_nc_to_ini() for better performance in #1258

  • added polygon argument to dfmt.refine_basegrid() in #1262

  • performance improvement for dfmt.get_Dataset_atdepths() in #1264

  • support for multiple 2D quantities in dfmt.Dataset_to_TimeSeries() and therefore in dfmt.plipointsDataset_to_ForcingModel() in #1268

0.39.0 (2025-08-21)#

Feat#

  • extrapolate depth first in dfmt.cmems_nc_to_ini() in #1231

  • avoid longpaths error in VSCode by removing direct fiona import in #1239

  • make all paths in dfm_tools settable in #1235

  • reduce filesize of bc files by rounding data to six decimal places in #1240

0.38.0 (2025-06-26)#

Fix#

  • made SSC linked stations in ssc_add_linked_stations() case-insensitive in #6b3949f

  • avoid usage of moved private hydrolib-core function in #1210

  • fixed z-sigma layer reconstruction for waterlevels below the z-sigma-interface in #1219

  • improved aspect ratio in dfmt.generate_basegrid() by updating meshkernel dependency in #1223

Feat#

  • enable separate retrieval of UHSLC rqds/fast observationdata again in dfmt.ssh_retrieve_data() (via uhslc_ssh_retrieve_data()) in #1205

  • separate ERA5 quantities (supported from Delft3D-FM 2024.02) in #1211

  • updated docker run script to efficiently work with Delft3D-FM 2025.02 (and later) docker containers in #1216

0.37.0 (2025-05-14)#

This release drops support for Python 3.9.

Fix#

  • retain encoding and long_name attribute also for variables converted with convert_meteo_units() in dfmt.merge_meteofiles() in #1164

  • updated GESLA3 datasource in #1173

  • stricter merging of datasets in dfmt.cmems_nc_to_ini() in #1174

  • support for copernicusmarine 2.1.0 in #1180

  • also construct sigmalayers when sigma-variables are coordinates #1183

  • improved performance of dfmt.uda_to_faces() by using xugrid alternative in #1177

  • check if all requested variables are present in merged dataset in dfmt.preprocess_merge_meteofiles_era5() in #1184

  • remove accents from UHSLC-fast insitu catalog so S64 datatype can be applied in _make_hydrotools_consistent() in #1186

Deprecated#

  • removed dfmt.preprocess_woa since WOA merging fails and is not used in #1161

  • support for python 3.9 is dropped in #1177

  • deprecated sources “uhslc-fast” and “uhslc-rqds” in dfmt.ssh_catalog_subset() and dfmt.ssh_retrieve_data() in favor of “uhslc” in #1193

Feat#

  • added cross-referencing SSC/IOC/UHSLC stations and distances in ssc_ssh_read_catalog() in #1191

  • set coords and reduce filesize of retrieved observations in dfmt.ssh_retrieve_data() in #1189

  • auto-merging of uhslc fast and rqds datasets in dfmt.ssh_retrieve_data() in #1193

0.36.0 (2025-03-18)#

Feat#

  • improved interpolation and extrapolation in dfmt.cmems_nc_to_ini() in #1152

  • added gtsm3-era5-cds data as a source of observations in dfmt.ssh_catalog_subset() and dfmt.ssh_retrieve_data() in #1153.

Fix#

  • improved performance of dfmt.merge_meteofiles() by adding xarray arguments and ensuring alignment in #1148

  • allow for number coordinate variable in dfmt.merge_meteofiles() in #1157

  • proper time slicing and out of bounds error of dfmt.merge_meteofiles() in #1149

0.35.0 (2025-02-20)#

Feat#

  • add constant waterlevel offset with dfmt.constant_to_bc() in #1130

  • download GSHHS data from github instead in #1132

  • support multiple grid refinements via kwargs in``dfmt.refine_basegrid()`` in #1136

Fix#

  • correct ssr conversion factor in convert_meteo_units() in #1134

  • support for pathlib.Path in file_to_list() in #1139

  • rename new (feb 2025) CDS varnames avg_ie and avg_tprate back to mer and mtpr in dfmt.preprocess_ERA5() in #1141

0.34.0 (2025-02-05)#

Feat#

  • usage of outside time buffer in dfmt.cmems_nc_to_ini() so noon-centered or monthly timestamps are also supported in #1087

  • correct CMEMS daily mean data (“P1D-m”) from midnight to noon by adding a 12-hour offset in dfmt.download_CMEMS() in #1088

  • updated cdsapi request to new format in #1103

  • added UHSLC backup for GSHHS data in #1112

  • auto download CMEMS phyc reanalysis from monthly mean dataset and auto convert to freq=M in #622

Fix#

  • made p-drive paths for tide models and gesla3 work on linux also in #1083 and #1085

  • support for different face dimension names in dfmt.enrich_rst_with_map in #1114

0.33.0 (2025-01-20)#

Feat#

  • optimized performance for getting CMEMS time extents and spatial buffer in #1059

  • replaced buffer and floor/ceil with copernicusmarine coordinates_selection_method, this deprecated the buffer argument for dfmt.download_CMEMS() #1061

  • inclusive selection of outside timesteps in open_prepare_dataset() and thus in cmems_nc_to_bc() in #1062

Fix#

  • fixed inexact latlon bbox in modelbuilder with dfmt.meshkernel_get_bbox() in #1067

  • included polyfile basename in bc-files from tide and cmems to prevent overwrite with multiple polyfiles in #1071

  • renamed ext_bnd argument to ext_new in dfmt.cmems_nc_to_bc() for consistency in #1071

0.32.0 (2025-01-14)#

Feat#

  • updated to copernicusmarine v2 in #1046

  • optimized CMEMS download performance in #1049

0.31.0 (2024-10-28)#

Fix#

  • fixed CDS login via prompt for new users in cds_credentials() in #1035

0.30.0 (2024-10-20)#

Fix#

  • fixed cmems-nrt insitu again by dropping station with varying coordinates again in #1023

  • update executable paths in batfile created by dfmt.create_model_exec_files() in #1030

0.29.0 (2024-09-27)#

Feat#

  • update from CDS-Beta to new CDS in #1013

0.28.0 (2024-09-26)#

Fix#

  • more robust CDS/ECMWF authentication check and support for new cdsapi versions in #1004

  • set default copernicusmarine buffer to 0.5 instead of 0 in #1009

0.27.0 (2024-09-09)#

Fix#

  • simplified dfmt.meshkernel_to_UgridDataset() by using new xugrid version in #991

  • updated dfmt.uda_to_faces() to accomodate for new fill_value of xugrid (this version includes fix for contour/contourf) #992

0.26.0 (2024-09-03)#

Fix#

  • properly assigning units attribute in ds_apply_conversion_dict() (it did not always stick) in #965

  • skipping initialwaterlevel in dfmt.cmems_nc_to_ini() #970

  • update to cdsapi 0.7.2 and properly catching error for dummy dataset in #972

  • deprecated dfmt.open_dataset_extra() (partly replaced by dfmt.open_prepare_dataset()) in #974

  • improved nan-conversion in dfmt.forcinglike_to_Dataset() in #982

  • improved performance of dfmt.open_partitioned_dataset() for datasets with many variables in #984

0.25.0 (2024-08-16)#

Feat#

  • making dfmt.open_dataset_extra() more modular by partly moving it to separate private functions in #913

  • added station_id variable to dataset returned by dfmt.interp_uds_to_plipoints() in #914

  • update private functions under dfmt.download_ERA5() to CDS-Beta (requires ECMWF apikey instead) in #925

  • simplified prevention of int dtypes in dfmt.preprocess_ERA5() in #943

  • simplified dfmt.open_dataset_extra() by dropping multi-quantity support and converted to private function in #946

  • improved dfmt.interp_uds_to_plipoints() by supporting outofbound points and new xugrid version in #948

  • neater chunking warning filter in dfmt.open_partitioned_dataset() in #952

  • performance improvement of dfmt.open_partitioned_dataset() by setting remove_edges=False in #960

Fix#

  • also apply convert_360to180 to longitude variable in dfmt.open_dataset_curvilinear() in #913

  • prevented adding of time dimension and dropped 0-sized cells in dfmt.open_dataset_curvilinear() in #929

  • dropping lat/lon/verts variables to maintain single source of truth in dfmt.open_dataset_curvilinear() in #931

  • fix for 3D initial fields in dfmt.cmems_nc_to_ini() to avoid top layer values over the entire depth in #955

0.24.0 (2024-07-12)#

Feat#

  • improved usability of dfmt.LineBuilder() in #854

  • added workaround for grids that are not orthogonal after cutting the land with dfmt.meshkernel_get_illegalcells() in #866

  • updated CMEMS bcg multiyear dataset name in #880

  • added CMEMS reananalysis-interim (myint) datasets to dfmt.download_CMEMS() in #883 and #903

  • avoid duplicate and empty polyline names in dfmt.geodataframe_to_PolyFile() in #896

  • support for multiple polylines per polyfile in interpolate_tide_to_bc() and cmems_nc_to_bc() in #906

  • add to ext as part of interpolate_tide_to_bc(), it now requires ext_new and has no return value anymore in #906

Fix#

  • cleanups for datasets retrieved with dfmt.ssh_retrieve_data() in #867

0.23.0 (2024-05-15)#

Feat#

  • added conversion_dict parameter to dfmt.cmems_nc_to_bc() in #841

Fix#

  • improved dfmt.download.copernicusmarine_reset() by changing order of tasks in #833

  • improved performance of dfmt.merge_meteofiles() by using more generic chunking method in #840

  • improved dfmt.open_dataset_delft3d4() by dropping original grid variable and attrs in #844

0.22.0 (2024-04-09)#

Feat#

  • generate kml file from ssh_catalog (geopandas.GeoDataFrame) with dfmt.ssh_catalog_tokmlfile() in #809

  • generate time availability overview and statistics from insitu netcdf files with dfmt.ssh_netcdf_overview() in #807

  • simplified rwsddl_ssh_retrieve_data() by using ddlpy.dataframe_to_xarray() in #805

  • updated cmems analysisforecast bio dataset ids in #816

  • from ftp to copernicusmarine files service for insitu data in #818

  • simplified dfmt.download.copernicusmarine_credentials() in #819

Fix#

  • fixed xy coordinates for cmems insitu netcdf files in #807

0.21.0 (2024-03-14)#

Feat#

  • added ddl/rws insitu data to dfmt.ssh_catalog_subset() and dfmt.ssh_retrieve_data() in #791 and #796

  • added cmems nrt insitu data to dfmt.ssh_catalog_subset() and dfmt.ssh_retrieve_data() in #791 and #793

  • support for meshkernel>=4.1.0 in #801

0.20.0 (2024-02-22)#

Fix#

  • solve xarray FutureWarning about ds.dims[dim] by @veenstrajelmer in #779

  • support copernicusmarine>=1.0.2 in copernicusmarine_credentials() and made copernicusmarine_reset() modular by @veenstrajelmer in #787

0.19.0 (2024-02-08)#

Feat#

  • automatically parse epsg from FM mapfile to crs in dfmt.open_partitioned_dataset() by @veenstrajelmer in #685

  • added spatial/temporal subsetting and retrieving of insitu observation data with dfmt.ssh_catalog_subset() and dfmt.ssh_retrieve_data() by @veenstrajelmer in #711

  • added conversion of absolute to relative paths for modelbuilder with dfmt.make_paths_relative() by @veenstrajelmer in #723

  • add retrieval and plotting of borders with dfmt.get_borders_gdb() and dfmt.plot_borders() by @veenstrajelmer in #750

  • improve z and sigmaz layer reconstruction (nans below bed and above waterlevel and centers as mean of interfaces) in reconstruct_zw_zcc() by @veenstrajelmer in #755 and #759

  • simplified correctly merging of ERA5 int16 meteofiles in merge_meteofiles() by @veenstrajelmer in #738

  • expecting da instead of ds in refine_basegrid() by @veenstrajelmer in #762

  • added support for Python 3.12 by @veenstrajelmer in #763

  • moved from copernicus-marine-client to copernicusmarine in dfmt.download_CMEMS() by @veenstrajelmer in #721 and #772

  • added reset of Copernicus Marine Toolbox with dfmt.download.copernicusmarine_reset() by @veenstrajelmer in #714

Fix#

Deprecated#

  • deprecated dfmt.preprocess_hirlam() since xarray>=2023.9.0 supports multidimensional coordinates by @veenstrajelmer in #770

0.18.0 (2023-12-08)#

Feat#

  • replaced is_geographic with crs argument in dfmt.make_basegrid() by @veenstrajelmer in #696

  • moved from OPeNDAP to copernicus-marine-client in dfmt.download_CMEMS() including safer authentication by @veenstrajelmer in #691

  • added timerange check of resulting dataset of dfmt.open_dataset_extra() by @veenstrajelmer in #708

0.17.0 (2023-11-17)#

Support for Python 3.8 was dropped (more info)

Feat#

Fix#

0.16.0 (2023-11-03)#

Feat#

  • more robust support for CMEMS/CDS credentials including environment variables by @veenstrajelmer in #633

  • enrich rst file with topology from corresponding mapfile with dfmt.enrich_rst_with_map() by @veenstrajelmer in #626

  • add cellinfo to minimal 2D networks (with 1D topology) with dfmt.add_network_cellinfo() by @veenstrajelmer in #624

  • xugrid feature uds.ugrid.to_nonperiodic() deprecates dfmt.remove_periodic_cells() by @veenstrajelmer in #503

  • support for initial fields for variables other than salinity/temperature with dfmt.cmems_nc_to_ini() by @veenstrajelmer in #619

Fix#

  • fix initial fields for salinity/temperature with dfmt.cmems_nc_to_ini() by @veenstrajelmer in #619

  • increased buffer in dfmt.download_ERA5() by @veenstrajelmer in #612

  • support for Polygon geometries in dfmt.geodataframe_to_PolyFile() by @veenstrajelmer in #610

  • fill nan-values in initial salinity/temperature netcdf dataset in dfmt.preprocess_ini_cmems_to_nc() by @veenstrajelmer in #617

  • skip all-nan boundary support points instead of converting to zeros in dfmt.plipointsDataset_to_ForcingModel() by @veenstrajelmer in #637

0.15.0 (2023-10-19)#

Feat#

  • create dimr_config.xml and run_parallel.bat with dfmt.create_model_exec_files() by @veenstrajelmer in #599

  • conversion of hcdfm.ForcingModel to xr.Dataset with dfmt.ForcingModel_to_plipointsDataset() by @veenstrajelmer in #567

  • support for reading asc files with dfmt.read_asc() by @veenstrajelmer in #571

  • added dfmt.meshkernel_delete_withshp() to delete parts of a mesh with a shapefile, while only reading the shapefile within the bounding box of the mesh by @rqwang in #548 and #566

  • improved spatial interpolation in dfmt.interp_regularnc_to_plipoints() by combining linear with nearest interpolation by @veenstrajelmer in #561

  • added GTSMv4.1 and GTSMv4.1_opendap datasets for tide interpolation with dfmt.interpolate_tide_to_bc() by @veenstrajelmer in #544

  • support for preprocess argument in dfmt.open_partitioned_dataset() by @veenstrajelmer in #530

Fix#

  • increased buffer for WAQ cmems variables by @veenstrajelmer in #601

  • prevent concatenation of datasets with slightly different coordinates by @veenstrajelmer in #595

  • aligned ncbnd dataset with FM/FEWS conventions by @veenstrajelmer in #567

  • documented minimal dependency versions to avoid issues with installation in existing environments containing old dependency versions by @veenstrajelmer in #581

0.14.0 (2023-09-15)#

Feat#

  • decoding of default fillvalues with dfmt.decode_default_fillvals() by @veenstrajelmer in #496

  • interpolation of edge variables to faces with dfmt.uda_edges_to_faces() by @veenstrajelmer in #482

  • interpolation of variables on layer interfaces to layer centers with dfmt.uda_interfaces_to_centers() by @veenstrajelmer in #482

  • generate polyline only for open boundary with dfmt.generate_bndpli_cutland() (deprecates dfmt.generate_bndpli()) by @veenstrajelmer in #466

  • automatically remove unassociated edges with dfmt.remove_unassociated_edges() by @veenstrajelmer in #470 and #494

  • clarifying error message when providing incorrectly formatted cds api key by @JulienGroenenboom in #514

Fix#

0.13.1 (2023-07-12)#

Fix#

  • proper fix for FM-compatible network file (1-based face_node_connectivity) by @veenstrajelmer in #461

0.13.0 (2023-07-12)#

Feat#

Fix#

  • added dfmt.uds_to_1based_ds() to ensure FM-compatible network file (1-based face_node_connectivity) by @veenstrajelmer in #456

0.12.0 (2023-07-07)#

Feat#

  • added support for curvilinear datasets like CMCC and WAQUA with open_dataset_curvilinear() by @veenstrajelmer in #397

  • added support for curvilinear Delft3D4 datasets with open_dataset_delft3d4() by @veenstrajelmer in #398

  • retrieving GSHHS landboundary with get_coastlines_gdb() and cartopy-alternative plot_coastlines() with this dataset by @veenstrajelmer in #342

  • added interpolation from UgridDataset (D-FlowFM mapfile) to plipoints with interp_uds_to_plipoints() by @veenstrajelmer #387

  • added geodataframe support from polyfile with PolyFile_to_geodataframe_points() by @veenstrajelmer in #387

Fix#

  • proper selection for either CMEMS reanalysis/forecast product in download_CMEMS() by @JulienGroenenboom in #308

  • slightly extend spatial and time domain when downloading ERA5 and CMEMS data by @JulienGroenenboom in #390

  • renamed uxuy to uxuyadvectionvelocitybnd in open_dataset_extra() and related example script by @veenstrajelmer in #366

  • add warning when interpolating to a boundary that contains multiple polylines by @Tammo-Zijlker-Deltares in #372

  • improved netcdf writing from meshkernel+xugrid generated network with add_crs_to_dataset() by @veenstrajelmer in #422

0.11.0 (2023-04-24)#

Replaced large parts of the code with HYDROLIB-core, xugrid and xarray. This means a non-backwards compatible change in the API but improved experience. xugrid gives loads of new features like coordinate conversion, shapefile export, cropping a unstructured dataset, regridding, rasterizing.

Feat#

  • improved support for ugrid datasets (D-FlowFM, D-HYDRO) with open_partitioned_dataset() by @veenstrajelmer in several PRs, this deprecates get_ncmodeldata(), get_netdata(), plot_netmapdata() and get_ugrid_verts()

  • improved rasterization of ugrid datasets with rasterize_ugrid() by @veenstrajelmer in #314, this deprecates scatter_to_regulargrid()

  • depth slicing of ugrid datasets with get_Dataset_atdepths() by @veenstrajelmer in #261

  • polygon slicing of ugrid datasets with polyline_mapslice() by @veenstrajelmer in #266 and #273

  • reconstruction of fullgrid ugrid variable from sigma, zsigma and z-layers with reconstruct_zw_zcc() by @veenstrajelmer in #291 and #293

  • added support for D-FlowFM mapformat 1 by @veenstrajelmer in #249

  • support for plotting of periodic grid with remove_periodic_cells() by @veenstrajelmer in #286

  • renaming water quality variables in dataset with rename_waqvars() by @veenstrajelmer in #259, this deprecates get_varnamefromattrs()

  • renaming variables in fourier dataset with rename_fouvars() by @veenstrajelmer in #283

  • simplified background plotting with contextily and cartopy by @veenstrajelmer in #328, this deprecates plot_background()

  • create release and pypi/pip installable by @veenstrajelmer in #338

  • using HYDROLIB-core for reading/writing of D-FlowFM files, this deprecates write_bcfile(), read_bcfile(), Polygon(), write_timfile() and read_timfile()

  • interpolation of regularly gridded dataset (CMEMS, HYCOM etc) to plipoints with interp_regularnc_to_plipoints() by @veenstrajelmer

  • interpolation of regularly gridded tide datasets to plipoints with interpolate_tide_to_bc() by @veenstrajelmer in #288

  • added modelbuilder by @veenstrajelmer in #305

  • added meshkernelpy helper functions like make_basegrid(), refine_basegrid() and meshkernel_to_UgridDataset() by @veenstrajelmer in #329

Fix#

  • made bedlevel variable optional in get_Dataset_atdepths(), if not explicitly needing it by @veenstrajelmer in #241