What’s new#

UNRELEASED#

0.31.0 (2024-10-28)#

Fix#

  • fixed CDS login via prompt for new users in cds_credentials() in #1035

0.30.0 (2024-10-20)#

Fix#

  • fixed cmems-nrt insitu again by dropping station with varying coordinates again in #1023

  • update executable paths in batfile created by dfmt.create_model_exec_files() in #1030

0.29.0 (2024-09-27)#

Feat#

  • update from CDS-Beta to new CDS in #1013

0.28.0 (2024-09-26)#

Fix#

  • more robust CDS/ECMWF authentication check and support for new cdsapi versions in #1004

  • set default copernicusmarine buffer to 0.5 instead of 0 in #1009

0.27.0 (2024-09-09)#

Fix#

  • simplified dfmt.meshkernel_to_UgridDataset() by using new xugrid version in #991

  • updated dfmt.uda_to_faces() to accomodate for new fill_value of xugrid (this version includes fix for contour/contourf) #992

0.26.0 (2024-09-03)#

Fix#

  • properly assigning units attribute in ds_apply_conversion_dict() (it did not always stick) in #965

  • skipping initialwaterlevel in dfmt.cmems_nc_to_ini() #970

  • update to cdsapi 0.7.2 and properly catching error for dummy dataset in #972

  • deprecated dfmt.open_dataset_extra() (partly replaced by dfmt.open_prepare_dataset()) in #974

  • improved nan-conversion in dfmt.forcinglike_to_Dataset() in #982

  • improved performance of dfmt.open_partitioned_dataset() for datasets with many variables in #984

0.25.0 (2024-08-16)#

Feat#

  • making dfmt.open_dataset_extra() more modular by partly moving it to separate private functions in #913

  • added station_id variable to dataset returned by dfmt.interp_uds_to_plipoints() in #914

  • update private functions under dfmt.download_ERA5() to CDS-Beta (requires ECMWF apikey instead) in #925

  • simplified prevention of int dtypes in dfmt.preprocess_ERA5() in #943

  • simplified dfmt.open_dataset_extra() by dropping multi-quantity support and converted to private function in #946

  • improved dfmt.interp_uds_to_plipoints() by supporting outofbound points and new xugrid version in #948

  • neater chunking warning filter in dfmt.open_partitioned_dataset() in #952

  • performance improvement of dfmt.open_partitioned_dataset() by setting remove_edges=False in #960

Fix#

  • also apply convert_360to180 to longitude variable in dfmt.open_dataset_curvilinear() in #913

  • prevented adding of time dimension and dropped 0-sized cells in dfmt.open_dataset_curvilinear() in #929

  • dropping lat/lon/verts variables to maintain single source of truth in dfmt.open_dataset_curvilinear() in #931

  • fix for 3D initial fields in dfmt.cmems_nc_to_ini() to avoid top layer values over the entire depth in #955

0.24.0 (2024-07-12)#

Feat#

  • improved usability of dfmt.LineBuilder() in #854

  • added workaround for grids that are not orthogonal after cutting the land with dfmt.meshkernel_get_illegalcells() in #866

  • updated CMEMS bcg multiyear dataset name in #880

  • added CMEMS reananalysis-interim (myint) datasets to dfmt.download_CMEMS() in #883 and #903

  • avoid duplicate and empty polyline names in dfmt.geodataframe_to_PolyFile() in #896

  • support for multiple polylines per polyfile in interpolate_tide_to_bc() and cmems_nc_to_bc() in #906

  • add to ext as part of interpolate_tide_to_bc(), it now requires ext_new and has no return value anymore in #906

Fix#

  • cleanups for datasets retrieved with dfmt.ssh_retrieve_data() in #867

0.23.0 (2024-05-15)#

Feat#

  • added conversion_dict parameter to dfmt.cmems_nc_to_bc() in #841

Fix#

  • improved dfmt.download.copernicusmarine_reset() by changing order of tasks in #833

  • improved performance of dfmt.merge_meteofiles() by using more generic chunking method in #840

  • improved dfmt.open_dataset_delft3d4() by dropping original grid variable and attrs in #844

0.22.0 (2024-04-09)#

Feat#

  • generate kml file from ssh_catalog (geopandas.GeoDataFrame) with dfmt.ssh_catalog_tokmlfile() in #809

  • generate time availability overview and statistics from insitu netcdf files with dfmt.ssh_netcdf_overview() in #807

  • simplified rwsddl_ssh_retrieve_data() by using ddlpy.dataframe_to_xarray() in #805

  • updated cmems analysisforecast bio dataset ids in #816

  • from ftp to copernicusmarine files service for insitu data in #818

  • simplified dfmt.download.copernicusmarine_credentials() in #819

Fix#

  • fixed xy coordinates for cmems insitu netcdf files in #807

0.21.0 (2024-03-14)#

Feat#

  • added ddl/rws insitu data to dfmt.ssh_catalog_subset() and dfmt.ssh_retrieve_data() in #791 and #796

  • added cmems nrt insitu data to dfmt.ssh_catalog_subset() and dfmt.ssh_retrieve_data() in #791 and #793

  • support for meshkernel>=4.1.0 in #801

0.20.0 (2024-02-22)#

Fix#

  • solve xarray FutureWarning about ds.dims[dim] by @veenstrajelmer in #779

  • support copernicusmarine>=1.0.2 in copernicusmarine_credentials() and made copernicusmarine_reset() modular by @veenstrajelmer in #787

0.19.0 (2024-02-08)#

Feat#

  • automatically parse epsg from FM mapfile to crs in dfmt.open_partitioned_dataset() by @veenstrajelmer in #685

  • added spatial/temporal subsetting and retrieving of insitu observation data with dfmt.ssh_catalog_subset() and dfmt.ssh_retrieve_data() by @veenstrajelmer in #711

  • added conversion of absolute to relative paths for modelbuilder with dfmt.make_paths_relative() by @veenstrajelmer in #723

  • add retrieval and plotting of borders with dfmt.get_borders_gdb() and dfmt.plot_borders() by @veenstrajelmer in #750

  • improve z and sigmaz layer reconstruction (nans below bed and above waterlevel and centers as mean of interfaces) in reconstruct_zw_zcc() by @veenstrajelmer in #755 and #759

  • simplified correctly merging of ERA5 int16 meteofiles in merge_meteofiles() by @veenstrajelmer in #738

  • expecting da instead of ds in refine_basegrid() by @veenstrajelmer in #762

  • added support for Python 3.12 by @veenstrajelmer in #763

  • moved from copernicus-marine-client to copernicusmarine in dfmt.download_CMEMS() by @veenstrajelmer in #721 and #772

  • added reset of Copernicus Marine Toolbox with dfmt.download.copernicusmarine_reset() by @veenstrajelmer in #714

Fix#

Deprecated#

  • deprecated dfmt.preprocess_hirlam() since xarray>=2023.9.0 supports multidimensional coordinates by @veenstrajelmer in #770

0.18.0 (2023-12-08)#

Feat#

  • replaced is_geographic with crs argument in dfmt.make_basegrid() by @veenstrajelmer in #696

  • moved from OPeNDAP to copernicus-marine-client in dfmt.download_CMEMS() including safer authentication by @veenstrajelmer in #691

  • added timerange check of resulting dataset of dfmt.open_dataset_extra() by @veenstrajelmer in #708

0.17.0 (2023-11-17)#

Support for Python 3.8 was dropped (more info)

Feat#

Fix#

0.16.0 (2023-11-03)#

Feat#

  • more robust support for CMEMS/CDS credentials including environment variables by @veenstrajelmer in #633

  • enrich rst file with topology from corresponding mapfile with dfmt.enrich_rst_with_map() by @veenstrajelmer in #626

  • add cellinfo to minimal 2D networks (with 1D topology) with dfmt.add_network_cellinfo() by @veenstrajelmer in #624

  • xugrid feature uds.ugrid.to_nonperiodic() deprecates dfmt.remove_periodic_cells() by @veenstrajelmer in #503

  • support for initial fields for variables other than salinity/temperature with dfmt.cmems_nc_to_ini() by @veenstrajelmer in #619

Fix#

  • fix initial fields for salinity/temperature with dfmt.cmems_nc_to_ini() by @veenstrajelmer in #619

  • increased buffer in dfmt.download_ERA5() by @veenstrajelmer in #612

  • support for Polygon geometries in dfmt.geodataframe_to_PolyFile() by @veenstrajelmer in #610

  • fill nan-values in initial salinity/temperature netcdf dataset in dfmt.preprocess_ini_cmems_to_nc() by @veenstrajelmer in #617

  • skip all-nan boundary support points instead of converting to zeros in dfmt.plipointsDataset_to_ForcingModel() by @veenstrajelmer in #637

0.15.0 (2023-10-19)#

Feat#

  • create dimr_config.xml and run_parallel.bat with dfmt.create_model_exec_files() by @veenstrajelmer in #599

  • conversion of hcdfm.ForcingModel to xr.Dataset with dfmt.ForcingModel_to_plipointsDataset() by @veenstrajelmer in #567

  • support for reading asc files with dfmt.read_asc() by @veenstrajelmer in #571

  • added dfmt.meshkernel_delete_withshp() to delete parts of a mesh with a shapefile, while only reading the shapefile within the bounding box of the mesh by @rqwang in #548 and #566

  • improved spatial interpolation in dfmt.interp_regularnc_to_plipoints() by combining linear with nearest interpolation by @veenstrajelmer in #561

  • added GTSMv4.1 and GTSMv4.1_opendap datasets for tide interpolation with dfmt.interpolate_tide_to_bc() by @veenstrajelmer in #544

  • support for preprocess argument in dfmt.open_partitioned_dataset() by @veenstrajelmer in #530

Fix#

  • increased buffer for WAQ cmems variables by @veenstrajelmer in #601

  • prevent concatenation of datasets with slightly different coordinates by @veenstrajelmer in #595

  • aligned ncbnd dataset with FM/FEWS conventions by @veenstrajelmer in #567

  • documented minimal dependency versions to avoid issues with installation in existing environments containing old dependency versions by @veenstrajelmer in #581

0.14.0 (2023-09-15)#

Feat#

  • decoding of default fillvalues with dfmt.decode_default_fillvals() by @veenstrajelmer in #496

  • interpolation of edge variables to faces with dfmt.uda_edges_to_faces() by @veenstrajelmer in #482

  • interpolation of variables on layer interfaces to layer centers with dfmt.uda_interfaces_to_centers() by @veenstrajelmer in #482

  • generate polyline only for open boundary with dfmt.generate_bndpli_cutland() (deprecates dfmt.generate_bndpli()) by @veenstrajelmer in #466

  • automatically remove unassociated edges with dfmt.remove_unassociated_edges() by @veenstrajelmer in #470 and #494

  • clarifying error message when providing incorrectly formatted cds api key by @JulienGroenenboom in #514

Fix#

  • removed matplotlib._api import to also support matplotlib<3.4.0 by @veenstrajelmer in #465

0.13.1 (2023-07-12)#

Fix#

  • proper fix for FM-compatible network file (1-based face_node_connectivity) by @veenstrajelmer in #461

0.13.0 (2023-07-12)#

Feat#

Fix#

  • added dfmt.uds_to_1based_ds() to ensure FM-compatible network file (1-based face_node_connectivity) by @veenstrajelmer in #456

0.12.0 (2023-07-07)#

Feat#

  • added support for curvilinear datasets like CMCC and WAQUA with open_dataset_curvilinear() by @veenstrajelmer in #397

  • added support for curvilinear Delft3D4 datasets with open_dataset_delft3d4() by @veenstrajelmer in #398

  • retrieving GSHHS landboundary with get_coastlines_gdb() and cartopy-alternative plot_coastlines() with this dataset by @veenstrajelmer in #342

  • added interpolation from UgridDataset (D-FlowFM mapfile) to plipoints with interp_uds_to_plipoints() by @veenstrajelmer #387

  • added geodataframe support from polyfile with PolyFile_to_geodataframe_points() by @veenstrajelmer in #387

Fix#

  • proper selection for either CMEMS reanalysis/forecast product in download_CMEMS() by @JulienGroenenboom in #308

  • slightly extend spatial and time domain when downloading ERA5 and CMEMS data by @JulienGroenenboom in #390

  • renamed uxuy to uxuyadvectionvelocitybnd in open_dataset_extra() and related example script by @veenstrajelmer in #366

  • add warning when interpolating to a boundary that contains multiple polylines by @Tammo-Zijlker-Deltares in #372

  • improved netcdf writing from meshkernel+xugrid generated network with add_crs_to_dataset() by @veenstrajelmer in #422

0.11.0 (2023-04-24)#

Replaced large parts of the code with HYDROLIB-core, xugrid and xarray. This means a non-backwards compatible change in the API but improved experience. xugrid gives loads of new features like coordinate conversion, shapefile export, cropping a unstructured dataset, regridding, rasterizing.

Feat#

  • improved support for ugrid datasets (D-FlowFM, D-HYDRO) with open_partitioned_dataset() by @veenstrajelmer in several PRs, this deprecates get_ncmodeldata(), get_netdata(), plot_netmapdata() and get_ugrid_verts()

  • improved rasterization of ugrid datasets with rasterize_ugrid() by @veenstrajelmer in #314, this deprecates scatter_to_regulargrid()

  • depth slicing of ugrid datasets with get_Dataset_atdepths() by @veenstrajelmer in #261

  • polygon slicing of ugrid datasets with polyline_mapslice() by @veenstrajelmer in #266 and #273

  • reconstruction of fullgrid ugrid variable from sigma, zsigma and z-layers with reconstruct_zw_zcc() by @veenstrajelmer in #291 and #293

  • added support for D-FlowFM mapformat 1 by @veenstrajelmer in #249

  • support for plotting of periodic grid with remove_periodic_cells() by @veenstrajelmer in #286

  • renaming water quality variables in dataset with rename_waqvars() by @veenstrajelmer in #259, this deprecates get_varnamefromattrs()

  • renaming variables in fourier dataset with rename_fouvars() by @veenstrajelmer in #283

  • simplified background plotting with contextily and cartopy by @veenstrajelmer in #328, this deprecates plot_background()

  • create release and pypi/pip installable by @veenstrajelmer in #338

  • using HYDROLIB-core for reading/writing of D-FlowFM files, this deprecates write_bcfile(), read_bcfile(), Polygon(), write_timfile() and read_timfile()

  • interpolation of regularly gridded dataset (CMEMS, HYCOM etc) to plipoints with interp_regularnc_to_plipoints() by @veenstrajelmer

  • interpolation of regularly gridded tide datasets to plipoints with interpolate_tide_to_bc() by @veenstrajelmer in #288

  • added modelbuilder by @veenstrajelmer in #305

  • added meshkernelpy helper functions like make_basegrid(), refine_basegrid() and meshkernel_to_UgridDataset() by @veenstrajelmer in #329

Fix#

  • made bedlevel variable optional in get_Dataset_atdepths(), if not explicitly needing it by @veenstrajelmer in #241