veriflow.datasinks.cf_compliant_zarr#
Read and write netcdf files in a fews compatible format.
Classes
|
For writing data to a CF-compliant zarr store. |
|
A cf-compliant Zarr output config element. |
- class veriflow.datasinks.cf_compliant_zarr.CFCompliantZarr(config)[source]#
For writing data to a CF-compliant zarr store.
This datasinks is compatible with both local filesystems and remote object stores.
This datasink writes the full pipeline output to a single zarr store. Each verification pair will be stored as a separate group within the zarr store. Each subgroup corresponds to a specific verification pair and contains both input data and output data.
Note
CF-compliancy is not yet fully implemented.
- Parameters:
config (CFCompliantZarrConfig)
- config_class#
alias of
CFCompliantZarrConfig
- class veriflow.datasinks.cf_compliant_zarr.CFCompliantZarrConfig(*, path, group=None, auth_config=None, storage_options=None, consolidated=None, export_adapter, force_overwrite=True, include_output=True, include_aligned_input_data=True, include_input_data=False, crs=None, general, conventions='CF-1.7', title='Verification results created by veriflow', institution, comment='Verification results created by veriflow', **extra_data)[source]#
A cf-compliant Zarr output config element.
- Parameters:
group (str | None)
auth_config (S3AuthConfig | None)
consolidated (bool | None)
export_adapter (Literal[DataSinkKind.cf_compliant_zarr])
force_overwrite (bool)
include_output (bool)
include_aligned_input_data (bool)
include_input_data (bool)
crs (Annotated[str, FieldInfo(annotation=NoneType, required=True, description="A coordinate reference system as a string compatible with pyproj.CRS.from_string, i.e. a PROJ string, a CRS WKT string, or an authority string such as 'EPSG:4326'. The string is not parsed at configuration time so that pyproj remains an optional dependency; it is only parsed when an actual coordinate transformation is required.", metadata=[MinLen(min_length=1)])] | None)
general (Annotated[GeneralInfoConfig, SkipJsonSchema()])
conventions (Literal['CF-1.7'])
title (str)
institution (str)
comment (str)
extra_data (Any)
- is_remote_path()#
Return True if
pathlooks like a remote/fsspec URL (e.g.s3://).- Return type:
- property resolved_storage_options: dict[str, object] | None#
Build the final storage_options for
xr.open_zarr/to_zarr.Merges ‘auth_config’ (auto-populated from
S3_-prefixed environment variables for remote paths) with the explicit ‘storage_options’ override. ReturnsNonefor local paths so that xarray opens the store directly from the local filesystem. This is the single place that should ever build storage_options from aBaseZarrConfig- duplicating this logic elsewhere risks silently dropping ‘auth_config’ (and thus falling back to boto3’s default credential discovery instead of the configuredS3_-prefixed credentials).
- validate_zarr_path_accessible()#
Check that a local cache dir exists, or initialize S3 auth for remote paths.
- Return type:
- property verification_period: TimePeriod#
- conventions#
- title#
- institution#
- comment#
- force_overwrite#
- include_output#
- include_aligned_input_data#
- include_input_data#
- crs#
- general#
- path#
- group#
- auth_config#
- storage_options#
- consolidated#