veriflow.datasinks.cf_compliant_zarr#

Read and write netcdf files in a fews compatible format.

Classes

CFCompliantZarr(config)

For writing data to a CF-compliant zarr store.

CFCompliantZarrConfig(*, path[, group, ...])

A cf-compliant Zarr output config element.

class veriflow.datasinks.cf_compliant_zarr.CFCompliantZarr(config)[source]#

For writing data to a CF-compliant zarr store.

This datasinks is compatible with both local filesystems and remote object stores.

This datasink writes the full pipeline output to a single zarr store. Each verification pair will be stored as a separate group within the zarr store. Each subgroup corresponds to a specific verification pair and contains both input data and output data.

Note

CF-compliancy is not yet fully implemented.

Parameters:

config (CFCompliantZarrConfig)

kind: str = 'cf_compliant_zarr'#
config_class#

alias of CFCompliantZarrConfig

write_data(dt)[source]#

Write the data in the xarray DataTree to zarr as specified in the output config.

Parameters:

dt (DataTree)

Return type:

None

classmethod from_config(raw_config)#

Initialize class from config dict.

Parameters:

raw_config (dict[str, Any])

Return type:

Self

class veriflow.datasinks.cf_compliant_zarr.CFCompliantZarrConfig(*, path, group=None, auth_config=None, storage_options=None, consolidated=None, export_adapter, force_overwrite=True, include_output=True, include_aligned_input_data=True, include_input_data=False, crs=None, general, conventions='CF-1.7', title='Verification results created by veriflow', institution, comment='Verification results created by veriflow', **extra_data)[source]#

A cf-compliant Zarr output config element.

Parameters:
  • path (Annotated[str, MinLen(min_length=1)])

  • group (str | None)

  • auth_config (S3AuthConfig | None)

  • storage_options (dict[str, str] | None)

  • consolidated (bool | None)

  • export_adapter (Literal[DataSinkKind.cf_compliant_zarr])

  • force_overwrite (bool)

  • include_output (bool)

  • include_aligned_input_data (bool)

  • include_input_data (bool)

  • crs (Annotated[str, FieldInfo(annotation=NoneType, required=True, description="A coordinate reference system as a string compatible with pyproj.CRS.from_string, i.e. a PROJ string, a CRS WKT string, or an authority string such as 'EPSG:4326'. The string is not parsed at configuration time so that pyproj remains an optional dependency; it is only parsed when an actual coordinate transformation is required.", metadata=[MinLen(min_length=1)])] | None)

  • general (Annotated[GeneralInfoConfig, SkipJsonSchema()])

  • conventions (Literal['CF-1.7'])

  • title (str)

  • institution (str)

  • comment (str)

  • extra_data (Any)

export_adapter: Literal[DataSinkKind.cf_compliant_zarr]#
is_remote_path()#

Return True if path looks like a remote/fsspec URL (e.g. s3://).

Return type:

bool

property resolved_storage_options: dict[str, object] | None#

Build the final storage_options for xr.open_zarr/to_zarr.

Merges ‘auth_config’ (auto-populated from S3_-prefixed environment variables for remote paths) with the explicit ‘storage_options’ override. Returns None for local paths so that xarray opens the store directly from the local filesystem. This is the single place that should ever build storage_options from a BaseZarrConfig - duplicating this logic elsewhere risks silently dropping ‘auth_config’ (and thus falling back to boto3’s default credential discovery instead of the configured S3_-prefixed credentials).

validate_zarr_path_accessible()#

Check that a local cache dir exists, or initialize S3 auth for remote paths.

Return type:

Self

property verification_period: TimePeriod#
conventions#
title#
institution#
comment#
force_overwrite#
include_output#
include_aligned_input_data#
include_input_data#
crs#
general#
path#
group#
auth_config#
storage_options#
consolidated#