Skip to content

INI format

The ini module provides the generic logic for parsing Deltares ini based files, such as the mdu, structures files, as well as more complex files such as the boundary condition (bc) files.

Note that specific attribute files that employ this ini format often have their own dedicated module (and separate API doc page). These include:

Following below is the documentation for the INI format base classes.

Model

DataBlockINIBasedModel

Bases: INIBasedModel

DataBlockINIBasedModel defines the base model for ini models with datablocks.

This class extends the functionality of INIBasedModel to handle structured data blocks commonly found in INI files. It provides validation, serialization, and conversion methods for working with these data blocks.

Attributes:

Name Type Description
datablock Datablock

(class attribute) the actual data columns.

datablock (List[List[Union[float, str]]]): A two-dimensional list representing the data block. Each sub-list corresponds to a row in the data block, and the values can be either floats or strings.

Parameters:

Name Type Description Default
datablock List[List[Union[float, str]]]

The initial data block for the model. Defaults to an empty list.

required

Raises:

Type Description
ValueError

If a NaN value is found within the data block.

See Also

INIBasedModel: The parent class for models representing INI-based configurations. INISerializerConfig: Provides configuration for INI serialization.

Examples:

Create a model and validate its data block:

>>> from hydrolib.core.dflowfm.ini.models import DataBlockINIBasedModel
>>> model = DataBlockINIBasedModel(datablock=[[1.0, 2.0], [3.0, 4.0]])
>>> print(model.datablock)
[[1.0, 2.0], [3.0, 4.0]]

Attempt to create a model with invalid data:

>>> try:
...     model = DataBlockINIBasedModel(datablock=[[1.0, None]])
... except Exception as e:
...     print(e)
1 validation error for DataBlockINIBasedModel
datablock -> 0 -> 1
  none is not an allowed value (type=type_error.none.not_allowed)

Notes
  • The class includes a validator to ensure that no NaN values are present in the data block.
  • Data blocks are converted to a serialized format for writing to INI files.
Source code in hydrolib/core/dflowfm/ini/models.py
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
class DataBlockINIBasedModel(INIBasedModel):
    """DataBlockINIBasedModel defines the base model for ini models with datablocks.

    This class extends the functionality of INIBasedModel to handle structured data blocks
    commonly found in INI files. It provides validation, serialization, and conversion methods
    for working with these data blocks.

    Attributes:
        datablock (Datablock): (class attribute) the actual data columns.

    Attributes:
    datablock (List[List[Union[float, str]]]):
        A two-dimensional list representing the data block. Each sub-list corresponds to
        a row in the data block, and the values can be either floats or strings.

    Args:
        datablock (List[List[Union[float, str]]], optional):
            The initial data block for the model. Defaults to an empty list.

    Raises:
        ValueError: If a NaN value is found within the data block.

    See Also:
        INIBasedModel: The parent class for models representing INI-based configurations.
        INISerializerConfig: Provides configuration for INI serialization.

    Examples:
        Create a model and validate its data block:
            ```python
            >>> from hydrolib.core.dflowfm.ini.models import DataBlockINIBasedModel
            >>> model = DataBlockINIBasedModel(datablock=[[1.0, 2.0], [3.0, 4.0]])
            >>> print(model.datablock)
            [[1.0, 2.0], [3.0, 4.0]]

            ```

        Attempt to create a model with invalid data:
            ```python
            >>> try:
            ...     model = DataBlockINIBasedModel(datablock=[[1.0, None]])
            ... except Exception as e:
            ...     print(e)
            1 validation error for DataBlockINIBasedModel
            datablock -> 0 -> 1
              none is not an allowed value (type=type_error.none.not_allowed)

            ```

    Notes:
        - The class includes a validator to ensure that no NaN values are present in the data block.
        - Data blocks are converted to a serialized format for writing to INI files.
    """

    datablock: Datablock = Field(default_factory=list)

    _make_lists = make_list_validator("datablock")

    @classmethod
    def _get_unknown_keyword_error_manager(cls) -> Optional[UnknownKeywordErrorManager]:
        """
        The DataBlockINIBasedModel does not need to raise an error on unknown keywords.

        Returns:
            Optional[UnknownKeywordErrorManager]: Returns None as unknown keywords are ignored.
        """
        return None

    def as_dataframe(self) -> DataFrame:
        """Convert the datablock as a pandas DataFrame

        - The first number from each list in the block as an index for that row.

        Returns:
            DataFrame: The datablock as a pandas DataFrame.

        Examples:
                >>> from hydrolib.core.dflowfm.ini.models import DataBlockINIBasedModel
                >>> model = DataBlockINIBasedModel(datablock=[[0, 10, 100], [1, 20, 200]])
                >>> df = model.as_dataframe()
                >>> print(df)
                        0      1
                0.0  10.0  100.0
                1.0  20.0  200.0
        """
        df = DataFrame(self.datablock).set_index(0)
        df.index.name = None
        df.columns = range(len(df.columns))
        return df

    def _to_section(
        self,
        config: DataBlockINIBasedSerializerConfig,
        save_settings: ModelSaveSettings,
    ) -> Section:
        """
        Converts the current model to an INI Section representation.

        Args:
            config (DataBlockINIBasedSerializerConfig): Configuration for serializing the data block.
            save_settings (ModelSaveSettings): Settings for saving the model.

        Returns:
            Section: The INI Section containing serialized data and the data block.
        """
        section = super()._to_section(config, save_settings)
        section.datablock = self._to_datablock(config)
        return section

    def _to_datablock(self, config: DataBlockINIBasedSerializerConfig) -> List[List]:
        """
        Converts the data block to a serialized format based on the configuration.

        Args:
            config (DataBlockINIBasedSerializerConfig): Configuration for serializing the data block.

        Returns:
            List[List]: A serialized representation of the data block.
        """
        converted_datablock = []

        for row in self.datablock:
            converted_row = (
                DataBlockINIBasedModel.convert_value(value, config) for value in row
            )
            converted_datablock.append(list(converted_row))

        return converted_datablock

    @classmethod
    def convert_value(
        cls, value: Union[float, str], config: DataBlockINIBasedSerializerConfig
    ) -> str:
        """
        Converts a value in the data block to its serialized string representation.

        Args:
            value (Union[float, str]): The value to be converted.
            config (DataBlockINIBasedSerializerConfig): Configuration for the conversion.

        Returns:
            str: The serialized string representation of the value.
        """
        if isinstance(value, float):
            return f"{value:{config.float_format_datablock}}"

        return value

    @validator("datablock")
    def _validate_no_nans_are_present(cls, datablock: Datablock) -> Datablock:
        """Validate that the datablock does not have any NaN values.

        Args:
            datablock (Datablock): The datablock to validate.

        Raises:
            ValueError: When a NaN is present in the datablock.

        Returns:
            Datablock: The validated datablock.
        """
        if any(cls._is_float_and_nan(value) for row in datablock for value in row):
            raise ValueError("NaN is not supported in datablocks.")

        return datablock

    @staticmethod
    def _is_float_and_nan(value: float) -> bool:
        """
        Determines whether a value is a float and is NaN.

        Args:
            value (float): The value to check.

        Returns:
            bool: True if the value is a NaN float; otherwise, False.
        """
        return isinstance(value, float) and isnan(value)

as_dataframe()

Convert the datablock as a pandas DataFrame

  • The first number from each list in the block as an index for that row.

Returns:

Name Type Description
DataFrame DataFrame

The datablock as a pandas DataFrame.

Examples:

>>> from hydrolib.core.dflowfm.ini.models import DataBlockINIBasedModel
>>> model = DataBlockINIBasedModel(datablock=[[0, 10, 100], [1, 20, 200]])
>>> df = model.as_dataframe()
>>> print(df)
        0      1
0.0  10.0  100.0
1.0  20.0  200.0
Source code in hydrolib/core/dflowfm/ini/models.py
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
def as_dataframe(self) -> DataFrame:
    """Convert the datablock as a pandas DataFrame

    - The first number from each list in the block as an index for that row.

    Returns:
        DataFrame: The datablock as a pandas DataFrame.

    Examples:
            >>> from hydrolib.core.dflowfm.ini.models import DataBlockINIBasedModel
            >>> model = DataBlockINIBasedModel(datablock=[[0, 10, 100], [1, 20, 200]])
            >>> df = model.as_dataframe()
            >>> print(df)
                    0      1
            0.0  10.0  100.0
            1.0  20.0  200.0
    """
    df = DataFrame(self.datablock).set_index(0)
    df.index.name = None
    df.columns = range(len(df.columns))
    return df

convert_value(value, config) classmethod

Converts a value in the data block to its serialized string representation.

Parameters:

Name Type Description Default
value Union[float, str]

The value to be converted.

required
config DataBlockINIBasedSerializerConfig

Configuration for the conversion.

required

Returns:

Name Type Description
str str

The serialized string representation of the value.

Source code in hydrolib/core/dflowfm/ini/models.py
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
@classmethod
def convert_value(
    cls, value: Union[float, str], config: DataBlockINIBasedSerializerConfig
) -> str:
    """
    Converts a value in the data block to its serialized string representation.

    Args:
        value (Union[float, str]): The value to be converted.
        config (DataBlockINIBasedSerializerConfig): Configuration for the conversion.

    Returns:
        str: The serialized string representation of the value.
    """
    if isinstance(value, float):
        return f"{value:{config.float_format_datablock}}"

    return value

INIBasedModel

Bases: BaseModel, ABC

INIBasedModel defines the base model for blocks/chapters inside an INIModel (*.ini file).

  • Abstract base class for representing INI-style configuration file blocks or chapters.
  • This class serves as the foundational model for handling blocks within INI configuration files. It supports creating instances from parsed INI sections, adding arbitrary fields, and ensuring well-defined serialization and deserialization behavior. Subclasses are expected to define specific behavior and headers for their respective INI blocks.

Attributes:

Name Type Description
comments Optional[Comments]

Optional Comments if defined by the user, containing descriptions for all data fields.

Parameters:

Name Type Description Default
comments Optional[Comments]

Comments for the model fields. Defaults to None.

required

Raises:

Type Description
ValueError

If unknown fields are encountered during validation.

See Also

BaseModel: The Pydantic base model extended by this class. INISerializerConfig: Provides configuration for INI serialization.

Examples:

Define a custom INI block subclass:

>>> from hydrolib.core.dflowfm.ini.models import INIBasedModel
>>> class MyModel(INIBasedModel):
...     _header = "MyHeader"
...     field_a: str = "default_value"

Parse an INI section:

>>> from hydrolib.core.dflowfm.ini.io_models import Section
>>> section = Section(header="MyHeader", content=[{"key": "field_a", "value": "value"}])
>>> model = MyModel.parse_obj(section.flatten())
>>> print(model.field_a)
value

Serialize a model to an INI format:

>>> from hydrolib.core.dflowfm.ini.serializer import INISerializerConfig
>>> from hydrolib.core.base.models import ModelSaveSettings
>>> config = INISerializerConfig()
>>> section = model._to_section(config, save_settings=ModelSaveSettings())
>>> print(section.header)
MyHeader

Notes
  • Subclasses can override the _header attribute to define the INI block header.
  • Arbitrary fields can be added dynamically and are included during serialization.
Source code in hydrolib/core/dflowfm/ini/models.py
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
class INIBasedModel(BaseModel, ABC):
    """INIBasedModel defines the base model for blocks/chapters
    inside an INIModel (*.ini file).

    - Abstract base class for representing INI-style configuration file blocks or chapters.
    - This class serves as the foundational model for handling blocks within INI configuration files.
    It supports creating instances from parsed INI sections, adding arbitrary fields, and ensuring
    well-defined serialization and deserialization behavior. Subclasses are expected to define
    specific behavior and headers for their respective INI blocks.

    Attributes:
        comments (Optional[Comments]):
            Optional Comments if defined by the user, containing descriptions for all data fields.

    Args:
        comments (Optional[Comments], optional):
            Comments for the model fields. Defaults to None.

    Raises:
        ValueError: If unknown fields are encountered during validation.

    See Also:
        BaseModel: The Pydantic base model extended by this class.
        INISerializerConfig: Provides configuration for INI serialization.


    Examples:
        Define a custom INI block subclass:
            ```python
            >>> from hydrolib.core.dflowfm.ini.models import INIBasedModel
            >>> class MyModel(INIBasedModel):
            ...     _header = "MyHeader"
            ...     field_a: str = "default_value"

            ```

        Parse an INI section:
            ```python
            >>> from hydrolib.core.dflowfm.ini.io_models import Section
            >>> section = Section(header="MyHeader", content=[{"key": "field_a", "value": "value"}])
            >>> model = MyModel.parse_obj(section.flatten())
            >>> print(model.field_a)
            value

            ```

        Serialize a model to an INI format:
            ```python
            >>> from hydrolib.core.dflowfm.ini.serializer import INISerializerConfig
            >>> from hydrolib.core.base.models import ModelSaveSettings
            >>> config = INISerializerConfig()
            >>> section = model._to_section(config, save_settings=ModelSaveSettings())
            >>> print(section.header)
            MyHeader

            ```

    Notes:
        - Subclasses can override the `_header` attribute to define the INI block header.
        - Arbitrary fields can be added dynamically and are included during serialization.
    """

    _header: str = ""
    _file_path_style_converter = FilePathStyleConverter()
    _scientific_notation_regex = compile(
        r"([\d.]+)([dD])([+-]?\d{1,3})"
    )  # matches a float: 1d9, 1D-3, 1.D+4, etc.

    class Config:
        extra = Extra.ignore
        arbitrary_types_allowed = False

    @classmethod
    def _get_unknown_keyword_error_manager(cls) -> Optional[UnknownKeywordErrorManager]:
        """
        Retrieves the error manager for handling unknown keywords in INI files.

        Returns:
            Optional[UnknownKeywordErrorManager]:
                An instance of the error manager or None if unknown keywords are allowed.
        """
        return UnknownKeywordErrorManager()

    @classmethod
    def _supports_comments(cls) -> bool:
        """
        Indicates whether the model supports comments for its fields.

        Returns:
            bool: True if comments are supported; otherwise, False.
        """
        return True

    @classmethod
    def _duplicate_keys_as_list(cls) -> bool:
        """
        Indicates whether duplicate keys in INI sections should be treated as lists.

        Returns:
            bool: True if duplicate keys should be treated as lists; otherwise, False.
        """
        return False

    @classmethod
    def get_list_delimiter(cls) -> str:
        """List delimiter string that will be used for serializing
        list field values for any IniBasedModel, **if** that field has
        no custom list delimiter.

        This function should be overridden by any subclass for a particular
        filetype that needs a specific/different list separator.
        """
        return " "

    @classmethod
    def get_list_field_delimiter(cls, field_key: str) -> str:
        """List delimiter string that will be used for serializing
        the given field's value.
        The returned delimiter is either the field's custom list delimiter
        if that was specified using Field(.., delimiter=".."), or the
        default list delimiter for the model class that this field belongs
        to.

        Args:
            field_key (str): the original field key (not its alias).

        Returns:
            str: the delimiter string to be used for serializing the given field.
        """
        delimiter = None
        if (field := cls.__fields__.get(field_key)) and isinstance(field, ModelField):
            delimiter = field.field_info.extra.get("delimiter")
        if not delimiter:
            delimiter = cls.get_list_delimiter()

        return delimiter

    class Comments(BaseModel, ABC):
        """
        Represents the comments associated with fields in an INIBasedModel.

        Attributes:
            Arbitrary fields can be added dynamically to store comments.

        Config:
            extra: Extra.allow
                Allows dynamic fields for comments.
            arbitrary_types_allowed: bool
                Indicates that only known types are allowed.
        """

        class Config:
            extra = Extra.allow
            arbitrary_types_allowed = False

    comments: Optional[Comments] = Comments()

    @root_validator(pre=True)
    def _validate_unknown_keywords(cls, values):
        """
        Validates fields and raises errors for unknown keywords.

        Args:
            values (dict): Dictionary of field values to validate.

        Returns:
            dict: Validated field values.

        Raises:
            ValueError: If unknown keywords are found.
        """
        unknown_keyword_error_manager = cls._get_unknown_keyword_error_manager()
        do_not_validate = cls._exclude_from_validation(values)
        if unknown_keyword_error_manager:
            unknown_keyword_error_manager.raise_error_for_unknown_keywords(
                values,
                cls._header,
                cls.__fields__,
                cls._exclude_fields() | do_not_validate,
            )
        return values

    @root_validator(pre=True)
    def _skip_nones_and_set_header(cls, values):
        """Drop None fields for known fields.

        Filters out None values and sets the model header.

        Args:
            values (dict): Dictionary of field values.

        Returns:
            dict: Updated field values with None values removed.
        """
        dropkeys = []
        for k, v in values.items():
            if v is None and k in cls.__fields__.keys():
                dropkeys.append(k)

        logger.info(f"Dropped unset keys: {dropkeys}")
        for k in dropkeys:
            values.pop(k)

        if "_header" in values:
            values["_header"] = cls._header

        return values

    @validator("comments", always=True, allow_reuse=True)
    def comments_matches_has_comments(cls, v):
        """
        Validates the presence of comments if supported by the model.

        Args:
            v (Any): The comments field value.

        Returns:
            Any: Validated comments field value.
        """
        if not cls._supports_comments() and v is not None:
            logging.warning(f"Dropped unsupported comments from {cls.__name__} init.")
            v = None
        return v

    @validator("*", pre=True, allow_reuse=True)
    def replace_fortran_scientific_notation_for_floats(cls, value, field):
        """
        Converts FORTRAN-style scientific notation to standard notation for float fields.

        Args:
            value (Any): The field value to process.
            field (Field): The field being processed.

        Returns:
            Any: The processed field value.
        """
        if field.type_ != float:
            return value

        return cls._replace_fortran_scientific_notation(value)

    @classmethod
    def _replace_fortran_scientific_notation(cls, value):
        """
        Replaces FORTRAN-style scientific notation in a value.

        Args:
            value (Any): The value to process.

        Returns:
            Any: The processed value.
        """
        if isinstance(value, str):
            return cls._scientific_notation_regex.sub(r"\1e\3", value)
        if isinstance(value, list):
            for i, v in enumerate(value):
                if isinstance(v, str):
                    value[i] = cls._scientific_notation_regex.sub(r"\1e\3", v)

        return value

    @classmethod
    def validate(cls: Type["INIBasedModel"], value: Any) -> "INIBasedModel":
        """
        Validates a value as an instance of INIBasedModel.

        Args:
            value (Any): The value to validate.

        Returns:
            INIBasedModel: The validated instance.
        """
        if isinstance(value, Section):
            value = value.flatten(
                cls._duplicate_keys_as_list(), cls._supports_comments()
            )

        return super().validate(value)

    @classmethod
    def _exclude_from_validation(cls, input_data: Optional[dict] = None) -> Set:
        """
        Fields that should not be checked when validating existing fields as they will be dynamically added.

        Args:
            input_data (Optional[dict]): Input data to process.

        Returns:
            Set: Set of field names to exclude from validation.
        """
        return set()

    @classmethod
    def _exclude_fields(cls) -> Set:
        """
        Defines fields to exclude from serialization.

        Returns:
            Set: Set of field names to exclude.
        """
        return {"comments", "datablock", "_header"}

    def _convert_value(
        self,
        key: str,
        v: Any,
        config: INISerializerConfig,
        save_settings: ModelSaveSettings,
    ) -> str:
        """
        Converts a field value to its serialized string representation.

        Args:
            key (str): The field key.
            v (Any): The field value.
            config (INISerializerConfig): Configuration for serialization.
            save_settings (ModelSaveSettings): Settings for saving the model.

        Returns:
            str: The serialized value.
        """
        if isinstance(v, bool):
            return str(int(v))
        elif isinstance(v, list):
            convert_value = lambda x: self._convert_value(key, x, config, save_settings)
            return self.__class__.get_list_field_delimiter(key).join(
                [convert_value(x) for x in v]
            )
        elif isinstance(v, Enum):
            return v.value
        elif isinstance(v, float):
            return f"{v:{config.float_format}}"
        elif isinstance(v, FileModel) and v.filepath is not None:
            return self._file_path_style_converter.convert_from_os_style(
                v.filepath, save_settings.path_style
            )
        elif v is None:
            return ""
        else:
            return str(v)

    def _to_section(
        self, config: INISerializerConfig, save_settings: ModelSaveSettings
    ) -> Section:
        """
        Converts the model to an INI section.

        Args:
            config (INISerializerConfig): Configuration for serialization.
            save_settings (ModelSaveSettings): Settings for saving the model.

        Returns:
            Section: The INI section representation of the model.
        """
        props = []
        for key, value in self:
            if not self._should_be_serialized(key, value, save_settings):
                continue

            field_key = key
            if key in self.__fields__:
                key = self.__fields__[key].alias

            prop = Property(
                key=key,
                value=self._convert_value(field_key, value, config, save_settings),
                comment=getattr(self.comments, field_key, None),
            )
            props.append(prop)
        return Section(header=self._header, content=props)

    def _should_be_serialized(
        self, key: str, value: Any, save_settings: ModelSaveSettings
    ) -> bool:
        """
        Determines if a field should be serialized.

        Args:
            key (str): The field key.
            value (Any): The field value.
            save_settings (ModelSaveSettings): Settings for saving the model.

        Returns:
            bool: True if the field should be serialized; otherwise, False.
        """
        if key in self._exclude_fields():
            return False

        if save_settings._exclude_unset and key not in self.__fields_set__:
            return False

        field = self.__fields__.get(key)
        if not field:
            return value is not None

        field_type = field.type_
        if self._is_union(field_type):
            return value is not None or self._union_has_filemodel(field_type)

        if self._is_list(field_type):
            field_type = get_args(field_type)[0]

        return self._value_is_not_none_or_type_is_filemodel(field_type, value)

    @staticmethod
    def _is_union(field_type: type) -> bool:
        """
        Checks if a type is a Union.

        Args:
            field_type (type): The type to check.

        Returns:
            bool: True if the type is a Union; otherwise, False.
        """
        return get_origin(field_type) is Union

    @staticmethod
    def _union_has_filemodel(field_type: type) -> bool:
        """
        Checks if a Union type includes a FileModel subtype.

        Args:
            field_type (type): The type to check.

        Returns:
            bool: True if the Union includes a FileModel; otherwise, False.
        """
        return any(
            isclass(arg) and issubclass(arg, FileModel) for arg in get_args(field_type)
        )

    @staticmethod
    def _is_list(field_type: type) -> bool:
        """
        Checks if a type is a list.

        Args:
            field_type (type): The type to check.

        Returns:
            bool: True if the type is a list; otherwise, False.
        """
        return get_origin(field_type) is List

    @staticmethod
    def _value_is_not_none_or_type_is_filemodel(field_type: type, value: Any) -> bool:
        """
        Checks if a value is not None or if its type is FileModel.

        Args:
        field_type (type): The expected type of the field.
        value (Any): The value to check.

        Returns:
            bool: True if the value is valid; otherwise, False.
        """
        return value is not None or issubclass(field_type, FileModel)

Comments

Bases: BaseModel, ABC

Represents the comments associated with fields in an INIBasedModel.

Config

extra: Extra.allow Allows dynamic fields for comments. arbitrary_types_allowed: bool Indicates that only known types are allowed.

Source code in hydrolib/core/dflowfm/ini/models.py
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
class Comments(BaseModel, ABC):
    """
    Represents the comments associated with fields in an INIBasedModel.

    Attributes:
        Arbitrary fields can be added dynamically to store comments.

    Config:
        extra: Extra.allow
            Allows dynamic fields for comments.
        arbitrary_types_allowed: bool
            Indicates that only known types are allowed.
    """

    class Config:
        extra = Extra.allow
        arbitrary_types_allowed = False

comments_matches_has_comments(v)

Validates the presence of comments if supported by the model.

Parameters:

Name Type Description Default
v Any

The comments field value.

required

Returns:

Name Type Description
Any

Validated comments field value.

Source code in hydrolib/core/dflowfm/ini/models.py
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
@validator("comments", always=True, allow_reuse=True)
def comments_matches_has_comments(cls, v):
    """
    Validates the presence of comments if supported by the model.

    Args:
        v (Any): The comments field value.

    Returns:
        Any: Validated comments field value.
    """
    if not cls._supports_comments() and v is not None:
        logging.warning(f"Dropped unsupported comments from {cls.__name__} init.")
        v = None
    return v

get_list_delimiter() classmethod

List delimiter string that will be used for serializing list field values for any IniBasedModel, if that field has no custom list delimiter.

This function should be overridden by any subclass for a particular filetype that needs a specific/different list separator.

Source code in hydrolib/core/dflowfm/ini/models.py
156
157
158
159
160
161
162
163
164
165
@classmethod
def get_list_delimiter(cls) -> str:
    """List delimiter string that will be used for serializing
    list field values for any IniBasedModel, **if** that field has
    no custom list delimiter.

    This function should be overridden by any subclass for a particular
    filetype that needs a specific/different list separator.
    """
    return " "

get_list_field_delimiter(field_key) classmethod

List delimiter string that will be used for serializing the given field's value. The returned delimiter is either the field's custom list delimiter if that was specified using Field(.., delimiter=".."), or the default list delimiter for the model class that this field belongs to.

Parameters:

Name Type Description Default
field_key str

the original field key (not its alias).

required

Returns:

Name Type Description
str str

the delimiter string to be used for serializing the given field.

Source code in hydrolib/core/dflowfm/ini/models.py
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
@classmethod
def get_list_field_delimiter(cls, field_key: str) -> str:
    """List delimiter string that will be used for serializing
    the given field's value.
    The returned delimiter is either the field's custom list delimiter
    if that was specified using Field(.., delimiter=".."), or the
    default list delimiter for the model class that this field belongs
    to.

    Args:
        field_key (str): the original field key (not its alias).

    Returns:
        str: the delimiter string to be used for serializing the given field.
    """
    delimiter = None
    if (field := cls.__fields__.get(field_key)) and isinstance(field, ModelField):
        delimiter = field.field_info.extra.get("delimiter")
    if not delimiter:
        delimiter = cls.get_list_delimiter()

    return delimiter

replace_fortran_scientific_notation_for_floats(value, field)

Converts FORTRAN-style scientific notation to standard notation for float fields.

Parameters:

Name Type Description Default
value Any

The field value to process.

required
field Field

The field being processed.

required

Returns:

Name Type Description
Any

The processed field value.

Source code in hydrolib/core/dflowfm/ini/models.py
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
@validator("*", pre=True, allow_reuse=True)
def replace_fortran_scientific_notation_for_floats(cls, value, field):
    """
    Converts FORTRAN-style scientific notation to standard notation for float fields.

    Args:
        value (Any): The field value to process.
        field (Field): The field being processed.

    Returns:
        Any: The processed field value.
    """
    if field.type_ != float:
        return value

    return cls._replace_fortran_scientific_notation(value)

validate(value) classmethod

Validates a value as an instance of INIBasedModel.

Parameters:

Name Type Description Default
value Any

The value to validate.

required

Returns:

Name Type Description
INIBasedModel INIBasedModel

The validated instance.

Source code in hydrolib/core/dflowfm/ini/models.py
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
@classmethod
def validate(cls: Type["INIBasedModel"], value: Any) -> "INIBasedModel":
    """
    Validates a value as an instance of INIBasedModel.

    Args:
        value (Any): The value to validate.

    Returns:
        INIBasedModel: The validated instance.
    """
    if isinstance(value, Section):
        value = value.flatten(
            cls._duplicate_keys_as_list(), cls._supports_comments()
        )

    return super().validate(value)

INIModel

Bases: ParsableFileModel

INI Model representation of a *.ini file.

Typically subclasses will implement the various sorts of ini files, specifically for their fileType/contents. Child elements of this class associated with chapters/blocks in the ini file will be (sub)class of INIBasedModel.

Source code in hydrolib/core/dflowfm/ini/models.py
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
class INIModel(ParsableFileModel):
    """INI Model representation of a *.ini file.

    Typically subclasses will implement the various sorts of ini files,
    specifically for their fileType/contents.
    Child elements of this class associated with chapters/blocks in the
    ini file will be (sub)class of INIBasedModel.
    """

    serializer_config: INISerializerConfig = INISerializerConfig()

    general: INIGeneral

    @classmethod
    def _ext(cls) -> str:
        return ".ini"

    @classmethod
    def _filename(cls) -> str:
        return "fm"

    @classmethod
    def _get_serializer(cls):
        pass  # unused in favor of direct _serialize

    @classmethod
    def _get_parser(cls) -> Callable:
        return Parser.parse_as_dict

    def _to_document(self, save_settings: ModelSaveSettings) -> Document:
        header = CommentBlock(lines=[f"written by HYDROLIB-core {version}"])
        sections = []
        for key, value in self:
            if key in self._exclude_fields() or value is None:
                continue
            if save_settings._exclude_unset and key not in self.__fields_set__:
                continue
            if isinstance(value, list):
                for v in value:
                    sections.append(
                        v._to_section(self.serializer_config, save_settings)
                    )
            else:
                sections.append(
                    value._to_section(self.serializer_config, save_settings)
                )
        return Document(header_comment=[header], sections=sections)

    def _serialize(self, _: dict, save_settings: ModelSaveSettings) -> None:
        """
        Create a `Document` from the model and write it to the file.
        """
        write_ini(
            self._resolved_filepath,
            self._to_document(save_settings),
            config=self.serializer_config,
        )

Parser

Parser

Parser defines a generic Parser for Deltares ini files.

The Parser can be configured with a ParserConfig object.

Source code in hydrolib/core/dflowfm/ini/parser.py
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
class Parser:
    """Parser defines a generic Parser for Deltares ini files.

    The Parser can be configured with a ParserConfig object.
    """

    class _StateType(IntEnum):
        NO_SECTION_FOUND = 0
        PARSING_PROPERTIES = 1
        PARSING_DATABLOCK = 2

    def __init__(self, config: ParserConfig) -> None:
        """Creates a new Parser configured with the provided config

        Args:
            config (ParserConfig): The configuration of this Parser
        """
        self._config = config
        self._document = Document()
        self._current_section: Optional[_IntermediateSection] = None
        self._current_header_block: Optional[_IntermediateCommentBlock] = None

        self._state = self._StateType.NO_SECTION_FOUND
        self._line_index = 0

        # TODO add invalid blocks
        self._feed_line: Dict[
            Parser._StateType, List[Tuple[Callable[[str], bool], Callable[[str], None]]]
        ] = {
            Parser._StateType.NO_SECTION_FOUND: [
                (self._is_comment, self._handle_header_comment),
                (self._is_section_header, self._handle_next_section_header),
            ],
            Parser._StateType.PARSING_PROPERTIES: [
                (self._is_comment, self._handle_section_comment),
                (self._is_section_header, self._handle_next_section_header),
                (self._is_property, self._handle_property),
                (self._is_datarow, self._handle_new_datarow),
            ],
            Parser._StateType.PARSING_DATABLOCK: [
                (self._is_section_header, self._handle_next_section_header),
                (self._is_datarow, self._handle_datarow),
            ],
        }

        self._handle_emptyline: Dict[Parser._StateType, Callable[[], None]] = {
            self._StateType.NO_SECTION_FOUND: self._finish_current_header_block,
            self._StateType.PARSING_PROPERTIES: self._noop,
            self._StateType.PARSING_DATABLOCK: self._noop,
        }

    def feed_line(self, line: str) -> None:
        """Parse the next line with this Parser.

        Args:
            line (str): The line to parse
        """
        if not self._is_empty_line(line):
            for is_line_type, handle_line_type in self._feed_line[self._state]:
                if is_line_type(line):
                    handle_line_type(line)
                    break
            else:
                # handle exception
                pass
        else:
            self._handle_emptyline[self._state]()

        self._increment_line()

    def finalize(self) -> Document:
        """Finalize parsing and return the constructed Document.

        Returns:
            Document:
                A Document describing the parsed ini file.
        """
        # TODO handle invalid block
        self._finish_current_header_block()
        self._finalise_current_section()
        return self._document

    def _increment_line(self) -> None:
        self._line_index += 1

    def _handle_next_section_header(self, line: str) -> None:
        self._finalise_current_section()
        self._handle_new_section_header(line)

        self._state = Parser._StateType.PARSING_PROPERTIES

    def _handle_new_section_header(self, line: str) -> None:
        section_header = line.strip()[1:-1].strip()
        self._current_section = _IntermediateSection(
            header=section_header, start_line=self._line_index
        )

    def _finalise_current_section(self) -> None:
        if self._current_section is not None:
            self._document.sections.append(self._current_section.finalize())

    def _handle_header_comment(self, line: str) -> None:
        if self._current_header_block is None:
            self._current_header_block = _IntermediateCommentBlock(
                start_line=self._line_index
            )

        comment = self._convert_to_comment(line)
        self._current_header_block.add_comment_line(comment)

    def _handle_section_comment(self, line: str) -> None:
        comment = self._convert_to_comment(line)
        self._current_section.add_comment(comment, self._line_index)  # type: ignore

    def _handle_property(self, line: str) -> None:
        key, valuepart = self._retrieve_key_value(line)
        if valuepart is not None:
            comment, value = self._retrieve_property_comment(valuepart.strip())
        else:
            comment, value = None, None

        prop = Property(key=key, value=value, comment=comment)
        self._current_section.add_property(prop)  # type: ignore

    def _handle_new_datarow(self, line: str) -> None:
        self._handle_datarow(line)
        self._state = Parser._StateType.PARSING_DATABLOCK

    def _handle_datarow(self, line: str) -> None:
        self._current_section.add_datarow(line.split())  # type: ignore

    def _retrieve_property_comment(self, line: str) -> Tuple[Optional[str], str]:
        """Retrieve the comment and value part from the valuestring of a key-value pair.

        The comment retrieval is complicated by the fact that in the Deltares-INI
        dialect, the comment delimiter '#' plays a double role: it may also be used
        to quote string values (for example if the contain spaces).

        Example lines that are supported:
        key = valueAndNoComment
        key = valueA  # and a simple comment
        key = #valueA with possible spaces#
        key = #valueA#  # and a simple comment
        key = #valueA# # and a complicated comment with hashes #1 example
        key = value # and a complicated comment with hashes #2.

        Keywords arguments:
            line (str) -- the partial string of the line containing both value and
                possibly a comment at the end. Note that the "key =" part must already
                have been split off, for example by _retrieve_key_value

        Returns:
            Tuple with the comment and string value, respectively. If no comment is
            present, the first tuple element is None.
        """

        if self._config.parse_comments and self._config.comment_delimiter in line:
            line = line.strip()
            parts = line.split(self._config.comment_delimiter)
            numhash = line.count(self._config.comment_delimiter)
            if numhash == 1:
                # normal value, simple comment: "key =  somevalue # and a comment "
                comment = parts[-1]
                value = parts[0]
            elif line.startswith(self._config.comment_delimiter):
                # hashed value, possible with comment: "key = #somevalue# ..."
                comment = (
                    self._config.comment_delimiter.join(parts[3:])
                    if numhash >= 3
                    else ""
                )

                value = self._config.comment_delimiter.join(parts[0:3])
            else:
                # normal value, comment with maybe more hashes: "key = somevalue #This is comment #2, or two "
                comment = self._config.comment_delimiter.join(parts[1:])
                value = parts[0]
        else:
            comment = ""
            value = line

        return (
            comment if len(comment := comment.strip()) > 0 else None,
            value if len(value := value.strip()) > 0 else None,
        )

    def _retrieve_key_value(self, line: str) -> Tuple[str, Optional[str]]:
        if "=" in line:
            key, value = line.split("=", 1)
            return key.strip(), value if len(value := value.strip()) > 0 else None
        else:
            # if no = exists, due to the previous check we know it will just be a
            # single value
            return line, None

    def _finish_current_header_block(self) -> None:
        if self._current_header_block is not None:
            self._document.header_comment.append(self._current_header_block.finalize())
            self._current_header_block = None

    def _noop(self, *_, **__) -> None:
        # no operation
        pass

    def _is_empty_line(self, line: str) -> bool:
        return len(line.strip()) == 0

    def _is_comment(self, line: str) -> bool:
        return line.strip().startswith(self._config.comment_delimiter)

    def _convert_to_comment(self, line: str) -> str:
        return line.strip()[1:].strip()

    def _is_section_header(self, line: str) -> bool:
        # a header is defined as "[ any-value ]"
        stripped = line.strip()
        return stripped.startswith("[") and stripped.endswith("]")

    def _is_property(self, line: str) -> bool:
        # we assume that we already checked wether it is a comment or
        # a section header.
        return self._config.allow_only_keywords or "=" in line

    def _is_datarow(self, _: str) -> bool:
        # we assume that we already checked whether it is either a comment,
        # section header or a property
        return self._config.parse_datablocks

    @classmethod
    def parse_as_dict(cls, filepath: Path, config: ParserConfig = None) -> dict:
        """
        Parses an INI file without a specific model type and returns it as a dictionary.

        Args:
            filepath (Path): File path to the INI-format file.
            config (ParserConfig, optional): Parser configuration to use. Defaults to None.

        Returns:
            dict: Representation of the parsed INI-file.
        """
        return cls.parse(filepath, config).flatten()

    @classmethod
    def parse(cls, filepath: Path, config: ParserConfig = None) -> Document:
        """
        Parses an INI file without a specific model type and returns it as a Document.

        Args:
            filepath (Path): File path to the INI-format file.
            config (ParserConfig, optional): Parser configuration to use. Defaults to None.

        Returns:
            Document: Representation of the parsed INI-file.
        """
        if not config:
            config = ParserConfig()
        parser = cls(config)

        with filepath.open(encoding="utf8") as f:
            for line in f:
                parser.feed_line(line)

        return parser.finalize()

__init__(config)

Creates a new Parser configured with the provided config

Parameters:

Name Type Description Default
config ParserConfig

The configuration of this Parser

required
Source code in hydrolib/core/dflowfm/ini/parser.py
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
def __init__(self, config: ParserConfig) -> None:
    """Creates a new Parser configured with the provided config

    Args:
        config (ParserConfig): The configuration of this Parser
    """
    self._config = config
    self._document = Document()
    self._current_section: Optional[_IntermediateSection] = None
    self._current_header_block: Optional[_IntermediateCommentBlock] = None

    self._state = self._StateType.NO_SECTION_FOUND
    self._line_index = 0

    # TODO add invalid blocks
    self._feed_line: Dict[
        Parser._StateType, List[Tuple[Callable[[str], bool], Callable[[str], None]]]
    ] = {
        Parser._StateType.NO_SECTION_FOUND: [
            (self._is_comment, self._handle_header_comment),
            (self._is_section_header, self._handle_next_section_header),
        ],
        Parser._StateType.PARSING_PROPERTIES: [
            (self._is_comment, self._handle_section_comment),
            (self._is_section_header, self._handle_next_section_header),
            (self._is_property, self._handle_property),
            (self._is_datarow, self._handle_new_datarow),
        ],
        Parser._StateType.PARSING_DATABLOCK: [
            (self._is_section_header, self._handle_next_section_header),
            (self._is_datarow, self._handle_datarow),
        ],
    }

    self._handle_emptyline: Dict[Parser._StateType, Callable[[], None]] = {
        self._StateType.NO_SECTION_FOUND: self._finish_current_header_block,
        self._StateType.PARSING_PROPERTIES: self._noop,
        self._StateType.PARSING_DATABLOCK: self._noop,
    }

feed_line(line)

Parse the next line with this Parser.

Parameters:

Name Type Description Default
line str

The line to parse

required
Source code in hydrolib/core/dflowfm/ini/parser.py
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
def feed_line(self, line: str) -> None:
    """Parse the next line with this Parser.

    Args:
        line (str): The line to parse
    """
    if not self._is_empty_line(line):
        for is_line_type, handle_line_type in self._feed_line[self._state]:
            if is_line_type(line):
                handle_line_type(line)
                break
        else:
            # handle exception
            pass
    else:
        self._handle_emptyline[self._state]()

    self._increment_line()

finalize()

Finalize parsing and return the constructed Document.

Returns:

Name Type Description
Document Document

A Document describing the parsed ini file.

Source code in hydrolib/core/dflowfm/ini/parser.py
186
187
188
189
190
191
192
193
194
195
196
def finalize(self) -> Document:
    """Finalize parsing and return the constructed Document.

    Returns:
        Document:
            A Document describing the parsed ini file.
    """
    # TODO handle invalid block
    self._finish_current_header_block()
    self._finalise_current_section()
    return self._document

parse(filepath, config=None) classmethod

Parses an INI file without a specific model type and returns it as a Document.

Parameters:

Name Type Description Default
filepath Path

File path to the INI-format file.

required
config ParserConfig

Parser configuration to use. Defaults to None.

None

Returns:

Name Type Description
Document Document

Representation of the parsed INI-file.

Source code in hydrolib/core/dflowfm/ini/parser.py
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
@classmethod
def parse(cls, filepath: Path, config: ParserConfig = None) -> Document:
    """
    Parses an INI file without a specific model type and returns it as a Document.

    Args:
        filepath (Path): File path to the INI-format file.
        config (ParserConfig, optional): Parser configuration to use. Defaults to None.

    Returns:
        Document: Representation of the parsed INI-file.
    """
    if not config:
        config = ParserConfig()
    parser = cls(config)

    with filepath.open(encoding="utf8") as f:
        for line in f:
            parser.feed_line(line)

    return parser.finalize()

parse_as_dict(filepath, config=None) classmethod

Parses an INI file without a specific model type and returns it as a dictionary.

Parameters:

Name Type Description Default
filepath Path

File path to the INI-format file.

required
config ParserConfig

Parser configuration to use. Defaults to None.

None

Returns:

Name Type Description
dict dict

Representation of the parsed INI-file.

Source code in hydrolib/core/dflowfm/ini/parser.py
344
345
346
347
348
349
350
351
352
353
354
355
356
@classmethod
def parse_as_dict(cls, filepath: Path, config: ParserConfig = None) -> dict:
    """
    Parses an INI file without a specific model type and returns it as a dictionary.

    Args:
        filepath (Path): File path to the INI-format file.
        config (ParserConfig, optional): Parser configuration to use. Defaults to None.

    Returns:
        dict: Representation of the parsed INI-file.
    """
    return cls.parse(filepath, config).flatten()

ParserConfig

Bases: BaseModel

ParserConfig defines the configuration options of the Parser

Note that we cannot set both allow_only_keywords and parse_datablocks to True because we cannot distinguish between datablocks and key only properties. As such this will lead to a validation error.

Attributes:

Name Type Description
allow_only_keywords bool

Whether to allow properties with only keys (no '=' or value). Defaults to False.

parse_datablocks bool

Whether to allow parsing of datablocks at the bottom of sections. Defaults to False.

parse_comments bool

Whether we allow parsing of comments defined with the comment_delimeter. Defaults to True.

comment_delimiter str

The character or sequence of character used to define a comment. Defaults to '#'.

Source code in hydrolib/core/dflowfm/ini/parser.py
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
class ParserConfig(BaseModel):
    """ParserConfig defines the configuration options of the Parser

    Note that we cannot set both allow_only_keywords and parse_datablocks to True
    because we cannot distinguish between datablocks and key only properties. As
    such this will lead to a validation error.

    Attributes:
        allow_only_keywords (bool):
            Whether to allow properties with only keys (no '=' or value).
            Defaults to False.
        parse_datablocks (bool):
            Whether to allow parsing of datablocks at the bottom of sections.
            Defaults to False.
        parse_comments (bool):
            Whether we allow parsing of comments defined with the comment_delimeter.
            Defaults to True.
        comment_delimiter (str):
            The character or sequence of character used to define a comment.
            Defaults to '#'.
    """

    allow_only_keywords: bool = False
    parse_datablocks: bool = False
    parse_comments: bool = True
    comment_delimiter: str = "#"

    @validator("parse_datablocks")
    def allow_only_keywods_and_parse_datablocks_leads_should_not_both_be_true(
        cls, parse_datablocks, values
    ):
        # if both allow_only_keywords and parse_datablocks is true, we cannot
        # distinguish between the two, and the parsing will not recognise either
        # properly
        if (
            parse_datablocks
            and "allow_only_keywords" in values
            and values["allow_only_keywords"]
        ):
            raise ValueError(
                "Both parse_datablocks and allow_only_keywords should not be both True."
            )
        return parse_datablocks

Serializer

DataBlockINIBasedSerializerConfig

Bases: INISerializerConfig

Class that holds the configuration settings for INI files with data blocks serialization.

Source code in hydrolib/core/dflowfm/ini/serializer.py
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
class DataBlockINIBasedSerializerConfig(INISerializerConfig):
    """Class that holds the configuration settings for INI files with data blocks serialization."""

    float_format_datablock: str = ""
    """str: The string format that will be used for float serialization of the datablock. If empty, the original number will be serialized. Defaults to an empty string.

    Examples:
        Input value = 123.456

        Format    | Output          | Description
        -------------------------------------------------------------------------------------------------------------------------------------
        ".0f"     | 123             | Format float with 0 decimal places.
        "f"       | 123.456000      | Format float with default (=6) decimal places.
        ".2f"     | 123.46          | Format float with 2 decimal places.
        "+.1f"    | +123.5          | Format float with 1 decimal place with a + or  sign.
        "e"       | 1.234560e+02    | Format scientific notation with the letter 'e' with default (=6) decimal places.
        "E"       | 1.234560E+02    | Format scientific notation with the letter 'E' with default (=6) decimal places.
        ".3e"     | 1.235e+02       | Format scientific notation with the letter 'e' with 3 decimal places.
        "<15"     | 123.456         | Left aligned in space with width 15
        "^15.0f"  |       123       | Center aligned in space with width 15 with 0 decimal places.
        ">15.1e"  |         1.2e+02 | Right aligned in space with width 15 with scientific notation with 1 decimal place.
        "*>15.1f" | **********123.5 | Right aligned in space with width 15 with 1 decimal place and fill empty space with *
        "%"       | 12345.600000%   | Format percentage with default (=6) decimal places.     
        ".3%"     | 12345.600%      | Format percentage with 3 decimal places.  

        More information: https://docs.python.org/3/library/string.html#format-specification-mini-language
    """

float_format_datablock = '' class-attribute instance-attribute

str: The string format that will be used for float serialization of the datablock. If empty, the original number will be serialized. Defaults to an empty string.

Examples:

Input value = 123.456

Format | Output | Description

".0f" | 123 | Format float with 0 decimal places. "f" | 123.456000 | Format float with default (=6) decimal places. ".2f" | 123.46 | Format float with 2 decimal places. "+.1f" | +123.5 | Format float with 1 decimal place with a + or sign. "e" | 1.234560e+02 | Format scientific notation with the letter 'e' with default (=6) decimal places. "E" | 1.234560E+02 | Format scientific notation with the letter 'E' with default (=6) decimal places. ".3e" | 1.235e+02 | Format scientific notation with the letter 'e' with 3 decimal places. "<15" | 123.456 | Left aligned in space with width 15 "^15.0f" | 123 | Center aligned in space with width 15 with 0 decimal places. ">15.1e" | 1.2e+02 | Right aligned in space with width 15 with scientific notation with 1 decimal place. ">15.1f" | ***123.5 | Right aligned in space with width 15 with 1 decimal place and fill empty space with * "%" | 12345.600000% | Format percentage with default (=6) decimal places.
".3%" | 12345.600% | Format percentage with 3 decimal places.

More information: https://docs.python.org/3/library/string.html#format-specification-mini-language

INISerializerConfig

Bases: SerializerConfig

SerializerConfig defines the configuration options of the Serializer

Attributes:

Name Type Description
section_indent int

The number of spaces with which whole sections should be indented. Defaults to 0.

property_indent int

The number of spaces with which properties should be indented relative to the section header (i.e. the full indent equals the section_indent plus property_indent). Defaults to 4.

datablock_indent int

The number of spaces with which datablock rows are indented relative to the section header (i.e. the full indent equals the section_indent plus datablock_indent). Defaults to 8.

datablock_spacing int

The number of spaces between datablock columns. Note that there might be additional offset to ensure . is lined out. Defaults to 2.

comment_delimiter str

The character used to delimit comments. Defaults to '#'.

skip_empty_properties bool

Whether or not to skip properties with a value that is empty or None. Defaults to True.

Source code in hydrolib/core/dflowfm/ini/serializer.py
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
class INISerializerConfig(SerializerConfig):
    """SerializerConfig defines the configuration options of the Serializer

    Attributes:
        section_indent (int):
            The number of spaces with which whole sections should be indented.
            Defaults to 0.
        property_indent (int):
            The number of spaces with which properties should be indented relative to
            the section header (i.e. the full indent equals the section_indent plus
            property_indent). Defaults to 4.
        datablock_indent (int):
            The number of spaces with which datablock rows are indented relative to
            the section header (i.e. the full indent equals the section_indent plus
            datablock_indent). Defaults to 8.
        datablock_spacing (int):
            The number of spaces between datablock columns. Note that there might be
            additional offset to ensure . is lined out. Defaults to 2.
        comment_delimiter (str):
            The character used to delimit comments. Defaults to '#'.
        skip_empty_properties (bool):
            Whether or not to skip properties with a value that is empty or None. Defaults to True.
    """

    section_indent: int = 0
    property_indent: int = 0
    datablock_indent: int = 8
    datablock_spacing: int = 2
    comment_delimiter: str = "#"
    skip_empty_properties: bool = True

    @property
    def total_property_indent(self) -> int:
        """The combined property indentation, i.e. section_indent + property_indent"""
        return self.section_indent + self.property_indent

    @property
    def total_datablock_indent(self) -> int:
        """The combined datablock indentation, i.e. section_indent + datablock_indent"""
        return self.section_indent + self.datablock_indent

total_datablock_indent property

The combined datablock indentation, i.e. section_indent + datablock_indent

total_property_indent property

The combined property indentation, i.e. section_indent + property_indent

MaxLengths

Bases: BaseModel

MaxLengths defines the maxmimum lengths of the parts of a section

Attributes:

Name Type Description
key int

The maximum length of all the keys of the properties within a section. If no properties are present it should be 0.

value int

The maximum length of all the non None values of the properties within a section. If no properties are present, or all values are None, it should be 0.

datablock Optional[Sequence[int]]

The maximum length of the values of each column of the Datablock. If no datablock is present it defaults to None.

Source code in hydrolib/core/dflowfm/ini/serializer.py
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
class MaxLengths(BaseModel):
    """MaxLengths defines the maxmimum lengths of the parts of a section

    Attributes:
        key (int):
            The maximum length of all the keys of the properties within a section.
            If no properties are present it should be 0.
        value (int):
            The maximum length of all the non None values of the properties within a
            section. If no properties are present, or all values are None, it should
            be 0.
        datablock (Optional[Sequence[int]]):
            The maximum length of the values of each column of the Datablock.
            If no datablock is present it defaults to None.
    """

    key: int
    value: int
    datablock: Optional[Sequence[int]] = None

    @classmethod
    def from_section(cls, section: Section) -> "MaxLengths":
        """Generate a MaxLengths instance from the given Section

        Args:
            section (Section): The section of which the MaxLengths are calculated

        Returns:
            MaxLengths: The MaxLengths corresponding with the provided section
        """
        properties = list(p for p in section.content if isinstance(p, Property))

        keys = (prop.key for prop in properties)
        values = (prop.value for prop in properties if prop.value is not None)

        max_key_length = max((len(k) for k in keys), default=0)
        max_value_length = max((len(v) for v in values), default=0)
        max_datablock_lengths = MaxLengths._of_datablock(section.datablock)

        return cls(
            key=max_key_length,
            value=max_value_length,
            datablock=max_datablock_lengths,
        )

    @staticmethod
    def _of_datablock(datablock: Optional[Datablock]) -> Optional[Sequence[int]]:
        if datablock is None or len(datablock) < 1:
            return None

        datablock_columns = map(list, zip(*datablock))
        datablock_column_lengths = (map(len, column) for column in datablock_columns)  # type: ignore
        max_lengths = (max(column) for column in datablock_column_lengths)

        return tuple(max_lengths)

from_section(section) classmethod

Generate a MaxLengths instance from the given Section

Parameters:

Name Type Description Default
section Section

The section of which the MaxLengths are calculated

required

Returns:

Name Type Description
MaxLengths MaxLengths

The MaxLengths corresponding with the provided section

Source code in hydrolib/core/dflowfm/ini/serializer.py
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
@classmethod
def from_section(cls, section: Section) -> "MaxLengths":
    """Generate a MaxLengths instance from the given Section

    Args:
        section (Section): The section of which the MaxLengths are calculated

    Returns:
        MaxLengths: The MaxLengths corresponding with the provided section
    """
    properties = list(p for p in section.content if isinstance(p, Property))

    keys = (prop.key for prop in properties)
    values = (prop.value for prop in properties if prop.value is not None)

    max_key_length = max((len(k) for k in keys), default=0)
    max_value_length = max((len(v) for v in values), default=0)
    max_datablock_lengths = MaxLengths._of_datablock(section.datablock)

    return cls(
        key=max_key_length,
        value=max_value_length,
        datablock=max_datablock_lengths,
    )

SectionSerializer

SectionSerializer provides the serialize method to serialize a Section

The entrypoint of this method is the serialize method, which will construct an actual instance and serializes the Section with it.

Source code in hydrolib/core/dflowfm/ini/serializer.py
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
class SectionSerializer:
    """SectionSerializer provides the serialize method to serialize a Section

    The entrypoint of this method is the serialize method, which will construct
    an actual instance and serializes the Section with it.
    """

    def __init__(self, config: INISerializerConfig, max_length: MaxLengths):
        """Create a new SectionSerializer

        Args:
            config (SerializerConfig): The config describing the serialization options
            max_length (MaxLengths): The max lengths of the section being serialized
        """
        self._config = config
        self._max_length = max_length

    @classmethod
    def serialize(cls, section: Section, config: INISerializerConfig) -> Lines:
        """Serialize the provided section with the given config

        Args:
            section (Section): The section to serialize
            config (SerializerConfig): The config describing the serialization options

        Returns:
            Lines: The iterable lines of the serialized section
        """
        serializer = cls(config, MaxLengths.from_section(section))
        return serializer._serialize_section(section)

    @property
    def config(self) -> INISerializerConfig:
        """The SerializerConfig used while serializing the section."""
        return self._config

    @property
    def max_length(self) -> MaxLengths:
        """The MaxLengths of the Section being serialized by this SectionSerializer."""
        return self._max_length

    def _serialize_section(self, section: Section) -> Lines:
        header_iterable = self._serialize_section_header(section.header)
        properties = self._serialize_content(section.content)
        datablock = self._serialize_datablock(section.datablock)

        return chain(header_iterable, properties, datablock)

    def _serialize_section_header(self, section_header: str) -> Lines:
        indent = " " * (self.config.section_indent)
        yield f"{indent}[{section_header}]"

    def _serialize_content(self, content: Iterable[ContentElement]) -> Lines:
        elements = (self._serialize_content_element(elem) for elem in content)
        return chain.from_iterable(elements)

    def _serialize_content_element(self, elem: ContentElement) -> Lines:
        if isinstance(elem, Property):
            return self._serialize_property(elem)
        else:
            indent = self.config.total_property_indent
            delimiter = self.config.comment_delimiter
            return _serialize_comment_block(elem, delimiter, indent)

    def _serialize_property(self, property: Property) -> Lines:
        if self.config.skip_empty_properties and str_is_empty_or_none(property.value):
            return

        indent = " " * (self._config.total_property_indent)
        key_ws = _get_offset_whitespace(property.key, self.max_length.key)
        key = f"{property.key}{key_ws} = "

        value_ws = _get_offset_whitespace(property.value, self.max_length.value)

        if property.value is not None:
            value = f"{property.value}{value_ws}"
        else:
            value = value_ws

        comment = (
            f" # {property.comment}"
            if not str_is_empty_or_none(property.comment)
            else ""
        )

        yield f"{indent}{key}{value}{comment}".rstrip()

    def _serialize_datablock(self, datablock: Optional[Datablock]) -> Lines:
        if datablock is None or self.max_length.datablock is None:
            return []

        indent = " " * self._config.total_datablock_indent
        return (self._serialize_row(row, indent) for row in datablock)

    def _serialize_row(self, row: DatablockRow, indent: str) -> str:
        elem_spacing = " " * self.config.datablock_spacing
        elems = (self._serialize_row_element(elem, i) for elem, i in zip(row, count()))

        return indent + elem_spacing.join(elems).rstrip()

    def _serialize_row_element(self, elem: str, index: int) -> str:
        max_length = self.max_length.datablock[index]  # type: ignore
        whitespace = _get_offset_whitespace(elem, max_length)
        return elem + whitespace

config property

The SerializerConfig used while serializing the section.

max_length property

The MaxLengths of the Section being serialized by this SectionSerializer.

__init__(config, max_length)

Create a new SectionSerializer

Parameters:

Name Type Description Default
config SerializerConfig

The config describing the serialization options

required
max_length MaxLengths

The max lengths of the section being serialized

required
Source code in hydrolib/core/dflowfm/ini/serializer.py
170
171
172
173
174
175
176
177
178
def __init__(self, config: INISerializerConfig, max_length: MaxLengths):
    """Create a new SectionSerializer

    Args:
        config (SerializerConfig): The config describing the serialization options
        max_length (MaxLengths): The max lengths of the section being serialized
    """
    self._config = config
    self._max_length = max_length

serialize(section, config) classmethod

Serialize the provided section with the given config

Parameters:

Name Type Description Default
section Section

The section to serialize

required
config SerializerConfig

The config describing the serialization options

required

Returns:

Name Type Description
Lines Lines

The iterable lines of the serialized section

Source code in hydrolib/core/dflowfm/ini/serializer.py
180
181
182
183
184
185
186
187
188
189
190
191
192
@classmethod
def serialize(cls, section: Section, config: INISerializerConfig) -> Lines:
    """Serialize the provided section with the given config

    Args:
        section (Section): The section to serialize
        config (SerializerConfig): The config describing the serialization options

    Returns:
        Lines: The iterable lines of the serialized section
    """
    serializer = cls(config, MaxLengths.from_section(section))
    return serializer._serialize_section(section)

Serializer

Serializer serializes Document to its corresponding lines.

Source code in hydrolib/core/dflowfm/ini/serializer.py
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
class Serializer:
    """Serializer serializes Document to its corresponding lines."""

    def __init__(self, config: INISerializerConfig):
        """Creates a new Serializer with the provided configuration.

        Args:
            config (SerializerConfig): The configuration of this Serializer.
        """
        self._config = config

    def serialize(self, document: Document) -> Lines:
        """Serialize the provided document into an iterable of lines.

        Args:
            document (Document): The Document to serialize.

        Returns:
            Lines: An iterable returning each line of the serialized Document.
        """
        header_iterable = self._serialize_document_header(document.header_comment)

        serialize_section = lambda s: SectionSerializer.serialize(s, self._config)
        sections = (serialize_section(section) for section in document.sections)
        sections_with_spacing = Serializer._interweave(sections, [""])
        sections_iterable = chain.from_iterable(sections_with_spacing)

        return chain(header_iterable, sections_iterable)

    def _serialize_document_header(self, header: Iterable[CommentBlock]) -> Lines:
        delimiter = self._config.comment_delimiter
        serialize = lambda cb: _serialize_comment_block(cb, delimiter)
        blocks = (serialize(block) for block in header)
        blocks_with_spacing = Serializer._interweave(blocks, [""])

        return chain.from_iterable(blocks_with_spacing)

    @staticmethod
    def _interweave(iterable: Iterable, val: Any) -> Iterable:
        # Interweave the provided iterable with the provided value:
        # iterable_element, val, iterable_element, val, ...

        # Note that this will interweave with val without making copies
        # as such it is the same object being interweaved.
        return chain.from_iterable(zip(iterable, repeat(val)))

__init__(config)

Creates a new Serializer with the provided configuration.

Parameters:

Name Type Description Default
config SerializerConfig

The configuration of this Serializer.

required
Source code in hydrolib/core/dflowfm/ini/serializer.py
272
273
274
275
276
277
278
def __init__(self, config: INISerializerConfig):
    """Creates a new Serializer with the provided configuration.

    Args:
        config (SerializerConfig): The configuration of this Serializer.
    """
    self._config = config

serialize(document)

Serialize the provided document into an iterable of lines.

Parameters:

Name Type Description Default
document Document

The Document to serialize.

required

Returns:

Name Type Description
Lines Lines

An iterable returning each line of the serialized Document.

Source code in hydrolib/core/dflowfm/ini/serializer.py
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
def serialize(self, document: Document) -> Lines:
    """Serialize the provided document into an iterable of lines.

    Args:
        document (Document): The Document to serialize.

    Returns:
        Lines: An iterable returning each line of the serialized Document.
    """
    header_iterable = self._serialize_document_header(document.header_comment)

    serialize_section = lambda s: SectionSerializer.serialize(s, self._config)
    sections = (serialize_section(section) for section in document.sections)
    sections_with_spacing = Serializer._interweave(sections, [""])
    sections_iterable = chain.from_iterable(sections_with_spacing)

    return chain(header_iterable, sections_iterable)

write_ini(path, document, config)

Write the provided document to the specified path

If the provided path already exists, it will be overwritten. If the parent folder do not exist, they will be created.

Parameters:

Name Type Description Default
path Path

The path to which the document should be written.

required
document Document

The document to serialize to the specified path.

required
config INISerializerConfig

The configuration settings for the serializer.

required
Source code in hydrolib/core/dflowfm/ini/serializer.py
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
def write_ini(path: Path, document: Document, config: INISerializerConfig) -> None:
    """Write the provided document to the specified path

    If the provided path already exists, it will be overwritten. If the parent folder
    do not exist, they will be created.

    Args:
        path (Path): The path to which the document should be written.
        document (Document): The document to serialize to the specified path.
        config (INISerializerConfig): The configuration settings for the serializer.
    """

    serializer = Serializer(config)

    path.parent.mkdir(parents=True, exist_ok=True)

    with path.open("w", encoding="utf8") as f:

        for line in serializer.serialize(document):
            f.write(line + "\n")