Karana.KUtils.DataStruct ======================== .. py:module:: Karana.KUtils.DataStruct .. autoapi-nested-parse:: Classes and functions related to DataStruct. DataStruct is a wrapper around `pydantic.BaseModel`. It contains extra functionality to: * Easily save/load `DataStruct`s from various file types. * Compare `DataStruct`s with one another either identically, using `__eq__`, or approximately, using `isApprox`. * Populate a Kclick CLI applications options from a `DataStruct` * Create an instance of a `DataStruct` from Kclick CLI application's options. * Generate an asciidoc with the data of a `DataStruct`. The saving/loading and comparison can be done recursively, meaning they accept `DataStruct`s whose fields are other `DataStructs`. In addition, they supported nested Python types, such as a dict of lists of numpy arrays. Attributes ---------- .. autoapisummary:: Karana.KUtils.DataStruct.SentinelValue Karana.KUtils.DataStruct.T Classes ------- .. autoapisummary:: Karana.KUtils.DataStruct.DataStruct Karana.KUtils.DataStruct.SentinelValueClass Karana.KUtils.DataStruct.DSLinker Karana.KUtils.DataStruct.AllObjsBase Karana.KUtils.DataStruct.IdMixin Karana.KUtils.DataStruct.PyClassDS Karana.KUtils.DataStruct.NestedBaseMixin Karana.KUtils.DataStruct.Meta Module Contents --------------- .. py:class:: DataStruct(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` Wrapper around `pydantic.BaseModel` that adds functionality useful for modeling and simulation. This class adds functionality to the `pydantic.BaseModel`, including: * Easily save/load `DataStruct`s from various file types. * Compare `DataStruct`s with one another either identically, using `__eq__`, or approximately, using `isApprox`. * Generate an asciidoc with the data of a `DataStruct`. The saving/loading and comparison can be done recursively, meaning they accept `DataStruct`s whose fields are other `DataStructs`. In addition, they supported nested Python types, such as a dict of lists of numpy arrays. :param version: Holds the version of this DataStruct. Users should override the current version using the _version_default class variable. DataStructs that use this should also add a field validator or model validator to handle version mismatches. :type version: tuple[int, int] .. py:attribute:: version :type: tuple[int, int] :value: None .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:method:: generateAsciidoc() -> str :classmethod: Generate asciidoc for the DataStruct. .. py:method:: cli(exclude: list[str] = [], extra_info: dict[str, Any] = {}) :classmethod: Add this DataStruct's fields to a cli. :param exclude: List of fields to exclude from the cli. :type exclude: list[str] :param extra_info: Extra information that is used to generate the cli. Examples include: * name - Name the field. * help - Description for the field. * type - Type for the field. * default - Default for the field. :type extra_info: dict[str, Any] .. py:method:: fromLinker(vals_orig: dict[str, Any], excluded_vals: dict[str, Any] = {}, ds_linker: Optional[DSLinker] = None) -> Self :classmethod: Create an instance of the DataStruct from a CLI dictionary. :param vals_orig: CLI dictionary to use to build the DataStruct. :type vals_orig: dict[str, Any] :param excluded_vals: Included any vals that were excluded from the DSLinker. :type excluded_vals: dict[str, Any] :param ds_linker: DSLinker to use to build the DataStruct. This is optional, and is only necessary if there are multiple DSLinker's for a DataStruct. :type ds_linker: Optional[DSLinker] .. py:method:: toFile(file: pathlib.Path | str | IO[bytes], file_type: Optional[Literal['json', 'yaml', 'yml', 'h5', 'hdf5', 'pickle', 'pck', 'pcl']] = None, meta: Meta | None = None) -> None toFile(g: h5py.Group, meta: Meta | None = None) -> None Write the DataStruct to a H5 group. See overloads for details. .. py:method:: fromFile(file: pathlib.Path | str | IO[bytes], *, file_type: Optional[Literal['json', 'yaml', 'yml', 'h5', 'hdf5', 'pickle', 'pck', 'pcl']] = None, get_meta: Literal[False] = False) -> Self fromFile(file: pathlib.Path | str | IO[bytes], *, file_type: Optional[Literal['json', 'yaml', 'yml', 'h5', 'hdf5', 'pickle', 'pck', 'pcl']] = None, get_meta: Literal[True]) -> tuple[Self, Meta] fromFile(g: h5py.Group, *, get_meta: Literal[False] = False) -> Self fromFile(g: h5py.Group, *, get_meta: Literal[True]) -> tuple[Self, Meta] :classmethod: Create an instance of this DataStruct a file. See overloads for details. .. py:method:: isApprox(other: Self, prec: float = MATH_EPSILON) -> bool Check if this DataStruct is approximately equal to another DataStruct of the same type. This recursively moves through the public fields only. Note that Pydantic's __eq__ checks private and public fields. Recursively here means if the field is an iterator, another DataStruct, an iterator of DataStructs, etc. we will go into the nested structure calling isApprox where appropriate on the items in the iterator, fields in the DataStruct, etc. If the field (or iterated value, etc.) does not have an isApprox method, then we fallback to using the __eq__. If all values of isApprox (or __eq__) return True, then this returns True; otherwise, this returns false. :param other: A DataStruct of the same type. :type other: Self :param prec: The precision to use. Karana.Math.MATH_EPSILON is the default. :type prec: float :returns: True if the two DataStructs are approximately equal. False otherwise. :rtype: bool .. py:method:: model_copy(*args, **kwargs) -> Self Create a copy of the model. This uses the typical pydantic model_copy method, but iterates through everything to call .copy() on numpy arrays. This creates an actual copy of the array, rather than keeping the data members linked. :returns: A copy of this DataStruct. :rtype: Self .. py:method:: __eq__(other: object) -> bool Check if this DataStruct is equal to another DataStruct of the same type. First a normal == is tried on the fields. If that doesn't work, then we recurse into the fields looking for numpy arrays. Recursively here means if the field is an iterator, another DataStruct, an iterator of DataStructs, etc. we will go into the nested structure calling the appropriate operator on the items in the iterator, fields in the DataStruct, etc. This is done mainly for numpy arrays, where we want to call np.array_equal rather than use ==. :param other: The object to compare with. :type other: object :returns: True if the two DataStructs are equal. False otherwise. :rtype: bool .. py:class:: SentinelValueClass Bases: :py:obj:`object` Dummy class uses as a sentinel value. .. py:data:: SentinelValue .. py:class:: DSLinker(data_struct: type[DataStruct], exclude: list[str], extra_info: dict[str, Any]) Class used to populate a CLI application with `DataStruct` fields and create a `DataStruct` instance from CLI options. .. py:attribute:: data_struct .. py:attribute:: exclude .. py:attribute:: extra_info .. py:attribute:: name_link :type: dict[str, str] .. py:method:: toCli() Use to convert a DataStruct to a Kclick cli. .. py:data:: T .. py:class:: AllObjsBase Base class for the IdMixin. Pydantic re-writes the namespace of the class each time, which means we can't have a shared class variable across all IdMixins. Having a non-pydantic base class allows for this. .. attribute:: _all_objects_from_ids Shared dictionary of all objects created from classes that use the IdMixin. :type: dict[int, ref["IdMixin"]] .. py:class:: IdMixin(/, **data: Any) Bases: :py:obj:`AllObjsBase`, :py:obj:`pydantic.BaseModel`, :py:obj:`Generic`\ [\ :py:obj:`T`\ ] Mixin to add ID tracking to a DataStruct. For book keeping, it is common to want to track the id of the object used to create the DataStruct. This mixin makes it easy to do so. It adds the private _id variable with default value None, adds a KaranaId property for it, and overrides the appropriate methods so that _id is serialized/deserialized if set. It is class using the Mixin, it is its job to add objects to via the addObjectFromId(...) method whenever appropriate. .. py:property:: karana_id :type: int | None Retrieve the private _id variable. .. py:property:: objects_from_id :type: list[Karana.Core.CppWeakRef[T]] Get any objects created from this DataStruct. :returns: A list of CppWeakRefs to objects created from this DataStruct. :rtype: list[CppWeakRef[T]] .. py:method:: addObjectFromId(obj: T) Store a CppWeakRef to an object created from this DataStruct. This is used internally to find objects created from this DataStruct. :param obj: The object created from this DataStruct. :type obj: T .. py:method:: resetAllObjectsFromIds() :classmethod: Reset the global bookkeeping for objects created from IDs. This should rarely be needed. Only in cases were you are doing something like creating two simulations, two StatePropagators, etc. should you need this. .. py:method:: __getstate__() Override __getstate__ to include _id if set. .. py:method:: getObjectsCreatedById(id: int, /) -> list[T] | None getObjectsCreatedById(val: Any, id: int, /) -> list[T] | None :classmethod: Find objects created by DataStructs that match the given ID. 1. Find any object created by any DataStruct since the last reset. See resetAllObjectsFromIds for details on how to perform a reset. This assumes unique IDs (and a unique DataStruct with that ID). Therefore, the search stops at the first ID. :param id: The ID to use in the search. :type id: int :returns: * *list[T] | None* -- None if the ID was not found. Otherwise, a list of objects created with the ID. * *2. Find any objects in the val data structure that were created for the ID given by id.* * *This assumes unique IDs (and a unique DataStruct with that ID). Therefore, the search stops* * *at the first ID.* :param val: The data structure to recurse through. This can be a composite type consisting of DataStructs, lists, tuples, sets, and dictionaries. :type val: Any :param id: The ID to use in the search. :type id: int :returns: None if the ID was not found. Otherwise, a list of objects created with the ID. :rtype: list[T] | None .. py:method:: getObjectCreatedById(id: int, /) -> T getObjectCreatedById(val: Any, id: int, /) -> T :classmethod: Find the unique object created by DataStructs that match the given ID. 1. Find the unique object created by any DataStruct matching the given ID since the last reset. See resetAllObjectsFromIds for details on how to perform a reset. :param id: The ID to use in the search. :type id: int :returns: * *T* -- The unique object created for the given ID. * *2. Find the unique objects in the val data structure created for the given ID.* :param val: The data structure to recurse through. This can be a composite type consisting of DataStructs, lists, tuples, sets, and dictionaries. :type val: Any :param id: The ID to use in the search. :type id: int :returns: The unique object created for the given ID. :rtype: T .. py:method:: objectHasDS(id: int, /) -> bool objectHasDS(obj: T, /) -> bool :classmethod: Return True if this was turned into a DataStruct, False otherwise. 1. Perform search by ID. See resetAllObjectsFromIds for details on how to perform a reset. This assumes unique IDs (and a unique DataStruct with that ID). Therefore, the search stops at the first ID. :param id: The ID to use in the search. :type id: int :returns: * *bool* -- True if an object with this ID was turned into a DataStruct, False otherwise. * *2. Perform search by object.* * *This assumes unique IDs (and a unique DataStruct with that ID). Therefore, the search stops* * *at the first ID.* :param val: The object to check if was turned into a DataStruct. :type val: T :returns: True if an object with this ID was turned into a DataStruct, False otherwise. :rtype: bool .. py:class:: PyClassDS(/, **data: Any) Bases: :py:obj:`DataStruct` Represent a Python class as a DataStruct. This is used to serialize and deserialize the Python class type. :param module: The module the class is located on. :type module: str :param qualname: The fully qualified name of the class. :type qualname: str :param origin: The origin name of the class. :type origin: str :param args: Any arguments for the class. Used for things like Generics. :type args: list[PyClassDS] .. py:attribute:: module :type: str .. py:attribute:: qualname :type: str .. py:attribute:: origin :type: str .. py:attribute:: args :type: list[PyClassDS] .. py:method:: fromClass(klass: Any) -> Self :classmethod: Create a PyClassDS from the provided class. :param klass: The class to convert to a PyClassDS. :type klass: Any :returns: An instance of PyClassDS that represents the provided class. :rtype: Self .. py:method:: fromObj(obj: Any) -> Self :classmethod: Create a PyClassDS for the class of the provided object. :param obj: The object whose class a PyClassDS will be created for. :type obj: Any :returns: A PyClassDS that represents the class of provided object. :rtype: Self .. py:method:: toClass() -> Any Convert this PyClassDS to the class it represents. :returns: The class that this PyClassDS represents. :rtype: Any .. py:method:: toString() -> str Return the string representation of this PyDataClass. :returns: The string representation of this PyDataClass. :rtype: str .. py:class:: NestedBaseMixin(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` A mixin used by DataStructs that are the base class for other DataStructs. This ensures that the most derived type is used when serializing and deserializing. .. py:property:: ds_python_class :type: PyClassDS Get the fully qualified Python class name of the DataStruct. :returns: The fully qualified Python class name of the DataStruct. :rtype: str .. py:class:: Meta(/, **data: Any) Bases: :py:obj:`DataStruct`, :py:obj:`NestedBaseMixin` DataStruct that contains meta data when dumping/loading from a file. :param description: The description of the information being saved/loaded from a file. :type description: str :param date_time: The date and time at which the file was saved. :type date_time: datetime :param type: The type of DataStruct being saved. :type type: str .. py:attribute:: description :type: str :value: '' .. py:attribute:: date_time :type: datetime.datetime :value: None .. py:attribute:: type :type: PyClassDS :value: None