Storages¶

Storages are classes based on TinyDB that handles file management, raw parsing and other things like that, if you’d like to create support for more file types, all you have to do is to inherit the BaseDB and follow the instructions in the API of that class.

Directory Utils¶

panda_core_data.storages.auto_convert_to_pathlib(path)¶

Check if the path is valid and automatically convert it into a Path object

Parameters:	path (str or `Path`) – source folder
Returns:	The Path object
Return type:	`Path`
Raises:	PCDFolderNotFound – If the folder is invalid PCDInvalidPath – If the file is invalid

panda_core_data.storages.get_extension(path)¶

Get file extension from the path

Return str:	The file extension

panda_core_data.storages.get_raw_extensions()¶

Get all available extensions the package supports

Return list(str):
	A list of available extensions

panda_core_data.storages.get_storage_from_extension(extension)¶

Returns the storage based on the file extension

Returns:	Returns a storage object that handles the raw file
Return type:	`Storage`

panda_core_data.storages.is_excluded_extension(path, exclude_ext)¶

Check if the file has an ignored extension

Parameters:	path (str or `Path`) – source folder exclude_ext (list(str)) – If the path should be from a folder or file
Return bool:	returns True if it’s a excluded extension, False otherwise

panda_core_data.storages.raw_glob_iterator(path, excluded_ext=False)¶

Iterate along the path yielding the raw file.

:yields Path: The file path

YAMLDB¶

class panda_core_data.storages.yaml_db.YamlDB(*args, **kwargs)¶

Parser storage class used to read yaml files

__init__(*args, **kwargs)¶

Open file as Data Base

Parameters:	str (path) – path pointing to a yaml file

read()¶: Method used by TinyDB to read the file

write(data)¶

Write the current state of the database to the storage.

Any kind of serialization should go here.

Parameters:	data (dict) – The current state of the database.

JsonDB¶

class panda_core_data.storages.json_db.JsonDB(path, **kwargs)¶

read()¶

Read the last stored state.

Any kind of deserialization should go here. Return None here to indicate that the storage is empty.

Return type:	dict

write(data)¶

Write the current state of the database to the storage.

Any kind of serialization should go here.

Parameters:	data (dict) – The current state of the database.

BaseDB¶

class panda_core_data.storages.base_db.BaseDB(path, **kwargs)¶

Base storage class that reads which extensions are available to feed the path handling functions

To create a new storage, you will need to inherit this class, create a extensions variable containing a list of extensions the storage will support for example:

extensions = ["yml", "yaml"]

then implement a read and write method using the methods base_read() and base_write() all you need to do is follow the instructions contained in them

__init__(path, **kwargs)¶

Create a new instance.

Parameters:	path (str) – Path to file

classmethod __init_subclass__()¶: Automatically generate an extension list containing the available raw extensions available together with their storage.

base_read(load_method, use_handle)¶

Base method used by children classes to read the file and transforms the string into a list of dictionaries, a good example of this method is the built in python json.load() however, since it needs a string as an input (or handler) you would need to set the parameter use_handler so the string, which is the contents of the raw file, will be passed to that method. For example the read method of our yaml parser:

def read(self):
    return self.base_read(yaml.safe_load, True)

And since the function yaml.safe_load() needs a string as an input, we set use_handle to True.

An example of list of dictionaries would be like this:

{"data": [
    {
        'field_name': 'value',
        'another_field': 10,
    },
    {
        'field_name': 'value',
        'another_field': 10,
    },
]}

The dict keys are fields of a dataclass and the value, well, values.

Parameters:	load_method (`function`) – method used to transform the raw file into a list of dictionaries. use_handle (bool) – TinyDB offers a handle (More specifically, the handle of the class `JSONStorage`) to load the file and turn into a string automatically if you’d like to use it, just set this parameter to True.

base_write(write_method, data, use_handle)¶

Transforms the data dictionary to a raw representation.

Parameters:	data (dict(str, object)) – data dictionary.