Storages¶

Storages are classes based on TinyDB that handles file management, raw parsing and other things like that, if you’d like to create support for more file types, all you have to do is to inherit the BaseDB and follow the instructions in the API of that class.

Directory Utils¶

panda_core_data.storages.auto_convert_to_pathlib(path)¶

Check if the path is valid and automatically convert it into a Path object

Parameters:	path (`Union`[`Path`, `str`]) – source folder
Return type:	`Path`
Returns:	The Path object
Raises:	PCDInvalidPath – If the file is invalid

panda_core_data.storages.get_extension(path)¶

Get file extension from the path

Parameters:	path (`Union`[`Path`, `str`]) – path to a file or extension
Return type:	`str`
Returns:	The file extension

panda_core_data.storages.get_raw_extensions()¶

Get all available extensions the package supports

Return type:	`List`[`str`]
Returns:	A list of available extensions

panda_core_data.storages.get_storage_from_extension(extension)¶

Returns the storage based on the file extension

Return type:	tinydb.storages.Storage
Returns:	Returns a storage object that handles the raw file

panda_core_data.storages.is_excluded_extension(path, exclude_ext)¶

Check if the file has an ignored extension

Parameters:	path (`Union`[`Path`, `str`]) – source folder exclude_ext (`List`[`str`]) – If the path should be from a folder or file
Return type:	`bool`
Returns:	returns True if it’s a excluded extension, False otherwise

panda_core_data.storages.raw_glob_iterator(path, excluded_ext=False)¶

Iterate along the path yielding the raw file.

Yields:	The file
Return type:	`Iterator`[`Path`]

YamlDB¶

class panda_core_data.storages.yaml_db.YamlDB(*args, **kwargs)¶

Parser storage class used to read yaml files

__init__(*args, **kwargs)¶

Open file as Data Base

Parameters:	str (path) – path pointing to a yaml file

read()¶

Method used by TinyDB to read the file

Return type:	`List`[`Dict`[`str`, `Any`]]

write(data)¶

Write the current state of the database to the storage.

Any kind of serialization should go here.

Parameters:	data (dict) – The current state of the database.

JsonDB¶

class panda_core_data.storages.json_db.JsonDB(path, **kwargs)¶

read()¶

Read the last stored state.

Any kind of deserialization should go here. Return None here to indicate that the storage is empty.

Return type:	dict

write(data)¶

Write the current state of the database to the storage.

Any kind of serialization should go here.

Parameters:	data (dict) – The current state of the database.

BaseDB¶

class panda_core_data.storages.base_db.BaseDB(path, **kwargs)¶

Base storage class that reads which extensions are available to feed the path handling functions

To create a new storage, you will need to inherit this class, create a extensions variable containing a list of extensions the storage will support for example:

extensions = ["yml", "yaml"]

then implement a read and write method using the methods base_read() and base_write() all you need to do is follow the instructions contained in them

__init__(path, **kwargs)¶

Create a new instance

Parameters:	path (str) – Path to file

classmethod __init_subclass__()¶: Automatically generate an extension list containing the available raw extensions available together with their storage

base_read(load_method, use_handle)¶

Base method used by children classes to read the file and transforms the string into a list of dictionaries, a good example of this method is the built in python json.load() however, since it needs a string as an input (or handler) you would need to set the parameter use_handler so the string, which is the contents of the raw file, will be passed to that method. For example the read method of our yaml parser:

def read(self):
    return self.base_read(yaml.safe_load, True)

And since the function yaml.safe_load() needs a string as an input, we set use_handle to True.

An example of list of dictionaries would be like this:

{"data": [
    {
        'field_name': 'value',
        'another_field': 10,
    },
    {
        'field_name': 'value',
        'another_field': 10,
    },
]}

The dict keys are fields of a dataclass and the value, well, values

Parameters:	load_method (`Callable`) – method used to transform the raw file into a list of dictionaries use_handle (`bool`) – TinyDB offers a handle (More specifically, the handle of the class `JSONStorage`) to load the file and turn into a string automatically if you’d like to use it, just set this parameter to True
Return type:	`List`[`Dict`[`str`, `Any`]]
Returns:	The generated data

base_write(write_method, data, use_handle)¶

Transforms the data dictionary to a raw representation

Parameters:	write_method (`Callable`) – method used to transform the raw file into a list of dictionaries data (`List`[`Dict`[`str`, `Any`]]) – data dictionary