access_nri_intake.catalog.manager

`access_nri_intake.catalog.manager`#

Manager for adding/updating intake sources in an intake-dataframe-catalog like the ACCESS-NRI catalog

Add/update intake sources in an intake-dataframe-catalog like the ACCESS-NRI catalog

exception access_nri_intake.catalog.manager.CatalogManagerError#

Bases: Exception

Generic Exception for the CatalogManager class

Initialize self. See help(type(self)) for accurate signature.

with_traceback()#: Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class access_nri_intake.catalog.manager.CatalogManager(path)#

Add/update intake sources in an intake-dataframe-catalog like the ACCESS-NRI catalog

Initialise a CatalogManager instance to add/update intake sources in a intake-dataframe-catalog like the ACCESS-NRI catalog

Parameters:

build_esm(name, description, builder, path, translator=DefaultTranslator, metadata=None, directory=None, overwrite=False, **kwargs)#

Build an Intake-ESM datastore and add it to the catalog

Parameters:

name: str: The name of the Intake-ESM datastore
description: str: Description of the contents of the Intake-ESM datastore
builder: subclass of :py:class:`access_nri_intake.source.builders.BaseBuilder`: The builder to use to build the Intake-ESM datastore
path: str or list of str: Path or list of paths to crawl for assets/files to add to the Intake-ESM datastore.
translator: :py:class:`~access_nri_intake.catalog.translators.DefaultTranslator`, optional: An instance of the DefaultTranslator class for translating info in the Intake-ESM datastore into intake-dataframe-catalog column metadata. Defaults to access_nri_intake.catalog.translators.DefaultTranslator
metadata: dict, optional: Additional info to store in the intake cat.metadata attribute. This info will be available to the translator and to users of the Intake-ESM datastore
directory: str: The directory to save the Intake-ESM datastore to. If None, use the current directory
overwrite: bool, optional: Whether to overwrite if an Intake-ESM datastore with the same name already exists
kwargs: dict: Additional kwargs to pass to the builder

load(name, description, path, driver='esm_datastore', translator=DefaultTranslator, metadata=None, **kwargs)#

Load an existing data source using Intake and add it to the catalog

Parameters:

name: str: The name of the data source
description: str: Description of the contents of the data source
path: str: The path to the Intake data source
driver: str: The name of the Intake driver to use to open the data source
translator: :py:class:`~access_nri_catalog.metacat.translators.DefaultTranslator`, optional: An instance of the DefaultTranslator class for translating data source metadata into intake-dataframe-catalog column metadata. Defaults to access_nri_intake.catalog.translators.DefaultTranslator
metadata: dict, optional: Additional info to store in the intake metadata attribute for this data source. This info will be available to the translator and to users of the catalog
kwargs: dict, optional: Additional kwargs to pass to ~intake.open_

save(**kwargs)#

Save the catalog

Parameters: