access_nri_intake.catalog.manager#

Manager for adding/updating intake sources in an intake-dataframe-catalog like the ACCESS-NRI catalog

Exceptions#

CatalogManagerError

Generic Exception for the CatalogManager class

Classes#

CatalogManager

Add/update intake sources in an intake-dataframe-catalog like the ACCESS-NRI catalog

Module Contents#

exception access_nri_intake.catalog.manager.CatalogManagerError#

Bases: Exception

Generic Exception for the CatalogManager class

Initialize self. See help(type(self)) for accurate signature.

class access_nri_intake.catalog.manager.CatalogManager(path)#

Add/update intake sources in an intake-dataframe-catalog like the ACCESS-NRI catalog

Initialise a CatalogManager instance to add/update intake sources in a intake-dataframe-catalog like the ACCESS-NRI catalog

Parameters:
path: str

The path to the intake-dataframe-catalog

path#
mode#
dfcat#
source = None#
source_metadata = None#
build_esm(name, description, builder, path, translator=DefaultTranslator, metadata=None, directory=None, overwrite=False, **kwargs)#

Build an Intake-ESM datastore and add it to the catalog

Parameters:
name: str

The name of the Intake-ESM datastore

description: str

Description of the contents of the Intake-ESM datastore

builder: subclass of :py:class:`access_nri_intake.source.builders.BaseBuilder`

The builder to use to build the Intake-ESM datastore

path: str or list of str

Path or list of paths to crawl for assets/files to add to the Intake-ESM datastore.

translator: :py:class:`~access_nri_intake.catalog.translators.DefaultTranslator`, optional

An instance of the DefaultTranslator class for translating info in the Intake-ESM datastore into intake-dataframe-catalog column metadata. Defaults to access_nri_intake.catalog.translators.DefaultTranslator

metadata: dict, optional

Additional info to store in the intake cat.metadata attribute. This info will be available to the translator and to users of the Intake-ESM datastore

directory: str

The directory to save the Intake-ESM datastore to. If None, use the current directory

overwrite: bool, optional

Whether to overwrite if an Intake-ESM datastore with the same name already exists

kwargs: dict

Additional kwargs to pass to the builder

load(name, description, path, driver='esm_datastore', translator=DefaultTranslator, metadata=None, **kwargs)#

Load an existing data source using Intake and add it to the catalog

Parameters:
name: str

The name of the data source

description: str

Description of the contents of the data source

path: str

The path to the Intake data source

driver: str

The name of the Intake driver to use to open the data source

translator: :py:class:`~access_nri_catalog.metacat.translators.DefaultTranslator`, optional

An instance of the DefaultTranslator class for translating data source metadata into intake-dataframe-catalog column metadata. Defaults to access_nri_intake.catalog.translators.DefaultTranslator

metadata: dict, optional

Additional info to store in the intake metadata attribute for this data source. This info will be available to the translator and to users of the catalog

kwargs: dict, optional

Additional kwargs to pass to ~intake.open_

save(**kwargs)#

Save the catalog

Parameters:
kwargs: dict, optional

Additional keyword arguments passed to to_csv().