Building sources#
Each entry in an intake-dataframe-catalog refers to an Intake source. Sources may be pre-generated (e.g.
the CMIP5 and CMIP6 Intake-ESM datastore sources managed by NCI)
or they can be built using the access_nri_intake.source
sub-package. Currently,
access_nri_intake.source
only supports building Intake-ESM datastore sources.
When access_nri_intake.source
is first imported, it downloads and parses a specific commit of the
schema at ACCESS-NRI/schema and stores it in the variable
access_nri_intake.source.ESM_JSONSCHEMA
. This schema defines what metadata must be included in an
Intake-ESM catalog, and what types and fields are allowed. Subsequent imports read the downloaded schema,
unless the schema is changed (see Updating schema), in which case the new schema is downloaded.
Note
The “required” field in the downloaded schema is replaced with access_nri_intake.source.CORE_COLUMNS
prior to saving to access_nri_intake.source.ESM_JSONSCHEMA
to allow this field to be customized
within access-nri-intake
Builders#
New Intake-ESM datastores can be created for ACCESS model output using the Builder classes in the sub-module
access_nri_intake.source.builders
. All public Builders inherit from
access_nri_intake.source.builders.BaseBuilder
, which inherits from ecgtools.builder.Builder
.
access_nri_intake.source.builders.BaseBuilder
simply reorganises the functionality in
ecgtools.builder.Builder
so that self-contained Builder classes can be easily written for different
types of model output. All Builders validate against the schema in
access_nri_intake.source.ESM_JSONSCHEMA
.
Creating a new Builder#
New Builders should inherit from access_nri_intake.source.builders.BaseBuilder
. In general, all that
should be required is to:
Hardcode some of the optional
kwargs
onBaseBuilder.__init__
in the__init__
method of the new Builder. The exact choice of whichkwargs
are hardcoded will depend on the specifics of the target model output. It might be useful to look at existing Builder class implementations and at theecgtools.builder.Builder
class, which is inhereted byaccess_nri_intake.source.builders.BaseBuilder
- see theecgtools
documentation here. The Intake-ESM specifications will also be helpful for interpreting many of thekwargs
Overwrite the static
parser
method. This method should receive a single file (usually netcdf) and return a dictionary containing all of the metadata to include in the Intake-ESM datastore. As described above the dictionary is validated againstaccess_nri_intake.source.ESM_JSONSCHEMA
. Theparser
method serves the same function as custom parsers inecgtools
, so the documentation on these may be useful. Again, it will also be helpful to look at existing Builder class implementations.
API for access_nri_intake.source
#
This documentation has been auto-generated using sphinx-autoapi