How do I use it?#
The ACCESS-NRI Intake catalog is intended to be used in Jupyter notebooks running on Gadi. It can be used outside of Jupyter notebooks, but interactive user experience will be impacted if HTML output formatting is not supported.
Prerequisites#
In order to use the catalog, you will need to have the following:
An account at NCI: see the NCI documentation for creating an account if you don’t have one. Note you will need to join a project with a compute allocation. If you don’t know what project is appropriate you will need to seek help from your local group or IT support.
Access to the
xp65
project: this project houses the catalog table files. See the NCI documentation for how to join projects.Access to the projects that house the data you’re interested in: the catalog references data products across multiple projects on Gadi. Currently, data is included from the following projects:
al33
cj50
dk92
fs38
ik11
oi10
p73
rr3
xp65
If you wish to be able to access all the data in the catalog, you will need to be a member of all these projects.
Attention
Catalog users will only be able to load data from projects that they have access to.
An installation of the catalog: the catalog is pre-installed in the CLEX CMS “analysis3” conda environment (>=23.04) and the ACCESS-NRI “access-med” conda environment (all versions). Users are encouraged to use one of these environments to use the catalog. Alternatively, you can install the catalog into your own environment as described below.
Attention
In order to use the CLEX CMS conda environments, you will need to also be a member of the
hh5
project
Installing the catalog#
Most users will not need to install the catalog themselves and will instead use the catalog through one
of the public analysis environments provided in either hh5
or xp65
(see below).
Advanced users that want to install the catalog into their own environment can do so in three ways:
Install method |
Code |
---|---|
$ conda install -c accessnri access-nri-intake
|
|
$ python -m pip install access-nri-intake
|
|
From source |
$ git clone git@github.com:ACCESS-NRI/access-nri-intake.git
$ cd access-nri-intake
$ python -m pip install -e .
|
Using the catalog on the ARE#
The easiest way to use the catalog is via the NCI Australian Research Environment (ARE). The ARE is a web-based graphical interface that makes it easy for NCI users to start JupyterLab sessions on Gadi compute nodes. NCI documentation for the ARE JupyterLab app can be found here.
After logging in to the ARE using your NCI credentials and clicking the JupyterLab icon, you’ll arrive at a page to configure your JupyterLab session. Many of the configuration options are hopefully self-explanatory, but a few must be set carefully to ensure the catalog and the data it references are available from your session. In particular:
Setting the storage flags: in addition to being a member of the projects you want to access, you also have to explicity tell the JupyterLab app that you want to access them in your session. Specify the project storage paths by entering them in the “Storage” dropdown. To allow access to the catalog and all the data products it contains enter:
gdata/al33+gdata/cj50+gdata/dk92+gdata/fs38+gdata/ik11+gdata/oi10+gdata/p73+gdata/rr3+gdata/xp65
If you want to use the
hh5
analysis environment, you’ll also need to addgdata/hh5
.Attention
You need to be a member of all projects you enter here. You can see what projects you are part of at https://my.nci.org.au/mancini.
Setting the environment: you need to make sure that the catalog is installed in your JupyterLab session. As mentioned above, the easiest way to do this is to use either the
hh5
orxp65
public analysis environments. You can activate thehh5
environment within your JupyterLab session using the “Advanced options” to set the “Module directories” to/g/data/hh5/public/modules
and “Modules” toconda/analysis3-unstable
. Similarly, to use thexp65
environment, set “Module directories” to/g/data/xp65/public/modules
and “Modules” toconda/are
.