clef.model¶

Model of NCI’s MAS database

The MAS database has two main tables - path and metadata. These base tables are available in the model as Path and Metadata, they have a SQLAlchemy relationship so that the two table can be joined in queries.

There may be multiple Metadata entries for a single Path, these represent different metadata types, such as checksums, netCDF attributes and POSIX file attributes. The type can be identified from Metadata.type, and is used as a polymorphic identity to SQLAlchemy’s single table inheritance, creating the Checksum, Netcdf and Posix models.

The C5Dataset and C6Dataset models represent datasets like you would find on ESGF, although without a version. They are created in the database from a DISTINCT view of the NetCDF attributes, and can be used to group paths on the filesystem into datasets.

class clef.model.C5Dataset(**kwargs)[source]¶

A CMIP5-era ESGF dataset

This class only has access to attributes from the file itself, so version information is not present.

See the CMIP documentation for descriptions of the attributes

cmor_table¶

ensemble¶

experiment¶

institute¶

model¶

project¶

realm¶

time_frequency¶

class clef.model.C6Dataset(**kwargs)[source]¶

A CMIP6-era ESGF dataset

This class only has access to attributes from the file itself, so version information is not present.

See the CMIP documentation for descriptions of the attributes

activity_id¶

experiment_id¶

frequency¶

grid_label¶

institution_id¶

member_id¶

nominal_resolution¶

project¶

realm¶

source_id¶

source_type¶

sub_experiment_id¶

table_id¶

variable_id¶

variant_label¶

class clef.model.Checksum(**kwargs)[source]¶

Checksum of a file on Raijin

md5¶: md5 checksum

path¶: type: Path

sha256¶: sha256 checksum

class clef.model.ExtendedMetadata(**kwargs)[source]¶: Extra metadata not present in the file’s attributes

class clef.model.Info(**kwargs)[source]¶

General information about a dataset file

This is a database view, its columns shouldn’t be used for searching as they are large and not indexed.

contact¶

description¶

further_info_url¶

license¶

parent_experiment_id¶

source¶

title¶

tracking_id¶

variant_info¶

class clef.model.Metadata(**kwargs)[source]¶

Generic base class for Metadata of a file on Raijin

See Posix and Netcdf for specific metadata information

json¶: Metadata value

path¶: type: Path

type¶: Metadata type

class clef.model.Netcdf(**kwargs)[source]¶

NetCDF metadata of a file on Raijin

As would be found by ncdump -h

attributes¶: File attributes

dimensions¶: File dimensions

variables¶: File variables

class clef.model.Path(**kwargs)[source]¶

Path of a file on Raijin, with links to metadata

c5dataset¶: type: C5Dataset

c6dataset¶: type: C6Dataset

checksum¶: type: Checksum

netcdf¶: type: Netcdf

path¶: File path at NCI

class clef.model.Posix(**kwargs)[source]¶

Posix metadata of a file on Raijin

As would be found by ls

class clef.model.pg_json_property(attr_name, index, cast_type)[source]¶