clef.model

Model of NCI’s MAS database

The MAS database has two main tables - path and metadata. These base tables are available in the model as Path and Metadata, they have a SQLAlchemy relationship so that the two table can be joined in queries.

There may be multiple Metadata entries for a single Path, these represent different metadata types, such as checksums, netCDF attributes and POSIX file attributes. The type can be identified from Metadata.type, and is used as a polymorphic identity to SQLAlchemy’s single table inheritance, creating the Checksum, Netcdf and Posix models.

The C5Dataset and C6Dataset models represent datasets like you would find on ESGF, although without a version. They are created in the database from a DISTINCT view of the NetCDF attributes, and can be used to group paths on the filesystem into datasets.

class clef.model.C5Dataset(**kwargs)[source]

A CMIP5-era ESGF dataset

This class only has access to attributes from the file itself, so version information is not present.

See the CMIP documentation for descriptions of the attributes

cmor_table
ensemble
experiment
institute
model
project
realm
time_frequency
class clef.model.C6Dataset(**kwargs)[source]

A CMIP6-era ESGF dataset

This class only has access to attributes from the file itself, so version information is not present.

See the CMIP documentation for descriptions of the attributes

activity_id
experiment_id
frequency
grid_label
institution_id
member_id
nominal_resolution
project
realm
source_id
source_type
sub_experiment_id
table_id
variable_id
variant_label
class clef.model.Checksum(**kwargs)[source]

Checksum of a file on Raijin

md5

md5 checksum

path

type: Path

sha256

sha256 checksum

class clef.model.ExtendedMetadata(**kwargs)[source]

Extra metadata not present in the file’s attributes

class clef.model.Info(**kwargs)[source]

General information about a dataset file

This is a database view, its columns shouldn’t be used for searching as they are large and not indexed.

contact
description
further_info_url
license
parent_experiment_id
source
title
tracking_id
variant_info
class clef.model.Metadata(**kwargs)[source]

Generic base class for Metadata of a file on Raijin

See Posix and Netcdf for specific metadata information

json

Metadata value

path

type: Path

type

Metadata type

class clef.model.Netcdf(**kwargs)[source]

NetCDF metadata of a file on Raijin

As would be found by ncdump -h

attributes

File attributes

dimensions

File dimensions

variables

File variables

class clef.model.Path(**kwargs)[source]

Path of a file on Raijin, with links to metadata

c5dataset

type: C5Dataset

c6dataset

type: C6Dataset

checksum

type: Checksum

netcdf

type: Netcdf

path

File path at NCI

class clef.model.Posix(**kwargs)[source]

Posix metadata of a file on Raijin

As would be found by ls

class clef.model.pg_json_property(attr_name, index, cast_type)[source]