rmellipse.workflows¶

Workflows module, utilized by the command line interface.

The workflows module exists to suppor the command line interface for running and publishing workflows. Documentation is provided for transparency and to aid development, however the python API is not intended for use.

Submodules¶

Attributes¶

SCHEMAS

Exceptions¶

`UninitializedRMEProject`	Common base class for all non-exit exceptions.
`JobFailure`	Common base class for all non-exit exceptions.

Classes¶

`ProjectSettings`	Store and access project settings.
`FILE_TYPES`	Create a collection of name/value pairs.
`WORKFLOW_SPECIAL_KEYS`	Create a collection of name/value pairs.
`DIRECTORY_ITEMS`	Create a collection of name/value pairs.
`WorkflowConfig`	Store and access a workflow configuration.
`RMESettings`	Settings file for the RME program.

Package Contents¶

exception rmellipse.workflows.UninitializedRMEProject(**kwargs)¶

Bases: Exception

Common base class for all non-exit exceptions.

Initialize self. See help(type(self)) for accurate signature.

exception rmellipse.workflows.JobFailure(**kwargs)¶

Bases: Exception

Common base class for all non-exit exceptions.

Initialize self. See help(type(self)) for accurate signature.

class rmellipse.workflows.ProjectSettings(project: pathlib.Path | str | dict)¶

Bases: dict

Store and access project settings.

Can be serialized to a JSON like file.

Initialize self. See help(type(self)) for accurate signature.

property requires_files: dict¶

Get a list of required files

Returns:: dictionary of required files or folder patterns, keys are the new name of the datset within environment.
Return type:: dict[str]

property requires_releases¶

Releases required by a project.

Returns:: list of version expression requirements, empty if no requirements.
Return type:: list

property requires_env¶

Releases required by a project.

Returns:: list of version expression requirements, empty if no requirements.
Return type:: list

property rmesettings: RMESettings¶

Get RMESettings for this project.

Returns:: _description_
Return type:: dict

property project_dir: pathlib.Path¶

Root directory of the project.

Returns:: Path to the projects root directory.
Return type:: Path

property active_workflow: pathlib.Path¶

Return information about the active workflow.

Returns:: Path to the active workflow file.
Return type:: Path

property project_map¶

Returns a project map.

Returns:: Path to the project map.
Return type:: Path

property data_env_dir¶: Directory containing the dataest environment.

property data_env_packages¶: Directory containing the full linked packages in data_env_dir.

property processlogdir: pathlib.Path¶: Direcotry that stores process logs.

property dotrmedir¶

Directory for storing hidden files within a project.

Return type:: Path

property wft_jsondir¶

Directory for storing workfow solution JSON files.

Returns:: _description_
Return type:: _type_

property rmeproj_file¶: Path to the project file within a directory.

static is_rmeproject(directory: pathlib.Path)¶: True if directory is an RME project (i.e. has a rmeproject.yml file.)

rmellipse.workflows.SCHEMAS¶

class rmellipse.workflows.FILE_TYPES(*args, **kwds)¶

Bases: enum.Enum

Create a collection of name/value pairs.

Example enumeration:

>>> class Color(Enum):
...     RED = 1
...     BLUE = 2
...     GREEN = 3

Access them by:

attribute access:
```
>>> Color.RED
<Color.RED: 1>
```
value lookup:
```
>>> Color(1)
<Color.RED: 1>
```
name lookup:
```
>>> Color['RED']
<Color.RED: 1>
```

Enumerations can be iterated over, and know how many members they have:

>>> len(Color)
3

>>> list(Color)
[<Color.RED: 1>, <Color.BLUE: 2>, <Color.GREEN: 3>]

Methods can be added to enumerations, and members can have their own attributes – see the documentation for details.

WORKFLOW = 'workflow'¶

CODE = 'code'¶

DOC = 'doc'¶

MISC = 'misc'¶

DATASET = 'dataset'¶

class rmellipse.workflows.WORKFLOW_SPECIAL_KEYS(*args, **kwds)¶

Bases: enum.Enum

Create a collection of name/value pairs.

Example enumeration:

>>> class Color(Enum):
...     RED = 1
...     BLUE = 2
...     GREEN = 3

Access them by:

attribute access:
```
>>> Color.RED
<Color.RED: 1>
```
value lookup:
```
>>> Color(1)
<Color.RED: 1>
```
name lookup:
```
>>> Color['RED']
<Color.RED: 1>
```

Enumerations can be iterated over, and know how many members they have:

>>> len(Color)
3

>>> list(Color)
[<Color.RED: 1>, <Color.BLUE: 2>, <Color.GREEN: 3>]

Methods can be added to enumerations, and members can have their own attributes – see the documentation for details.

JOB = 'jobs'¶

RELEASE = 'release'¶

DATASETS = 'datasets'¶

class rmellipse.workflows.DIRECTORY_ITEMS(*args, **kwds)¶

Bases: enum.Enum

Create a collection of name/value pairs.

Example enumeration:

>>> class Color(Enum):
...     RED = 1
...     BLUE = 2
...     GREEN = 3

Access them by:

attribute access:
```
>>> Color.RED
<Color.RED: 1>
```
value lookup:
```
>>> Color(1)
<Color.RED: 1>
```
name lookup:
```
>>> Color['RED']
<Color.RED: 1>
```

Enumerations can be iterated over, and know how many members they have:

>>> len(Color)
3

>>> list(Color)
[<Color.RED: 1>, <Color.BLUE: 2>, <Color.GREEN: 3>]

Methods can be added to enumerations, and members can have their own attributes – see the documentation for details.

DIRECTORY = 'Directory'¶

FILE = 'File'¶

H5_FILE = 'H5_File'¶

H5_GROUP = 'H5_Group'¶

H5_DATASET = 'H5_DATASET'¶

class rmellipse.workflows.WorkflowConfig(workflow_file: pathlib.Path | str | dict, project_settings: ProjectSettings)¶

Bases: dict

Store and access a workflow configuration.

Parameters:

dict (_type_) – _description_

Returns:

_description_

Return type:

_type_

Raises:

Exception – _description_
:param workflow_file – Workflow file.:
:type workflow_file – Path | str | dict:
:param project_settings – Project settings that contain the: workflow.
:type project_settings – ProjectSettings:
:raises ValueError – Missing environment variables.:
:raises ValueError – Empry workflow file.:

project_settings¶

raw¶

env_variables¶

expand_variables(obj: str | dict | list, expand_env: bool = True, expand_datasets: bool = True) → str | dict | list¶

Expand variables of any object that can be serialized to JSON.

Encodes obj to a JSON string, replaces any variable expressions, then de-encodes the stirng. Order of operations is encode -> env variables -> dataset names -> decode.

JSON is a faster serialization format, so using it here instead of YAML.

Parameters:

obj (str | dict | list) – _description_
expand_env (bool, optional) – If True, expands environment variables. Default is true.
expand_datasets (bool, optional) – If True, expands dataset variables. Default is True

Returns:

Same type as input with variables expanded.

Return type:

object

property title: str¶

property release_version: str¶

property requires_files: dict¶

Get a list of required files

Includes required files of the project, any name-space conflicts are overwritten with the workspace requires_files.

Returns:: dictionary of required files or folder patterns, keys are the new name of the datset within environment.
Return type:: dict[str]

property requires_releases: None¶

Releases requirements defined in the workflow file.

Returns:: _description_
Return type:: _type_

property requires_env: None¶

Environment variable requirements.

Returns:: _description_
Return type:: _type_

property formatting_keys: dict¶: Dictionary of kwargs for string formating.

property datasets¶

List of defined datsets in the workflow config.

Returns:: _description_
Return type:: _type_

property jobs¶

List of defined datsets in the workflow config.

Returns:: _description_
Return type:: _type_

property release¶

Release related settings

Returns:: _description_
Return type:: _type_

class rmellipse.workflows.RMESettings(project_dir: pathlib.Path)¶

Bases: dict

Settings file for the RME program.

rme.yml files are used for storing settings that can be defined at the user level. For example, what archives to use by default, and aliases for the archives.

Settings files are overloaded by higher priority settings files.

Discovery order (lowest to highest priority) User config -> project config

Parameters:

dict (_type_) – _description_
signature. (Initialize self. See help(type(self)) for accurate)

get_host_settings(host_name: str = None)¶

Get the host settings by host name.

User and password are returned as None if they aren’t present in the settings.

Does not attempt to get credentials based on host.

Parameters:

host_name (str, optional) – _description_, by default None

Returns:

host – str
user – str
password – str

Raises:

LookupError: – If the host isn’t in the settings.

property hosts¶

property default_host¶

rmellipse.workflows¶

Submodules¶

Attributes¶

Exceptions¶

Classes¶

Package Contents¶

Table of Contents

This Page

Versions