rmellipse.workflows¶

Workflows module, utilized by the command line interface.

The workflows module exists to suppor the command line interface for running and publishing workflows. Documentation is provided for transparency and to aid development, however the python API is not intended for use.

Submodules¶

Attributes¶

SCHEMAS

Exceptions¶

`UninitializedRMEProject`	Common base class for all non-exit exceptions.
`JobFailure`	Common base class for all non-exit exceptions.

Classes¶

`ProjectSettings`	Store and access project settings.
`FILE_TYPES`	Create a collection of name/value pairs.
`WORKFLOW_SPECIAL_KEYS`	Create a collection of name/value pairs.
`DIRECTORY_ITEMS`	Create a collection of name/value pairs.
`WorkflowConfig`	Store and access a workflow configuration.
`RMESettings`	Settings file for the RME program.

Package Contents¶

exception rmellipse.workflows.UninitializedRMEProject(**kwargs)¶

Bases: Exception

Common base class for all non-exit exceptions.

Initialize self. See help(type(self)) for accurate signature.

exception rmellipse.workflows.JobFailure(**kwargs)¶

Bases: Exception

Common base class for all non-exit exceptions.

Initialize self. See help(type(self)) for accurate signature.

class rmellipse.workflows.ProjectSettings(project: pathlib.Path | str | dict)¶

Bases: dict

Store and access project settings.

Can be serialized to a JSON like file.

Initialize self. See help(type(self)) for accurate signature.

property requires_files: dict¶

Get a list of required files

Returns:

dict[str]: dictionary of required files or folder patterns, keys are the new name of the datset within environment.

property requires_releases¶

Releases required by a project.

Returns:

list: list of version expression requirements, empty if no requirements.

property requires_env¶

Releases required by a project.

Returns:

list: list of version expression requirements, empty if no requirements.

property rmesettings: RMESettings¶

Get RMESettings for this project.

Returns:

dict: _description_

property project_dir: pathlib.Path¶

Root directory of the project.

Returns:

Path: Path to the projects root directory.

property active_workflow: pathlib.Path¶

Return information about the active workflow.

Returns:

Path:: Path to the active workflow file.

property project_map¶

Returns a project map.

Returns:

Path: Path to the project map.

property data_env_dir¶: Directory containing the dataest environment.

property data_env_packages¶: Directory containing the full linked packages in data_env_dir.

property processlogdir: pathlib.Path¶: Direcotry that stores process logs.

property dotrmedir¶

Directory for storing hidden files within a project.

Returns:

Path

property wft_jsondir¶

Directory for storing workfow solution JSON files.

Returns:

_type_: _description_

property rmeproj_file¶: Path to the project file within a directory.

static is_rmeproject(directory: pathlib.Path)¶: True if directory is an RME project (i.e. has a rmeproject.yml file.)

rmellipse.workflows.SCHEMAS¶

class rmellipse.workflows.FILE_TYPES(*args, **kwds)¶

Bases: enum.Enum

Create a collection of name/value pairs.

Example enumeration:

>>> class Color(Enum):
...     RED = 1
...     BLUE = 2
...     GREEN = 3

Access them by:

attribute access:
```
>>> Color.RED
<Color.RED: 1>
```
value lookup:
```
>>> Color(1)
<Color.RED: 1>
```
name lookup:
```
>>> Color['RED']
<Color.RED: 1>
```

Enumerations can be iterated over, and know how many members they have:

>>> len(Color)
3

>>> list(Color)
[<Color.RED: 1>, <Color.BLUE: 2>, <Color.GREEN: 3>]

Methods can be added to enumerations, and members can have their own attributes – see the documentation for details.

WORKFLOW = 'workflow'¶

CODE = 'code'¶

DOC = 'doc'¶

MISC = 'misc'¶

DATASET = 'dataset'¶

class rmellipse.workflows.WORKFLOW_SPECIAL_KEYS(*args, **kwds)¶

Bases: enum.Enum

Create a collection of name/value pairs.

Example enumeration:

>>> class Color(Enum):
...     RED = 1
...     BLUE = 2
...     GREEN = 3

Access them by:

attribute access:
```
>>> Color.RED
<Color.RED: 1>
```
value lookup:
```
>>> Color(1)
<Color.RED: 1>
```
name lookup:
```
>>> Color['RED']
<Color.RED: 1>
```

Enumerations can be iterated over, and know how many members they have:

>>> len(Color)
3

>>> list(Color)
[<Color.RED: 1>, <Color.BLUE: 2>, <Color.GREEN: 3>]

Methods can be added to enumerations, and members can have their own attributes – see the documentation for details.

JOB = 'jobs'¶

RELEASE = 'release'¶

DATASETS = 'datasets'¶

class rmellipse.workflows.DIRECTORY_ITEMS(*args, **kwds)¶

Bases: enum.Enum

Create a collection of name/value pairs.

Example enumeration:

>>> class Color(Enum):
...     RED = 1
...     BLUE = 2
...     GREEN = 3

Access them by:

attribute access:
```
>>> Color.RED
<Color.RED: 1>
```
value lookup:
```
>>> Color(1)
<Color.RED: 1>
```
name lookup:
```
>>> Color['RED']
<Color.RED: 1>
```

Enumerations can be iterated over, and know how many members they have:

>>> len(Color)
3

>>> list(Color)
[<Color.RED: 1>, <Color.BLUE: 2>, <Color.GREEN: 3>]

Methods can be added to enumerations, and members can have their own attributes – see the documentation for details.

DIRECTORY = 'Directory'¶

FILE = 'File'¶

H5_FILE = 'H5_File'¶

H5_GROUP = 'H5_Group'¶

H5_DATASET = 'H5_DATASET'¶

class rmellipse.workflows.WorkflowConfig(workflow_file: pathlib.Path | str | dict, project_settings: ProjectSettings)¶

Bases: dict

Store and access a workflow configuration.

Parameters¶

dict_type_: _description_

Returns¶

_type_: _description_

Raises¶

Exception: _description_

Parameters:

workflow_filePath | str | dict: Workflow file.
project_settingsProjectSettings: Project settings that contain the workflow.

Raises:

ValueError: Missing environment variables.
ValueError: Empry workflow file.

project_settings¶

raw¶

env_variables¶

expand_variables(obj: str | dict | list, expand_env: bool = True, expand_datasets: bool = True) → str | dict | list¶

Expand variables of any object that can be serialized to JSON.

Encodes obj to a JSON string, replaces any variable expressions, then de-encodes the stirng. Order of operations is encode -> env variables -> dataset names -> decode.

JSON is a faster serialization format, so using it here instead of YAML.

Parameters:

objstr | dict | list: _description_
expand_env: bool, optional: If True, expands environment variables. Default is true.
expand_datasets: bool, optional: If True, expands dataset variables. Default is True

Returns:

object: Same type as input with variables expanded.

property title: str¶

property release_version: str¶

property requires_files: dict¶

Get a list of required files

Includes required files of the project, any name-space conflicts are overwritten with the workspace requires_files.

Returns:

dict[str]: dictionary of required files or folder patterns, keys are the new name of the datset within environment.

property requires_releases: None¶

Releases requirements defined in the workflow file.

Returns:

_type_: _description_

property requires_env: None¶

Environment variable requirements.

Returns:

_type_: _description_

property formatting_keys: dict¶: Dictionary of kwargs for string formating.

property datasets¶

List of defined datsets in the workflow config.

Returns:

_type_: _description_

property jobs¶

List of defined datsets in the workflow config.

Returns:

_type_: _description_

property release¶

Release related settings

Returns:

_type_: _description_

class rmellipse.workflows.RMESettings(project_dir: pathlib.Path)¶

Bases: dict

Settings file for the RME program.

rme.yml files are used for storing settings that can be defined at the user level. For example, what archives to use by default, and aliases for the archives.

Settings files are overloaded by higher priority settings files.

Discovery order (lowest to highest priority) User config -> project config

Parameters¶

dict_type_: _description_

Initialize self. See help(type(self)) for accurate signature.

get_host_settings(host_name: str = None)¶

Get the host settings by host name.

User and password are returned as None if they aren’t present in the settings.

Does not attempt to get credentials based on host.

Parameters:

host_namestr, optional: _description_, by default None

Returns:

host:: str
user:: str
password:: str

Raises:

LookupError:: If the host isn’t in the settings.

property hosts¶

property default_host¶

rmellipse.workflows¶

Submodules¶

Attributes¶

Exceptions¶

Classes¶

Package Contents¶

Parameters¶

Returns¶

Raises¶

Parameters¶

Table of Contents

This Page

Versions