Import Resources#
This how-to explains how to import Dioptra Resources (e.g. Plugins, Entrypoints, Types) to a Dioptra deployment via an externally provided Resource Import TOML file.
Prerequisites#
Prepare Your Deployment - A deployment of Dioptra is required.
Set Up Dioptra in the GUI - Access Dioptra services in the GUI, create a user, and login.
Prepare Your Deployment - A deployment of Dioptra is required.
Set Up the Python Client - Connect to the Python Client in a Jupyter Notebook.
Resource Import Workflow#
Follow these steps to import resources (plugins, entrypoints, and types) into Dioptra.
Note
There are several name conflict resolution strategies for imported resources, which apply to all import options:
Fail - The import will fail if there is a name conflict.
Update - The previously registered resources will be updated with new definitions.
Overwrite - The previously registered resources will be deleted, and new resources will be created.
Step 1: Select the source to import from. (GUI only)#
Import resources from a Git repository.
In the Dioptra GUI, click the wrench symbol on the navigation bar. Select Import Resources from the drop down.
Select the either GIT REPO, UPLOAD ARCHIVE or UPLOAD DIRECTORY depending on where you would like to import resources from.
Select the access location for the resources:
For a Git repository:
Enter the HTTPS URL of a public repository on github.com or gitlab.com. For example:
https://github.com/usnistgov/dioptra.git#mainorhttps://gitlab.com/group/subgroup/project#main. The.gitsuffix is optional. Append#REFto select a branch, tag, or commit; omit it to use the repository’s default branch.Dioptra resolves the ref to a commit and downloads that commit’s tar snapshot over HTTPS. The response’s
hashis the resolved commit ID. The provider’s wrapper directory is removed, so the config path remains relative to the repository root.Downloads use 5-second connection and 30-second read timeouts to detect stalled connections, with no overall download-duration limit.
For uploading an Archive:
Click Archive File Upload and select a tar file containing the files. Uploaded archives retain their directory layout; their top-level directory is not stripped. Plain tar and gzip, bzip2, and xz-compressed tar archives use the same validation and extraction as repository imports.
For uploading a Directory:
Click Directory Upload and select the directory containing the files.
The common limits are 50 MiB per file, 500 MiB total file contents, and 10,000 explicit entries (files and archive directories). Downloaded and uploaded archives are limited to 100 MiB before decompression. The content limits apply to extracted files, not tar metadata or padding. Multipart files are subject to the same file/content limits. Both downloaded and uploaded archives receive a 120-second extraction budget
configPath is relative to the import root. Plugin and entrypoint paths in the
config are relative to the config’s directory and must resolve within that same
import root. For example, configs/dioptra.toml can refer to ../plugins.
References that escape the import root are rejected.
The response’s hash remains the resolved commit ID for repository imports,
the SHA-256 of the uploaded archive bytes for archive uploads, or the SHA-256 of
file contents concatenated in path-sorted order for multipart uploads.
Step 2: Import the Resources.#
Select a group to import these resources under.
Note
Dioptra does not currently support the creation of additional groups. All resources are under the same default public group.
Enter the path to the TOML file. (For example -
extra/dioptra.toml)Select a name conflict resolution strategy.
Select IMPORT.
Client Method:
Use the client to import the resources.
- WorkflowsCollectionClient.import_resources(group_id: int, source: str | dioptra.client.base.DioptraFile | list[dioptra.client.base.DioptraFile], config_path: str | None = None, resolve_name_conflicts_strategy: Optional[Literal['fail', 'update', 'overwrite']] = None)[source]
Import resources from a archive file or git repository
- Parameters
group_id – The group to import resources into
source – A public github.com or gitlab.com HTTPS repository URL, a DioptraFile containing an archive file, or a list[DioptraFile] containing resource files. Repository URLs accept an optional .git suffix and #REF for a branch, tag, or commit. Omitting #REF selects the default branch. Repository imports download a tar snapshot; semicolon path selections are not supported.
config_path – The path to the toml configuration file in the import source. If None, the API will use “dioptra.toml” as the default. Defaults to None. This path is relative to the import root. Plugin and entrypoint paths are relative to the config directory and must stay inside the import root.
resolve_name_conflicts_strategy – The strategy for resolving name conflicts. Either “fail”, “update”, or “overwrite”. If None, the API will use “fail” as the default. Defaults to None.
- Raises
IllegalArgumentError – If more than one import source is provided or if no import source is provided.
See Also#
Resource Import Reference - More information on syntax requirements for the Resource Import file
Plugins Reference - More information on syntax requirements for Plugins
Entrypoints Reference - More information on syntax requirements for Entrypoints