Logging Metrics#
This how-to explains how to log metrics in Dioptra.
Prerequisites#
Prepare Your Deployment - A deployment of Dioptra is required.
Set Up the Python Client - Connect to the Python Client from within the job.
Metric Logging Workflow#
Metrics can be logged as part of a plugin, during the execution of a job, by using the python client, and can be retrieved via the API or viewed through the Job details page.
Note
This guide only explains how to log metrics through the client, and not how to incorporate metric logging into your custom plugin tasks.
Log a Metric to a Job#
Log a metric as part of a job using the Python client.
Client Method:
Use the client to create a metric.
Note that from a job, you can use os.environ["__JOB_ID"] to get the current job’s ID.
- JobsCollectionClient.append_metric_by_id(job_id: str | int, metric_name: str, metric_value: float, metric_step: int | None = None, timestamp: datetime.datetime | str | None = None) dioptra.client.jobs.T[source]#
Posts a new metric to a job.
- Parameters
job_id – The job id, an integer.
metric_name – The name of the metric.
metric_value – The value of the metric. Can be a float, NaN, Infinity, or -Infinity. NaN values will be converted to the string “nan” and infinite values will be converted to the strings “inf” and “-inf” before sending to the API.
metric_step – The step value for the metric, optional. Sequentially increasing step values are used to track how a metric changes over time. Indexed from 0. If not provided, defaults to 0.
timestamp – A timestamp value to associate with the metric. If not provided, defaults to the server time when the metric is received.
- Returns
The response from the Dioptra API.
Retrieving Metrics for Jobs#
Retrieve metrics from a job or view them in the GUI.
In the Dioptra GUI, navigate to the Jobs tab. Click the row for a job. Click the Metrics tab on the Job Dashboard page. A list of logged metrics will be displayed here.
Client Method:
Use the client to retrieve the latest metrics associated with the job.
- JobsCollectionClient.get_metrics_by_id(job_id: str | int) dioptra.client.jobs.T[source]#
Gets all the latest metrics for a job.
The returned metric value is either a float or a string representing the special values of NaN, Infinity, or -Infinity. NaN values are represented as the string “nan”, positive infinity as “inf”, and negative infinity as “-inf”. To convert these string values back to their corresponding float representations in Python, wrap the returned value with
float().- Parameters
job_id – The job id, an integer.
- Returns
The response from the Dioptra API.
Alternatively, you can retrieve the full metric history for a job.
- JobsCollectionClient.get_metrics_snapshots_by_id(job_id: str | int, metric_name: str | int, index: int = 0, page_length: int = 10) dioptra.client.jobs.T[source]#
Gets the metric history for a job with a specific metric name.
Each returned metric value is either a float or a string representing the special values of NaN, Infinity, or -Infinity. NaN values are represented as the string “nan”, positive infinity as “inf”, and negative infinity as “-inf”. To convert these string values back to their corresponding float representations in Python, wrap the returned value with
float().- Parameters
job_id – The job id, an integer.
metric_name – The name of the metric.
index – The paging index. Optional, defaults to 0.
page_length – The maximum number of metrics to return in the paged response. Optional, defaults to 10.
- Returns
The response from the Dioptra API.
It is also possible to retrieve the metrics across all jobs in an experiment.
- ExperimentsCollectionClient.get_metrics_by_id(experiment_id: str | int, index: int = 0, page_length: int = 10, sort_by: str | None = None, descending: bool | None = None, search: str | None = None) dioptra.client.experiments.T[source]#
Get the metrics for the jobs in this experiment.
Each returned metric value is either a float or a string representing the special values of NaN, Infinity, or -Infinity. NaN values are represented as the string “nan”, positive infinity as “inf”, and negative infinity as “-inf”. To convert these string values back to their corresponding float representations in Python, wrap the returned value with
float().- Parameters
experiment_id – The experiment id, an integer.
index – The paging index. Optional, defaults to 0.
page_length – The maximum number of experiments to return in the paged response. Optional, defaults to 10.
sort_by – The field to use to sort the returned list. Optional, defaults to None.
descending – Sort the returned list in descending order. Optional, defaults to None.
search – Search for jobs using the Dioptra API’s query language. Optional, defaults to None.
- Returns
The response from the Dioptra API.
See Also#
Metrics Reference - Reference page for metrics.