Metrics#

Metric Definition#

A Metric in Dioptra is a representation of a measurement taken during a job.

Metric Attributes#

A metric has the following attributes. Note that the name and step values together form a primary key for metrics.

  • Name: (string) The name associated with the metric.

  • Value: (float or string) The value of the metric. Can be a float, NaN, Infinity, or -Infinity. When sent to the API, the following values will be converted:

    • NaN will become "nan"

    • Infinity will become "inf"

    • -Infinity will become "-inf"

Optional Attributes#

  • Step: (integer, optional) an optional value which can be used to track the change of a metric over time (using sequentially increasing integers). Defaults to 0.

  • Timestamp: (datetime or string, optional) - a timestamp value to associate with the metric. If not provided, defaults to the server time when the metric is logged.

Retrieval Interfaces#

Metrics can be retrieved using the python client or the RESTAPI. Alternatively, the Job Dashboard page in the UI can show an overview of metrics for the job.

Using Python Client#

Retrieve the metrics with the highest step number for the job.

JobsCollectionClient.get_metrics_by_id(job_id: str | int) → dioptra.client.jobs.T[source]#

Gets all the latest metrics for a job.

The returned metric value is either a float or a string representing the special values of NaN, Infinity, or -Infinity. NaN values are represented as the string “nan”, positive infinity as “inf”, and negative infinity as “-inf”. To convert these string values back to their corresponding float representations in Python, wrap the returned value with float().

Parameters

job_id – The job id, an integer.

Returns

The response from the Dioptra API.

Retrieve the full metric history for a job for a given name.

JobsCollectionClient.get_metrics_snapshots_by_id(job_id: str | int, metric_name: str | int, index: int = 0, page_length: int = 10) → dioptra.client.jobs.T[source]#

Gets the metric history for a job with a specific metric name.

Each returned metric value is either a float or a string representing the special values of NaN, Infinity, or -Infinity. NaN values are represented as the string “nan”, positive infinity as “inf”, and negative infinity as “-inf”. To convert these string values back to their corresponding float representations in Python, wrap the returned value with float().

Parameters

  • job_id – The job id, an integer.

  • metric_name – The name of the metric.

  • index – The paging index. Optional, defaults to 0.

  • page_length – The maximum number of metrics to return in the paged response. Optional, defaults to 10.

Returns

The response from the Dioptra API.

Retrieve the metrics with the highest step number for all jobs in an experiment

ExperimentsCollectionClient.get_metrics_by_id(experiment_id: str | int, index: int = 0, page_length: int = 10, sort_by: str | None = None, descending: bool | None = None, search: str | None = None) → dioptra.client.experiments.T[source]#

Get the metrics for the jobs in this experiment.

Each returned metric value is either a float or a string representing the special values of NaN, Infinity, or -Infinity. NaN values are represented as the string “nan”, positive infinity as “inf”, and negative infinity as “-inf”. To convert these string values back to their corresponding float representations in Python, wrap the returned value with float().

Parameters

  • experiment_id – The experiment id, an integer.

  • index – The paging index. Optional, defaults to 0.

  • page_length – The maximum number of experiments to return in the paged response. Optional, defaults to 10.

  • sort_by – The field to use to sort the returned list. Optional, defaults to None.

  • descending – Sort the returned list in descending order. Optional, defaults to None.

  • search – Search for jobs using the Dioptra API’s query language. Optional, defaults to None.

Returns

The response from the Dioptra API.

Using REST API#

Metrics can be retrieved directly via the HTTP API.

Retrieve Latest Metrics for a Job

See the GET /api/v1/jobs/{int:id}/metrics endpoint documentation for payload requirements.

Retrieve Full Metric History for a Job for a given Metric

See the GET /api/v1/jobs/{int:id}/metrics/{str:name}/snapshots endpoint documentation for payload requirements.

Retrieve Latest Metrics for all Jobs in an Experiment

See the GET /api/v1/experiments/{int:id}/metrics endpoint documentation for payload requirements.

Registration Interfaces#

Metrics can be logged to a job using either the python client or the REST API.

Using Python Client#

Post metrics for a job.

JobsCollectionClient.append_metric_by_id(job_id: str | int, metric_name: str, metric_value: float, metric_step: int | None = None, timestamp: datetime.datetime | str | None = None) → dioptra.client.jobs.T[source]#

Posts a new metric to a job.

Parameters

  • job_id – The job id, an integer.

  • metric_name – The name of the metric.

  • metric_value – The value of the metric. Can be a float, NaN, Infinity, or -Infinity. NaN values will be converted to the string “nan” and infinite values will be converted to the strings “inf” and “-inf” before sending to the API.

  • metric_step – The step value for the metric, optional. Sequentially increasing step values are used to track how a metric changes over time. Indexed from 0. If not provided, defaults to 0.

  • timestamp – A timestamp value to associate with the metric. If not provided, defaults to the server time when the metric is received.

Returns

The response from the Dioptra API.

Using REST API#

Metrics can be logged using the RESTAPI.

Log Metrics

See the POST /api/v1/jobs/{int:id}/metrics endpoint documentation for payload requirements.

See Also#

  • <how-to-logging-metrics>