System Architecture

The NIST Test and Evaluation team provides a shared testbed to evaluate solutions against sequestered datasets.

Test Servers

NIST maintains two test servers.

Both ‘servers’ operate on the same physical hardware, with identical software setups. The only difference between the two ‘servers’ is which processing queue the job is submitted to, what compute time limits there are, and how much data will be evaluated.

Evaluation Server

⚠️ details pending ⚠️

Smoke Test Server

⚠️ details pending ⚠️

Job Timeout

Job time limits are enforced using the linux command timeout -s SIGKILL which can generate an exit status code of -9 for your job.

[vm-executor.py:197] Execute status = -9
[vm-executor.py:199] VM "gpu-vm-db" Execute submission "container.simg" timed out

Compute Resources

Your solution will have access to the following server resources allocated for the duration of the compute time limit to evaluate the relevant dataset.

⚠️ details pending ⚠️