TrojAI Leaderboard


Welcome to the TrojAI Leaderboard!

Using machine learning, an artificial intelligence (AI) is trained on data, learns relationships in that data, and then is deployed to the world to operate on new data. The problem is that an adversary that can disrupt the training pipeline can insert Trojan behaviors into the AI. TrojAI’s goal is to detect Trojans hidden in trained AI models. This page is a leaderboard of how well different Trojan detectors work against a population of AIs with and without Trojans. Read more about the problem, see the full submission documentation, or get started with a free minimal example.

Current Round: 10

Round 10 is the Object Detection round. Each AI is trained to perform Object Detection either using a single stage (SSD), or a two stage detector (Faster-RCNN). Submitted Trojan detectors must produce a probability of Trojan presence for 144 AIs within 2160 minutes of compute time. Each submission is executed using 3 GPUs which evaluate the models in parallel using round robin, so the wall time limit for a submission is 720 minutes (2160 / 3). For those AIs that have been attacked, the presence of the pattern will cause the AI to reliably produce the wrong extractive answer. The Round 10 Training Data Download consists of 144 reference AIs (exactly 50% are poisoned) with example input data. More info here.

Missing Zebra Evasion Trigger

Example poisoned image, where the green evasion trigger on the zebra causes the box to dissapear. This image is drawn from COCO (image 117897.jpg).


Reload page to update tables. Content is pushed every 10 minutes. Timestamps are presented in UTC.

Last modified:

Evaluation Server Status

Nodes: idle; running; down.
Accepting Submissions:

Smoke Test Server Status

Nodes: idle; running; down.
Accepting Submissions:


Teams/Jobs

Team Execution Timestamp Job Status File Status File Timestamp Time until next execution
trojai-example 2022-07-28T01:30:07 None Ok 2022-07-28T01:30:03 0 d, 0 h, 0 m, 0 s
IceTorch 2021-05-03T17:40:02 None Ok 2020-07-27T04:34:41 0 d, 0 h, 0 m, 0 s
trojaicy 2021-05-03T17:40:02 None Ok 2020-07-27T01:44:11 0 d, 0 h, 0 m, 0 s
Hector 2021-05-03T17:40:02 None Ok 2020-07-14T00:09:58 0 d, 0 h, 0 m, 0 s
Cassandra-XF 2021-05-03T17:40:02 None Ok 2020-07-27T19:44:57 0 d, 0 h, 0 m, 0 s
Perspecta 2022-08-09T13:10:01 None Ok 2022-08-09T13:04:43 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI 2022-08-11T06:50:02 None Ok 2022-08-11T06:41:58 0 d, 0 h, 0 m, 0 s
NSEC 2021-05-03T17:40:02 None None 2020-10-17T13:16:52 0 d, 0 h, 0 m, 0 s
DRAFFT 2021-05-03T17:40:02 None Ok 2020-09-19T21:01:10 0 d, 0 h, 0 m, 0 s
PL-GIFT 2022-08-09T03:50:01 None Ok 2022-08-09T03:42:13 0 d, 0 h, 0 m, 0 s
Perspecta-PurdueRutgers 2022-07-29T01:00:02 None Ok 2022-07-29T00:52:14 0 d, 0 h, 0 m, 0 s
Perspecta-IUB 2022-05-31T15:10:01 None Ok 2022-05-31T15:01:13 0 d, 0 h, 0 m, 0 s
ARM-UCSD 2022-05-05T01:00:01 None None 2022-05-05T00:50:39 0 d, 0 h, 0 m, 0 s
gooddefense None None None None 0 d, 0 h, 0 m, 0 s
ARM 2022-05-17T16:00:01 None Ok 2022-05-17T15:51:53 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-SBU 2022-08-11T16:30:01 None Ok 2022-08-11T16:24:12 0 d, 0 h, 0 m, 0 s
ARM-UMBC 2021-06-02T14:20:01 None Ok 2021-06-02T14:19:17 0 d, 0 h, 0 m, 0 s
cyberinfrastructure None None None None 0 d, 0 h, 0 m, 0 s
ICSI-1 2022-04-27T15:10:01 None Ok 2022-04-27T15:05:16 0 d, 0 h, 0 m, 0 s
ICSI-2 2022-08-11T21:30:01 None Ok 2022-08-11T21:22:33 0 d, 0 h, 0 m, 0 s
glaros None None None None 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-BostonU 2022-02-25T17:00:01 None Multiple files shared 2022-02-25T16:54:55 0 d, 0 h, 0 m, 0 s
Ultraman None None None None 0 d, 0 h, 0 m, 0 s
BackSec None None None None 0 d, 0 h, 0 m, 0 s
Palladium None None None None 0 d, 0 h, 0 m, 0 s

Results


Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Execution Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 1.1422 0.20328 0.35385 0.4767 909 2022-07-28T01:30:07 2022-07-28T01:30:03 None None
TrinitySRITrojAI 0.4334 0.06755 0.14125 0.8692 3937 2022-08-11T06:50:02 2022-08-11T06:41:58 None None
ICSI-2 0.3407 0.08762 0.10681 0.9378 573 2022-08-09T07:00:02 2022-08-09T05:59:27 None None
Perspecta-PurdueRutgers 0.2469 0.08555 0.05968 0.9405 20556 2022-07-29T01:00:02 2022-07-29T00:52:14 None None
PL-GIFT 0.3832 0.08957 0.12258 0.9099 2654 2022-08-09T03:50:01 2022-08-09T03:42:13 None :Container Parameters (jsonschema checker):
Perspecta 0.6632 0.02806 0.23535 0.6133 17442 2022-08-09T13:10:01 2022-08-09T13:04:43 None :Container Parameters (jsonschema checker):
TrinitySRITrojAI-SBU 0.6902 0.02699 0.24895 0.5545 968 2022-08-11T16:30:01 2022-08-11T16:24:12 None None

All Results


Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Execution Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 1.1422 0.20328 0.35385 0.4767 909 2022-07-28T01:30:07 2022-07-28T01:30:03 None None
TrinitySRITrojAI 0.4456 0.07967 0.14031 0.8835 11072 2022-07-28T03:50:01 2022-07-28T03:46:36 None None
TrinitySRITrojAI 0.4583 0.14674 0.13468 0.8875 11567 2022-08-01T20:00:01 2022-08-01T19:51:21 None None
TrinitySRITrojAI 0.4957 0.09840 0.16507 0.8336 1261 2022-08-08T23:10:01 2022-08-08T23:08:01 None None
TrinitySRITrojAI 0.4334 0.06755 0.14125 0.8692 3937 2022-08-11T06:50:02 2022-08-11T06:41:58 None None
ICSI-2 0.4892 0.06161 0.16241 0.8503 535 2022-07-28T21:00:02 2022-07-28T20:56:51 None :Container Parameters (jsonschema checker):
ICSI-2 0.4552 0.09580 0.14463 0.8638 532 2022-07-29T23:20:01 2022-07-29T23:12:21 None None
ICSI-2 0.4023 0.07610 0.12481 0.9045 569 2022-08-03T03:20:01 2022-08-03T03:17:09 None None
ICSI-2 0.3827 0.07356 0.11598 0.9153 569 2022-08-05T00:40:01 2022-08-05T00:39:34 None None
ICSI-2 0.3698 0.07443 0.11192 0.9288 590 2022-08-09T01:00:01 2022-08-09T00:52:50 None None
ICSI-2 0.3407 0.08762 0.10681 0.9378 573 2022-08-09T07:00:02 2022-08-09T05:59:27 None None
ICSI-2 0.4961 0.20392 0.12848 0.9263 586 2022-08-09T23:50:01 2022-08-09T23:47:56 None None
ICSI-2 0.3413 0.09577 0.10844 0.9376 603 2022-08-10T22:50:02 2022-08-10T22:12:01 None None
ICSI-2 0.3413 0.08756 0.10686 0.9378 588 2022-08-11T05:30:02 2022-08-11T05:29:28 None None
ICSI-2 0.3452 0.09544 0.10873 0.9376 594 2022-08-11T21:30:01 2022-08-11T21:22:33 None None
Perspecta-PurdueRutgers 0.2469 0.08555 0.05968 0.9405 20556 2022-07-29T01:00:02 2022-07-29T00:52:14 None None
PL-GIFT 0.6931 0.00000 0.25000 0.5000 242 2022-07-29T20:00:02 2022-07-29T19:44:36 :No Results: None
PL-GIFT 3.2188 1.36132 0.23092 0.7489 2887 2022-07-30T02:00:02 2022-07-29T22:41:27 None None
PL-GIFT 0.3832 0.08957 0.12258 0.9099 2654 2022-08-09T03:50:01 2022-08-09T03:42:13 None :Container Parameters (jsonschema checker):
Perspecta 0.6632 0.02806 0.23535 0.6133 17442 2022-08-09T13:10:01 2022-08-09T13:04:43 None :Container Parameters (jsonschema checker):
TrinitySRITrojAI-SBU 0.6931 0.00000 0.25000 0.5000 242 2022-08-11T00:50:02 2022-08-11T00:42:03 :No Results: None
TrinitySRITrojAI-SBU 0.6902 0.02699 0.24895 0.5545 968 2022-08-11T16:30:01 2022-08-11T16:24:12 None None

Test Teams/Jobs


Team Execution Timestamp Job Status File Status File Timestamp Time until next execution
trojai-example 2022-07-28T01:40:04 None Ok 2022-07-28T01:31:29 0 d, 0 h, 0 m, 0 s
IceTorch None None None None 0 d, 0 h, 0 m, 0 s
trojaicy 2022-07-28T01:40:04 None Ok 2020-07-27T01:16:59 0 d, 0 h, 0 m, 0 s
Hector None None None None 0 d, 0 h, 0 m, 0 s
Cassandra-XF None None None None 0 d, 0 h, 0 m, 0 s
Perspecta 2022-08-09T04:50:11 None None 2022-08-09T04:49:25 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI 2022-08-11T06:20:08 None None 2022-08-11T06:18:25 0 d, 0 h, 0 m, 0 s
NSEC None None None None 0 d, 0 h, 0 m, 0 s
DRAFFT 2022-07-28T01:40:04 None Ok 2020-09-18T23:49:20 0 d, 0 h, 0 m, 0 s
PL-GIFT 2022-08-09T03:30:11 None None 2022-08-09T03:26:46 0 d, 0 h, 0 m, 0 s
Perspecta-PurdueRutgers 2022-07-28T23:40:10 None None 2022-07-28T23:38:35 0 d, 0 h, 0 m, 0 s
Perspecta-IUB None None None None 0 d, 0 h, 0 m, 0 s
ARM-UCSD None None None None 0 d, 0 h, 0 m, 0 s
gooddefense None None None None 0 d, 0 h, 0 m, 0 s
ARM 2022-07-28T01:40:04 None Ok 2022-05-17T15:52:07 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-SBU 2022-08-11T16:10:08 None None 2022-08-11T16:06:50 0 d, 0 h, 0 m, 0 s
ARM-UMBC None None None None 0 d, 0 h, 0 m, 0 s
cyberinfrastructure None None None None 0 d, 0 h, 0 m, 0 s
ICSI-1 None None None None 0 d, 0 h, 0 m, 0 s
ICSI-2 2022-08-11T05:30:08 None None 2022-08-11T05:29:48 0 d, 0 h, 0 m, 0 s
glaros None None None None 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-BostonU 2022-07-28T01:40:04 None Ok 2022-02-26T19:30:00 0 d, 0 h, 0 m, 0 s
Ultraman None None None None 0 d, 0 h, 0 m, 0 s
BackSec None None None None 0 d, 0 h, 0 m, 0 s
Palladium None None None None 0 d, 0 h, 0 m, 0 s

Test Results


Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Execution Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 1.09775 0.3572 0.42324 0.250 180 2022-07-28T01:40:04 2022-07-28T01:31:29 None None
trojaicy 0.69315 0.0000 0.25000 0.500 2022-07-28T01:40:04 2020-07-27T01:16:59 :No Results::Info File Missing: None
TrinitySRITrojAI 0.32941 0.3183 0.09420 0.958 353 2022-08-01T19:20:21 2022-08-01T19:18:38 None None
DRAFFT 0.69315 0.0000 0.25000 0.500 2022-07-28T01:40:04 2020-09-18T23:49:20 :No Results::Info File Missing: None
PL-GIFT 0.14267 0.0853 0.02792 1.000 153 2022-08-09T03:30:11 2022-08-09T03:26:46 None :Container Parameters (jsonschema checker):
ARM 0.69315 0.0000 0.25000 0.500 89 2022-07-28T01:40:04 2022-05-17T15:52:07 :No Results: :Copy out:
ICSI-2 0.01096 0.0091 0.00032 1.000 120 2022-08-09T23:50:11 2022-08-09T23:48:44 None None
TrinitySRITrojAI-BostonU 0.69315 0.0000 0.25000 0.500 89 2022-07-28T01:40:04 2022-02-26T19:30:00 :No Results: :Container Parameters (jsonschema checker)::Copy out:
Perspecta-PurdueRutgers 0.30571 0.3666 0.08632 0.979 3809 2022-07-28T23:40:10 2022-07-28T23:38:35 None None
Perspecta 0.68164 0.1222 0.24407 0.542 3129 2022-08-09T04:50:11 2022-08-09T04:49:25 None :Container Parameters (jsonschema checker):
TrinitySRITrojAI-SBU 0.69315 0.0000 0.25000 0.500 51 2022-08-10T22:50:10 2022-08-10T16:32:37 :No Results: :Copy out:

All Test Results


Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Execution Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 1.09775 0.357 0.42324 0.250 180 2022-07-28T01:40:04 2022-07-28T01:31:29 None None
trojaicy 0.69315 0.000 0.25000 0.500 2022-07-28T01:40:04 2020-07-27T01:16:59 :No Results::Info File Missing: None
TrinitySRITrojAI 0.69315 0.000 0.25000 0.500 50 2022-07-28T01:40:04 2022-07-28T00:28:59 :No Results: :Copy out:
TrinitySRITrojAI 0.78433 0.513 0.25589 0.625 346 2022-07-28T03:20:11 2022-07-28T03:12:35 None None
TrinitySRITrojAI 0.32941 0.318 0.09420 0.958 353 2022-08-01T19:20:21 2022-08-01T19:18:38 None None
TrinitySRITrojAI 0.49337 0.416 0.16030 0.792 239 2022-08-08T22:00:11 2022-08-08T21:50:37 None None
TrinitySRITrojAI 0.47218 0.262 0.16019 0.792 639 2022-08-11T06:20:08 2022-08-11T06:18:25 None None
DRAFFT 0.69315 0.000 0.25000 0.500 2022-07-28T01:40:04 2020-09-18T23:49:20 :No Results::Info File Missing: None
PL-GIFT 0.69315 0.000 0.25000 0.500 60 2022-07-28T01:40:04 2022-06-20T18:31:58 :No Results: :Container Parameters (jsonschema checker)::Copy out:
PL-GIFT 0.69315 0.000 0.25000 0.500 50 2022-07-29T20:00:14 2022-07-29T19:34:00 :No Results: :Container Parameters (jsonschema checker)::Copy out:
PL-GIFT 0.69315 0.000 0.25000 0.500 39 2022-07-29T20:50:13 2022-07-29T20:41:58 :No Results: :Container Parameters (jsonschema checker)::Copy out:
PL-GIFT 0.32958 0.233 0.11667 0.937 151 2022-07-29T22:30:12 2022-07-29T22:27:40 None :Container Parameters (jsonschema checker):
PL-GIFT 0.69315 0.000 0.25000 0.500 152 2022-08-09T01:50:09 2022-08-09T01:45:55 :No Results: :Container Parameters (jsonschema checker)::Copy out:
PL-GIFT 0.14267 0.085 0.02792 1.000 153 2022-08-09T03:30:11 2022-08-09T03:26:46 None :Container Parameters (jsonschema checker):
ARM 0.69315 0.000 0.25000 0.500 89 2022-07-28T01:40:04 2022-05-17T15:52:07 :No Results: :Copy out:
ICSI-2 0.69315 0.000 0.25000 0.500 2022-07-28T01:40:04 2022-07-28T00:26:17 :No Results::Info File Missing: None
ICSI-2 0.32147 0.158 0.09516 1.000 110 2022-07-28T07:00:17 2022-07-28T06:53:04 None :Container Parameters (jsonschema checker):
ICSI-2 0.18506 0.069 0.03507 1.000 110 2022-07-29T23:20:07 2022-07-29T23:11:24 None None
ICSI-2 0.28589 0.111 0.07201 1.000 110 2022-08-03T03:20:11 2022-08-03T03:16:35 None None
ICSI-2 0.27118 0.114 0.06778 1.000 116 2022-08-05T00:20:08 2022-08-05T00:12:44 None None
ICSI-2 0.19901 0.064 0.03778 1.000 120 2022-08-09T01:00:11 2022-08-09T00:52:39 None None
ICSI-2 0.12565 0.041 0.01669 1.000 120 2022-08-09T05:20:09 2022-08-09T05:18:02 None None
ICSI-2 0.01096 0.009 0.00032 1.000 120 2022-08-09T23:50:11 2022-08-09T23:48:44 None None
ICSI-2 0.16796 0.112 0.03613 1.000 115 2022-08-10T22:50:10 2022-08-10T22:12:42 None None
ICSI-2 0.12565 0.041 0.01669 1.000 120 2022-08-11T05:30:08 2022-08-11T05:29:48 None None
TrinitySRITrojAI-BostonU 0.69315 0.000 0.25000 0.500 89 2022-07-28T01:40:04 2022-02-26T19:30:00 :No Results: :Container Parameters (jsonschema checker)::Copy out:
Perspecta-PurdueRutgers 0.69315 0.000 0.25000 0.500 20 2022-07-28T05:50:08 2022-07-28T05:42:58 :No Results: :Copy out:
Perspecta-PurdueRutgers 0.69315 0.000 0.25000 0.500 49 2022-07-28T06:30:10 2022-07-28T06:21:18 :No Results: :Copy out:
Perspecta-PurdueRutgers 0.69315 0.000 0.25000 0.500 50 2022-07-28T15:20:13 2022-07-28T15:19:37 :No Results: :Copy out:
Perspecta-PurdueRutgers 0.69315 0.000 0.25000 0.500 50 2022-07-28T16:40:11 2022-07-28T16:32:10 :No Results: :Copy out:
Perspecta-PurdueRutgers 0.69315 0.000 0.25000 0.500 59 2022-07-28T17:30:09 2022-07-28T17:27:32 :No Results: :Copy out:
Perspecta-PurdueRutgers 0.69315 0.000 0.25000 0.500 60 2022-07-28T18:20:09 2022-07-28T18:11:36 :No Results: :Copy out:
Perspecta-PurdueRutgers 0.69315 0.000 0.25000 0.500 60 2022-07-28T18:50:10 2022-07-28T18:48:12 :No Results: :Copy out:
Perspecta-PurdueRutgers 0.69315 0.000 0.25000 0.500 60 2022-07-28T19:20:11 2022-07-28T19:18:35 :No Results: :Copy out:
Perspecta-PurdueRutgers 0.32550 0.380 0.09063 0.771 4538 2022-07-28T20:00:10 2022-07-28T19:56:56 None None
Perspecta-PurdueRutgers 0.53000 0.524 0.16712 0.896 3811 2022-07-28T21:30:12 2022-07-28T21:26:39 None None
Perspecta-PurdueRutgers 0.30571 0.367 0.08632 0.979 3809 2022-07-28T23:40:10 2022-07-28T23:38:35 None None
Perspecta 0.69315 0.000 0.25000 0.500 2022-08-08T05:50:09 2022-08-08T05:48:08 :No Results::Info File Missing: None
Perspecta 0.69315 0.000 0.25000 0.500 63 2022-08-08T13:30:10 2022-08-08T13:28:28 :No Results: :Container Parameters (jsonschema checker)::Copy out:
Perspecta 0.69315 0.000 0.25000 0.500 137 2022-08-08T18:20:11 2022-08-08T18:18:52 :No Results: :Container Parameters (jsonschema checker)::Copy out:
Perspecta 0.69315 0.000 0.25000 0.500 142 2022-08-08T20:40:07 2022-08-08T20:31:31 :No Results: :Container Parameters (jsonschema checker)::Copy out:
Perspecta 0.69315 0.000 0.25000 0.500 3056 2022-08-08T21:40:10 2022-08-08T21:33:24 :No Results: :Container Parameters (jsonschema checker)::Copy out:
Perspecta 0.68164 0.122 0.24407 0.542 3129 2022-08-09T04:50:11 2022-08-09T04:49:25 None :Container Parameters (jsonschema checker):
TrinitySRITrojAI-SBU 0.69315 0.000 0.25000 0.500 51 2022-08-10T22:50:10 2022-08-10T16:32:37 :No Results: :Copy out:
TrinitySRITrojAI-SBU 0.69315 0.000 0.25000 0.500 50 2022-08-11T02:30:10 2022-08-11T02:27:55 :No Results: :Copy out:
TrinitySRITrojAI-SBU 0.69315 0.000 0.25000 0.500 182 2022-08-11T15:20:08 2022-08-11T15:12:18 :No Results: :Copy out:
TrinitySRITrojAI-SBU 0.70040 0.094 0.25390 0.813 192 2022-08-11T16:10:08 2022-08-11T16:06:50 None None

Status and Error Codes

Messages about the status of submissions that may be awaiting execution.

Code Description
None No jobs are submitted.
Queued Job has been queued for processing.
Awaiting Timeout Job is submitted, but must wait for time until next execution.
Pending Job is in processing queue and is pending availability of system resources.
Running Job is running
Disabled No longer accepting jobs from the team.

Messages about the status of files that are shared with the TrojAI google drive.

Code Description
None No shared files found in TrojAI google drive.
Multiple files submitted Team has more than one file shared with the TrojAI google drive account per evaluation server. You can share one file which starts with 'test' (for STS) and one which does not (for ES). Unshare/Delete shared files until only one is shared per server.
Ok Found one file shared with TrojAI google drive per server.

Error codes during processing result and metadata after a submission has completed.

Code Description
None No errors found while parsing the results of the submission.
:Result Parse: Unable to parse one ore more result files generated by the submission container.
:No Results: Unable to find and parse any results generated by the container.
:Executed File Update: Unable to update the submission metadata to reflect the Drive File which was actually executed, if different from the one initially submitted.
:Log File Missing: Unable to find the log file from the container execution.
:Info File Missing: Unable to find the job information file from the container execution.
:Confusion File Missing: Unable to find the confusion matrix file from the container execution.
:File Upload: Unable to upload a file from the test server to Drive.

Error codes during execution of a submission.

Code Description
None No errors found while launching the submission.
:Container Parameters: There was an error with the container, one or more required parameters may not be valid or not exist.
:Slurm Script Error: There was an error when submitting to the processing queue (Slurm may be offline)
Hypervisor offline Unable to establish connection to the virtual machine hypervisor.
:GPU: Failed to communicate with the GPU within the VM
:Copy in: Failed to copy in the actor-shared file into the VM.
:Timeout: The actor's execution timed out. Failed to finish processing all input models within allocated time.
:Execute: Other errors during execution of actor execution. (check slurm log file)
:Copy out: Failed to copy out results (they may not exist).
:Shutdown: Failed to shutdown the VM.
:VM: Issue with the VM resource (major error).

Disclaimer

NIST-developed software is provided by NIST as a public service. You may use, copy and distribute copies of the software in any medium, provided that you keep intact this entire notice. You may improve, modify and create derivative works of the software or any portion of the software, and you may copy and distribute such modifications or works. Modified works should carry a notice stating that you changed the software and should note the date and nature of any such change. Please explicitly acknowledge the National Institute of Standards and Technology as the source of the software.

NIST-developed software is expressly provided "AS IS." NIST MAKES NO WARRANTY OF ANY KIND, EXPRESS, IMPLIED, IN FACT OR ARISING BY OPERATION OF LAW, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT AND DATA ACCURACY. NIST NEITHER REPRESENTS NOR WARRANTS THAT THE OPERATION OF THE SOFTWARE WILL BE UNINTERRUPTED OR ERROR-FREE, OR THAT ANY DEFECTS WILL BE CORRECTED. NIST DOES NOT WARRANT OR MAKE ANY REPRESENTATIONS REGARDING THE USE OF THE SOFTWARE OR THE RESULTS THEREOF, INCLUDING BUT NOT LIMITED TO THE CORRECTNESS, ACCURACY, RELIABILITY, OR USEFULNESS OF THE SOFTWARE.

You are solely responsible for determining the appropriateness of using and distributing the software and you assume all risks associated with its use, including but not limited to the risks and costs of program errors, compliance with applicable laws, damage to or loss of data, programs or equipment, and the unavailability or interruption of operation. This software is not intended to be used in any situation where a failure could cause risk of injury or damage to property. The software developed by NIST employees is not subject to copyright protection within the United States.

The Cross Entropy loss values (and confidence intervals) reported by the TrojAI leaderboard are only indicative of trojan detector performance on the specific dataset the detector was evaluated on. The TrojAI leaderboard results do not indicate general purpose trojan detection algorithm quality.