TrojAI Leaderboards

Click Here to Join the Competition!

Using machine learning, an artificial intelligence (AI) is trained on data, learns relationships in that data, and then is deployed to the world to operate on new data. The problem is that an adversary that can disrupt the training pipeline can insert Trojan behaviors into the AI. TrojAI’s goal is to detect Trojans hidden in trained AI models. This page is a leaderboard of how well different Trojan detectors work against a population of AIs with and without Trojans. Read more about the problem, see the full submission documentation, or get started with a free minimal example.

Submission container names must use the following format: "<Leaderboard Name>_<Data Split>_<Container Name>.simg"

Reload page to update tables. Content is pushed every 10 minutes. Timestamps are presented in UTC.

Last modified:

Evaluation Server Status

Nodes: idle; running; down.
TrojAI Accepting Submissions:

Smoke Test Server Status

Nodes: idle; running; down.
TrojAI Accepting Submissions:


Round 10 leaderboard for object detection August 2022.

Each AI is trained to perform Object Detection either using a single stage (SSD), or a two stage detector (Faster-RCNN). For those AIs that have been attacked, the presence of the pattern will cause the AI to reliably produce the wrong extractive answer. The Round 10 Training Data Download consists of 144 reference AIs (exactly 50% are poisoned) with example input data. More info here.

Missing Zebra Evasion Trigger

Example poisoned image, where the green evasion trigger on the zebra causes the box to dissapear. This image is drawn from COCO (image 117897.jpg).

Required filename format: "object-detection-aug2022_train_<Submission Name>.simg"
Accepting submissions: True
Number of models in object-detection-aug2022, train: 144
Execution timeout (hh:mm:ss): 1 day, 12:00:00

Teams/Jobs

Team Submission Timestamp Type Job Status File Status General Status File Timestamp Time until next execution
Perspecta 2022-09-21T13:10:04 performer None None Ok 2022-09-21T13:01:18 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI 2022-09-23T05:40:05 performer None None Ok 2022-09-23T05:32:01 0 d, 0 h, 0 m, 0 s
DRAFFT performer None None Ok None 0 d, 0 h, 0 m, 0 s
PL-GIFT performer None None Ok None 0 d, 0 h, 0 m, 0 s
Perspecta-PurdueRutgers performer None None Ok None 0 d, 0 h, 0 m, 0 s
Perspecta-IUB 2022-09-20T19:20:07 performer None None Ok 2022-09-20T19:16:36 0 d, 0 h, 0 m, 0 s
ARM-UCSD 2022-10-06T22:50:10 performer None None Ok 2022-10-06T22:41:58 0 d, 0 h, 0 m, 0 s
ARM 2022-10-03T07:30:13 performer None None Ok 2022-10-03T07:25:28 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-SBU performer None None Ok None 0 d, 0 h, 0 m, 0 s
ARM-UMBC performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-1 performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-2 performer None None Ok None 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-BostonU performer None None Ok None 0 d, 0 h, 0 m, 0 s
trojai-example 2022-09-16T20:09:40 public None None Ok 2022-09-16T20:05:28 0 d, 0 h, 0 m, 0 s
IceTorch public None None Ok None 0 d, 0 h, 0 m, 0 s
trojaicy public None None Ok None 0 d, 0 h, 0 m, 0 s
Hector public None None Ok None 0 d, 0 h, 0 m, 0 s
Cassandra-XF public None None Ok None 0 d, 0 h, 0 m, 0 s
gooddefense public None None Ok None 0 d, 0 h, 0 m, 0 s
cyberinfrastructure public None None Ok None 0 d, 0 h, 0 m, 0 s
glaros public None None Ok None 0 d, 0 h, 0 m, 0 s
Ultraman public None None Ok None 0 d, 0 h, 0 m, 0 s
BackSec public None None Ok None 0 d, 0 h, 0 m, 0 s
Palladium public None None Ok None 0 d, 0 h, 0 m, 0 s
IIECAS public None None Ok None 0 d, 0 h, 0 m, 0 s
NSEC public None None Ok None 0 d, 0 h, 0 m, 0 s
UMBCb 2022-10-17T18:40:21 performer None None Ok 2022-10-17T18:34:26 0 d, 0 h, 0 m, 0 s

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 0.96655 0.15787 0.32514 0.52537 2592.99 2022-09-16T20:09:40 2022-09-16T20:05:28 None None
TrinitySRITrojAI 0.1878 0.04885 0.04729 0.9864 16587.61 2022-09-23T05:40:05 2022-09-23T05:32:01 None None
Perspecta 0.30791 0.06677 0.09363 0.93962 55923.26 2022-09-21T13:10:04 2022-09-21T13:01:18 None None
Perspecta-IUB 0.28794 0.06866 0.09122 0.94956 4631.59 2022-09-20T19:20:07 2022-09-20T19:16:36 None None
ARM 0.44716 0.0774 0.151 0.8751 2595.77 2022-10-03T05:20:13 2022-10-03T05:16:26 None None
ARM-UCSD 0.41538 0.0547 0.13453 0.89612 19482.57 2022-10-06T22:50:10 2022-10-06T22:41:58 None None
UMBCb 0.31604 0.04107 0.08516 0.97357 1956.02 2022-10-17T18:40:21 2022-10-17T18:34:26 :Container File Missing: None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 0.96655 0.15787 0.32514 0.52537 2592.99 2022-09-16T20:09:40 2022-09-16T20:05:28 None None
TrinitySRITrojAI 0.1878 0.04885 0.04729 0.9864 16587.61 2022-09-23T05:40:05 2022-09-23T05:32:01 None None
Perspecta 0.69068 0.00482 0.24883 0.50694 5997.8 2022-09-19T21:40:04 2022-09-19T21:31:36 :Result Parse: None
Perspecta 0.69315 0.0 0.25 0.5 875.54 2022-09-20T16:00:04 2022-09-20T15:52:03 None None
Perspecta 0.30791 0.06677 0.09363 0.93962 55923.26 2022-09-21T13:10:04 2022-09-21T13:01:18 None None
Perspecta-IUB 0.28794 0.06866 0.09122 0.94956 4631.59 2022-09-20T19:20:07 2022-09-20T19:16:36 None None
ARM 0.44716 0.0774 0.151 0.8751 2595.77 2022-10-03T05:20:13 2022-10-03T05:16:26 None None
ARM 0.51654 0.0708 0.17471 0.89082 2593.42 2022-10-03T07:30:13 2022-10-03T07:25:28 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 382.63 2022-09-19T19:10:07 2022-09-19T17:19:37 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 557.98 2022-09-19T23:30:07 2022-09-19T23:25:37 None None
ARM-UCSD 0.68683 0.00891 0.24721 0.51379 400.63 2022-09-19T23:50:34 2022-09-19T23:45:52 None None
ARM-UCSD 0.69889 0.02001 0.25175 0.50675 395.63 2022-09-20T03:50:07 2022-09-20T03:48:06 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 818.28 2022-09-20T16:10:04 2022-09-20T16:03:08 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 288.51 2022-09-20T16:30:28 2022-09-20T16:22:56 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T23:40:07 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T23:50:07 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T00:00:07 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T00:10:07 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T00:20:07 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T00:30:08 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T00:40:07 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T00:50:07 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T01:00:07 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T01:10:06 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T01:20:07 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T01:30:07 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T01:40:06 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T01:50:06 2022-09-20T23:32:15 :Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T02:00:06 2022-09-20T23:32:15 :Executed File Update::Info File Missing::Log File Missing::Container File Missing: None
ARM-UCSD 0.56854 0.08995 0.19865 0.78173 9436.56 2022-09-21T02:10:07 2022-09-20T23:32:15 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 1153.95 2022-10-04T04:20:09 2022-10-04T04:13:30 None None
ARM-UCSD 0.41538 0.0547 0.13453 0.89612 19482.57 2022-10-06T22:50:10 2022-10-06T22:41:58 None None
UMBCb 0.31604 0.04107 0.08516 0.97357 1956.02 2022-10-17T18:40:21 2022-10-17T18:34:26 :Container File Missing: None

Required filename format: "object-detection-aug2022_test_<Submission Name>.simg"
Accepting submissions: True
Number of models in object-detection-aug2022, test: 144
Execution timeout (hh:mm:ss): 1 day, 12:00:00

Teams/Jobs

Team Submission Timestamp Type Job Status File Status General Status File Timestamp Time until next execution
Perspecta 2022-09-20T20:00:03 performer None None Ok 2022-09-20T19:53:19 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI 2022-09-23T05:40:05 performer None None Ok 2022-09-23T05:32:01 0 d, 0 h, 0 m, 0 s
DRAFFT performer None None Ok None 0 d, 0 h, 0 m, 0 s
PL-GIFT performer None None Ok None 0 d, 0 h, 0 m, 0 s
Perspecta-PurdueRutgers performer None None Ok None 0 d, 0 h, 0 m, 0 s
Perspecta-IUB 2022-09-20T19:20:07 performer None None Ok 2022-09-20T19:16:36 0 d, 0 h, 0 m, 0 s
ARM-UCSD 2022-11-08T00:40:10 performer None Ok Ok 2022-11-08T00:37:39 0 d, 0 h, 0 m, 0 s
ARM 2022-10-03T07:30:13 performer None Ok Ok 2022-10-03T07:25:28 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-SBU performer None None Ok None 0 d, 0 h, 0 m, 0 s
ARM-UMBC performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-1 performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-2 performer None None Ok None 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-BostonU performer None None Ok None 0 d, 0 h, 0 m, 0 s
trojai-example 2022-09-20T18:30:09 public None Ok Ok 2022-09-20T18:27:57 0 d, 0 h, 0 m, 0 s
IceTorch public None None Ok None 0 d, 0 h, 0 m, 0 s
trojaicy public None None Ok None 0 d, 0 h, 0 m, 0 s
Hector public None None Ok None 0 d, 0 h, 0 m, 0 s
Cassandra-XF public None None Ok None 0 d, 0 h, 0 m, 0 s
gooddefense public None None Ok None 0 d, 0 h, 0 m, 0 s
cyberinfrastructure public None None Ok None 0 d, 0 h, 0 m, 0 s
glaros public None None Ok None 0 d, 0 h, 0 m, 0 s
Ultraman public None None Ok None 0 d, 0 h, 0 m, 0 s
BackSec public None None Ok None 0 d, 0 h, 0 m, 0 s
Palladium public None None Ok None 0 d, 0 h, 0 m, 0 s
IIECAS public None None Ok None 0 d, 0 h, 0 m, 0 s
NSEC public None None Ok None 0 d, 0 h, 0 m, 0 s
UMBCb 2022-10-17T18:40:21 performer None None Ok 2022-10-17T18:34:26 0 d, 0 h, 0 m, 0 s

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 0.69315 0.0 0.25 0.5 2022-09-20T18:30:09 2022-09-20T18:27:57 :Info File Missing: None
TrinitySRITrojAI 0.39378 0.08799 0.12842 0.89902 6019.45 2022-09-09T20:00:01 2022-09-09T19:54:15 :Executed File Update::Log File Missing: None
ICSI-2 0.33308 0.08907 0.104 0.93808 662.89 2022-09-02T10:10:01 2022-09-02T10:03:39 :Executed File Update::Log File Missing: None
Perspecta-PurdueRutgers 0.16787 0.09226 0.04146 0.96335 20369.45 2022-08-21T20:10:01 2022-08-21T20:01:40 :Executed File Update::Log File Missing: None
PL-GIFT 0.37038 0.0686 0.11748 0.92033 541.24 2022-09-08T18:20:02 2022-09-08T18:17:57 :Executed File Update::Log File Missing: None
Perspecta 0.38825 0.08085 0.12527 0.8914 55313.35 2022-09-20T20:00:03 2022-09-20T19:53:19 None None
TrinitySRITrojAI-SBU 0.44974 0.06643 0.14299 0.87953 1181.78 2022-09-04T19:40:01 2022-09-04T15:36:46 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.34531 0.0922 0.10788 0.92708 983.53 2022-09-02T05:00:01 2022-09-02T04:52:07 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-BostonU 0.44557 0.07967 0.14031 0.88349 11213.55 2022-08-23T14:30:01 2022-08-23T14:21:31 :Executed File Update::Log File Missing: None
ARM 0.62867 0.37593 0.15467 0.85957 2596.3 2022-10-03T06:20:13 2022-10-03T06:18:21 None None
ARM-UCSD 0.46467 0.05577 0.15539 0.85552 19518.63 2022-10-06T22:50:10 2022-10-06T22:41:58 None None
UMBCb 0.46014 0.06258 0.14668 0.87973 1947.54 2022-10-17T18:40:21 2022-10-17T18:34:26 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 1.14219 0.20328 0.35385 0.47666 909.16 2022-07-28T01:30:07 2022-07-28T01:30:03 :Executed File Update::Log File Missing: None
trojai-example 0.95963 0.14719 0.32745 0.48563 2591.05 2022-09-16T20:09:40 2022-09-16T20:05:28 None None
trojai-example 0.69315 0.0 0.25 0.5 2022-09-20T18:30:09 2022-09-20T18:27:57 :Info File Missing: None
TrinitySRITrojAI 0.44557 0.07967 0.14031 0.88349 11072.36 2022-07-28T03:50:01 2022-07-28T03:46:36 :Executed File Update::Log File Missing: None
TrinitySRITrojAI 0.45826 0.14674 0.13468 0.88754 11567.63 2022-08-01T20:00:01 2022-08-01T19:51:21 :Executed File Update::Log File Missing: None
TrinitySRITrojAI 0.49569 0.0984 0.16507 0.83362 1261.47 2022-08-08T23:10:01 2022-08-08T23:08:01 :Executed File Update::Log File Missing: None
TrinitySRITrojAI 0.43343 0.06755 0.14125 0.86921 3937.95 2022-08-11T06:50:02 2022-08-11T06:41:58 :Executed File Update::Log File Missing: None
TrinitySRITrojAI 0.40207 0.10005 0.13224 0.90075 3942.27 2022-08-27T06:00:01 2022-08-27T05:58:46 :Executed File Update::Log File Missing: None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 2022-09-06T22:50:01 2022-09-06T22:45:07 :Executed File Update::Info File Missing::Log File Missing: None
TrinitySRITrojAI 0.40557 0.10773 0.13063 0.89911 3927.8 2022-09-07T04:50:02 2022-09-07T04:34:50 :Executed File Update::Log File Missing: None
TrinitySRITrojAI 0.39378 0.08799 0.12842 0.89902 6019.45 2022-09-09T20:00:01 2022-09-09T19:54:15 :Executed File Update::Log File Missing: None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 2022-09-20T18:20:04 2022-09-20T18:15:08 :Info File Missing: None
TrinitySRITrojAI 0.40958 0.0964 0.13166 0.89361 16610.22 2022-09-23T05:40:05 2022-09-23T05:32:01 None None
ICSI-2 0.48919 0.06161 0.16241 0.85031 535.05 2022-07-28T21:00:02 2022-07-28T20:56:51 :Executed File Update::Log File Missing: :Container Parameters (jsonschema checker):
ICSI-2 0.45518 0.0958 0.14463 0.86381 532.58 2022-07-29T23:20:01 2022-07-29T23:12:21 :Executed File Update::Log File Missing: None
ICSI-2 0.4023 0.0761 0.12481 0.90451 569.74 2022-08-03T03:20:01 2022-08-03T03:17:09 :Executed File Update::Log File Missing: None
ICSI-2 0.38275 0.07356 0.11598 0.91532 569.39 2022-08-05T00:40:01 2022-08-05T00:39:34 :Executed File Update::Log File Missing: None
ICSI-2 0.36983 0.07443 0.11192 0.92882 590.89 2022-08-09T01:00:01 2022-08-09T00:52:50 :Executed File Update::Log File Missing: None
ICSI-2 0.34072 0.08762 0.10681 0.93779 573.48 2022-08-09T07:00:02 2022-08-09T05:59:27 :Executed File Update::Log File Missing: None
ICSI-2 0.49614 0.20392 0.12848 0.92631 586.74 2022-08-09T23:50:01 2022-08-09T23:47:56 :Executed File Update::Log File Missing: None
ICSI-2 0.34129 0.09577 0.10844 0.9376 603.02 2022-08-10T22:50:02 2022-08-10T22:12:01 :Executed File Update::Log File Missing: None
ICSI-2 0.34132 0.08756 0.10686 0.93779 588.8 2022-08-11T05:30:02 2022-08-11T05:29:28 :Executed File Update::Log File Missing: None
ICSI-2 0.34523 0.09544 0.10873 0.9376 594.7 2022-08-11T21:30:01 2022-08-11T21:22:33 :Executed File Update::Log File Missing: None
ICSI-2 0.35496 0.08273 0.11088 0.92429 1185.1 2022-08-16T14:50:01 2022-08-16T14:49:40 :Executed File Update::Log File Missing: None
ICSI-2 0.33536 0.089 0.105 0.93808 593.21 2022-08-22T23:50:01 2022-08-22T23:41:20 :Executed File Update::Log File Missing: None
ICSI-2 0.33536 0.089 0.105 0.93808 594.83 2022-08-23T05:50:01 2022-08-23T00:03:15 :Executed File Update::Log File Missing: None
ICSI-2 0.3406 0.08354 0.10502 0.93412 591.03 2022-08-23T11:50:01 2022-08-23T07:14:37 :Executed File Update::Log File Missing: None
ICSI-2 0.33308 0.08907 0.104 0.93808 662.89 2022-09-02T10:10:01 2022-09-02T10:03:39 :Executed File Update::Log File Missing: None
Perspecta-PurdueRutgers 0.24686 0.08555 0.05968 0.94049 20556.45 2022-07-29T01:00:02 2022-07-29T00:52:14 :Executed File Update::Log File Missing: None
Perspecta-PurdueRutgers 0.23909 0.08514 0.05642 0.94126 20181.37 2022-08-19T23:20:01 2022-08-19T23:13:07 :Executed File Update::Log File Missing: None
Perspecta-PurdueRutgers 0.24568 0.09897 0.05347 0.92612 20670.17 2022-08-20T21:40:02 2022-08-20T21:32:11 :Executed File Update::Log File Missing: None
Perspecta-PurdueRutgers 0.16787 0.09226 0.04146 0.96335 20369.45 2022-08-21T20:10:01 2022-08-21T20:01:40 :Executed File Update::Log File Missing: None
PL-GIFT 0.69315 0.0 0.25 0.5 242.71 2022-07-29T20:00:02 2022-07-29T19:44:36 :Executed File Update::Log File Missing: None
PL-GIFT 3.21884 1.36132 0.23092 0.74894 2887.89 2022-07-30T02:00:02 2022-07-29T22:41:27 :Executed File Update::Log File Missing: None
PL-GIFT 0.38316 0.08957 0.12258 0.90992 2654.61 2022-08-09T03:50:01 2022-08-09T03:42:13 :Executed File Update::Log File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.42492 0.11866 0.13435 0.88657 2498.94 2022-08-20T16:30:02 2022-08-20T16:24:23 :Executed File Update::Log File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.57207 0.16016 0.17005 0.82147 974.33 2022-08-25T23:20:01 2022-08-25T23:12:14 :Executed File Update::Log File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.4778 0.12937 0.15503 0.8968 1925.54 2022-09-03T16:20:02 2022-09-03T16:16:06 :Executed File Update::Log File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.4987 0.07674 0.17538 0.88262 1914.04 2022-09-03T22:20:02 2022-09-03T18:33:34 :Executed File Update::Log File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.38923 0.07448 0.12435 0.91223 517.53 2022-09-07T20:40:02 2022-09-07T20:32:17 :Executed File Update::Log File Missing: None
PL-GIFT 0.37038 0.0686 0.11748 0.92033 541.24 2022-09-08T18:20:02 2022-09-08T18:17:57 :Executed File Update::Log File Missing: None
PL-GIFT 0.38372 0.06729 0.12096 0.91995 544.48 2022-09-09T00:20:02 2022-09-08T23:46:27 :Executed File Update::Log File Missing: None
Perspecta 0.66321 0.02806 0.23535 0.61333 17442.4 2022-08-09T13:10:01 2022-08-09T13:04:43 :Executed File Update::Log File Missing: :Container Parameters (jsonschema checker):
Perspecta 0.6087 0.06574 0.20903 0.70737 24040.01 2022-08-30T00:40:02 2022-08-30T00:36:36 :Executed File Update::Log File Missing: :Container Parameters (jsonschema checker):
Perspecta 0.57544 0.06812 0.19543 0.7689 24217.35 2022-08-31T03:00:02 2022-08-31T02:55:12 :Executed File Update::Log File Missing: None
Perspecta 0.51267 0.08891 0.16341 0.79784 18676.66 2022-09-02T17:00:02 2022-09-02T16:59:06 :Executed File Update::Log File Missing: None
Perspecta 0.48061 0.07919 0.15509 0.83671 20922.93 2022-09-05T06:40:01 2022-09-05T06:32:24 :Executed File Update::Log File Missing: None
Perspecta 0.46803 0.07267 0.15087 0.84674 21076.29 2022-09-07T23:40:01 2022-09-07T23:39:31 :Executed File Update::Log File Missing: None
Perspecta 0.42916 0.08665 0.14171 0.84076 21315.05 2022-09-08T13:00:01 2022-09-08T12:57:41 :Executed File Update::Log File Missing: None
Perspecta 0.40836 0.07876 0.13245 0.87867 21297.73 2022-09-08T23:40:02 2022-09-08T23:32:13 :Executed File Update::Log File Missing: None
Perspecta 1.71484 0.38792 0.34274 0.72203 21329.54 2022-09-09T19:00:01 2022-09-09T18:53:06 :Executed File Update::Log File Missing: None
Perspecta 0.69315 0.0 0.25 0.5 569.95 2022-09-19T21:40:04 2022-09-19T21:31:36 None None
Perspecta 0.68092 0.0138 0.24509 0.52064 1137.85 2022-09-20T16:00:04 2022-09-20T15:52:03 None None
Perspecta 0.69315 0.0 0.25 0.5 2022-09-20T17:40:03 2022-09-20T17:31:14 :Info File Missing: None
Perspecta 0.38825 0.08085 0.12527 0.8914 55313.35 2022-09-20T20:00:03 2022-09-20T19:53:19 None None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 242.85 2022-08-11T00:50:02 2022-08-11T00:42:03 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-SBU 0.69018 0.02699 0.24895 0.55449 968.17 2022-08-11T16:30:01 2022-08-11T16:24:12 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-SBU 0.6859 0.05438 0.2487 0.84742 984.51 2022-08-19T01:40:02 2022-08-19T01:31:42 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 146.31 2022-09-02T02:30:01 2022-09-02T02:20:55 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 146.3 2022-09-02T16:00:01 2022-09-02T15:58:50 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-SBU 0.49163 0.05391 0.1645 0.8559 999.98 2022-09-02T23:50:02 2022-09-02T23:49:32 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-SBU 0.48748 0.06064 0.16207 0.85774 542.07 2022-09-03T16:40:02 2022-09-03T16:32:39 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 1000.01 2022-09-03T22:50:01 2022-09-03T17:58:57 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 997.4 2022-09-04T13:40:01 2022-09-04T13:32:50 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-SBU 0.44974 0.06643 0.14299 0.87953 1181.78 2022-09-04T19:40:01 2022-09-04T15:36:46 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-SBU 0.46014 0.06258 0.14668 0.87973 696.25 2022-09-05T04:20:01 2022-09-05T04:10:52 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.69315 0.0 0.25 0.5 2022-08-20T12:50:01 2022-08-20T05:20:27 :Executed File Update::Info File Missing::Log File Missing: None
Perspecta-IUB 0.69315 0.0 0.25 0.5 242.65 2022-08-20T18:50:01 2022-08-20T14:09:27 :Executed File Update::Log File Missing: :Container Parameters (jsonschema checker):
Perspecta-IUB 0.38398 0.07422 0.12311 0.91291 943.14 2022-08-21T01:10:01 2022-08-21T01:09:36 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.38489 0.07559 0.12368 0.9076 1155.3 2022-08-21T07:10:02 2022-08-21T04:43:25 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.38368 0.08351 0.12466 0.90538 1157.16 2022-08-21T13:20:01 2022-08-21T10:35:53 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.38151 0.09405 0.12178 0.91358 1107.12 2022-08-21T19:20:02 2022-08-21T16:09:32 :Executed File Update::Log File Missing: None
Perspecta-IUB 1.70346 0.88386 0.16209 0.8749 1110.03 2022-08-22T01:30:01 2022-08-21T21:31:52 :Executed File Update::Log File Missing: None
Perspecta-IUB 1.2325 0.81994 0.1274 0.89352 1108.13 2022-08-22T07:30:02 2022-08-22T03:47:24 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.38299 0.07476 0.12332 0.91387 1104.07 2022-08-22T15:50:01 2022-08-22T15:44:02 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.38345 0.09579 0.12526 0.90847 915.39 2022-08-22T21:50:01 2022-08-22T17:33:07 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.7091 0.52659 0.12237 0.90693 1106.19 2022-08-23T03:50:02 2022-08-23T03:43:11 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.5462 0.37943 0.12211 0.90789 1103.9 2022-08-23T15:40:01 2022-08-23T15:36:57 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.46154 0.14162 0.14309 0.89265 982.75 2022-09-01T21:00:02 2022-09-01T20:53:07 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.34531 0.0922 0.10788 0.92708 983.53 2022-09-02T05:00:01 2022-09-02T04:52:07 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.3471 0.08301 0.10943 0.92593 980.65 2022-09-02T11:00:02 2022-09-02T05:24:14 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.37432 0.11724 0.11518 0.91358 956.17 2022-09-02T17:00:02 2022-09-02T14:32:45 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.37834 0.12218 0.11548 0.91657 1021.11 2022-09-03T00:30:02 2022-09-03T00:23:40 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.35561 0.09127 0.11384 0.92216 1633.86 2022-09-03T06:30:02 2022-09-03T04:12:11 :Executed File Update::Log File Missing: None
Perspecta-IUB 0.35561 0.09127 0.11384 0.92216 4625.08 2022-09-20T19:20:07 2022-09-20T19:16:36 None None
TrinitySRITrojAI-BostonU 0.84611 0.09268 0.31589 0.5 11503.52 2022-08-22T16:30:01 2022-08-22T15:15:13 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-BostonU 0.84545 0.09245 0.31564 0.5 12486.37 2022-08-23T00:10:02 2022-08-22T23:53:29 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-BostonU 1.51843 0.27506 0.42086 0.6034 11137.86 2022-08-23T06:20:02 2022-08-23T04:01:32 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-BostonU 0.44557 0.07967 0.14031 0.88349 11213.55 2022-08-23T14:30:01 2022-08-23T14:21:31 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-BostonU 0.49831 0.10303 0.15791 0.84414 11119.38 2022-08-23T20:30:01 2022-08-23T17:55:58 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-BostonU 0.49946 0.04878 0.16172 0.86535 11116.17 2022-08-24T02:30:01 2022-08-24T00:09:32 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-BostonU 0.44557 0.07967 0.14031 0.88349 11469.17 2022-08-24T15:50:02 2022-08-24T15:42:02 :Executed File Update::Log File Missing: None
TrinitySRITrojAI-BostonU 0.44557 0.07967 0.14031 0.88349 11790.8 2022-08-25T14:00:01 2022-08-25T13:50:19 :Executed File Update::Log File Missing: None
ARM 0.69315 0.0 0.25 0.5 868.0 2022-08-23T07:50:01 2022-08-23T09:18:28 :Executed File Update::Log File Missing: None
ARM 0.81852 0.08419 0.30396 0.46672 868.19 2022-08-23T19:20:01 2022-08-23T19:19:22 :Executed File Update::Log File Missing: None
ARM 0.87806 0.37656 0.26375 0.8342 870.05 2022-08-24T01:20:02 2022-08-23T20:13:57 :Executed File Update::Log File Missing: None
ARM 0.69315 0.0 0.25 0.5 868.06 2022-09-06T03:10:02 2022-09-06T03:02:06 :Executed File Update::Log File Missing: None
ARM 0.90598 0.37697 0.27668 0.83227 867.9 2022-09-06T09:20:01 2022-09-06T06:40:56 :Executed File Update::Log File Missing: None
ARM 0.67795 0.37716 0.17678 0.81395 868.02 2022-09-06T16:10:01 2022-09-06T16:04:10 :Executed File Update::Log File Missing: None
ARM 0.62867 0.37593 0.15467 0.85957 2596.3 2022-10-03T06:20:13 2022-10-03T06:18:21 None None
ARM 0.71323 0.37262 0.18769 0.87249 2597.16 2022-10-03T07:30:13 2022-10-03T07:25:28 None None
ARM-UCSD 0.52751 0.09885 0.18077 0.81375 3439.11 2022-09-02T06:10:01 2022-09-02T06:02:30 :Executed File Update::Log File Missing: None
ARM-UCSD 0.56404 0.10495 0.19593 0.77951 3452.26 2022-09-02T21:10:02 2022-09-02T21:35:08 :Executed File Update::Log File Missing: None
ARM-UCSD 0.55948 0.09147 0.19342 0.76399 3440.33 2022-09-03T06:00:01 2022-09-03T05:55:52 :Executed File Update::Log File Missing: None
ARM-UCSD 0.69054 0.00509 0.24878 0.50694 8959.98 2022-09-19T19:10:07 2022-09-19T17:19:37 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 708.34 2022-09-19T23:30:08 2022-09-19T23:25:22 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 712.03 2022-09-19T23:50:35 2022-09-19T23:45:47 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 650.04 2022-09-20T03:50:08 2022-09-20T03:48:13 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 615.03 2022-09-20T16:30:29 2022-09-20T16:23:30 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T22:20:07 2022-09-20T22:14:55 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T22:30:07 2022-09-20T22:14:55 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T22:40:07 2022-09-20T22:14:55 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T22:50:07 2022-09-20T22:14:55 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T23:00:07 2022-09-20T22:14:55 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T23:10:07 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T23:20:07 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T23:30:06 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T23:40:08 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T23:50:08 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T00:00:08 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T00:10:08 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T00:20:08 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T00:30:09 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T00:40:08 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T00:50:08 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T01:00:08 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T01:10:07 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T01:20:08 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T01:30:08 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T01:40:07 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T01:50:07 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-21T02:00:08 2022-09-20T23:08:46 :Executed File Update::Info File Missing::Log File Missing: None
ARM-UCSD 0.62893 0.09361 0.22448 0.70804 9334.92 2022-09-21T02:10:08 2022-09-20T23:08:46 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 1155.89 2022-10-04T04:20:10 2022-10-04T04:13:40 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 1154.27 2022-10-04T23:10:10 2022-10-04T23:07:45 None None
ARM-UCSD 0.46467 0.05577 0.15539 0.85552 19518.63 2022-10-06T22:50:10 2022-10-06T22:41:58 None None
ARM-UCSD 0.66841 0.0274 0.23736 0.63677 34424.14 2022-11-02T05:40:11 2022-11-02T05:35:09 None :Container Parameters (jsonschema checker):
ARM-UCSD 0.48031 0.0513 0.16388 0.82128 34260.63 2022-11-08T00:40:10 2022-11-08T00:37:39 None :Container Parameters (jsonschema checker):
UMBCb 0.69315 0.0 0.25 0.5 865.64 2022-10-16T21:00:23 2022-10-16T20:57:55 None None
UMBCb 0.46014 0.06258 0.14668 0.87973 1947.54 2022-10-17T18:40:21 2022-10-17T18:34:26 None None

Required filename format: "object-detection-aug2022_sts_<Submission Name>.simg"
Accepting submissions: True
Number of models in object-detection-aug2022, sts: 10
Execution timeout (hh:mm:ss): 2:30:00

Teams/Jobs

Team Submission Timestamp Type Job Status File Status General Status File Timestamp Time until next execution
Perspecta 2022-09-20T16:20:03 performer None None Ok 2022-09-20T16:15:49 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI 2022-09-22T23:30:05 performer None None Ok 2022-09-22T23:24:44 0 d, 0 h, 0 m, 0 s
DRAFFT performer None None Ok None 0 d, 0 h, 0 m, 0 s
PL-GIFT performer None None Ok None 0 d, 0 h, 0 m, 0 s
Perspecta-PurdueRutgers performer None None Ok None 0 d, 0 h, 0 m, 0 s
Perspecta-IUB performer None None Ok None 0 d, 0 h, 0 m, 0 s
ARM-UCSD 2022-11-08T00:40:11 performer None Ok Ok 2022-11-08T00:37:31 0 d, 0 h, 0 m, 0 s
ARM performer None None Ok None 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-SBU performer None None Ok None 0 d, 0 h, 0 m, 0 s
ARM-UMBC performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-1 performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-2 performer None None Ok None 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-BostonU performer None None Ok None 0 d, 0 h, 0 m, 0 s
trojai-example 2022-10-05T15:10:17 public None Ok Ok 2022-10-05T14:59:32 0 d, 0 h, 0 m, 0 s
IceTorch public None None Ok None 0 d, 0 h, 0 m, 0 s
trojaicy public None None Ok None 0 d, 0 h, 0 m, 0 s
Hector public None None Ok None 0 d, 0 h, 0 m, 0 s
Cassandra-XF public None None Ok None 0 d, 0 h, 0 m, 0 s
gooddefense public None None Ok None 0 d, 0 h, 0 m, 0 s
cyberinfrastructure public None None Ok None 0 d, 0 h, 0 m, 0 s
glaros public None None Ok None 0 d, 0 h, 0 m, 0 s
Ultraman public None None Ok None 0 d, 0 h, 0 m, 0 s
BackSec public None None Ok None 0 d, 0 h, 0 m, 0 s
Palladium public None None Ok None 0 d, 0 h, 0 m, 0 s
IIECAS public None None Ok None 0 d, 0 h, 0 m, 0 s
NSEC public None None Ok None 0 d, 0 h, 0 m, 0 s
UMBCb 2022-10-17T18:20:22 performer None None Ok 2022-10-17T18:16:50 0 d, 0 h, 0 m, 0 s

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 0.51567 0.25566 0.17866 0.83333 180.51 2022-10-05T15:10:17 2022-10-05T14:59:32 None None
TrinitySRITrojAI 0.29559 0.27212 0.0956 0.95833 1180.67 2022-09-22T23:30:05 2022-09-22T23:24:44 None None
Perspecta 0.59455 0.133 0.21041 0.6875 1144.68 2022-09-20T16:20:03 2022-09-20T16:15:49 None None
ARM-UCSD 0.42054 0.15442 0.1275 1.0 1502.36 2022-10-06T22:50:11 2022-10-06T22:41:50 None None
UMBCb 0.43737 0.21442 0.14229 0.83333 135.39 2022-10-17T18:20:22 2022-10-17T18:16:50 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 0.69315 0.0 0.25 0.5 0.55 2022-09-16T16:49:12 2022-09-16T16:28:58 :Container File Missing: :Copy Out:
trojai-example 0.69315 0.0 0.25 0.5 50.15 2022-09-16T18:15:25 2022-09-16T16:28:58 None None
trojai-example 0.69315 0.0 0.25 0.5 50.16 2022-09-16T18:41:59 2022-09-16T16:28:58 None None
trojai-example 0.69315 0.0 0.25 0.5 50.15 2022-09-16T18:55:49 2022-09-16T16:28:58 None None
trojai-example 0.69315 0.0 0.25 0.5 2022-09-16T19:31:19 2022-09-16T16:28:58 :Executed File Update::Info File Missing::Container File Missing: None
trojai-example 0.69315 0.0 0.25 0.5 131.51 2022-09-16T19:33:25 2022-09-16T16:28:58 None None
trojai-example 1.24928 0.79595 0.38884 0.52083 179.93 2022-09-16T19:39:51 2022-09-16T16:28:58 None None
trojai-example 0.85088 0.56821 0.27957 0.625 180.06 2022-09-16T19:53:23 2022-09-16T16:28:58 None None
trojai-example 0.9125 0.55567 0.30594 0.58333 179.98 2022-09-16T20:13:46 2022-09-16T16:28:58 None None
trojai-example 0.87033 0.39561 0.32378 0.5 179.9 2022-09-20T18:30:10 2022-09-20T18:27:48 None None
trojai-example 0.63804 0.42147 0.21279 0.70833 181.61 2022-10-05T14:20:19 2022-10-05T14:14:26 :Container File Missing: None
trojai-example 1.59808 0.6812 0.52814 0.08333 180.31 2022-10-05T14:50:16 2022-10-05T14:41:53 :Container File Missing: None
trojai-example 0.51567 0.25566 0.17866 0.83333 180.51 2022-10-05T15:10:17 2022-10-05T14:59:32 None None
TrinitySRITrojAI 0.29559 0.27212 0.0956 0.95833 1180.67 2022-09-22T23:30:05 2022-09-22T23:24:44 None None
Perspecta 0.86773 0.77126 0.24779 0.625 1800.55 2022-09-19T19:10:03 2022-09-19T13:43:43 None :Timeout:
Perspecta 0.59455 0.133 0.21041 0.6875 1144.68 2022-09-20T16:20:03 2022-09-20T16:15:49 None None
ARM-UCSD 0.51516 0.28617 0.16466 0.79167 650.68 2022-09-19T23:40:06 2022-09-19T23:30:09 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-09-20T00:00:53 2022-09-19T23:45:42 :Info File Missing::Container File Missing: None
ARM-UCSD 0.69315 0.0 0.25 0.5 392.48 2022-09-20T16:30:30 2022-09-20T16:27:42 None None
ARM-UCSD 0.46552 0.16004 0.14563 0.97917 649.66 2022-09-20T17:50:06 2022-09-20T17:43:39 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 79.93 2022-10-05T20:20:10 2022-10-05T20:16:53 None None
ARM-UCSD 0.42054 0.15442 0.1275 1.0 1502.36 2022-10-06T22:50:11 2022-10-06T22:41:50 None None
ARM-UCSD 0.6713 0.09389 0.23895 0.45833 2690.42 2022-11-02T05:40:12 2022-11-02T05:35:19 None :Container Parameters (jsonschema checker):
ARM-UCSD 0.46868 0.15856 0.14981 0.9375 2724.25 2022-11-08T00:40:11 2022-11-08T00:37:31 None :Container Parameters (jsonschema checker):
UMBCb 0.69315 0.0 0.25 0.5 60.27 2022-10-17T14:50:22 2022-10-17T14:46:23 None None
UMBCb 0.69315 0.0 0.25 0.5 60.01 2022-10-17T16:20:21 2022-10-17T16:16:54 None None
UMBCb 0.69315 0.0 0.25 0.5 60.11 2022-10-17T17:10:22 2022-10-17T17:02:12 None None
UMBCb 0.43737 0.21442 0.14229 0.83333 135.39 2022-10-17T18:20:22 2022-10-17T18:16:50 None None

Required filename format: "object-detection-aug2022_holdout_<Submission Name>.simg"
Accepting submissions: False
Number of models in object-detection-aug2022, holdout: 144
Execution timeout (hh:mm:ss): 1 day, 12:00:00

Teams/Jobs

Team Submission Timestamp Type Job Status File Status General Status File Timestamp Time until next execution
Perspecta performer None None Ok 2022-09-20T19:53:19 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI performer None None Ok 2022-09-23T05:32:01 0 d, 0 h, 0 m, 0 s
DRAFFT performer None None Ok None 0 d, 0 h, 0 m, 0 s
PL-GIFT performer None None Ok None 0 d, 0 h, 0 m, 0 s
Perspecta-PurdueRutgers performer None None Ok None 0 d, 0 h, 0 m, 0 s
Perspecta-IUB performer None None Ok 2022-09-20T19:16:36 0 d, 0 h, 0 m, 0 s
ARM-UCSD performer None None Ok None 0 d, 0 h, 0 m, 0 s
ARM performer None None Ok None 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-SBU performer None None Ok None 0 d, 0 h, 0 m, 0 s
ARM-UMBC performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-1 performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-2 performer None None Ok None 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-BostonU performer None None Ok None 0 d, 0 h, 0 m, 0 s
trojai-example public None None Ok None 0 d, 0 h, 0 m, 0 s
IceTorch public None None Ok None 0 d, 0 h, 0 m, 0 s
trojaicy public None None Ok None 0 d, 0 h, 0 m, 0 s
Hector public None None Ok None 0 d, 0 h, 0 m, 0 s
Cassandra-XF public None None Ok None 0 d, 0 h, 0 m, 0 s
gooddefense public None None Ok None 0 d, 0 h, 0 m, 0 s
cyberinfrastructure public None None Ok None 0 d, 0 h, 0 m, 0 s
glaros public None None Ok None 0 d, 0 h, 0 m, 0 s
Ultraman public None None Ok None 0 d, 0 h, 0 m, 0 s
BackSec public None None Ok None 0 d, 0 h, 0 m, 0 s
Palladium public None None Ok None 0 d, 0 h, 0 m, 0 s
IIECAS public None None Ok None 0 d, 0 h, 0 m, 0 s
NSEC public None None Ok None 0 d, 0 h, 0 m, 0 s
UMBCb performer None None Ok None 0 d, 0 h, 0 m, 0 s

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
TrinitySRITrojAI 0.34193 0.09211 0.11231 0.92785 9247.52 2022-08-27T06:00:01 2022-08-27T05:58:46 :Executed File Update: :Cleanup:
ICSI-2 0.33164 0.09464 0.10578 0.9457 1665.71 2022-08-23T05:50:01 2022-08-23T00:03:15 :Executed File Update: None
Perspecta-PurdueRutgers 0.29129 0.13131 0.07822 0.92033 51374.46 2022-08-21T20:10:01 2022-08-21T20:01:40 :Executed File Update: None
PL-GIFT 0.35137 0.06594 0.11234 0.92921 1521.68 2022-09-09T00:20:02 2022-09-08T23:46:27 :Executed File Update: None
Perspecta 0.3146 0.07024 0.09687 0.93403 53539.44 2022-09-20T20:00:03 2022-09-20T19:53:19 None :Cleanup:
TrinitySRITrojAI-SBU 0.38251 0.05547 0.11542 0.93287 3312.27 2022-09-04T19:40:01 2022-09-04T15:36:46 :Executed File Update: None
Perspecta-IUB 0.33807 0.07182 0.11561 0.92024 2800.66 2022-09-02T11:00:02 2022-09-02T05:24:14 :Executed File Update: None
TrinitySRITrojAI-BostonU 0.46373 0.07871 0.15107 0.86121 4846.82 2022-08-25T14:00:01 2022-08-25T13:50:19 :Executed File Update: None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
TrinitySRITrojAI 0.46254 0.07874 0.15057 0.86265 5302.79 2022-07-28T03:50:01 2022-07-28T03:46:36 :Executed File Update: :Cleanup:
TrinitySRITrojAI 0.55614 0.16702 0.16426 0.84606 4994.49 2022-08-01T20:00:01 2022-08-01T19:51:21 :Executed File Update: :Cleanup:
TrinitySRITrojAI 0.39268 0.08522 0.12732 0.90181 3501.73 2022-08-08T23:10:01 2022-08-08T23:08:01 :Executed File Update: :Cleanup:
TrinitySRITrojAI 0.38327 0.06971 0.11866 0.9158 9072.04 2022-08-11T06:50:02 2022-08-11T06:41:58 :Executed File Update: :Cleanup:
TrinitySRITrojAI 0.34193 0.09211 0.11231 0.92785 9247.52 2022-08-27T06:00:01 2022-08-27T05:58:46 :Executed File Update: :Cleanup:
TrinitySRITrojAI 0.37083 0.10742 0.11905 0.9239 9210.02 2022-09-07T04:50:02 2022-09-07T04:34:50 :Executed File Update: :Cleanup:
TrinitySRITrojAI 0.38787 0.10167 0.12221 0.89988 15769.88 2022-09-09T20:00:01 2022-09-09T19:54:15 :Executed File Update: :Cleanup:
TrinitySRITrojAI 0.38999 0.10317 0.12272 0.90866 15906.08 2022-09-23T05:40:05 2022-09-23T05:32:01 None :Cleanup:
ICSI-2 0.48649 0.06557 0.1607 0.894 1524.68 2022-07-28T21:00:02 2022-07-28T20:56:51 :Executed File Update: :Container Parameters (jsonschema checker):
ICSI-2 0.40334 0.08355 0.12805 0.9076 1624.97 2022-07-29T23:20:01 2022-07-29T23:12:21 :Executed File Update: None
ICSI-2 0.38377 0.07245 0.1197 0.92747 1590.65 2022-08-03T03:20:01 2022-08-03T03:17:09 :Executed File Update: None
ICSI-2 0.37457 0.07378 0.11572 0.92853 1637.77 2022-08-05T00:40:01 2022-08-05T00:39:34 :Executed File Update: None
ICSI-2 0.35959 0.07624 0.11021 0.93895 1733.94 2022-08-09T01:00:01 2022-08-09T00:52:50 :Executed File Update: None
ICSI-2 0.34165 0.09282 0.10993 0.94358 1646.77 2022-08-09T07:00:02 2022-08-09T05:59:27 :Executed File Update: None
ICSI-2 0.52083 0.21233 0.13648 0.92978 1772.23 2022-08-09T23:50:01 2022-08-09T23:47:56 :Executed File Update: None
ICSI-2 0.34975 0.10279 0.11354 0.93981 1681.95 2022-08-10T22:50:02 2022-08-10T22:12:01 :Executed File Update: None
ICSI-2 0.34379 0.09267 0.11025 0.9428 1637.77 2022-08-11T05:30:02 2022-08-11T05:29:28 :Executed File Update: None
ICSI-2 0.35473 0.10238 0.11388 0.93818 1765.15 2022-08-11T21:30:01 2022-08-11T21:22:33 :Executed File Update: None
ICSI-2 0.34055 0.08735 0.1068 0.93972 2086.22 2022-08-16T14:50:01 2022-08-16T14:49:40 :Executed File Update: None
ICSI-2 0.34224 0.09618 0.11019 0.94252 1689.84 2022-08-22T23:50:01 2022-08-22T23:41:20 :Executed File Update: None
ICSI-2 0.33164 0.09464 0.10578 0.9457 1665.71 2022-08-23T05:50:01 2022-08-23T00:03:15 :Executed File Update: None
ICSI-2 0.34175 0.08927 0.10837 0.94232 1775.17 2022-08-23T11:50:01 2022-08-23T07:14:37 :Executed File Update: None
ICSI-2 0.33198 0.09391 0.10664 0.94599 1934.52 2022-09-02T10:10:01 2022-09-02T10:03:39 :Executed File Update: None
Perspecta-PurdueRutgers 0.35093 0.11059 0.09703 0.87789 52594.69 2022-07-29T01:00:02 2022-07-29T00:52:14 :Executed File Update: None
Perspecta-PurdueRutgers 0.34166 0.11264 0.09257 0.87539 52752.66 2022-08-19T23:20:01 2022-08-19T23:13:07 :Executed File Update: None
Perspecta-PurdueRutgers 0.32377 0.11454 0.08314 0.87558 52821.91 2022-08-20T21:40:02 2022-08-20T21:32:11 :Executed File Update: None
Perspecta-PurdueRutgers 0.29129 0.13131 0.07822 0.92033 51374.46 2022-08-21T20:10:01 2022-08-21T20:01:40 :Executed File Update: None
PL-GIFT 0.37879 0.10794 0.1179 0.91204 2495.98 2022-08-09T03:50:01 2022-08-09T03:42:13 :Executed File Update: :Container Parameters (jsonschema checker)::Cleanup:
PL-GIFT 0.44755 0.13221 0.13604 0.89159 2489.37 2022-08-20T16:30:02 2022-08-20T16:24:23 :Executed File Update: :Container Parameters (jsonschema checker)::Cleanup:
PL-GIFT 0.46114 0.13969 0.14312 0.90191 2440.2 2022-09-03T16:20:02 2022-09-03T16:16:06 :Executed File Update: :Container Parameters (jsonschema checker)::Cleanup:
PL-GIFT 0.49975 0.12266 0.16015 0.88185 2352.8 2022-09-03T22:20:02 2022-09-03T18:33:34 :Executed File Update: :Container Parameters (jsonschema checker):
PL-GIFT 0.38056 0.07568 0.12486 0.91435 1423.4 2022-09-07T20:40:02 2022-09-07T20:32:17 :Executed File Update: None
PL-GIFT 0.35618 0.06955 0.11671 0.91744 1506.59 2022-09-08T18:20:02 2022-09-08T18:17:57 :Executed File Update: None
PL-GIFT 0.35137 0.06594 0.11234 0.92921 1521.68 2022-09-09T00:20:02 2022-09-08T23:46:27 :Executed File Update: None
Perspecta 0.41019 0.07279 0.1261 0.87307 51947.72 2022-09-05T06:40:01 2022-09-05T06:32:24 :Executed File Update: :Cleanup:
Perspecta 0.39859 0.06019 0.12254 0.91001 53091.01 2022-09-07T23:40:01 2022-09-07T23:39:31 :Executed File Update: :Cleanup:
Perspecta 0.37932 0.07821 0.12492 0.88069 53263.27 2022-09-08T13:00:01 2022-09-08T12:57:41 :Executed File Update: :Cleanup:
Perspecta 0.31846 0.0604 0.0964 0.93682 54015.82 2022-09-08T23:40:02 2022-09-08T23:32:13 :Executed File Update: :Cleanup:
Perspecta 0.3146 0.07024 0.09687 0.93403 53539.44 2022-09-20T20:00:03 2022-09-20T19:53:19 None :Cleanup:
TrinitySRITrojAI-SBU 0.47247 0.05356 0.15479 0.9103 2877.61 2022-09-02T23:50:02 2022-09-02T23:49:32 :Executed File Update: None
TrinitySRITrojAI-SBU 0.48047 0.06418 0.15882 0.89323 1566.92 2022-09-03T16:40:02 2022-09-03T16:32:39 :Executed File Update: None
TrinitySRITrojAI-SBU 0.38251 0.05547 0.11542 0.93287 3312.27 2022-09-04T19:40:01 2022-09-04T15:36:46 :Executed File Update: None
TrinitySRITrojAI-SBU 0.41837 0.06148 0.12781 0.90856 1913.21 2022-09-05T04:20:01 2022-09-05T04:10:52 :Executed File Update: None
Perspecta-IUB 0.39707 0.07793 0.13217 0.89728 2724.53 2022-08-21T01:10:01 2022-08-21T01:09:36 :Executed File Update: None
Perspecta-IUB 0.39766 0.07717 0.13371 0.89448 3329.47 2022-08-21T07:10:02 2022-08-21T04:43:25 :Executed File Update: None
Perspecta-IUB 0.39908 0.0848 0.1358 0.89448 3300.45 2022-08-21T13:20:01 2022-08-21T10:35:53 :Executed File Update: None
Perspecta-IUB 0.39354 0.0924 0.13299 0.90008 3196.38 2022-08-21T19:20:02 2022-08-21T16:09:32 :Executed File Update: None
Perspecta-IUB 0.39662 0.07565 0.13317 0.89497 3135.08 2022-08-22T15:50:01 2022-08-22T15:44:02 :Executed File Update: None
Perspecta-IUB 0.40333 0.10126 0.13752 0.89757 2585.21 2022-08-22T21:50:01 2022-08-22T17:33:07 :Executed File Update: None
Perspecta-IUB 0.47426 0.14006 0.15223 0.89159 2808.51 2022-09-01T21:00:02 2022-09-01T20:53:07 :Executed File Update: None
Perspecta-IUB 0.34869 0.07776 0.12052 0.9159 2847.77 2022-09-02T05:00:01 2022-09-02T04:52:07 :Executed File Update: None
Perspecta-IUB 0.33807 0.07182 0.11561 0.92024 2800.66 2022-09-02T11:00:02 2022-09-02T05:24:14 :Executed File Update: None
Perspecta-IUB 0.38533 0.10376 0.13302 0.90992 2797.7 2022-09-02T17:00:02 2022-09-02T14:32:45 :Executed File Update: None
Perspecta-IUB 0.38954 0.10647 0.13406 0.90924 2883.67 2022-09-03T00:30:02 2022-09-03T00:23:40 :Executed File Update: None
Perspecta-IUB 0.3635 0.0833 0.12673 0.9102 4471.47 2022-09-03T06:30:02 2022-09-03T04:12:11 :Executed File Update: None
Perspecta-IUB 0.36022 0.08343 0.12573 0.91252 4553.35 2022-09-20T19:20:07 2022-09-20T19:16:36 None None
TrinitySRITrojAI-BostonU 0.46472 0.07862 0.15138 0.86034 4904.72 2022-08-23T14:30:01 2022-08-23T14:21:31 :Executed File Update: None
TrinitySRITrojAI-BostonU 0.49977 0.09807 0.16339 0.83922 4879.33 2022-08-23T20:30:01 2022-08-23T17:55:58 :Executed File Update: None
TrinitySRITrojAI-BostonU 0.50821 0.05585 0.16422 0.85774 4924.92 2022-08-24T02:30:01 2022-08-24T00:09:32 :Executed File Update: None
TrinitySRITrojAI-BostonU 0.46531 0.07855 0.15148 0.86053 4919.53 2022-08-24T15:50:02 2022-08-24T15:42:02 :Executed File Update: None
TrinitySRITrojAI-BostonU 0.46373 0.07871 0.15107 0.86121 4846.82 2022-08-25T14:00:01 2022-08-25T13:50:19 :Executed File Update: None

Round 11 leaderboard for image classification September 2022.

Each AI is trained to perform image classification. For those AIs that have been attacked, the presence of the trigger pattern will cause the AI to reliably produce the wrong prediction. The Round 11 Training Data Download consists of 288 reference AIs (exactly 50% are poisoned) with example input data. More info here.

Polygon Trigger on Street Sign

Example poisoned image, where the purple polygon trigger on the street sign causes misclassification.

Required filename format: "image-classification-sep2022_train_<Submission Name>.simg"
Accepting submissions: True
Number of models in image-classification-sep2022, train: 288
Execution timeout (hh:mm:ss): 2 days, 0:00:00

Teams/Jobs

Team Submission Timestamp Type Job Status File Status General Status File Timestamp Time until next execution
Perspecta 2022-12-04T00:00:05 performer None None Ok 2022-12-02T03:25:52 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI 2022-11-18T00:00:55 performer None None Ok 2022-11-17T23:59:38 0 d, 0 h, 0 m, 0 s
DRAFFT performer None None Ok None 0 d, 0 h, 0 m, 0 s
PL-GIFT 2022-11-15T23:50:08 performer None None Ok 2022-11-15T23:45:59 0 d, 0 h, 0 m, 0 s
Perspecta-PurdueRutgers 2022-11-19T04:20:08 performer None None Ok 2022-11-19T04:10:44 0 d, 0 h, 0 m, 0 s
Perspecta-IUB 2022-10-15T04:00:30 performer None None Ok 2022-10-14T21:09:47 0 d, 0 h, 0 m, 0 s
ARM-UCSD 2022-11-01T16:10:12 performer None None Ok 2022-11-01T16:04:02 0 d, 0 h, 0 m, 0 s
ARM 2022-11-28T05:10:15 performer None None Ok 2022-11-28T05:03:33 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-SBU 2022-11-29T18:50:17 performer None None Ok 2022-11-29T18:45:10 0 d, 0 h, 0 m, 0 s
ARM-UMBC performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-1 performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-2 2022-11-23T23:10:19 performer None None Ok 2022-11-23T23:07:52 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-BostonU 2022-11-15T19:51:44 performer None Ok Ok 2022-11-15T17:57:00 0 d, 0 h, 0 m, 0 s
trojai-example 2022-09-21T17:30:13 public None None Ok 2022-09-21T17:28:50 0 d, 0 h, 0 m, 0 s
IceTorch public None None Ok None 0 d, 0 h, 0 m, 0 s
trojaicy public None None Ok None 0 d, 0 h, 0 m, 0 s
Hector public None None Ok None 0 d, 0 h, 0 m, 0 s
Cassandra-XF public None None Ok None 0 d, 0 h, 0 m, 0 s
gooddefense public None None Ok None 0 d, 0 h, 0 m, 0 s
cyberinfrastructure public None None Ok None 0 d, 0 h, 0 m, 0 s
glaros public None None Ok None 0 d, 0 h, 0 m, 0 s
Ultraman public None None Ok None 0 d, 0 h, 0 m, 0 s
BackSec public None None Ok None 0 d, 0 h, 0 m, 0 s
Palladium public None None Ok None 0 d, 0 h, 0 m, 0 s
IIECAS 2022-12-01T15:01:55 public None None Ok 2022-12-01T13:44:48 0 d, 0 h, 0 m, 0 s
NSEC public None None Ok None 0 d, 0 h, 0 m, 0 s
UMBCb 2022-11-04T14:43:29 performer None None Ok 2022-11-03T20:20:40 0 d, 0 h, 0 m, 0 s

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
Perspecta 0.05764 0.01556 0.01155 1.0 2874.81 2022-11-16T19:00:04 2022-11-16T18:58:52 None None
TrinitySRITrojAI 0.32043 0.02606 0.08708 0.9809 9235.16 2022-09-30T20:50:05 2022-09-30T20:43:42 None None
PL-GIFT 0.02219 0.00408 0.00143 1.0 2494.6 2022-11-11T19:20:07 2022-11-11T19:19:08 None None
Perspecta-PurdueRutgers 0.25688 0.07209 0.06897 0.9396 76308.23 2022-10-18T06:50:22 2022-10-18T06:41:35 :Container File Missing: :Container Parameters (learned parameters):
Perspecta-IUB 0.01537 0.0062 0.00234 1.0 42205.08 2022-10-08T02:30:09 2022-10-08T02:23:53 None None
ARM-UCSD 0.45631 0.09297 0.13778 0.84028 5883.57 2022-10-07T18:20:09 2022-10-07T18:15:51 None None
ARM 0.52828 0.04556 0.17645 0.8255 4134.8 2022-11-01T21:10:13 2022-11-01T21:08:20 None None
TrinitySRITrojAI-SBU 0.2078 0.03321 0.06981 0.99841 14851.38 2022-11-29T18:50:17 2022-11-29T18:45:10 None None
ICSI-2 0.01882 0.00204 0.00064 1.0 3748.12 2022-11-14T23:30:19 2022-11-14T23:23:54 None None
TrinitySRITrojAI-BostonU 0.56018 0.05892 0.18865 0.81862 9466.89 2022-10-31T22:10:15 2022-10-31T22:07:23 :Log File Missing: None
trojai-example 0.95798 0.09962 0.32408 0.47912 3412.55 2022-09-21T17:30:13 2022-09-21T17:28:50 None None
IIECAS 0.69315 0.0 0.25 0.5 14666.41 2022-11-29T02:30:30 2022-11-29T02:30:04 :No Results: None
UMBCb 0.49955 0.02726 0.16128 0.89157 3656.39 2022-10-31T19:10:23 2022-10-31T19:00:37 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
Perspecta 0.6652 0.04391 0.23816 0.64374 2984.18 2022-10-13T03:30:03 2022-10-13T03:20:12 None None
Perspecta 0.80193 0.1474 0.237 0.70083 2855.94 2022-10-18T05:20:03 2022-10-18T05:11:57 :Container File Missing: None
Perspecta 1.33425 0.17995 0.38552 0.50388 2720.75 2022-10-18T17:30:03 2022-10-18T17:23:07 :Container File Missing: None
Perspecta 0.65184 0.04655 0.23064 0.65635 2715.51 2022-10-19T00:00:04 2022-10-18T23:56:57 :Container File Missing: None
Perspecta 0.62086 0.04185 0.21855 0.69739 2720.72 2022-10-19T18:10:04 2022-10-19T18:02:05 :Container File Missing: None
Perspecta 0.62212 0.04065 0.21871 0.69307 2697.39 2022-10-19T21:00:04 2022-10-19T20:53:59 :Container File Missing: None
Perspecta 0.59835 0.04744 0.20777 0.74074 2698.45 2022-10-19T23:40:03 2022-10-19T23:33:06 :Container File Missing: None
Perspecta 0.21427 0.05219 0.06112 0.97234 2831.66 2022-10-26T14:40:04 2022-10-26T14:32:03 :Container File Missing: None
Perspecta 0.12382 0.03737 0.02857 0.9919 2773.57 2022-10-27T00:50:03 2022-10-27T00:44:13 :Container File Missing: None
Perspecta 0.26149 0.12254 0.04851 0.96945 2764.53 2022-10-27T04:20:03 2022-10-27T04:15:11 :Container File Missing: None
Perspecta 0.18326 0.04828 0.05197 0.9816 2785.87 2022-10-27T15:40:04 2022-10-27T15:38:34 :Container File Missing: None
Perspecta 0.09854 0.03075 0.02451 0.99785 2762.58 2022-10-28T23:30:03 2022-10-28T23:22:21 :Container File Missing: None
Perspecta 0.11793 0.03488 0.03162 0.99696 2807.55 2022-11-02T18:04:43 2022-11-02T17:56:09 None None
Perspecta 0.11007 0.03466 0.02755 0.99614 2864.94 2022-11-07T21:10:05 2022-11-07T21:04:58 None None
Perspecta 0.09311 0.03294 0.02288 0.99725 2842.02 2022-11-15T04:00:04 2022-11-15T03:50:54 None None
Perspecta 0.07973 0.0299 0.02103 0.99916 2874.05 2022-11-16T01:20:04 2022-11-16T01:16:33 None None
Perspecta 0.05764 0.01556 0.01155 1.0 2874.81 2022-11-16T19:00:04 2022-11-16T18:58:52 None None
Perspecta 0.09848 0.03445 0.02498 0.99749 2866.06 2022-11-16T21:10:04 2022-11-16T21:09:20 None None
Perspecta 0.10029 0.01762 0.0194 1.0 2884.42 2022-11-17T22:30:04 2022-11-17T22:23:07 None None
Perspecta 0.09066 0.03062 0.02072 0.99904 2881.75 2022-11-22T05:20:04 2022-11-22T05:17:14 None None
Perspecta 0.10967 0.03244 0.02618 0.99508 2700.33 2022-11-22T07:30:04 2022-11-22T07:22:56 None None
Perspecta 0.06669 0.01045 0.00978 1.0 14760.75 2022-11-29T07:40:04 2022-11-29T07:38:19 None None
Perspecta 0.08064 0.02369 0.01815 0.99959 14762.07 2022-11-29T15:50:05 2022-11-29T15:43:06 None None
Perspecta 0.06829 0.02055 0.01398 1.0 14762.68 2022-11-30T20:00:04 2022-11-30T19:50:09 None None
Perspecta 0.08309 0.00683 0.00886 1.0 14907.68 2022-12-01T05:20:04 2022-12-01T05:16:29 None None
Perspecta 0.08545 0.00708 0.00926 1.0 15082.17 2022-12-01T16:50:09 2022-12-01T16:48:02 None None
Perspecta 0.08545 0.00708 0.00926 1.0 3681.39 2022-12-04T00:00:05 2022-12-02T03:25:52 None None
TrinitySRITrojAI 0.32043 0.02606 0.08708 0.9809 9235.16 2022-09-30T20:50:05 2022-09-30T20:43:42 None None
TrinitySRITrojAI 0.54917 0.02553 0.18469 0.85156 9342.89 2022-10-07T08:40:06 2022-10-07T08:37:36 None None
TrinitySRITrojAI 0.49442 0.05732 0.16598 0.81554 25912.92 2022-10-08T05:20:06 2022-10-08T05:19:38 None None
TrinitySRITrojAI 0.47598 0.05011 0.16215 0.81542 58631.5 2022-10-17T08:50:06 2022-10-17T08:43:17 :Container File Missing: None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 2511.06 2022-11-17T18:50:05 2022-11-17T18:40:23 :No Results: None
TrinitySRITrojAI 0.58453 0.05725 0.19929 0.76078 64127.18 2022-11-18T00:00:55 2022-11-17T23:59:38 None None
PL-GIFT 0.05384 0.00709 0.00561 1.0 1936.0 2022-09-23T22:10:06 2022-09-23T22:01:36 None None
PL-GIFT 0.69315 0.0 0.25 0.5 2022-10-12T15:10:08 2022-10-12T15:01:57 :Info File Missing::Container File Missing: None
PL-GIFT 0.06802 0.00735 0.00743 1.0 15352.97 2022-10-13T12:00:07 2022-10-13T11:55:39 None None
PL-GIFT 0.05165 0.01306 0.00939 1.0 1922.36 2022-10-14T18:20:08 2022-10-14T18:18:21 None None
PL-GIFT 0.05165 0.01306 0.00939 1.0 1914.95 2022-10-15T05:10:18 2022-10-15T01:10:08 None None
PL-GIFT 0.04584 0.01273 0.00858 1.0 2499.8 2022-10-25T23:40:07 2022-10-25T23:31:47 :Container File Missing: None
PL-GIFT 0.10749 0.02444 0.02664 0.99993 2016.56 2022-11-04T23:30:07 2022-11-04T23:20:37 None None
PL-GIFT 0.03642 0.01337 0.00779 0.99983 2499.44 2022-11-07T17:10:08 2022-11-07T17:10:02 None None
PL-GIFT 0.1386 0.0348 0.03823 0.9891 2483.46 2022-11-09T23:10:08 2022-11-09T23:03:07 None None
PL-GIFT 0.02219 0.00408 0.00143 1.0 2494.6 2022-11-11T19:20:07 2022-11-11T19:19:08 None None
PL-GIFT 0.02552 0.0107 0.00512 1.0 2494.44 2022-11-11T23:10:07 2022-11-11T23:09:05 None None
PL-GIFT 0.69315 0.0 0.25 0.5 2022-11-15T17:40:09 2022-11-15T17:33:52 :No Results::Info File Missing::Container File Missing: None
PL-GIFT 0.69315 0.0 0.25 0.5 2022-11-15T20:20:45 2022-11-15T20:17:08 :No Results::Info File Missing::Container File Missing: None
PL-GIFT 0.06478 0.02366 0.01772 0.99995 2465.8 2022-11-15T23:50:08 2022-11-15T23:45:59 None None
Perspecta-PurdueRutgers 0.3668 0.08189 0.10528 0.88129 100934.45 2022-10-03T08:20:08 2022-10-03T08:18:13 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.39643 0.08497 0.11582 0.871 101643.48 2022-10-04T22:40:07 2022-10-04T22:39:49 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.39643 0.08497 0.11582 0.871 101635.92 2022-10-06T23:30:22 2022-10-05T19:54:06 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.43815 0.08725 0.13387 0.87461 66302.31 2022-10-14T10:00:08 2022-10-14T09:52:56 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.46545 0.09003 0.14429 0.86574 126878.95 2022-10-15T15:20:09 2022-10-15T15:12:56 :Container File Missing: :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.25688 0.07209 0.06897 0.9396 76308.23 2022-10-18T06:50:22 2022-10-18T06:41:35 :Container File Missing: :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.29374 0.06724 0.07936 0.92559 78379.77 2022-10-24T01:40:09 2022-10-24T01:34:56 :Container File Missing: :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.43148 0.02807 0.13377 0.92836 2981.71 2022-11-04T15:20:09 2022-11-04T15:17:23 None None
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.91194 2022-11-04T20:40:10 2022-11-04T20:30:10 :No Results::Info File Missing::Container File Missing: None
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.9362 2022-11-06T21:00:08 2022-11-06T20:52:56 :No Results::Info File Missing::Container File Missing: None
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 2022-11-08T19:40:09 2022-11-08T19:33:41 :No Results::Info File Missing::Container File Missing: None
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 2022-11-11T17:20:08 2022-11-11T17:17:24 :No Results::Info File Missing::Container File Missing: None
Perspecta-PurdueRutgers 0.43884 0.06778 0.14517 0.87389 2567.7 2022-11-15T23:30:10 2022-11-15T23:21:05 None None
Perspecta-PurdueRutgers 0.93707 0.45288 0.14834 0.85335 2553.35 2022-11-16T05:00:08 2022-11-16T04:53:47 None None
Perspecta-PurdueRutgers 0.45463 0.06882 0.14889 0.86601 2549.1 2022-11-16T16:40:09 2022-11-16T16:36:52 None None
Perspecta-PurdueRutgers 0.43417 0.07036 0.14703 0.86601 2561.35 2022-11-17T02:00:07 2022-11-17T01:55:49 None None
Perspecta-PurdueRutgers 0.43178 0.07048 0.14644 0.86601 2502.94 2022-11-17T03:10:07 2022-11-17T03:09:26 None None
Perspecta-PurdueRutgers 0.41234 0.06165 0.14182 0.86789 2556.15 2022-11-17T04:41:19 2022-11-17T04:34:55 None None
Perspecta-PurdueRutgers 0.43116 0.04782 0.14813 0.85214 2508.91 2022-11-19T00:30:08 2022-11-18T21:26:33 None None
Perspecta-PurdueRutgers 0.42961 0.04791 0.14768 0.85214 2506.02 2022-11-19T02:01:15 2022-11-19T01:55:52 None None
Perspecta-PurdueRutgers 0.43561 0.04748 0.14917 0.85214 2519.24 2022-11-19T04:20:08 2022-11-19T04:10:44 None None
Perspecta-IUB 0.24616 0.0205 0.06302 1.0 49831.94 2022-10-06T23:40:09 2022-10-06T23:35:08 None None
Perspecta-IUB 0.23145 0.01981 0.0576 0.99995 41884.97 2022-10-07T14:00:09 2022-10-07T03:35:32 None None
Perspecta-IUB 0.01537 0.0062 0.00234 1.0 42205.08 2022-10-08T02:30:09 2022-10-08T02:23:53 None None
Perspecta-IUB 0.02305 0.01443 0.00543 0.99981 41364.65 2022-10-08T12:40:31 2022-10-08T04:37:48 None None
Perspecta-IUB 0.33401 0.00656 0.08135 0.99995 41703.38 2022-10-08T21:40:30 2022-10-08T18:08:16 None None
Perspecta-IUB 0.3314 0.00559 0.08016 1.0 41492.89 2022-10-09T06:30:35 2022-10-08T23:24:10 None None
Perspecta-IUB 0.22543 0.01824 0.05413 1.0 42087.76 2022-10-09T17:40:08 2022-10-09T17:30:28 None None
Perspecta-IUB 0.16931 0.0255 0.04101 0.99443 44000.76 2022-10-10T02:30:33 2022-10-09T22:35:57 None None
Perspecta-IUB 0.10872 0.02994 0.03099 0.9947 42151.94 2022-10-10T15:20:09 2022-10-10T15:11:15 None None
Perspecta-IUB 0.12328 0.02697 0.03342 0.9953 5536.52 2022-10-11T00:10:31 2022-10-10T18:15:05 None None
Perspecta-IUB 0.18142 0.02512 0.04338 0.99429 5557.0 2022-10-11T02:20:08 2022-10-11T02:13:19 None None
Perspecta-IUB 0.16654 0.0229 0.0383 0.9972 6644.42 2022-10-11T03:40:31 2022-10-11T03:02:15 None None
Perspecta-IUB 0.17701 0.02285 0.04105 0.99713 6695.53 2022-10-11T17:40:09 2022-10-11T17:23:06 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 2022-10-11T19:20:09 2022-10-11T19:13:18 :Info File Missing::Container File Missing: None
Perspecta-IUB 0.20619 0.018 0.0461 0.99988 54700.54 2022-10-12T21:00:20 2022-10-12T18:57:51 None None
Perspecta-IUB 0.2183 0.01802 0.05042 0.99995 53655.09 2022-10-13T09:40:24 2022-10-12T23:53:26 None None
Perspecta-IUB 0.21104 0.01674 0.04715 1.0 54386.04 2022-10-14T01:50:10 2022-10-13T23:08:43 None None
Perspecta-IUB 0.2008 0.01702 0.04374 1.0 54506.62 2022-10-14T13:20:33 2022-10-14T06:51:18 None None
Perspecta-IUB 0.19169 0.01818 0.04159 0.9999 54738.46 2022-10-15T04:00:30 2022-10-14T21:09:47 None None
ARM-UCSD 0.55305 0.079 0.18167 0.79167 5683.8 2022-10-07T06:10:11 2022-10-07T06:05:34 None None
ARM-UCSD 0.45631 0.09297 0.13778 0.84028 5883.57 2022-10-07T18:20:09 2022-10-07T18:15:51 None None
ARM-UCSD 0.60889 0.10666 0.19333 0.77083 32650.77 2022-10-12T21:00:21 2022-10-11T23:57:30 None None
ARM-UCSD 1.20397 0.12688 0.41 0.5 3367.51 2022-11-01T16:10:12 2022-11-01T16:04:02 None None
ARM 0.69315 0.0 0.25 0.5 1829.04 2022-10-03T07:00:12 2022-10-03T06:55:39 None None
ARM 0.69315 0.0 0.25 0.5 2936.19 2022-10-04T18:40:14 2022-10-04T18:31:54 None None
ARM 0.69315 0.0 0.25 0.5 3178.67 2022-10-12T21:40:11 2022-10-12T21:36:43 None None
ARM 0.64633 0.01447 0.22789 0.66112 3180.82 2022-10-17T18:20:10 2022-10-17T18:11:32 :Container File Missing: None
ARM 0.69315 0.0 0.25 0.5 2695.53 2022-11-01T05:50:13 2022-11-01T06:58:02 None None
ARM 0.69315 0.0 0.25 0.5 2702.23 2022-11-01T19:20:12 2022-11-01T19:16:09 None None
ARM 0.52828 0.04556 0.17645 0.8255 4134.8 2022-11-01T21:10:13 2022-11-01T21:08:20 None None
ARM 0.69315 0.0 0.25 0.5 2700.2 2022-11-02T00:10:51 2022-11-01T19:16:09 None None
ARM 0.59672 0.02598 0.20403 0.78106 14892.7 2022-11-28T05:10:15 2022-11-28T05:03:33 None None
TrinitySRITrojAI-SBU 0.2078 0.03321 0.06981 0.99841 14851.38 2022-11-29T18:50:17 2022-11-29T18:45:10 None None
ICSI-2 0.10749 0.00514 0.01166 1.0 6585.41 2022-09-26T03:50:13 2022-09-26T03:43:51 None None
ICSI-2 0.44762 0.04697 0.15718 0.84279 3512.36 2022-10-03T02:00:14 2022-10-03T01:59:58 None None
ICSI-2 0.05754 0.00475 0.00447 1.0 3426.85 2022-10-04T04:50:25 2022-10-04T04:43:13 None None
ICSI-2 0.05952 0.00476 0.00468 1.0 4259.7 2022-10-10T09:10:15 2022-10-10T09:09:08 None None
ICSI-2 0.05835 0.00673 0.00583 1.0 3935.69 2022-10-11T20:20:14 2022-10-11T20:11:37 None None
ICSI-2 0.08364 0.01985 0.01835 0.99867 4349.12 2022-10-13T01:30:13 2022-10-13T01:30:01 None None
ICSI-2 0.13076 0.02471 0.03505 0.99542 3984.43 2022-10-14T01:50:16 2022-10-13T20:29:39 None None
ICSI-2 0.16264 0.02676 0.04563 0.9947 3960.36 2022-10-14T23:20:22 2022-10-14T23:17:14 None None
ICSI-2 0.1081 0.02084 0.02548 0.99853 3952.7 2022-10-15T21:00:12 2022-10-15T20:53:16 None None
ICSI-2 0.17996 0.06299 0.05036 0.98563 3959.79 2022-10-16T05:40:13 2022-10-16T05:34:41 None None
ICSI-2 0.0405 0.00575 0.00349 1.0 4134.71 2022-10-25T21:20:13 2022-10-25T21:15:59 :Container File Missing: None
ICSI-2 0.04095 0.00654 0.00403 1.0 4612.87 2022-10-26T06:40:13 2022-10-26T06:40:02 :Container File Missing: None
ICSI-2 0.03098 0.00397 0.00195 1.0 3758.06 2022-11-09T18:50:18 2022-11-09T18:48:31 None None
ICSI-2 0.01882 0.00204 0.00064 1.0 3748.12 2022-11-14T23:30:19 2022-11-14T23:23:54 None None
ICSI-2 0.01882 0.00204 0.00064 1.0 3749.14 2022-11-15T03:20:58 2022-11-15T03:15:09 None None
ICSI-2 0.69315 0.0 0.25 0.5 2022-11-22T11:00:19 2022-11-22T10:58:01 :No Results::Info File Missing::Container File Missing: None
ICSI-2 0.69315 0.0 0.25 0.5 2022-11-23T00:30:20 2022-11-23T00:27:50 :No Results::Info File Missing::Container File Missing: None
ICSI-2 0.0626 0.01323 0.01152 1.0 84630.15 2022-11-23T23:10:19 2022-11-23T23:07:52 None None
TrinitySRITrojAI-BostonU 0.69315 0.0 0.25 0.5 1539.44 2022-10-18T15:20:15 2022-10-18T15:11:05 :Container File Missing: None
TrinitySRITrojAI-BostonU 0.60027 0.02818 0.20945 0.68981 4557.63 2022-10-29T07:10:14 2022-10-29T07:05:21 :Container File Missing: None
TrinitySRITrojAI-BostonU 0.69315 0.0 0.25 0.5 2022-10-31T20:10:16 2022-10-31T20:01:29 :Executed File Update::Info File Missing::Container File Missing: None
TrinitySRITrojAI-BostonU 0.56018 0.05892 0.18865 0.81862 9466.89 2022-10-31T22:10:15 2022-10-31T22:07:23 :Log File Missing: None
TrinitySRITrojAI-BostonU 0.56018 0.05892 0.18865 0.81862 9545.42 2022-11-01T08:30:53 2022-11-01T08:15:11 None None
TrinitySRITrojAI-BostonU 0.69325 0.00164 0.25005 0.5 9534.94 2022-11-01T21:20:48 2022-11-01T21:13:17 None None
TrinitySRITrojAI-BostonU 0.69315 8e-05 0.25 0.5 9745.25 2022-11-02T02:00:18 2022-11-02T01:53:52 None None
TrinitySRITrojAI-BostonU 0.79041 0.06774 0.28707 0.56855 9477.14 2022-11-02T07:10:20 2022-11-02T07:04:36 None None
TrinitySRITrojAI-BostonU 0.80599 0.06542 0.29317 0.5244 9565.26 2022-11-02T18:30:21 2022-11-02T18:13:50 None None
TrinitySRITrojAI-BostonU 0.56018 0.05892 0.18865 0.81862 9509.69 2022-11-03T03:40:21 2022-11-03T03:36:00 None None
TrinitySRITrojAI-BostonU 0.56018 0.05892 0.18865 0.81862 9603.86 2022-11-06T20:04:14 2022-11-06T16:40:19 None None
TrinitySRITrojAI-BostonU 0.56018 0.05892 0.18865 0.81862 9661.45 2022-11-07T23:21:17 2022-11-07T22:00:00 None None
TrinitySRITrojAI-BostonU 0.56018 0.05892 0.18865 0.81862 9682.3 2022-11-15T17:00:27 2022-11-15T16:57:31 None None
TrinitySRITrojAI-BostonU 0.60361 0.02366 0.20746 0.77361 9525.36 2022-11-15T19:51:44 2022-11-15T17:57:00 None None
trojai-example 0.95798 0.09962 0.32408 0.47912 3412.55 2022-09-21T17:30:13 2022-09-21T17:28:50 None None
IIECAS 12.3764 1.58693 0.44792 0.11073 7560.93 2022-11-17T12:30:29 2022-11-17T12:21:16 None None
IIECAS 1.09716 0.12628 0.37111 0.54861 116013.04 2022-11-18T12:20:26 2022-11-18T12:12:28 None None
IIECAS 1.00561 0.1248 0.33778 0.59028 64116.48 2022-11-27T14:00:30 2022-11-27T13:58:29 None None
IIECAS 0.69315 0.0 0.25 0.5 14666.41 2022-11-29T02:30:30 2022-11-29T02:30:04 :No Results: None
IIECAS 1.02087 0.12511 0.34333 0.58333 66585.78 2022-11-30T04:11:19 2022-11-30T02:21:07 None None
IIECAS 0.88354 0.12137 0.29333 0.64583 75608.21 2022-12-01T15:01:55 2022-12-01T13:44:48 None None
UMBCb 0.49955 0.02726 0.16128 0.89157 3656.39 2022-10-31T19:10:23 2022-10-31T19:00:37 None None
UMBCb 7.91988 1.34209 0.40696 0.53742 5067.91 2022-11-04T14:43:29 2022-11-03T20:20:40 None None

Required filename format: "image-classification-sep2022_test_<Submission Name>.simg"
Accepting submissions: True
Number of models in image-classification-sep2022, test: 216
Execution timeout (hh:mm:ss): 1 day, 12:00:00

Teams/Jobs

Team Submission Timestamp Type Job Status File Status General Status File Timestamp Time until next execution
Perspecta 2022-12-04T00:00:05 performer None Ok Ok 2022-12-02T03:25:52 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI 2022-11-18T00:00:55 performer None Ok Ok 2022-11-17T23:59:38 0 d, 0 h, 0 m, 0 s
DRAFFT performer None None Ok None 0 d, 0 h, 0 m, 0 s
PL-GIFT 2022-11-15T23:50:08 performer None Ok Ok 2022-11-15T23:45:59 0 d, 0 h, 0 m, 0 s
Perspecta-PurdueRutgers 2022-11-19T04:20:08 performer None Ok Ok 2022-11-19T04:10:44 0 d, 0 h, 0 m, 0 s
Perspecta-IUB 2022-11-18T18:45:04 performer None Ok Ok 2022-11-18T17:16:35 0 d, 0 h, 0 m, 0 s
ARM-UCSD 2022-11-01T20:30:11 performer None Ok Ok 2022-11-01T20:20:14 0 d, 0 h, 0 m, 0 s
ARM 2022-11-28T05:10:15 performer None Ok Ok 2022-11-28T05:03:33 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-SBU 2022-11-29T18:50:17 performer None Ok Ok 2022-11-29T18:45:10 0 d, 0 h, 0 m, 0 s
ARM-UMBC performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-1 performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-2 2022-12-04T23:50:19 performer RUNNING Ok Ok 2022-12-04T23:44:26 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-BostonU 2022-11-15T19:40:51 performer None Ok Ok 2022-11-15T17:50:14 0 d, 0 h, 0 m, 0 s
trojai-example 2022-09-21T17:30:13 public None Ok Ok 2022-09-21T17:28:50 0 d, 0 h, 0 m, 0 s
IceTorch public None None Ok None 0 d, 0 h, 0 m, 0 s
trojaicy public None None Ok None 0 d, 0 h, 0 m, 0 s
Hector public None None Ok None 0 d, 0 h, 0 m, 0 s
Cassandra-XF public None None Ok None 0 d, 0 h, 0 m, 0 s
gooddefense public None None Ok None 0 d, 0 h, 0 m, 0 s
cyberinfrastructure public None None Ok None 0 d, 0 h, 0 m, 0 s
glaros public None None Ok None 0 d, 0 h, 0 m, 0 s
Ultraman public None None Ok None 0 d, 0 h, 0 m, 0 s
BackSec public None None Ok None 0 d, 0 h, 0 m, 0 s
Palladium public None None Ok None 0 d, 0 h, 0 m, 0 s
IIECAS 2022-12-04T00:00:31 public None Ok Ok 2022-12-02T08:08:27 0 d, 0 h, 0 m, 0 s
NSEC public None None Ok None 0 d, 0 h, 0 m, 0 s
UMBCb 2022-11-04T14:43:29 performer None Ok Ok 2022-11-03T20:20:40 0 d, 0 h, 0 m, 0 s

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
Perspecta 0.3428 0.09285 0.09818 0.93514 11069.68 2022-11-29T15:50:05 2022-11-29T15:43:06 None None
TrinitySRITrojAI 0.4992 0.0595 0.16927 0.81297 43071.42 2022-10-17T08:50:06 2022-10-17T08:43:17 None None
PL-GIFT 0.21259 0.06967 0.06595 0.97878 1883.08 2022-11-15T17:40:09 2022-11-15T17:33:52 None None
Perspecta-PurdueRutgers 0.30179 0.08706 0.08387 0.91615 118130.14 2022-11-08T19:40:09 2022-11-08T19:33:41 None :Container Parameters (learned parameters):
Perspecta-IUB 0.37543 0.09072 0.11553 0.92678 3170.51 2022-11-18T18:45:04 2022-11-18T17:16:35 None :Container Parameters (jsonschema checker):
ARM-UCSD 0.54277 0.11701 0.16926 0.80093 4421.76 2022-10-07T18:20:09 2022-10-07T18:15:51 None None
ARM 0.63044 0.05541 0.2192 0.71348 3099.07 2022-11-01T21:10:13 2022-11-01T21:08:20 None None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 938.16 2022-11-29T05:21:12 2022-11-29T05:13:35 :No Results: None
ICSI-2 0.31188 0.07664 0.09225 0.94072 63796.16 2022-11-23T23:10:19 2022-11-23T23:07:52 None None
TrinitySRITrojAI-BostonU 0.69214 0.04466 0.24685 0.58312 3412.87 2022-10-29T07:10:15 2022-10-29T07:06:28 None None
trojai-example 0.84173 0.10527 0.28669 0.53477 2543.89 2022-09-21T17:30:13 2022-09-21T17:28:50 None None
IIECAS 0.69315 0.0 0.25 0.5 2022-11-27T12:10:30 2022-11-27T12:07:55 :No Results::Info File Missing: None
UMBCb 0.69034 0.0512 0.245 0.6178 2662.31 2022-10-31T19:10:23 2022-10-31T19:00:37 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
Perspecta 0.69585 0.00687 0.2513 0.49537 129603.31 2022-10-03T20:40:03 2022-10-03T20:31:27 None :Timeout:
Perspecta 0.71976 0.08286 0.24824 0.62731 2181.47 2022-10-13T03:30:03 2022-10-13T03:20:12 None None
Perspecta 0.91329 0.24076 0.23932 0.69217 2143.86 2022-10-18T05:20:03 2022-10-18T05:11:57 None None
Perspecta 1.76118 0.32537 0.41468 0.42074 2071.51 2022-10-18T17:30:03 2022-10-18T17:23:07 None None
Perspecta 0.73769 0.10956 0.24833 0.63953 2035.15 2022-10-19T00:00:04 2022-10-18T23:56:57 None None
Perspecta 0.68774 0.09754 0.22978 0.6788 2036.43 2022-10-19T18:10:04 2022-10-19T18:02:05 None None
Perspecta 0.6845 0.08769 0.23123 0.67357 2034.36 2022-10-19T21:00:04 2022-10-19T20:53:59 None None
Perspecta 0.64453 0.07715 0.21921 0.72308 2026.29 2022-10-19T23:40:03 2022-10-19T23:33:06 None None
Perspecta 0.75607 0.16437 0.21981 0.76097 2114.58 2022-10-26T14:40:04 2022-10-26T14:32:03 None None
Perspecta 0.8982 0.23516 0.21884 0.77761 2085.55 2022-10-27T00:50:03 2022-10-27T00:44:13 None None
Perspecta 1.02657 0.33428 0.2214 0.77516 2079.53 2022-10-27T04:20:03 2022-10-27T04:15:11 None None
Perspecta 0.59227 0.09079 0.2 0.77756 2091.55 2022-10-27T15:40:04 2022-10-27T15:38:34 None None
Perspecta 0.4583 0.0657 0.15224 0.86081 2074.57 2022-10-28T23:30:03 2022-10-28T23:22:21 None None
Perspecta 0.47358 0.06943 0.15484 0.85678 2156.95 2022-11-02T18:04:43 2022-11-02T17:56:09 None None
Perspecta 0.38893 0.06678 0.12453 0.90415 2150.32 2022-11-07T21:10:05 2022-11-07T21:04:58 None None
Perspecta 0.38314 0.07537 0.1232 0.90222 2121.85 2022-11-15T04:00:04 2022-11-15T03:50:54 None None
Perspecta 0.41223 0.07331 0.13351 0.89069 2152.8 2022-11-16T01:20:04 2022-11-16T01:16:33 None None
Perspecta 0.37649 0.14735 0.09713 0.9459 2156.74 2022-11-16T19:00:04 2022-11-16T18:58:52 None None
Perspecta 0.37985 0.07276 0.12031 0.90578 2150.64 2022-11-16T21:10:04 2022-11-16T21:09:20 None None
Perspecta 0.46206 0.15627 0.11401 0.90758 2062.49 2022-11-17T22:30:04 2022-11-17T22:23:07 None None
Perspecta 0.36412 0.06779 0.11863 0.91615 2159.55 2022-11-22T05:20:04 2022-11-22T05:17:14 None None
Perspecta 0.38556 0.06515 0.12332 0.90505 2025.27 2022-11-22T07:30:04 2022-11-22T07:22:56 None None
Perspecta 0.39845 0.08336 0.12282 0.90959 11071.45 2022-11-29T07:40:04 2022-11-29T07:38:19 None None
Perspecta 0.3428 0.09285 0.09818 0.93514 11069.68 2022-11-29T15:50:05 2022-11-29T15:43:06 None None
Perspecta 0.34504 0.07069 0.10928 0.92953 11066.84 2022-11-30T20:00:04 2022-11-30T19:50:09 None None
Perspecta 0.40367 0.03316 0.12032 0.95786 11174.79 2022-12-01T05:20:04 2022-12-01T05:16:29 None None
Perspecta 0.35539 0.04517 0.10568 0.94796 11316.39 2022-12-01T16:50:09 2022-12-01T16:48:02 None None
Perspecta 0.35539 0.04517 0.10568 0.94796 2852.21 2022-12-04T00:00:05 2022-12-02T03:25:52 None None
TrinitySRITrojAI 0.707 0.0916 0.23573 0.68639 6866.14 2022-09-30T06:30:06 2022-09-30T06:24:28 None :Cleanup::Cleanup:
TrinitySRITrojAI 0.75205 0.11422 0.24401 0.68266 6865.44 2022-09-30T20:50:05 2022-09-30T20:43:42 None None
TrinitySRITrojAI 0.73967 0.06156 0.26013 0.59821 7017.93 2022-10-07T08:40:06 2022-10-07T08:37:36 None None
TrinitySRITrojAI 0.59141 0.08373 0.19756 0.76595 19243.69 2022-10-08T05:20:06 2022-10-08T05:19:38 None None
TrinitySRITrojAI 0.4992 0.0595 0.16927 0.81297 43071.42 2022-10-17T08:50:06 2022-10-17T08:43:17 None None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 1813.63 2022-11-16T01:40:06 2022-11-16T01:35:44 :No Results: None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 1887.99 2022-11-16T07:00:06 2022-11-16T06:59:01 :No Results: None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 1816.96 2022-11-17T05:40:05 2022-11-17T05:33:04 :No Results: None
TrinitySRITrojAI 0.6194 0.06721 0.2151 0.72741 48083.46 2022-11-18T00:00:55 2022-11-17T23:59:38 None None
PL-GIFT 0.57702 0.07557 0.19379 0.78074 1429.5 2022-09-23T22:10:06 2022-09-23T22:01:36 None None
PL-GIFT 0.46674 0.05664 0.16051 0.84709 2038.31 2022-09-30T01:00:06 2022-09-30T00:56:52 None :Cleanup::Cleanup:
PL-GIFT 0.50873 0.10565 0.15658 0.85142 11566.66 2022-10-12T15:10:08 2022-10-12T15:01:57 None None
PL-GIFT 0.34255 0.07196 0.109 0.93236 1419.24 2022-10-14T18:20:08 2022-10-14T18:18:21 None None
PL-GIFT 0.7001 0.04798 0.24546 0.64489 4271.0 2022-10-20T22:00:07 2022-10-20T21:55:07 None :Container Parameters (jsonschema checker):
PL-GIFT 0.32598 0.07557 0.10175 0.9411 1863.26 2022-10-25T23:40:07 2022-10-25T23:31:47 None None
PL-GIFT 0.33886 0.09336 0.09653 0.94959 1516.36 2022-11-04T23:30:07 2022-11-04T23:20:37 None None
PL-GIFT 0.3101 0.09115 0.09195 0.94029 1875.68 2022-11-07T17:10:08 2022-11-07T17:10:02 None None
PL-GIFT 0.36153 0.08622 0.1082 0.91487 1871.2 2022-11-09T23:10:08 2022-11-09T23:03:07 None None
PL-GIFT 0.36958 0.09961 0.11224 0.92571 1874.25 2022-11-11T19:20:07 2022-11-11T19:19:08 None None
PL-GIFT 0.22401 0.07489 0.07151 0.97467 1870.67 2022-11-11T23:10:07 2022-11-11T23:09:05 None None
PL-GIFT 0.21259 0.06967 0.06595 0.97878 1883.08 2022-11-15T17:40:09 2022-11-15T17:33:52 None None
PL-GIFT 0.37087 0.11213 0.10117 0.91397 1872.44 2022-11-15T20:20:45 2022-11-15T20:17:08 None None
PL-GIFT 0.26396 0.07679 0.08149 0.96579 1866.32 2022-11-15T23:50:08 2022-11-15T23:45:59 None None
Perspecta-PurdueRutgers 0.44105 0.10542 0.13222 0.84722 72513.96 2022-10-03T08:20:08 2022-10-03T08:18:13 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.42974 0.10225 0.12829 0.85742 73585.52 2022-10-04T22:40:07 2022-10-04T22:39:49 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.49211 0.10619 0.15434 0.85322 47542.29 2022-10-14T10:00:08 2022-10-14T09:52:56 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.51457 0.11011 0.16168 0.83479 98129.1 2022-10-15T15:20:09 2022-10-15T15:12:56 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.36284 0.10839 0.10061 0.89489 61710.03 2022-10-18T06:50:22 2022-10-18T06:41:35 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.33742 0.08619 0.09471 0.90008 61248.41 2022-10-24T01:40:09 2022-10-24T01:34:56 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.35812 0.07447 0.10628 0.90771 129603.41 2022-11-01T17:50:09 2022-11-01T17:42:36 None :Container Parameters (learned parameters)::Timeout:
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 2022-11-03T13:10:09 2022-11-03T13:07:21 :Info File Missing: None
Perspecta-PurdueRutgers 0.65449 0.02534 0.23147 0.63773 2236.2 2022-11-04T15:20:09 2022-11-04T15:17:23 None None
Perspecta-PurdueRutgers 0.31039 0.07938 0.0853 0.91615 121209.15 2022-11-04T20:40:10 2022-11-04T20:30:10 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.58752 0.03242 0.20052 0.76432 1778.7 2022-11-06T21:00:08 2022-11-06T20:52:56 None None
Perspecta-PurdueRutgers 0.30179 0.08706 0.08387 0.91615 118130.14 2022-11-08T19:40:09 2022-11-08T19:33:41 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.67805 0.01868 0.24277 0.5796 1782.61 2022-11-11T17:20:08 2022-11-11T17:17:24 None None
Perspecta-PurdueRutgers 0.56947 0.07318 0.19096 0.78562 1920.95 2022-11-15T23:30:10 2022-11-15T23:21:05 None None
Perspecta-PurdueRutgers 0.76088 0.35207 0.185 0.80153 1914.11 2022-11-16T05:00:08 2022-11-16T04:53:47 None None
Perspecta-PurdueRutgers 0.57369 0.07366 0.19305 0.78888 1924.87 2022-11-16T16:40:09 2022-11-16T16:36:52 None None
Perspecta-PurdueRutgers 0.56123 0.07512 0.19141 0.78888 1914.21 2022-11-17T02:00:07 2022-11-17T01:55:49 None None
Perspecta-PurdueRutgers 0.5424 0.07689 0.18724 0.78888 1877.95 2022-11-17T03:10:07 2022-11-17T03:09:26 None None
Perspecta-PurdueRutgers 0.89219 0.42945 0.18875 0.76415 1889.69 2022-11-17T04:41:19 2022-11-17T04:34:55 None None
Perspecta-PurdueRutgers 0.50568 0.04985 0.17495 0.81083 1879.76 2022-11-19T00:30:08 2022-11-18T21:26:33 None None
Perspecta-PurdueRutgers 0.623 0.25074 0.17604 0.8074 1883.73 2022-11-19T02:01:15 2022-11-19T01:55:52 None None
Perspecta-PurdueRutgers 0.52693 0.04722 0.18031 0.81083 1885.74 2022-11-19T04:20:08 2022-11-19T04:10:44 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 1450.22 2022-10-05T22:40:09 2022-10-05T22:38:17 None None
Perspecta-IUB 0.45452 0.05118 0.15329 0.887 37485.16 2022-10-06T23:40:09 2022-10-06T23:35:08 None None
Perspecta-IUB 0.48582 0.05713 0.1648 0.84701 31197.94 2022-10-07T14:00:09 2022-10-07T03:35:32 None None
Perspecta-IUB 0.48491 0.10831 0.15727 0.87757 31816.89 2022-10-08T02:30:09 2022-10-08T02:23:53 None None
Perspecta-IUB 0.5696 0.1248 0.17848 0.84546 31353.94 2022-10-08T12:40:31 2022-10-08T04:37:48 None None
Perspecta-IUB 0.56846 0.0265 0.1904 0.84512 31110.88 2022-10-08T21:40:30 2022-10-08T18:08:16 None None
Perspecta-IUB 0.5535 0.02455 0.183 0.87646 31280.68 2022-10-09T06:30:35 2022-10-08T23:24:10 None None
Perspecta-IUB 0.46086 0.05045 0.15486 0.87903 31302.38 2022-10-09T17:40:08 2022-10-09T17:30:28 None None
Perspecta-IUB 0.4776 0.09831 0.14941 0.8657 33452.88 2022-10-10T02:30:33 2022-10-09T22:35:57 None None
Perspecta-IUB 0.77317 0.28022 0.16866 0.85434 31590.83 2022-10-10T15:20:09 2022-10-10T15:11:15 None None
Perspecta-IUB 0.60741 0.18245 0.16066 0.85721 4179.16 2022-10-11T00:10:31 2022-10-10T18:15:05 None None
Perspecta-IUB 0.46348 0.08648 0.14714 0.86737 4180.82 2022-10-11T02:20:08 2022-10-11T02:13:19 None None
Perspecta-IUB 0.44146 0.07907 0.14309 0.87534 4997.51 2022-10-11T03:40:31 2022-10-11T03:02:15 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 2022-10-11T05:10:33 2022-10-11T04:00:25 :Info File Missing: None
Perspecta-IUB 0.4397 0.07606 0.14243 0.87603 4984.81 2022-10-11T17:40:09 2022-10-11T17:23:06 None None
Perspecta-IUB 0.46103 0.04889 0.15507 0.88387 44245.22 2022-10-11T19:20:09 2022-10-11T19:13:18 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 2022-10-12T15:20:09 2022-10-12T15:10:37 :Executed File Update::Info File Missing: None
Perspecta-IUB 0.69315 0.0 0.25 0.5 2022-10-12T19:00:09 2022-10-12T18:57:51 :Executed File Update::Info File Missing: None
Perspecta-IUB 0.42076 0.04954 0.1379 0.89206 45067.73 2022-10-12T21:00:20 2022-10-12T18:57:51 None None
Perspecta-IUB 0.42745 0.04739 0.14082 0.89716 41399.1 2022-10-13T09:40:24 2022-10-12T23:53:26 None None
Perspecta-IUB 0.42878 0.04669 0.141 0.89819 40730.59 2022-10-14T01:50:10 2022-10-13T23:08:43 None None
Perspecta-IUB 0.42214 0.04923 0.13825 0.8936 40755.44 2022-10-14T13:20:33 2022-10-14T06:51:18 None None
Perspecta-IUB 0.42 0.05403 0.13757 0.88833 40810.27 2022-10-15T04:00:30 2022-10-14T21:09:47 None None
Perspecta-IUB 0.42 0.05403 0.13757 0.88833 41123.91 2022-11-15T17:10:08 2022-11-15T17:08:10 None None
Perspecta-IUB 0.65895 0.23166 0.13675 0.8879 4441.04 2022-11-17T03:50:45 2022-11-17T03:48:28 None :Container Parameters (jsonschema checker):
Perspecta-IUB 0.66235 0.10598 0.21817 0.74271 2593.28 2022-11-17T22:10:09 2022-11-17T22:03:12 None :Container Parameters (jsonschema checker):
Perspecta-IUB 0.50003 0.07657 0.16996 0.82472 2955.83 2022-11-18T04:20:09 2022-11-18T04:17:39 None :Container Parameters (jsonschema checker):
Perspecta-IUB 0.50283 0.13492 0.14662 0.90548 2954.72 2022-11-18T05:30:08 2022-11-18T05:26:14 None :Container Parameters (jsonschema checker):
Perspecta-IUB 0.37543 0.09072 0.11553 0.92678 3170.51 2022-11-18T18:45:04 2022-11-18T17:16:35 None :Container Parameters (jsonschema checker):
ARM-UCSD 0.69315 0.0 0.25 0.5 1804.24 2022-10-06T22:20:09 2022-10-06T22:16:03 None None
ARM-UCSD 0.5556 0.09155 0.18259 0.79012 4257.3 2022-10-07T06:10:11 2022-10-07T06:05:34 None None
ARM-UCSD 0.54277 0.11701 0.16926 0.80093 4421.76 2022-10-07T18:20:09 2022-10-07T18:15:51 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-10-12T13:32:35 2022-10-11T23:57:30 :Info File Missing: None
ARM-UCSD 0.7157 0.13125 0.23222 0.72222 24636.71 2022-10-12T21:00:21 2022-10-11T23:57:30 None None
ARM-UCSD 0.7157 0.13125 0.23222 0.72222 8961.43 2022-10-13T04:00:20 2022-10-12T21:28:41 None None
ARM-UCSD 1.20397 0.14651 0.41 0.5 2546.13 2022-11-01T15:40:11 2022-11-01T15:33:06 None None
ARM-UCSD 1.20397 0.14651 0.41 0.5 2514.79 2022-11-01T20:30:11 2022-11-01T20:20:14 None None
ARM 0.69315 0.0 0.25 0.5 1375.72 2022-10-03T07:30:14 2022-10-03T07:26:20 None None
ARM 0.66913 0.0129 0.23838 0.60237 2388.62 2022-10-17T18:20:10 2022-10-17T18:11:32 None None
ARM 0.63044 0.05541 0.2192 0.71348 3099.07 2022-11-01T21:10:13 2022-11-01T21:08:20 None None
ARM 0.65914 0.03107 0.23323 0.65835 11174.49 2022-11-28T05:10:15 2022-11-28T05:03:33 None None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 938.16 2022-11-29T05:21:12 2022-11-29T05:13:35 :No Results: None
TrinitySRITrojAI-SBU 1.22432 0.23004 0.33275 0.5691 11107.06 2022-11-29T18:50:17 2022-11-29T18:45:10 None None
ICSI-2 0.49894 0.07109 0.16271 0.83989 4954.9 2022-09-26T03:50:14 2022-09-26T03:42:55 None None
ICSI-2 0.64325 0.08268 0.22344 0.72441 2686.94 2022-10-03T02:00:15 2022-10-03T01:59:22 None None
ICSI-2 0.4133 0.06463 0.13373 0.89725 2529.52 2022-10-04T04:50:26 2022-10-04T04:43:23 None None
ICSI-2 0.69315 0.0 0.25 0.5 2022-10-10T08:40:16 2022-10-10T08:34:46 :Info File Missing: None
ICSI-2 0.40322 0.06265 0.12774 0.90261 3186.55 2022-10-10T09:10:15 2022-10-10T09:09:08 None None
ICSI-2 0.39858 0.06644 0.12588 0.90689 2980.6 2022-10-11T20:20:14 2022-10-11T20:11:37 None None
ICSI-2 0.43494 0.09453 0.13357 0.89965 3297.06 2022-10-13T01:30:13 2022-10-13T01:30:01 None None
ICSI-2 0.46428 0.07177 0.15497 0.85978 2988.68 2022-10-14T01:50:16 2022-10-13T20:29:39 None None
ICSI-2 0.45526 0.06498 0.1525 0.86368 2967.65 2022-10-14T23:20:22 2022-10-14T23:17:14 None None
ICSI-2 0.48321 0.08619 0.15631 0.86441 2977.13 2022-10-15T21:00:12 2022-10-15T20:53:16 None None
ICSI-2 0.63939 0.1329 0.19874 0.83025 2980.11 2022-10-16T05:40:13 2022-10-16T05:34:41 None None
ICSI-2 0.39105 0.07074 0.12407 0.90822 3098.31 2022-10-25T21:20:13 2022-10-25T21:15:59 None None
ICSI-2 0.41841 0.0842 0.13041 0.89973 3468.91 2022-10-26T06:40:13 2022-10-26T06:40:02 None None
ICSI-2 0.3635 0.07566 0.11306 0.92138 2818.3 2022-11-09T18:50:18 2022-11-09T18:48:31 None None
ICSI-2 0.36325 0.08031 0.11263 0.92245 2808.91 2022-11-14T23:30:19 2022-11-14T23:23:54 None None
ICSI-2 0.36325 0.08031 0.11263 0.92245 2807.1 2022-11-15T03:20:58 2022-11-15T03:15:09 None None
ICSI-2 0.34572 0.07765 0.10591 0.93128 35217.68 2022-11-22T11:00:19 2022-11-22T10:58:01 None None
ICSI-2 0.36183 0.08154 0.11309 0.9198 11242.62 2022-11-23T00:30:20 2022-11-23T00:27:50 None None
ICSI-2 0.31188 0.07664 0.09225 0.94072 63796.16 2022-11-23T23:10:19 2022-11-23T23:07:52 None None
TrinitySRITrojAI-BostonU 0.69315 0.0 0.25 0.5 1158.38 2022-10-18T00:00:15 2022-10-17T23:55:05 None None
TrinitySRITrojAI-BostonU 0.69315 0.0 0.25 0.5 1154.16 2022-10-18T00:30:24 2022-10-18T00:05:51 None None
TrinitySRITrojAI-BostonU 0.69214 0.04466 0.24685 0.58312 3412.87 2022-10-29T07:10:15 2022-10-29T07:06:28 None None
TrinitySRITrojAI-BostonU 0.82041 0.0869 0.29204 0.5643 7086.8 2022-10-31T20:10:17 2022-10-31T20:01:29 None None
TrinitySRITrojAI-BostonU 0.82041 0.0869 0.29204 0.5643 7123.09 2022-11-01T08:30:54 2022-11-01T08:15:26 None None
TrinitySRITrojAI-BostonU 0.69315 0.0001 0.25 0.5 7314.69 2022-11-02T02:00:19 2022-11-02T01:50:55 None None
TrinitySRITrojAI-BostonU 0.89365 0.0825 0.32826 0.46039 7091.41 2022-11-02T06:20:20 2022-11-02T06:13:15 None None
TrinitySRITrojAI-BostonU 0.98433 0.10842 0.34387 0.42627 7154.62 2022-11-02T18:30:22 2022-11-02T18:13:38 None None
TrinitySRITrojAI-BostonU 0.82041 0.0869 0.29204 0.5643 7104.33 2022-11-03T05:50:21 2022-11-03T05:47:16 None None
TrinitySRITrojAI-BostonU 0.82041 0.0869 0.29204 0.5643 7228.94 2022-11-06T20:04:15 2022-11-06T16:40:24 None None
TrinitySRITrojAI-BostonU 0.82041 0.0869 0.29204 0.5643 7160.91 2022-11-07T23:21:18 2022-11-07T22:00:09 None None
TrinitySRITrojAI-BostonU 0.82041 0.0869 0.29204 0.5643 7218.55 2022-11-15T17:30:58 2022-11-15T17:25:55 None None
TrinitySRITrojAI-BostonU 0.72758 0.04424 0.26281 0.53935 7197.8 2022-11-15T19:40:51 2022-11-15T17:50:14 None None
trojai-example 0.84173 0.10527 0.28669 0.53477 2543.89 2022-09-21T17:30:13 2022-09-21T17:28:50 None None
IIECAS 12.28046 1.83104 0.44444 0.11111 5691.78 2022-11-17T12:30:29 2022-11-17T12:21:16 None None
IIECAS 1.04122 0.1449 0.35074 0.57407 72411.13 2022-11-18T12:20:26 2022-11-18T12:12:28 None None
IIECAS 0.69315 0.0 0.25 0.5 2022-11-27T12:10:30 2022-11-27T12:07:55 :No Results::Info File Missing: None
IIECAS 0.99035 0.14372 0.33222 0.59722 45830.48 2022-11-27T14:00:30 2022-11-27T13:58:29 None None
IIECAS 0.69315 0.0 0.25 0.5 11044.76 2022-11-28T05:21:14 2022-11-28T05:18:08 :No Results: None
IIECAS 0.69315 0.0 0.25 0.5 11001.31 2022-11-28T09:30:30 2022-11-28T09:28:15 :No Results: None
IIECAS 0.69315 0.0 0.25 0.5 10999.59 2022-11-28T14:20:30 2022-11-28T14:16:37 :No Results: None
IIECAS 0.90898 0.14113 0.30259 0.63426 45553.76 2022-11-29T02:30:30 2022-11-29T02:30:04 None None
IIECAS 0.90898 0.14113 0.30259 0.63426 45545.31 2022-11-29T15:21:30 2022-11-29T02:59:18 None None
IIECAS 0.98018 0.14344 0.32852 0.60185 47294.07 2022-11-30T04:11:19 2022-11-30T02:21:07 None None
IIECAS 0.69315 0.0 0.25 0.5 10998.42 2022-12-01T06:20:28 2022-12-01T06:14:21 :No Results: None
IIECAS 0.69315 0.0 0.25 0.5 11001.18 2022-12-01T11:50:30 2022-12-01T11:44:58 :No Results: None
IIECAS 0.77673 0.13498 0.25444 0.69444 54445.88 2022-12-01T15:01:55 2022-12-01T13:44:48 None None
IIECAS 0.69315 0.0 0.25 0.5 1424.47 2022-12-04T00:00:31 2022-12-02T08:08:27 :No Results: :Container Parameters (jsonschema checker):
UMBCb 0.69034 0.0512 0.245 0.6178 2662.31 2022-10-31T19:10:23 2022-10-31T19:00:37 None None
UMBCb 7.60792 1.52536 0.39033 0.55856 3799.12 2022-11-04T14:43:29 2022-11-03T20:20:40 None None

Required filename format: "image-classification-sep2022_sts_<Submission Name>.simg"
Accepting submissions: True
Number of models in image-classification-sep2022, sts: 10
Execution timeout (hh:mm:ss): 1:40:00

Teams/Jobs

Team Submission Timestamp Type Job Status File Status General Status File Timestamp Time until next execution
Perspecta 2022-12-01T16:20:07 performer None None Ok 2022-12-01T16:12:07 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI 2022-11-17T04:40:05 performer None None Ok 2022-11-17T04:37:15 0 d, 0 h, 0 m, 0 s
DRAFFT performer None None Ok None 0 d, 0 h, 0 m, 0 s
PL-GIFT 2022-11-15T20:40:07 performer None None Ok 2022-11-15T20:35:27 0 d, 0 h, 0 m, 0 s
Perspecta-PurdueRutgers 2022-11-17T04:30:07 performer None None Ok 2022-11-17T04:20:21 0 d, 0 h, 0 m, 0 s
Perspecta-IUB 2022-10-12T13:32:33 performer None None Ok 2022-10-12T00:48:36 0 d, 0 h, 0 m, 0 s
ARM-UCSD 2022-10-12T13:32:36 performer None None Ok 2022-10-11T23:57:26 0 d, 0 h, 0 m, 0 s
ARM 2022-11-28T05:10:16 performer None Ok Ok 2022-11-28T05:03:08 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-SBU 2022-11-29T18:30:17 performer None None Ok 2022-11-29T18:27:38 0 d, 0 h, 0 m, 0 s
ARM-UMBC performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-1 performer None None Ok None 0 d, 0 h, 0 m, 0 s
ICSI-2 2022-12-04T00:00:21 performer None Ok Ok 2022-12-02T02:30:32 0 d, 0 h, 0 m, 0 s
TrinitySRITrojAI-BostonU 2022-11-03T05:50:22 performer None None Ok 2022-11-03T05:47:16 0 d, 0 h, 0 m, 0 s
trojai-example 2022-10-19T13:30:19 public None Ok Ok 2022-10-19T13:28:02 0 d, 0 h, 0 m, 0 s
IceTorch public None None Ok None 0 d, 0 h, 0 m, 0 s
trojaicy public None None Ok None 0 d, 0 h, 0 m, 0 s
Hector public None None Ok None 0 d, 0 h, 0 m, 0 s
Cassandra-XF public None None Ok None 0 d, 0 h, 0 m, 0 s
gooddefense public None None Ok None 0 d, 0 h, 0 m, 0 s
cyberinfrastructure public None None Ok None 0 d, 0 h, 0 m, 0 s
glaros public None None Ok None 0 d, 0 h, 0 m, 0 s
Ultraman public None None Ok None 0 d, 0 h, 0 m, 0 s
BackSec public None None Ok None 0 d, 0 h, 0 m, 0 s
Palladium public None None Ok None 0 d, 0 h, 0 m, 0 s
IIECAS 2022-11-28T14:20:31 public None None Ok 2022-11-28T14:16:24 0 d, 0 h, 0 m, 0 s
NSEC public None None Ok None 0 d, 0 h, 0 m, 0 s
UMBCb 2022-11-02T20:10:30 performer None None Ok 2022-11-02T17:11:22 0 d, 0 h, 0 m, 0 s

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
Perspecta 0.07122 0.07771 0.01429 1.0 94.51 2022-11-30T19:20:04 2022-11-30T19:13:22 None None
TrinitySRITrojAI 0.36564 0.15694 0.11009 0.91667 2438.18 2022-10-17T07:20:06 2022-10-17T07:15:57 None None
PL-GIFT 0.03577 0.0293 0.00308 1.0 63.61 2022-09-23T21:50:07 2022-09-23T21:43:37 None None
Perspecta-PurdueRutgers 0.08373 0.01642 0.007 1.0 1659.9 2022-10-18T05:40:08 2022-10-18T05:33:54 :Container File Missing: :Container Parameters (learned parameters):
Perspecta-IUB 0.00234 0.00083 1e-05 1.0 1435.5 2022-10-08T01:50:08 2022-10-08T01:41:26 None None
ARM-UCSD 0.22292 0.14573 0.058 1.0 198.3 2022-10-07T06:10:12 2022-10-07T06:05:52 None None
ARM 0.17945 0.03395 0.02873 1.0 142.38 2022-11-28T05:10:16 2022-11-28T05:03:08 None None
TrinitySRITrojAI-SBU 0.3605 0.18267 0.12165 1.0 113.01 2022-11-29T18:30:17 2022-11-29T18:27:38 None None
ICSI-2 0.01681 0.01354 0.00072 1.0 126.82 2022-11-14T23:30:20 2022-11-14T23:24:02 None None
TrinitySRITrojAI-BostonU 0.26979 0.10317 0.06495 1.0 319.35 2022-10-31T20:10:18 2022-10-31T20:01:29 None None
trojai-example 0.55684 0.24684 0.19247 0.79167 152.1 2022-09-30T17:00:19 2022-09-30T16:59:47 None None
IIECAS 0.51681 0.16695 0.178 0.79167 462.64 2022-11-26T14:20:30 2022-11-26T14:18:48 None None
UMBCb 0.588 0.1539 0.20394 0.70833 133.11 2022-10-31T18:40:25 2022-10-31T18:31:33 :Container File Missing: None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
Perspecta 0.66583 0.05079 0.23676 0.58333 6000.62 2022-10-03T18:30:04 2022-10-03T18:28:21 None :Timeout:
Perspecta 0.47751 0.17169 0.15468 0.91667 519.7 2022-10-13T02:50:04 2022-10-13T02:49:18 None None
Perspecta 0.39355 0.18202 0.12011 0.91667 92.7 2022-10-18T05:00:04 2022-10-18T04:51:17 :Container File Missing: None
Perspecta 0.88063 0.37914 0.33177 0.66667 94.64 2022-10-18T14:20:04 2022-10-18T14:14:36 :Container File Missing: None
Perspecta 0.08808 0.04512 0.01099 1.0 93.74 2022-10-26T14:20:04 2022-10-26T14:12:49 :Container File Missing: None
Perspecta 0.12029 0.09595 0.02479 1.0 92.77 2022-10-27T00:30:05 2022-10-27T00:25:51 :Container File Missing: None
Perspecta 0.10195 0.07925 0.0193 1.0 94.71 2022-10-27T15:20:05 2022-10-27T15:11:54 :Container File Missing: None
Perspecta 0.16894 0.19425 0.05365 1.0 93.41 2022-10-28T23:10:03 2022-10-28T23:02:30 :Container File Missing: None
Perspecta 0.25789 0.30528 0.09238 1.0 93.69 2022-11-02T17:10:04 2022-11-02T17:09:08 None None
Perspecta 0.12036 0.10149 0.02734 1.0 93.71 2022-11-07T20:10:07 2022-11-07T20:06:40 None None
Perspecta 0.09413 0.1256 0.02667 1.0 93.68 2022-11-15T03:10:04 2022-11-15T03:04:49 None None
Perspecta 0.13628 0.18049 0.04508 1.0 96.86 2022-11-16T00:50:04 2022-11-16T00:43:48 None None
Perspecta 0.07444 0.09283 0.01784 1.0 95.47 2022-11-16T18:20:03 2022-11-16T18:17:56 None None
Perspecta 0.12297 0.12806 0.03305 1.0 92.44 2022-11-16T20:50:05 2022-11-16T20:47:39 None None
Perspecta 0.15885 0.10778 0.03503 1.0 94.5 2022-11-17T22:10:04 2022-11-17T22:02:11 None None
Perspecta 0.11923 0.1495 0.03465 1.0 100.55 2022-11-22T05:00:05 2022-11-22T04:56:07 None None
Perspecta 0.12935 0.16694 0.04025 1.0 92.72 2022-11-22T05:30:04 2022-11-22T05:24:42 None None
Perspecta 0.0846 0.05546 0.0122 1.0 96.52 2022-11-29T07:20:05 2022-11-29T07:17:16 None None
Perspecta 0.08018 0.0691 0.01422 1.0 98.28 2022-11-29T15:20:04 2022-11-29T15:18:03 None None
Perspecta 0.07122 0.07771 0.01429 1.0 94.51 2022-11-30T19:20:04 2022-11-30T19:13:22 None None
Perspecta 0.69315 0.0 0.25 0.5 97.79 2022-12-01T01:20:05 2022-12-01T01:12:44 :No Results: None
Perspecta 0.09387 0.03818 0.01072 1.0 105.72 2022-12-01T04:40:04 2022-12-01T04:38:18 None None
Perspecta 0.08 0.04113 0.00897 1.0 122.14 2022-12-01T16:20:07 2022-12-01T16:12:07 None None
TrinitySRITrojAI 0.43286 0.13095 0.13538 1.0 1064.99 2022-10-08T04:50:07 2022-10-08T04:47:28 None None
TrinitySRITrojAI 0.36564 0.15694 0.11009 0.91667 2438.18 2022-10-17T07:20:06 2022-10-17T07:15:57 None None
TrinitySRITrojAI 0.49685 0.17761 0.16473 0.85417 2142.53 2022-11-15T19:00:07 2022-11-15T18:55:12 None None
TrinitySRITrojAI 0.39175 0.16235 0.11944 0.95833 2148.7 2022-11-17T04:40:05 2022-11-17T04:37:15 None None
PL-GIFT 0.69315 0.0 0.25 0.5 65.03 2022-09-22T19:30:06 2022-09-22T19:28:35 None None
PL-GIFT 0.69315 0.0 0.25 0.5 61.17 2022-09-23T19:40:06 2022-09-23T19:31:46 None None
PL-GIFT 0.03577 0.0293 0.00308 1.0 63.61 2022-09-23T21:50:07 2022-09-23T21:43:37 None None
PL-GIFT 0.05908 0.04643 0.00768 1.0 94.48 2022-09-29T22:50:07 2022-09-29T22:42:10 None :Cleanup::Cleanup:
PL-GIFT 0.69315 0.0 0.25 0.5 107.02 2022-10-11T17:40:08 2022-10-11T17:40:00 None None
PL-GIFT 0.69315 0.0 0.25 0.5 103.15 2022-10-11T21:30:08 2022-10-11T21:29:03 None None
PL-GIFT 0.69315 0.0 0.25 0.5 98.72 2022-10-12T13:32:16 2022-10-12T01:33:00 None None
PL-GIFT 0.05288 0.04288 0.00634 1.0 107.91 2022-10-12T14:50:08 2022-10-12T14:41:11 None None
PL-GIFT 0.1197 0.15674 0.0362 1.0 67.56 2022-10-14T18:00:08 2022-10-14T17:56:35 None None
PL-GIFT 0.69315 0.0 0.25 0.5 22.11 2022-10-18T00:40:08 2022-10-18T00:32:10 :Container File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.69315 0.0 0.25 0.5 72.29 2022-10-18T16:20:08 2022-10-18T16:16:16 :Container File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.69315 0.0 0.25 0.5 72.51 2022-10-18T18:30:08 2022-10-18T18:24:03 :Container File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.69315 0.0 0.25 0.5 72.44 2022-10-18T19:10:07 2022-10-18T19:00:59 :Container File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.69315 0.0 0.25 0.5 1355.03 2022-10-18T20:00:07 2022-10-18T19:51:07 :Container File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.69315 0.0 0.25 0.5 1354.69 2022-10-18T22:30:07 2022-10-18T22:21:19 :Container File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.1197 0.15674 0.0362 1.0 67.49 2022-10-19T15:40:07 2022-10-19T15:35:59 :Container File Missing: None
PL-GIFT 0.69315 0.0 0.25 0.5 1352.16 2022-10-19T17:00:07 2022-10-19T16:55:18 :Container File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.69315 0.0 0.25 0.5 187.16 2022-10-20T20:20:07 2022-10-20T20:12:41 :Container File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.67108 0.08147 0.23929 0.70833 213.22 2022-10-20T21:40:07 2022-10-20T21:30:51 :Container File Missing: :Container Parameters (jsonschema checker):
PL-GIFT 0.10691 0.15583 0.03417 1.0 83.18 2022-10-25T23:20:07 2022-10-25T23:14:49 :Container File Missing: None
PL-GIFT 0.20959 0.21446 0.06896 1.0 74.75 2022-11-04T22:10:08 2022-11-04T22:07:34 None None
PL-GIFT 0.08938 0.13006 0.02676 1.0 83.82 2022-11-07T16:40:07 2022-11-07T16:31:25 None None
PL-GIFT 0.29009 0.32496 0.10695 1.0 83.85 2022-11-09T22:50:07 2022-11-09T22:49:06 None None
PL-GIFT 0.05746 0.06468 0.01012 1.0 83.73 2022-11-11T19:00:07 2022-11-11T18:50:19 None None
PL-GIFT 0.07544 0.12159 0.02363 1.0 83.9 2022-11-11T22:50:08 2022-11-11T22:44:14 None None
PL-GIFT 0.08143 0.13207 0.02647 1.0 83.08 2022-11-15T17:20:14 2022-11-15T17:13:41 None None
PL-GIFT 0.07852 0.10764 0.02078 1.0 83.69 2022-11-15T18:10:08 2022-11-15T18:03:02 None None
PL-GIFT 0.1361 0.21533 0.0487 1.0 84.33 2022-11-15T20:40:07 2022-11-15T20:35:27 None None
Perspecta-PurdueRutgers 0.10536 0.0 0.01 1.0 3276.01 2022-10-03T07:10:08 2022-10-03T07:03:26 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.11108 0.01063 0.01125 1.0 3335.9 2022-10-04T21:30:08 2022-10-04T21:24:00 :Container File Missing: :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.67775 0.37314 0.234 0.79167 2474.85 2022-10-14T05:30:09 2022-10-14T05:29:02 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.45803 0.17848 0.154 1.0 1074.43 2022-10-14T07:40:09 2022-10-14T07:39:48 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.10536 0.0 0.01 1.0 1702.59 2022-10-14T09:10:08 2022-10-14T09:04:24 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 2022-10-15T06:00:07 2022-10-15T05:57:13 :Executed File Update::Info File Missing::Container File Missing: None
Perspecta-PurdueRutgers 0.3308 0.40751 0.09125 0.89583 3134.32 2022-10-15T20:50:09 2022-10-15T20:47:36 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.08373 0.01642 0.007 1.0 1659.9 2022-10-18T05:40:08 2022-10-18T05:33:54 :Container File Missing: :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.10983 0.00339 0.01084 1.0 1640.69 2022-10-24T00:50:09 2022-10-24T00:41:14 :Container File Missing: :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.10983 0.00339 0.01084 1.0 5220.01 2022-11-01T05:50:09 2022-11-01T03:51:19 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.10983 0.00339 0.01084 1.0 3818.45 2022-11-01T16:10:09 2022-11-01T16:06:40 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 42.5 2022-11-03T15:50:09 2022-11-03T15:49:58 None None
Perspecta-PurdueRutgers 0.37625 0.11831 0.10675 1.0 114.68 2022-11-04T14:43:05 2022-11-03T23:08:36 None None
Perspecta-PurdueRutgers 0.10983 0.00339 0.01084 1.0 2991.91 2022-11-04T19:00:09 2022-11-04T18:57:28 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 52.54 2022-11-05T17:30:09 2022-11-05T17:28:49 :No Results: None
Perspecta-PurdueRutgers 0.34248 0.13594 0.09605 1.0 90.7 2022-11-06T20:40:09 2022-11-06T20:39:23 None None
Perspecta-PurdueRutgers 0.09044 0.01981 0.00826 1.0 2974.03 2022-11-08T18:20:14 2022-11-08T18:16:08 None :Container Parameters (learned parameters):
Perspecta-PurdueRutgers 0.55049 0.09903 0.18139 1.0 87.67 2022-11-11T16:50:10 2022-11-11T16:44:02 None None
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 2022-11-15T21:50:09 2022-11-15T21:40:30 :Executed File Update::No Results::Info File Missing::Container File Missing: None
Perspecta-PurdueRutgers 0.19949 0.15463 0.05531 1.0 95.73 2022-11-15T22:31:19 2022-11-15T22:23:56 None None
Perspecta-PurdueRutgers 0.61681 0.0716 0.21225 0.875 92.64 2022-11-16T04:00:09 2022-11-16T03:51:01 None None
Perspecta-PurdueRutgers 0.23014 0.19935 0.08043 1.0 92.42 2022-11-16T04:40:09 2022-11-16T04:31:04 None None
Perspecta-PurdueRutgers 0.24011 0.15465 0.0673 1.0 92.62 2022-11-16T16:20:09 2022-11-16T16:12:58 None None
Perspecta-PurdueRutgers 0.21181 0.16604 0.06443 1.0 92.39 2022-11-17T01:40:07 2022-11-17T01:38:36 None None
Perspecta-PurdueRutgers 0.17007 0.16432 0.05278 1.0 92.32 2022-11-17T02:40:43 2022-11-17T02:39:46 None None
Perspecta-PurdueRutgers 0.17672 0.16277 0.05366 1.0 92.7 2022-11-17T04:30:07 2022-11-17T04:20:21 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 66.27 2022-10-05T23:20:09 2022-10-05T23:19:25 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 67.69 2022-10-06T01:10:09 2022-10-06T01:00:54 None None
Perspecta-IUB 1.03784 0.81466 0.29067 0.5 152.31 2022-10-06T03:00:10 2022-10-06T02:56:53 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 68.57 2022-10-06T04:20:09 2022-10-06T04:16:53 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 71.76 2022-10-06T19:00:10 2022-10-06T18:58:43 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 1405.4 2022-10-06T21:20:09 2022-10-06T21:11:45 None None
Perspecta-IUB 0.17421 0.13729 0.048 1.0 1712.78 2022-10-06T22:50:09 2022-10-06T22:48:40 None None
Perspecta-IUB 0.15665 0.1255 0.04211 1.0 1436.15 2022-10-07T02:50:09 2022-10-07T02:42:56 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 2022-10-07T22:10:08 2022-10-07T22:10:00 :Executed File Update::Info File Missing::Container File Missing: None
Perspecta-IUB 0.00234 0.00083 1e-05 1.0 1435.5 2022-10-08T01:50:08 2022-10-08T01:41:26 None None
Perspecta-IUB 0.00787 0.0107 0.00034 1.0 1417.67 2022-10-08T02:30:10 2022-10-08T02:24:02 None None
Perspecta-IUB 0.21078 0.19553 0.06301 1.0 1501.1 2022-10-09T22:00:09 2022-10-09T21:56:39 None None
Perspecta-IUB 0.18093 0.24366 0.064 1.0 1441.74 2022-10-09T23:50:09 2022-10-09T23:46:26 None None
Perspecta-IUB 0.1863 0.22723 0.06502 1.0 187.45 2022-10-10T18:00:09 2022-10-10T17:51:55 None None
Perspecta-IUB 0.19405 0.13466 0.05351 1.0 1694.97 2022-10-11T17:50:23 2022-10-11T17:45:12 None None
Perspecta-IUB 0.19251 0.1391 0.0523 1.0 2007.69 2022-10-12T13:32:33 2022-10-12T00:48:36 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 51.85 2022-10-06T19:30:10 2022-10-06T19:25:14 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 61.97 2022-10-06T21:20:10 2022-10-06T21:16:26 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 82.39 2022-10-06T22:20:10 2022-10-06T22:16:36 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 82.34 2022-10-07T05:50:10 2022-10-07T05:43:25 None None
ARM-UCSD 0.22292 0.14573 0.058 1.0 198.3 2022-10-07T06:10:12 2022-10-07T06:05:52 None None
ARM-UCSD 0.32508 0.40856 0.09 0.91667 201.95 2022-10-07T18:20:10 2022-10-07T18:15:40 None None
ARM-UCSD 0.32508 0.40856 0.09 0.91667 415.03 2022-10-12T13:32:36 2022-10-11T23:57:26 None None
ARM 0.69315 0.0 0.25 0.5 130.58 2022-10-17T19:00:10 2022-10-17T18:54:57 None None
ARM 0.67295 0.33496 0.22489 0.83333 154.87 2022-10-17T20:29:59 2022-10-17T20:00:13 :Container File Missing: None
ARM 0.60843 0.30525 0.19867 0.83333 142.27 2022-11-01T05:50:14 2022-11-01T06:52:33 None None
ARM 0.60843 0.30525 0.19867 0.83333 142.27 2022-11-01T18:20:13 2022-11-01T18:00:29 None None
ARM 0.60843 0.30525 0.19867 0.83333 142.02 2022-11-01T20:50:13 2022-11-01T20:40:16 None None
ARM 0.69315 0.0 0.25 0.5 124.73 2022-11-28T04:40:17 2022-11-28T04:39:02 :No Results: None
ARM 0.17945 0.03395 0.02873 1.0 142.38 2022-11-28T05:10:16 2022-11-28T05:03:08 None None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 98.59 2022-10-21T01:30:12 2022-10-21T01:22:39 :Container File Missing: None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 98.57 2022-10-21T02:30:12 2022-10-21T02:26:54 :Container File Missing: None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 98.6 2022-10-21T03:10:11 2022-10-21T03:08:37 :Container File Missing: None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 98.35 2022-10-31T21:30:12 2022-10-31T21:14:17 None None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 98.61 2022-11-01T16:30:56 2022-11-01T16:26:24 None None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 97.6 2022-11-01T18:20:14 2022-11-01T17:53:57 None None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 98.6 2022-11-01T19:00:14 2022-11-01T18:56:14 None None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 132.7 2022-11-01T19:30:13 2022-11-01T19:22:07 None None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 109.62 2022-11-01T22:10:14 2022-11-01T22:03:34 None None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 112.62 2022-11-29T02:30:17 2022-11-29T02:28:33 :No Results: None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 42.27 2022-11-29T03:31:13 2022-11-29T03:08:42 :No Results: None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 42.05 2022-11-29T04:10:17 2022-11-29T04:01:50 :No Results: None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 42.36 2022-11-29T04:40:17 2022-11-29T04:33:20 :No Results: None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 42.02 2022-11-29T05:10:17 2022-11-29T05:01:48 :No Results: None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 107.51 2022-11-29T05:41:01 2022-11-29T05:36:40 :Result Parse::No Results: None
TrinitySRITrojAI-SBU 0.3605 0.18267 0.12165 1.0 113.01 2022-11-29T18:30:17 2022-11-29T18:27:38 None None
ICSI-2 0.10425 0.02226 0.01071 1.0 215.83 2022-09-26T03:30:13 2022-09-26T03:21:28 None None
ICSI-2 0.40206 0.252 0.13958 1.0 114.13 2022-10-03T02:50:23 2022-10-03T02:41:48 None None
ICSI-2 0.05906 0.02135 0.00423 1.0 109.83 2022-10-04T04:30:12 2022-10-04T04:29:58 None None
ICSI-2 0.06352 0.02138 0.00473 1.0 137.33 2022-10-10T08:20:15 2022-10-10T08:18:51 None None
ICSI-2 0.04803 0.0323 0.00445 1.0 139.15 2022-10-11T20:20:15 2022-10-11T20:10:41 None None
ICSI-2 0.17217 0.20058 0.05687 1.0 165.56 2022-10-13T01:30:14 2022-10-13T01:29:31 None None
ICSI-2 0.14005 0.14438 0.04125 1.0 143.38 2022-10-14T01:50:17 2022-10-13T20:29:28 None None
ICSI-2 0.15442 0.16011 0.04867 1.0 137.59 2022-10-14T23:20:23 2022-10-14T23:17:41 None None
ICSI-2 0.12037 0.12021 0.03196 1.0 137.37 2022-10-15T21:00:13 2022-10-15T20:53:40 None None
ICSI-2 0.03534 0.02349 0.00244 1.0 136.26 2022-10-16T05:40:14 2022-10-16T05:35:09 None None
ICSI-2 0.04256 0.03763 0.00458 1.0 138.34 2022-10-25T21:10:14 2022-10-25T21:10:02 :Container File Missing: None
ICSI-2 0.03431 0.03022 0.0031 1.0 174.26 2022-10-26T06:40:14 2022-10-26T06:39:30 :Container File Missing: None
ICSI-2 0.02791 0.02287 0.00197 1.0 130.53 2022-11-09T18:50:19 2022-11-09T18:47:04 None None
ICSI-2 0.01681 0.01354 0.00072 1.0 126.82 2022-11-14T23:30:20 2022-11-14T23:24:02 None None
ICSI-2 0.04539 0.04701 0.00633 1.0 1028.83 2022-11-22T10:20:19 2022-11-22T10:19:46 None None
ICSI-2 0.03404 0.0326 0.0034 1.0 411.35 2022-11-23T00:30:21 2022-11-23T00:27:22 None None
ICSI-2 0.04264 0.04659 0.00602 1.0 2105.16 2022-11-23T23:10:20 2022-11-23T23:08:22 None None
ICSI-2 0.24397 0.1877 0.08067 0.95833 6000.69 2022-12-04T00:00:21 2022-12-02T02:30:32 None :Timeout:
TrinitySRITrojAI-BostonU 0.69315 0.0 0.25 0.5 52.38 2022-10-28T19:30:15 2022-10-28T19:27:45 :Container File Missing: None
TrinitySRITrojAI-BostonU 0.69315 0.0 0.25 0.5 52.33 2022-10-29T05:10:15 2022-10-28T21:13:14 :Container File Missing: None
TrinitySRITrojAI-BostonU 0.49047 0.12879 0.15734 1.0 224.28 2022-10-29T06:50:15 2022-10-29T06:43:21 :Container File Missing: None
TrinitySRITrojAI-BostonU 0.26979 0.10317 0.06495 1.0 319.35 2022-10-31T20:10:18 2022-10-31T20:01:29 None None
TrinitySRITrojAI-BostonU 0.693 0.00044 0.24993 0.5 328.73 2022-11-02T02:10:19 2022-11-02T02:00:29 None None
TrinitySRITrojAI-BostonU 0.31027 0.02827 0.07185 1.0 315.39 2022-11-02T06:30:54 2022-11-02T06:29:02 None None
TrinitySRITrojAI-BostonU 0.31027 0.02827 0.07185 1.0 320.59 2022-11-02T19:00:21 2022-11-02T18:56:27 None None
TrinitySRITrojAI-BostonU 0.26979 0.10317 0.06495 1.0 325.11 2022-11-03T05:50:22 2022-11-03T05:47:16 None None
trojai-example 1.25867 0.63594 0.4195 0.375 131.05 2022-09-21T17:40:14 2022-09-21T17:39:18 None None
trojai-example 0.69315 0.0 0.25 0.5 0.01 2022-09-27T16:50:18 2022-09-27T16:44:39 :Container File Missing: :Execute::Copy Out:
trojai-example 0.98262 0.52056 0.35336 0.54167 130.01 2022-09-27T17:10:18 2022-09-27T17:02:51 None None
trojai-example 0.69315 0.0 0.25 0.5 2022-09-28T18:00:18 2022-09-28T18:00:05 :Info File Missing::Container File Missing: None
trojai-example 0.65689 0.22961 0.23136 0.66667 132.28 2022-09-28T18:20:18 2022-09-28T18:16:11 None None
trojai-example 0.9352 0.3863 0.34494 0.47917 152.02 2022-09-28T19:10:18 2022-09-28T19:00:39 None None
trojai-example 0.61578 0.27849 0.2172 0.75 155.32 2022-09-28T21:20:18 2022-09-28T21:11:57 None None
trojai-example 0.55684 0.24684 0.19247 0.79167 152.1 2022-09-30T17:00:19 2022-09-30T16:59:47 None None
trojai-example 1.07807 0.47955 0.38411 0.33333 152.1 2022-10-19T13:30:19 2022-10-19T13:28:02 :Container File Missing: None
IIECAS 0.69315 0.0 0.25 0.5 61.07 2022-11-17T04:50:26 2022-11-17T05:03:38 :No Results: None
IIECAS 8.42794 7.79351 0.35 0.5 240.68 2022-11-17T07:50:28 2022-11-17T07:49:03 None None
IIECAS 13.81552 8.56295 0.5 0.16667 247.73 2022-11-17T11:50:28 2022-11-17T11:46:34 None None
IIECAS 1.20397 0.68093 0.41 0.58333 392.51 2022-11-18T05:00:26 2022-11-18T04:59:14 None None
IIECAS 0.69315 0.0 0.25 0.5 59.03 2022-11-18T08:00:26 2022-11-18T07:53:01 :No Results: None
IIECAS 0.69315 0.0 0.25 0.5 59.97 2022-11-18T09:50:27 2022-11-18T09:45:38 :No Results: None
IIECAS 1.20397 0.68093 0.41 0.58333 3416.16 2022-11-18T11:00:28 2022-11-18T10:52:48 None None
IIECAS 0.57559 0.14573 0.202 0.75 404.2 2022-11-23T10:00:30 2022-11-23T09:58:26 None None
IIECAS 0.57559 0.14573 0.202 0.75 424.99 2022-11-23T13:30:30 2022-11-23T13:24:16 None None
IIECAS 0.69315 0.0 0.25 0.5 2022-11-23T14:00:30 2022-11-23T13:50:58 :No Results::Info File Missing::Container File Missing: None
IIECAS 0.57559 0.14573 0.202 0.75 412.25 2022-11-23T16:10:30 2022-11-23T16:10:06 None None
IIECAS 0.57559 0.14573 0.202 0.75 426.15 2022-11-23T17:50:30 2022-11-23T17:47:26 None None
IIECAS 0.57559 0.14573 0.202 0.75 513.69 2022-11-24T04:40:29 2022-11-24T04:31:39 None None
IIECAS 0.69315 0.0 0.25 0.5 2022-11-24T12:00:30 2022-11-24T11:55:02 :No Results::Info File Missing::Container File Missing: None
IIECAS 0.57559 0.14573 0.202 0.75 496.63 2022-11-24T12:40:29 2022-11-24T12:34:38 None None
IIECAS 0.57559 0.14573 0.202 0.75 459.27 2022-11-24T17:10:30 2022-11-24T17:05:42 None None
IIECAS 0.69315 0.0 0.25 0.5 92.23 2022-11-26T08:10:30 2022-11-26T08:06:40 :No Results: None
IIECAS 0.69315 0.0 0.25 0.5 2022-11-26T09:30:30 2022-11-26T09:28:44 :No Results::Info File Missing::Container File Missing: None
IIECAS 0.69315 0.0 0.25 0.5 2022-11-26T10:30:30 2022-11-26T10:21:12 :No Results::Info File Missing::Container File Missing: None
IIECAS 0.69315 0.0 0.25 0.5 91.97 2022-11-26T11:20:30 2022-11-26T11:12:32 :No Results: None
IIECAS 0.57559 0.14573 0.202 0.75 464.97 2022-11-26T12:30:32 2022-11-26T12:28:44 None None
IIECAS 0.51681 0.16695 0.178 0.79167 462.64 2022-11-26T14:20:30 2022-11-26T14:18:48 None None
IIECAS 0.69315 0.0 0.25 0.5 2022-11-26T15:30:29 2022-11-26T15:30:17 :No Results::Info File Missing::Container File Missing: None
IIECAS 0.51681 0.16695 0.178 0.79167 511.03 2022-11-26T16:30:29 2022-11-26T16:25:20 None None
IIECAS 0.51681 0.16695 0.178 0.79167 1429.5 2022-11-27T03:30:29 2022-11-27T03:29:03 None None
IIECAS 0.54481 0.54474 0.17 0.75 2576.02 2022-11-27T06:40:36 2022-11-27T06:39:55 None None
IIECAS 0.54481 0.54474 0.17 0.75 1896.66 2022-11-27T11:10:28 2022-11-27T11:02:27 None None
IIECAS 0.54481 0.54474 0.17 0.75 1830.4 2022-11-28T14:20:31 2022-11-28T14:16:24 None None
UMBCb 0.588 0.1539 0.20394 0.70833 133.11 2022-10-31T18:40:25 2022-10-31T18:31:33 :Container File Missing: None
UMBCb 7.89749 6.54432 0.53367 0.58333 193.37 2022-11-02T20:10:30 2022-11-02T17:11:22 None None

Placeholder text for image-classification-jun2020

Placeholder image

Placeholder image description

Accepting submissions: False
Number of models in image-classification-jun2020, test: 100

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
IceTorch 0.32804 0.12372 0.09454 0.945 2020-07-24T04:20:01 2020-07-24T04:17:54 None None
Cassandra-XF 0.34258 0.10809 0.0998 0.917 2020-07-25T03:50:01 2020-07-25T03:46:30 None None
trojaicy 0.34646 0.12179 0.1002 0.9076 2020-07-25T20:30:02 2020-07-25T20:27:15 None None
Hector 0.44008 0.11423 0.13852 0.8734 2020-07-14T00:10:01 2020-07-14T00:09:58 None None
Perspecta 0.30311 0.12325 0.082 0.91 2020-07-25T15:30:01 2020-07-25T15:20:50 None None
ICSI-1 0.5909 0.13032 0.19967 0.7746 2020-07-26T03:00:01 2020-07-26T02:52:23 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
IceTorch 0.4268 0.18144 0.11967 0.8784 2020-07-10T01:00:01 2020-07-10T00:56:49 None None
IceTorch 0.43448 0.15057 0.13475 0.8796 2020-07-20T16:10:01 2020-07-20T16:02:52 None None
IceTorch 0.32804 0.12372 0.09454 0.945 2020-07-24T04:20:01 2020-07-24T04:17:54 None None
IceTorch 0.39232 0.21124 0.09074 0.934 2020-07-25T03:00:02 2020-07-25T02:58:48 None None
IceTorch 0.96762 0.1462 0.3559 0.2044 2020-07-25T18:10:02 2020-07-25T18:03:42 None None
IceTorch 0.381 0.17337 0.1019 0.9292 2020-07-26T06:20:02 2020-07-26T00:48:05 None None
IceTorch 0.36499 0.15063 0.10139 0.9314 2020-07-26T21:10:02 2020-07-26T21:01:48 None None
IceTorch 0.34106 0.12587 0.09859 0.9392 2020-07-27T09:20:01 2020-07-27T04:34:41 None None
Cassandra-XF 0.69315 0.0 0.25 0.5 2020-06-16T22:36:01 2020-06-10T21:12:53 :No Results: None
Cassandra-XF 0.5237 0.29714 0.1091 0.9144 2020-07-14T18:20:01 2020-07-14T18:18:52 None None
Cassandra-XF 0.51984 0.29784 0.10587 0.9136 2020-07-15T21:10:02 2020-07-15T21:05:53 None None
Cassandra-XF 0.61117 0.32676 0.12041 0.9004 2020-07-17T01:20:02 2020-07-17T01:10:25 None None
Cassandra-XF 0.36903 0.13995 0.106 0.88 2020-07-18T01:20:02 2020-07-17T19:35:57 None None
Cassandra-XF 0.43494 0.15377 0.13 0.85 2020-07-19T01:20:02 2020-07-18T17:07:24 None None
Cassandra-XF 0.41297 0.14943 0.122 0.86 2020-07-20T03:10:01 2020-07-19T19:44:20 None None
Cassandra-XF 0.67284 0.32996 0.14522 0.8938 2020-07-21T03:10:02 2020-07-20T21:38:03 None None
Cassandra-XF 0.69918 0.36595 0.12414 0.8836 2020-07-22T06:10:01 2020-07-22T06:05:07 None None
Cassandra-XF 0.34258 0.10809 0.0998 0.917 2020-07-25T03:50:01 2020-07-25T03:46:30 None None
Cassandra-XF 0.35834 0.12192 0.1033 0.9014 2020-07-25T19:10:02 2020-07-25T19:09:21 None None
Cassandra-XF 0.37913 0.13518 0.1102 0.8808 2020-07-27T02:20:01 2020-07-27T02:19:50 None None
trojaicy 0.69239 0.16767 0.22578 0.7198 2020-07-10T03:10:02 2020-07-10T03:06:38 None None
trojaicy 0.61689 0.23688 0.16614 0.835 2020-07-11T13:20:02 2020-07-11T13:18:18 None None
trojaicy 0.71348 0.23725 0.20364 0.7726 2020-07-12T21:10:02 2020-07-12T21:08:08 None None
trojaicy 0.55121 0.32009 0.10378 0.9134 2020-07-13T21:20:01 2020-07-13T19:55:04 None None
trojaicy 0.69315 0.0 0.25 0.5 2020-07-21T03:30:02 2020-07-21T03:29:43 :No Results: None
trojaicy 0.38078 0.14003 0.1108 0.8768 2020-07-22T06:00:01 2020-07-22T05:47:05 :Result Parse: None
trojaicy 0.36299 0.12927 0.1047 0.8916 2020-07-23T17:50:01 2020-07-23T17:40:49 None None
trojaicy 0.58861 0.32343 0.11428 0.9012 2020-07-25T05:50:02 2020-07-25T05:42:28 None None
trojaicy 0.34646 0.12179 0.1002 0.9076 2020-07-25T20:30:02 2020-07-25T20:27:15 None None
trojaicy 0.69315 0.0 0.25 0.5 2020-07-27T01:50:02 2020-07-27T01:44:11 :No Results: :Execute:
Hector 0.44008 0.11423 0.13852 0.8734 2020-07-14T00:10:01 2020-07-14T00:09:58 None None
Perspecta 0.34706 0.13475 0.098 0.89 2020-07-23T00:40:02 2020-07-23T00:35:21 None None
Perspecta 0.73828 0.14824 0.26292 0.842 2020-07-23T21:00:01 2020-07-23T15:01:45 None None
Perspecta 0.69315 0.0 0.25 0.5 2020-07-24T23:40:01 2020-07-24T20:38:10 :Executed File Update::No Results: None
Perspecta 2.29593 0.5863 0.424 0.738 2020-07-25T01:00:01 2020-07-24T20:38:10 None None
Perspecta 0.30311 0.12325 0.082 0.91 2020-07-25T15:30:01 2020-07-25T15:20:50 None None
Perspecta 0.42635 0.15485 0.12625 0.8974 2020-07-27T07:40:01 2020-07-27T07:35:51 None None
ICSI-1 0.69315 0.0 0.25 0.5 2020-07-23T22:00:01 2020-07-23T21:56:13 :No Results: :Execute:
ICSI-1 0.65845 0.1762 0.21782 0.7888 2020-07-25T10:50:02 2020-07-25T10:42:42 None None
ICSI-1 0.5909 0.13032 0.19967 0.7746 2020-07-26T03:00:01 2020-07-26T02:52:23 None None
ICSI-1 0.69315 0.0 0.25 0.5 2020-07-27T09:00:01 2020-07-27T08:53:42 :No Results: :Execute:

Placeholder text for image-classification-aug2020

Placeholder image

Placeholder image description

Accepting submissions: False
Number of models in image-classification-aug2020, test: 144

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
IceTorch 0.69315 0.0 0.25 0.5 2020-08-03T22:10:01 2020-07-27T04:34:41 None None
trojaicy 0.69315 0.0 0.25 0.5 2020-08-03T22:10:01 2020-07-27T01:44:11 None None
Hector 0.70232 0.01791 0.25348 0.49306 2020-08-03T22:10:01 2020-07-14T00:09:58 None None
Cassandra-XF 0.69315 0.0 0.25 0.5 2020-08-03T22:10:01 2020-07-27T19:44:57 None None
PL-GIFT 0.51319 0.09327 0.17333 0.80797 2020-09-25T17:00:01 2020-09-25T13:37:02 None None
ICSI-1 0.5894 0.064 0.20262 0.74498 2020-09-21T21:30:01 2020-09-21T21:26:03 None None
Perspecta-PurdueRutgers 0.32409 0.09493 0.09745 0.89198 2020-10-25T12:20:02 2020-10-25T12:11:37 None None
TrinitySRITrojAI 0.46688 0.11095 0.15378 0.87876 2020-10-23T16:20:02 2020-10-23T16:18:41 None None
DRAFFT 0.63337 0.07392 0.22446 0.68895 2020-08-21T00:20:02 2020-08-21T00:12:48 None None
NSEC 0.69315 0.0 0.25 0.5 2020-08-20T15:00:01 2020-08-20T14:50:21 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 2020-08-28T17:30:01 2020-08-28T17:23:51 None None
ARM-UCSD 0.66982 0.10001 0.23435 0.68393 2020-09-22T04:10:01 2020-09-22T04:06:42 None None
trojai-example 2.37711 1.06703 0.30121 0.50492 2020-09-22T15:20:02 2020-09-22T15:19:25 None None
Perspecta 0.69315 0.0 0.25 0.5 2020-10-05T05:40:01 2020-10-05T05:39:56 None None
ICSI-2 1.00108 0.24182 0.3059 0.64998 2020-10-23T22:30:01 2020-10-23T22:23:46 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
IceTorch 0.69315 0.0 0.25 0.5 2020-08-03T22:10:01 2020-07-27T04:34:41 None None
trojaicy 0.69315 0.0 0.25 0.5 2020-08-03T22:10:01 2020-07-27T01:44:11 None None
Hector 0.70232 0.01791 0.25348 0.49306 2020-08-03T22:10:01 2020-07-14T00:09:58 None None
Cassandra-XF 0.69315 0.0 0.25 0.5 2020-08-03T22:10:01 2020-07-27T19:44:57 None None
PL-GIFT 0.74453 0.04917 0.27348 0.48351 2020-08-03T22:10:01 2020-08-01T14:03:40 None None
PL-GIFT 0.65358 0.05216 0.23289 0.66049 2020-08-19T20:30:02 2020-08-19T20:24:12 None None
PL-GIFT 0.61533 0.07759 0.21652 0.70737 2020-08-27T16:10:01 2020-08-27T16:00:43 None None
PL-GIFT 0.73021 0.13665 0.24875 0.7689 2020-09-04T01:40:01 2020-09-04T01:30:06 None None
PL-GIFT 0.61661 0.09319 0.21305 0.72029 2020-09-06T13:20:02 2020-09-06T13:15:24 None None
PL-GIFT 0.82929 0.37312 0.23541 0.65837 2020-09-06T19:20:02 2020-09-06T14:57:00 None None
PL-GIFT 0.5652 0.07705 0.19812 0.76669 2020-09-22T02:00:02 2020-09-22T01:59:42 None None
PL-GIFT 0.54447 0.07217 0.18936 0.78115 2020-09-22T08:10:01 2020-09-22T03:42:38 None None
PL-GIFT 0.54969 0.09441 0.1853 0.79215 2020-09-24T03:10:02 2020-09-24T03:00:31 None None
PL-GIFT 0.69315 0.0 0.25 0.5 2020-09-25T16:20:01 2020-09-25T13:37:02 None None
PL-GIFT 0.51319 0.09327 0.17333 0.80797 2020-09-25T17:00:01 2020-09-25T13:37:02 None None
PL-GIFT 0.69315 0.0 0.25 0.5 2020-10-17T14:00:01 2020-10-17T13:56:41 None None
PL-GIFT 0.52013 0.08688 0.17689 0.80855 2020-10-18T15:20:02 2020-10-18T15:12:33 None None
ICSI-1 0.71562 0.04073 0.2602 0.50444 2020-08-13T06:30:01 2020-08-13T06:24:58 None None
ICSI-1 0.64664 0.04582 0.22895 0.65394 2020-08-17T15:50:02 2020-08-17T07:01:30 None None
ICSI-1 0.73677 0.11248 0.25512 0.64062 2020-08-19T00:30:02 2020-08-19T00:21:03 None None
ICSI-1 0.66942 0.05017 0.23832 0.63387 2020-08-19T20:10:01 2020-08-19T20:01:19 None None
ICSI-1 0.63847 0.0475 0.22461 0.67197 2020-08-22T01:30:02 2020-08-22T01:25:46 None None
ICSI-1 0.69407 0.07813 0.24694 0.68162 2020-08-24T08:10:01 2020-08-24T08:04:36 None None
ICSI-1 0.62521 0.05758 0.22064 0.68441 2020-08-25T06:30:01 2020-08-25T06:25:14 None None
ICSI-1 0.62624 0.0483 0.22073 0.68056 2020-08-27T22:00:02 2020-08-27T21:52:05 None None
ICSI-1 0.62716 0.04783 0.22099 0.67834 2020-09-01T09:40:02 2020-09-01T09:35:28 None None
ICSI-1 0.61755 0.05496 0.2166 0.70052 2020-09-06T23:00:01 2020-09-06T22:58:31 None None
ICSI-1 0.61737 0.04614 0.21504 0.71103 2020-09-07T10:40:01 2020-09-07T10:39:22 None None
ICSI-1 0.63036 0.0421 0.22089 0.69271 2020-09-08T05:20:01 2020-09-08T05:13:28 None None
ICSI-1 0.62336 0.05419 0.21887 0.68924 2020-09-10T08:00:01 2020-09-10T07:55:01 None None
ICSI-1 0.61898 0.05777 0.21524 0.71422 2020-09-17T08:20:01 2020-09-17T08:16:58 None None
ICSI-1 0.5894 0.064 0.20262 0.74498 2020-09-21T21:30:01 2020-09-21T21:26:03 None None
ICSI-1 0.61131 0.06609 0.21142 0.72039 2020-09-22T21:10:02 2020-09-22T21:09:48 None None
ICSI-1 0.60917 0.0679 0.21053 0.72483 2020-09-23T22:30:01 2020-09-23T22:23:31 None None
ICSI-1 0.60464 0.05746 0.20908 0.72685 2020-09-24T23:20:02 2020-09-24T23:15:09 None None
ICSI-1 0.60233 0.06324 0.20796 0.72897 2020-09-26T04:40:02 2020-09-26T04:33:47 None None
ICSI-1 0.59895 0.06223 0.20659 0.73274 2020-09-27T04:40:02 2020-09-27T04:33:00 None None
ICSI-1 0.60216 0.06161 0.20834 0.72637 2020-09-27T21:00:01 2020-09-27T20:58:30 None None
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 2020-08-14T04:50:01 2020-08-14T04:42:07 None None
Perspecta-PurdueRutgers 0.68518 0.15817 0.22111 0.73611 2020-08-16T16:30:01 2020-08-16T16:28:47 None None
Perspecta-PurdueRutgers 0.48015 0.10342 0.14972 0.826 2020-08-18T04:40:01 2020-08-18T04:32:48 None None
Perspecta-PurdueRutgers 0.64054 0.10232 0.22083 0.69898 2020-08-20T22:40:02 2020-08-20T22:12:54 None None
Perspecta-PurdueRutgers 0.66599 0.10557 0.23167 0.68056 2020-08-21T08:40:02 2020-08-21T08:38:08 None None
Perspecta-PurdueRutgers 0.69657 0.12521 0.24243 0.73466 2020-08-23T07:50:02 2020-08-23T07:45:20 None None
Perspecta-PurdueRutgers 0.65962 0.10479 0.22896 0.68557 2020-08-25T06:10:01 2020-08-25T06:01:43 None None
Perspecta-PurdueRutgers 0.53121 0.09414 0.17333 0.77778 2020-08-28T06:00:01 2020-08-28T05:54:53 None None
Perspecta-PurdueRutgers 0.52389 0.11506 0.16833 0.79167 2020-08-29T03:00:01 2020-08-29T02:57:03 None None
Perspecta-PurdueRutgers 0.62026 0.12487 0.20722 0.73611 2020-08-29T13:50:02 2020-08-29T13:41:13 None None
Perspecta-PurdueRutgers 0.50641 0.12046 0.16 0.81597 2020-09-01T04:20:01 2020-09-01T04:16:06 None None
Perspecta-PurdueRutgers 0.5546 0.12567 0.17944 0.7934 2020-09-01T15:50:01 2020-09-01T15:49:20 None None
Perspecta-PurdueRutgers 0.76947 0.08011 0.27979 0.53511 2020-09-03T03:30:01 2020-09-03T03:21:59 None None
Perspecta-PurdueRutgers 0.88138 0.11057 0.32521 0.50916 2020-09-05T00:50:02 2020-09-05T00:41:13 None None
Perspecta-PurdueRutgers 0.52389 0.11506 0.16833 0.79167 2020-09-07T03:40:01 2020-09-07T03:35:30 None None
Perspecta-PurdueRutgers 0.50412 0.11182 0.15736 0.81607 2020-09-08T03:50:01 2020-09-08T03:42:43 None None
Perspecta-PurdueRutgers 0.52419 0.11358 0.1659 0.81289 2020-09-09T03:20:02 2020-09-09T03:19:55 None None
Perspecta-PurdueRutgers 0.46807 0.10251 0.14493 0.83941 2020-09-12T04:50:01 2020-09-12T04:45:09 None None
Perspecta-PurdueRutgers 0.41512 0.09434 0.12292 0.86285 2020-09-13T05:40:01 2020-09-13T05:35:06 None None
Perspecta-PurdueRutgers 0.38244 0.07411 0.10949 0.88985 2020-09-14T03:10:01 2020-09-14T03:08:58 None None
Perspecta-PurdueRutgers 0.39206 0.07592 0.11366 0.89236 2020-09-21T14:30:01 2020-09-21T14:28:18 None None
Perspecta-PurdueRutgers 0.41132 0.07932 0.12199 0.86651 2020-09-23T05:30:01 2020-09-23T05:21:35 None None
Perspecta-PurdueRutgers 0.41132 0.07932 0.12199 0.87886 2020-09-24T04:00:01 2020-09-24T04:00:02 None None
Perspecta-PurdueRutgers 0.47871 0.08937 0.15116 0.8532 2020-09-25T03:30:02 2020-09-25T03:21:56 None None
Perspecta-PurdueRutgers 0.48833 0.0906 0.15532 0.86806 2020-09-27T05:50:01 2020-09-27T05:41:56 None None
Perspecta-PurdueRutgers 0.45945 0.08676 0.14282 0.86362 2020-09-27T19:50:01 2020-09-27T19:42:40 None None
Perspecta-PurdueRutgers 0.44983 0.08538 0.13866 0.85494 2020-09-28T12:00:01 2020-09-28T11:51:32 None None
Perspecta-PurdueRutgers 0.39823 0.11555 0.11939 0.88175 2020-09-29T06:20:02 2020-09-29T06:13:19 None None
Perspecta-PurdueRutgers 0.40069 0.10161 0.12659 0.87674 2020-09-30T02:10:02 2020-09-30T02:02:13 None None
Perspecta-PurdueRutgers 0.35389 0.09323 0.10767 0.90027 2020-10-01T04:10:01 2020-10-01T04:07:56 None None
Perspecta-PurdueRutgers 0.33545 0.09132 0.09952 0.90702 2020-10-01T19:20:01 2020-10-01T19:19:13 None None
Perspecta-PurdueRutgers 0.45203 0.09199 0.14841 0.83102 2020-10-05T23:30:01 2020-10-05T23:21:27 None None
Perspecta-PurdueRutgers 0.35885 0.09588 0.10898 0.89525 2020-10-08T16:10:01 2020-10-08T16:02:38 None None
Perspecta-PurdueRutgers 0.37676 0.10114 0.11534 0.8831 2020-10-09T04:30:02 2020-10-09T04:29:05 None None
Perspecta-PurdueRutgers 0.34839 0.09432 0.10457 0.89988 2020-10-09T16:50:01 2020-10-09T16:45:57 None None
Perspecta-PurdueRutgers 0.3897 0.10372 0.12039 0.87596 2020-10-10T05:20:01 2020-10-10T05:19:24 None None
Perspecta-PurdueRutgers 0.41665 0.10184 0.13409 0.87249 2020-10-11T01:20:01 2020-10-11T01:19:21 None None
Perspecta-PurdueRutgers 0.35885 0.09588 0.10898 0.89525 2020-10-11T23:50:01 2020-10-11T23:49:22 None None
Perspecta-PurdueRutgers 0.40061 0.09609 0.12574 0.88127 2020-10-13T07:10:02 2020-10-13T07:00:54 None None
Perspecta-PurdueRutgers 0.37126 0.10217 0.11224 0.88272 2020-10-14T05:40:01 2020-10-14T05:31:54 None None
Perspecta-PurdueRutgers 0.39768 0.10386 0.12414 0.87384 2020-10-15T05:10:01 2020-10-15T05:02:18 None None
Perspecta-PurdueRutgers 0.3608 0.10074 0.10784 0.88735 2020-10-16T04:30:01 2020-10-16T04:27:25 None None
Perspecta-PurdueRutgers 0.3608 0.10074 0.10784 0.88735 2020-10-18T04:20:01 2020-10-18T04:11:15 None None
Perspecta-PurdueRutgers 0.49862 0.08864 0.16906 0.86111 2020-10-19T06:10:02 2020-10-19T06:00:38 None None
Perspecta-PurdueRutgers 0.40793 0.10505 0.12805 0.86863 2020-10-20T02:10:01 2020-10-20T02:03:56 None None
Perspecta-PurdueRutgers 0.44254 0.1066 0.1442 0.85822 2020-10-21T05:40:02 2020-10-21T05:32:25 None None
Perspecta-PurdueRutgers 0.3663 0.09971 0.11093 0.88773 2020-10-22T05:40:01 2020-10-22T05:31:32 None None
Perspecta-PurdueRutgers 0.35074 0.11116 0.10525 0.88735 2020-10-23T13:50:01 2020-10-23T13:46:25 None None
Perspecta-PurdueRutgers 0.34162 0.1001 0.10333 0.88735 2020-10-24T14:20:02 2020-10-24T14:14:56 None None
Perspecta-PurdueRutgers 0.32409 0.09493 0.09745 0.89198 2020-10-25T12:20:02 2020-10-25T12:11:37 None None
TrinitySRITrojAI 0.67896 0.00715 0.24292 0.67361 2020-08-15T05:40:01 2020-08-15T05:36:34 None None
TrinitySRITrojAI 0.69708 0.0169 0.25196 0.63194 2020-08-28T04:50:02 2020-08-28T04:44:12 None None
TrinitySRITrojAI 1.9776 0.67208 0.32842 0.64034 2020-08-30T00:50:02 2020-08-30T00:40:10 None None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 2020-08-31T02:00:01 2020-08-31T01:26:31 None None
TrinitySRITrojAI 0.68964 0.02147 0.24819 0.57147 2020-08-31T17:50:01 2020-08-31T17:43:33 None None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 2020-09-07T18:10:02 2020-09-07T18:03:16 None None
TrinitySRITrojAI 0.62102 0.03672 0.21857 0.68789 2020-09-08T00:10:02 2020-09-07T18:57:37 None None
TrinitySRITrojAI 0.69801 0.01276 0.25241 0.48785 2020-09-10T01:40:02 2020-09-10T01:34:48 None None
TrinitySRITrojAI 7.32527 1.74856 0.43641 0.62288 2020-09-10T16:30:02 2020-09-10T16:26:33 None None
TrinitySRITrojAI 0.58754 0.05188 0.20588 0.71258 2020-09-10T22:40:01 2020-09-10T20:06:36 None None
TrinitySRITrojAI 0.55538 0.06299 0.1907 0.77672 2020-09-15T13:10:01 2020-09-15T13:09:20 None None
TrinitySRITrojAI 1.0982 0.55522 0.22226 0.75039 2020-10-09T18:50:01 2020-10-09T18:43:53 None None
TrinitySRITrojAI 0.62068 0.09287 0.21341 0.72926 2020-10-19T19:50:02 2020-10-19T19:44:59 None None
TrinitySRITrojAI 0.73415 0.0955 0.25311 0.60542 2020-10-20T17:30:02 2020-10-20T17:29:22 None None
TrinitySRITrojAI 0.66241 0.08429 0.22983 0.63638 2020-10-21T06:20:02 2020-10-21T06:16:30 None None
TrinitySRITrojAI 0.67071 0.1832 0.19729 0.7827 2020-10-21T22:50:02 2020-10-21T22:43:35 None None
TrinitySRITrojAI 0.46688 0.11095 0.15378 0.87876 2020-10-23T16:20:02 2020-10-23T16:18:41 None None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 2020-10-24T16:00:01 2020-10-24T15:59:22 None None
TrinitySRITrojAI 0.52591 0.14695 0.16624 0.87201 2020-10-24T22:00:02 2020-10-24T17:06:59 None None
DRAFFT 0.6385 0.07314 0.22751 0.67477 2020-08-20T02:50:02 2020-08-20T02:44:52 None None
DRAFFT 0.63337 0.07392 0.22446 0.68895 2020-08-21T00:20:02 2020-08-21T00:12:48 None None
DRAFFT 0.65754 0.09633 0.22863 0.68818 2020-09-21T13:40:02 2020-09-19T21:01:10 None None
NSEC 0.69315 0.0 0.25 0.5 2020-08-20T15:00:01 2020-08-20T14:50:21 None None
NSEC 0.69315 0.0 0.25 0.5 2020-09-12T02:30:01 2020-09-12T02:27:46 None None
NSEC 0.69315 0.0 0.25 0.5 2020-09-13T06:00:01 2020-09-13T05:58:54 None None
NSEC 0.69315 0.0 0.25 0.5 2020-09-14T11:30:02 2020-09-14T11:22:59 None None
NSEC 0.94321 0.1389 0.32712 0.51881 2020-09-16T10:50:01 2020-09-16T10:43:23 None None
NSEC 0.75944 0.0884 0.2765 0.50048 2020-09-18T08:10:01 2020-09-18T08:00:30 None None
NSEC 10.00741 2.13212 0.43403 0.37172 2020-10-09T00:50:01 2020-10-09T00:47:52 None None
NSEC 8.87537 2.06589 0.39931 0.50829 2020-10-11T11:20:01 2020-10-11T11:17:50 None None
NSEC 14.58304 2.25305 0.52778 0.34761 2020-10-15T06:20:01 2020-10-15T06:14:14 None None
NSEC 12.47234 2.24584 0.45139 0.44155 2020-10-17T13:50:01 2020-10-17T13:16:52 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 2020-08-28T17:30:01 2020-08-28T17:23:51 None None
Perspecta-IUB 2.14638 1.00992 inf 0.5 2020-09-01T01:30:01 2020-09-01T01:24:57 None None
Perspecta-IUB 3.38847 1.35084 2.8958608832720736e+34 0.52228 2020-09-02T02:40:01 2020-09-01T05:24:26 None None
Perspecta-IUB 7.58514 1.96096 1179015.16022 0.52855 2020-09-03T06:20:01 2020-09-03T06:16:40 None None
Perspecta-IUB 4.0554 1.40854 0.33079 0.57542 2020-09-04T07:30:01 2020-09-03T16:47:13 None None
Perspecta-IUB 4.08947 1.39508 0.35647 0.52778 2020-09-06T01:20:02 2020-09-06T01:16:35 None None
Perspecta-IUB 6.19687 1.75088 0.36225 0.66705 2020-09-08T03:00:02 2020-09-08T02:55:21 None None
Perspecta-IUB 6.633 1.79682 0.37516 0.62288 2020-09-08T19:40:02 2020-09-08T16:48:29 None None
Perspecta-IUB 7.0681 1.87743 0.38435 0.62596 2020-09-09T07:00:01 2020-09-09T06:53:16 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 2020-09-14T17:20:01 2020-09-14T17:18:26 None None
Perspecta-IUB 2.64507 0.77077 0.32905 0.64111 2020-09-15T01:10:02 2020-09-15T01:08:10 None None
Perspecta-IUB 2.64456 0.77063 0.32904 0.63985 2020-09-15T18:40:01 2020-09-15T18:31:16 None None
Perspecta-IUB 3.57737 0.82746 0.415 0.5271 2020-09-17T16:40:01 2020-09-17T16:34:35 None None
Perspecta-IUB 3.13725 0.74276 0.40472 0.58902 2020-09-18T03:40:02 2020-09-18T03:38:36 None None
Perspecta-IUB 1.6162 0.59674 0.26696 0.66368 2020-09-21T13:40:02 2020-09-20T03:57:09 None None
Perspecta-IUB 1.82287 0.48258 0.34788 0.54475 2020-09-22T17:10:01 2020-09-22T17:03:24 None None
Perspecta-IUB 1.08081 0.35112 0.27993 0.61294 2020-09-27T14:40:01 2020-09-27T14:35:35 None None
Perspecta-IUB 1.13728 0.33566 0.29991 0.56597 2020-09-28T15:50:01 2020-09-28T15:45:04 None None
Perspecta-IUB 1.10624 0.2993 0.29764 0.5787 2020-09-29T02:30:02 2020-09-29T02:27:48 None None
Perspecta-IUB 1.60908 0.6642 0.29379 0.60629 2020-09-30T14:10:01 2020-09-30T14:04:54 None None
Perspecta-IUB 1.54629 0.61635 0.28994 0.60262 2020-10-02T14:40:01 2020-10-02T14:39:01 None None
Perspecta-IUB 1.45855 0.58435 0.2949 0.59076 2020-10-04T12:20:01 2020-10-04T12:15:19 None None
Perspecta-IUB 0.79142 0.15141 0.25512 0.52614 2020-10-04T18:20:01 2020-10-04T17:48:02 None None
Perspecta-IUB 0.77498 0.1557 0.24804 0.59404 2020-10-05T01:20:01 2020-10-05T01:11:15 None None
Perspecta-IUB 1.32748 0.56399 0.27353 0.58777 2020-10-05T07:20:01 2020-10-05T07:00:38 None None
Perspecta-IUB 1.53278 0.66937 0.28145 0.5979 2020-10-05T13:20:02 2020-10-05T09:30:47 None None
Perspecta-IUB 0.72823 0.05353 0.26086 0.4809 2020-10-05T19:30:01 2020-10-05T17:05:52 None None
Perspecta-IUB 0.74998 0.13464 0.25175 0.52575 2020-10-06T01:40:01 2020-10-06T01:38:11 None None
Perspecta-IUB 0.69801 0.00772 0.2523 0.48621 2020-10-06T07:40:01 2020-10-06T05:45:15 None None
Perspecta-IUB 0.70334 0.03105 0.2552 0.5001 2020-10-06T13:40:02 2020-10-06T13:14:32 None None
Perspecta-IUB 0.7123 0.03452 0.25734 0.46431 2020-10-08T15:00:02 2020-10-07T02:45:34 None None
Perspecta-IUB 2.48756 1.07913 0.28366 0.60002 2020-10-13T02:50:01 2020-10-13T02:40:14 None None
ARM-UCSD 0.66982 0.10001 0.23435 0.68393 2020-09-22T04:10:01 2020-09-22T04:06:42 None None
ARM-UCSD 1.40083 0.23963 0.415 0.54167 2020-09-22T19:40:01 2020-09-22T19:04:45 None None
ARM-UCSD 5.94361 1.7797 0.33875 0.47106 2020-09-24T01:10:01 2020-09-24T01:04:03 None None
ARM-UCSD 1.01161 0.16435 0.32703 0.50694 2020-09-24T23:30:01 2020-09-24T23:28:56 None None
ARM-UCSD 0.95395 0.15036 0.31297 0.50694 2020-09-25T18:50:02 2020-09-25T18:47:38 None None
ARM-UCSD 0.9091 0.16771 0.30444 0.62307 2020-10-02T19:50:01 2020-10-02T19:47:37 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 2020-10-22T16:10:02 2020-10-22T15:47:43 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 2020-10-22T17:40:02 2020-10-22T16:54:25 None None
ARM-UCSD 0.72688 0.08551 0.26062 0.64583 2020-10-23T13:10:02 2020-10-23T01:18:49 :Executed File Update::Result Parse::Info File Missing::Log File Missing: None
ARM-UCSD 0.72688 0.08551 0.26062 0.64583 2020-10-24T07:00:01 2020-10-23T18:31:40 :Executed File Update::Result Parse::Info File Missing::Log File Missing: None
trojai-example 2.37711 1.06703 0.30121 0.50492 2020-09-22T15:20:02 2020-09-22T15:19:25 None None
trojai-example 3.16237 1.2653 0.31597 0.56549 2020-10-06T13:40:02 2020-10-06T13:34:38 None None
Perspecta 0.69994 0.02443 0.25329 0.51206 2020-09-29T00:20:01 2020-09-29T00:15:45 None None
Perspecta 0.69315 0.0 0.25 0.5 2020-10-05T05:40:01 2020-10-05T05:39:56 None None
Perspecta 0.69315 0.0 0.25 0.5 2020-10-08T15:00:02 2020-10-07T01:41:02 None None
Perspecta 0.74196 0.05157 0.272 0.50116 2020-10-09T04:00:01 2020-10-09T03:55:00 None None
Perspecta 0.89257 0.08363 0.33205 0.40123 2020-10-11T19:20:01 2020-10-11T19:19:29 None None
Perspecta 0.77378 0.07228 0.28494 0.5191 2020-10-22T00:50:02 2020-10-22T00:46:10 None None
ICSI-2 1.03362 0.26515 0.30233 0.64892 2020-10-23T13:10:02 2020-10-23T00:14:28 None None
ICSI-2 1.00108 0.24182 0.3059 0.64998 2020-10-23T22:30:01 2020-10-23T22:23:46 None None

Placeholder text for image-classification-dec2020

Placeholder image

Placeholder image description

Accepting submissions: False
Number of models in image-classification-dec2020, test: 288

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
IceTorch 0.69315 0.0 0.25 0.5 3837.76 2020-10-28T18:50:02 2020-07-27T04:34:41 :No Results: None
trojaicy 0.69315 0.0 0.25 0.5 120.59 2020-10-28T18:50:02 2020-07-27T01:44:11 :No Results: None
Hector 0.69315 0.0 0.25 0.5 3377.07 2020-10-28T18:50:02 2020-07-14T00:09:58 :No Results: None
Cassandra-XF 0.69315 0.0 0.25 0.5 634.51 2020-10-28T18:50:02 2020-07-27T19:44:57 :No Results: None
TrinitySRITrojAI 0.37257 0.05574 0.12025 0.90934 115229.34 2020-11-15T10:00:01 2020-11-15T09:52:26 None None
ICSI-1 0.54819 0.04688 0.1872 0.77681 89690.72 2020-11-07T08:30:01 2020-11-07T08:21:36 None None
NSEC 4.20621 1.07115 0.30295 0.52252 90000.01 2020-10-28T18:50:02 2020-10-17T13:16:52 None :Timeout:
DRAFFT 0.77138 0.09491 0.26494 0.65736 26384.45 2020-10-28T18:50:02 2020-09-19T21:01:10 None None
PL-GIFT 0.50831 0.06051 0.16956 0.81313 27715.67 2020-12-07T16:00:02 2020-12-07T15:52:15 None None
Perspecta-PurdueRutgers 0.29938 0.06858 0.08679 0.9143 120853.0 2020-11-13T06:30:01 2020-11-13T06:26:04 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 115137.36 2020-12-01T19:10:01 2020-12-01T19:04:03 :No Results: None
ARM-UCSD 0.31135 0.07397 0.085 0.90625 124936.92 2020-11-14T06:30:02 2020-11-14T06:20:19 None None
trojai-example 3.38798 0.92317 0.3397 0.51818 5374.98 2020-10-28T18:50:02 2020-10-06T13:34:38 None None
ICSI-2 0.69315 0.0 0.25 0.5 5600.11 2020-11-10T16:10:02 2020-11-09T20:35:51 :No Results: None
Perspecta 0.62075 0.04045 0.21495 0.71383 26615.78 2020-11-24T04:10:02 2020-11-23T21:06:47 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
IceTorch 0.69315 0.0 0.25 0.5 3837.76 2020-10-28T18:50:02 2020-07-27T04:34:41 :No Results: None
trojaicy 0.69315 0.0 0.25 0.5 120.59 2020-10-28T18:50:02 2020-07-27T01:44:11 :No Results: None
Hector 0.69315 0.0 0.25 0.5 3377.07 2020-10-28T18:50:02 2020-07-14T00:09:58 :No Results: None
Cassandra-XF 0.69315 0.0 0.25 0.5 634.51 2020-10-28T18:50:02 2020-07-27T19:44:57 :No Results: None
TrinitySRITrojAI 0.68308 0.09188 0.22794 0.70971 90000.01 2020-10-28T18:50:02 2020-10-24T17:06:59 None :Timeout:
TrinitySRITrojAI 0.57498 0.10331 0.18358 0.8615 133200.01 2020-11-03T20:20:02 2020-11-03T20:17:39 None :Timeout:
TrinitySRITrojAI 0.40923 0.05129 0.12888 0.89263 74619.88 2020-11-06T23:50:01 2020-11-06T23:41:15 None None
TrinitySRITrojAI 0.37257 0.05574 0.12025 0.90934 115229.34 2020-11-15T10:00:01 2020-11-15T09:52:26 None None
TrinitySRITrojAI 0.40037 0.07495 0.12876 0.89827 116509.35 2020-12-04T14:30:02 2020-12-04T14:25:00 None None
ICSI-1 0.70784 0.05134 0.25404 0.61384 90000.01 2020-10-28T18:50:02 2020-09-27T20:58:30 None :Timeout:
ICSI-1 0.67037 0.05409 0.23681 0.69717 133200.01 2020-10-30T05:30:01 2020-10-30T05:20:31 None :Timeout:
ICSI-1 0.58865 0.06062 0.2001 0.78945 133200.01 2020-11-03T22:20:02 2020-11-03T22:14:42 None :Timeout:
ICSI-1 0.56203 0.05206 0.19269 0.77365 120810.39 2020-11-05T20:00:02 2020-11-05T19:50:22 None None
ICSI-1 0.54819 0.04688 0.1872 0.77681 89690.72 2020-11-07T08:30:01 2020-11-07T08:21:36 None None
NSEC 4.20621 1.07115 0.30295 0.52252 90000.01 2020-10-28T18:50:02 2020-10-17T13:16:52 None :Timeout:
DRAFFT 0.77138 0.09491 0.26494 0.65736 26384.45 2020-10-28T18:50:02 2020-09-19T21:01:10 None None
PL-GIFT 0.94946 0.13961 0.29065 0.72478 41039.73 2020-10-28T18:50:02 2020-10-18T15:12:33 None None
PL-GIFT 0.5833 0.06391 0.20188 0.76309 28116.93 2020-10-29T19:50:02 2020-10-29T19:40:42 None None
PL-GIFT 0.5844 0.07087 0.20087 0.77006 31911.44 2020-10-30T14:20:01 2020-10-30T14:10:51 None None
PL-GIFT 1.25718 0.14801 0.40233 0.62023 32870.9 2020-11-04T21:10:02 2020-11-04T21:00:47 None None
PL-GIFT 0.69315 0.0 0.25 0.5 91.05 2020-11-05T13:20:02 2020-11-05T13:16:13 :No Results: None
PL-GIFT 0.61405 0.06638 0.21179 0.72678 32438.24 2020-11-05T19:30:02 2020-11-05T14:20:06 None None
PL-GIFT 0.63837 0.06969 0.22087 0.71957 32253.39 2020-11-09T18:00:02 2020-11-09T17:50:06 None None
PL-GIFT 0.58797 0.06948 0.1993 0.76982 72248.83 2020-11-12T03:40:01 2020-11-12T03:38:23 None None
PL-GIFT 0.65475 0.19242 0.19496 0.76442 16891.54 2020-11-13T16:20:02 2020-11-13T16:11:12 None None
PL-GIFT 0.56582 0.05735 0.1945 0.76447 16892.41 2020-11-14T04:10:01 2020-11-14T04:04:25 None None
PL-GIFT 0.51604 0.07583 0.16796 0.82723 128394.06 2020-11-14T15:10:01 2020-11-14T15:04:37 None None
PL-GIFT 0.67462 0.08554 0.23514 0.74023 26977.08 2020-11-23T18:20:01 2020-11-23T18:14:36 None None
PL-GIFT 0.59298 0.06976 0.20553 0.77168 23690.77 2020-11-24T04:50:02 2020-11-24T04:46:31 None None
PL-GIFT 0.55019 0.06214 0.18404 0.80543 11503.37 2020-11-24T11:50:02 2020-11-24T04:54:09 None None
PL-GIFT 0.53082 0.05009 0.18118 0.79555 11237.48 2020-11-24T17:50:02 2020-11-24T15:23:32 None None
PL-GIFT 0.56911 0.06489 0.19064 0.79437 11081.94 2020-11-25T06:00:01 2020-11-25T05:51:44 None None
PL-GIFT 0.59465 0.07368 0.20154 0.75285 16743.64 2020-11-30T22:00:02 2020-11-30T21:56:54 None None
PL-GIFT 0.55324 0.06235 0.18954 0.78438 11285.14 2020-12-01T14:50:02 2020-12-01T14:40:05 None None
PL-GIFT 0.5748 0.07511 0.19544 0.792 11523.63 2020-12-01T20:50:02 2020-12-01T18:06:31 None None
PL-GIFT 0.71498 0.09426 0.24186 0.71508 31366.27 2020-12-02T14:10:01 2020-12-02T14:02:54 None None
PL-GIFT 0.58062 0.06481 0.19824 0.7574 30954.31 2020-12-03T18:50:01 2020-12-03T18:40:24 None None
PL-GIFT 0.50831 0.06051 0.16956 0.81313 27715.67 2020-12-07T16:00:02 2020-12-07T15:52:15 None None
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 2020-10-28T18:50:02 2020-10-25T12:11:37 :Executed File Update::No Results::Info File Missing: None
Perspecta-PurdueRutgers 0.53198 0.07965 0.17492 0.81038 90000.01 2020-10-29T05:20:02 2020-10-29T05:14:22 None :Timeout:
Perspecta-PurdueRutgers 0.48406 0.09078 0.15062 0.8384 119178.72 2020-10-31T00:10:02 2020-10-31T00:00:34 None None
Perspecta-PurdueRutgers 0.43391 0.07325 0.13585 0.84295 117808.24 2020-11-03T14:30:02 2020-11-03T14:24:51 None None
Perspecta-PurdueRutgers 0.45173 0.06216 0.14516 0.83779 133200.02 2020-11-06T03:10:01 2020-11-06T03:06:56 None :Timeout:
Perspecta-PurdueRutgers 0.35111 0.06662 0.10311 0.90823 133200.01 2020-11-07T18:40:02 2020-11-07T18:34:56 None :Timeout:
Perspecta-PurdueRutgers 0.36479 0.07396 0.10892 0.89981 61966.84 2020-11-09T08:50:02 2020-11-09T08:50:00 None None
Perspecta-PurdueRutgers 0.31671 0.0662 0.09392 0.91758 66933.63 2020-11-10T07:20:02 2020-11-10T07:16:36 None None
Perspecta-PurdueRutgers 0.32274 0.07907 0.09082 0.90177 116386.42 2020-11-11T18:50:02 2020-11-11T18:41:04 None None
Perspecta-PurdueRutgers 0.29938 0.06858 0.08679 0.9143 120853.0 2020-11-13T06:30:01 2020-11-13T06:26:04 None None
Perspecta-PurdueRutgers 0.45798 0.07275 0.14454 0.89147 112598.38 2020-11-14T16:10:01 2020-11-14T05:07:55 None None
Perspecta-PurdueRutgers 0.54113 0.08011 0.17729 0.82251 93241.45 2020-11-16T01:30:02 2020-11-16T01:21:45 None None
Perspecta-PurdueRutgers 0.31684 0.06419 0.09411 0.9219 74824.06 2020-11-17T04:10:02 2020-11-17T04:03:36 None None
Perspecta-PurdueRutgers 0.32491 0.06792 0.09675 0.9158 73272.16 2020-11-24T02:30:01 2020-11-24T02:22:20 None None
Perspecta-PurdueRutgers 0.30639 0.06384 0.08722 0.91893 53975.75 2020-12-14T23:10:01 2020-12-14T23:02:02 None None
Perspecta-IUB 3.44142 0.9145 0.32218 0.5536 13603.41 2020-10-28T18:50:02 2020-10-13T02:40:14 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 115137.36 2020-12-01T19:10:01 2020-12-01T19:04:03 :No Results: None
Perspecta-IUB 1.5448 0.54581 0.26825 0.64523 102480.62 2020-12-07T06:00:02 2020-12-07T05:56:06 None None
ARM-UCSD 0.66211 0.05035 0.23344 0.69792 90000.01 2020-10-28T18:50:02 2020-10-23T18:31:40 :Result Parse: :Timeout:
ARM-UCSD 0.69315 0.0 0.25 0.5 1604.27 2020-10-30T08:10:01 2020-10-29T22:40:03 :No Results: None
ARM-UCSD 0.69315 0.0 0.25 0.5 1576.79 2020-10-30T16:10:02 2020-10-30T16:07:16 :No Results: None
ARM-UCSD 0.69315 0.0 0.25 0.5 1540.77 2020-10-30T18:40:02 2020-10-30T16:51:34 :No Results: None
ARM-UCSD 0.69315 0.0 0.25 0.5 1556.29 2020-10-31T02:00:01 2020-10-31T01:57:30 :No Results: None
ARM-UCSD 0.57074 0.10369 0.17944 0.78819 122107.27 2020-10-31T08:00:02 2020-10-31T03:34:25 None None
ARM-UCSD 0.59169 0.1024 0.18833 0.78036 133200.01 2020-11-01T20:50:02 2020-11-01T20:46:45 None :Timeout:
ARM-UCSD 0.56379 0.08144 0.18639 0.75694 127778.2 2020-11-03T17:30:01 2020-11-03T17:25:10 None None
ARM-UCSD 0.56009 0.06868 0.18583 0.75694 129005.31 2020-11-05T16:40:02 2020-11-05T16:38:30 None None
ARM-UCSD 0.73567 0.1107 0.24139 0.71296 133200.01 2020-11-09T19:50:02 2020-11-09T19:47:34 None :Timeout:
ARM-UCSD 0.33424 0.07752 0.09333 0.89583 121738.1 2020-11-12T20:00:02 2020-11-12T19:59:32 None None
ARM-UCSD 0.31135 0.07397 0.085 0.90625 124936.92 2020-11-14T06:30:02 2020-11-14T06:20:19 None None
ARM-UCSD 0.3495 0.07975 0.09889 0.88889 125932.45 2020-11-15T17:20:01 2020-11-15T01:59:17 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 205.93 2020-12-01T18:50:02 2020-12-01T18:48:51 :No Results: None
ARM-UCSD 0.38116 0.0975 0.1082 0.88959 118305.89 2020-12-01T20:30:02 2020-12-01T20:26:14 None None
ARM-UCSD 0.35272 0.09454 0.09839 0.8988 122672.13 2020-12-03T20:00:01 2020-12-03T19:57:04 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 200.34 2020-12-05T06:10:02 2020-12-03T20:24:56 :No Results: None
ARM-UCSD 0.51591 0.11503 0.15051 0.83693 118929.33 2020-12-05T17:30:02 2020-12-05T17:24:52 None None
ARM-UCSD 0.45582 0.09702 0.13623 0.84375 118625.52 2020-12-07T18:50:02 2020-12-07T18:42:57 None None
trojai-example 3.38798 0.92317 0.3397 0.51818 5374.98 2020-10-28T18:50:02 2020-10-06T13:34:38 None None
ICSI-2 1.03266 0.13235 0.33262 0.60284 56482.4 2020-10-28T18:50:02 2020-10-23T22:23:46 None None
ICSI-2 1.13714 0.13728 0.36544 0.58565 77342.25 2020-11-05T17:30:02 2020-11-05T17:13:54 None None
ICSI-2 1.91091 0.27497 0.43059 0.57841 73905.87 2020-11-07T20:50:02 2020-11-07T20:41:20 None None
ICSI-2 1.98176 0.29448 0.43289 0.57118 75418.6 2020-11-08T21:30:01 2020-11-08T21:25:49 None None
ICSI-2 0.69315 0.0 0.25 0.5 5600.11 2020-11-10T16:10:02 2020-11-09T20:35:51 :No Results: None
ICSI-2 0.69315 0.0 0.25 0.5 5621.3 2020-11-12T17:30:01 2020-11-12T17:20:38 :No Results: None
ICSI-2 1.48589 0.18752 0.41866 0.64181 80346.81 2020-11-14T19:10:01 2020-11-14T19:06:56 None None
ICSI-2 0.96 0.11132 0.32959 0.65304 81136.99 2020-11-15T19:00:02 2020-11-15T18:53:27 None None
ICSI-2 2.56228 0.32661 0.47711 0.58218 80680.08 2020-11-23T19:00:01 2020-11-23T18:52:29 None None
ICSI-2 0.69315 0.0 0.25 0.5 2020-11-24T18:50:02 2020-11-24T18:46:17 :No Results::Info File Missing: None
ICSI-2 1.85154 0.23297 0.45121 0.59435 77325.6 2020-11-24T20:00:02 2020-11-24T19:53:42 None None
ICSI-2 1.89832 0.23951 0.454 0.60072 75598.55 2020-11-25T17:50:02 2020-11-25T17:47:49 None None
ICSI-2 3.48434 0.50105 0.47222 0.55838 76033.33 2020-11-26T16:40:02 2020-11-26T16:33:53 None None
ICSI-2 1.51073 0.19686 0.41118 0.59816 75279.65 2020-11-27T16:40:02 2020-11-27T16:38:56 None None
ICSI-2 0.96827 0.11531 0.32828 0.65502 74897.6 2020-11-28T17:20:02 2020-11-28T17:12:37 None None
ICSI-2 0.97366 0.11367 0.33162 0.63821 74387.56 2020-11-29T16:00:01 2020-11-29T15:54:49 None None
ICSI-2 1.34898 0.20872 0.36121 0.55396 45116.52 2020-12-03T23:40:02 2020-12-03T23:32:28 None None
ICSI-2 1.29195 0.20368 0.34409 0.58608 44997.49 2020-12-04T13:20:02 2020-12-04T13:15:33 None None
ICSI-2 1.8264 0.30192 0.39749 0.56595 43509.51 2020-12-05T10:10:02 2020-12-05T10:04:36 None None
ICSI-2 1.85226 0.29785 0.40467 0.55604 42388.2 2020-12-06T15:00:01 2020-12-06T14:54:10 None None
ICSI-2 1.51475 0.19679 0.41657 0.65979 77764.96 2020-12-07T14:10:02 2020-12-07T13:18:57 None None
ICSI-2 1.2902 0.21182 0.34219 0.58673 44230.04 2020-12-08T13:10:01 2020-12-08T13:01:27 None None
ICSI-2 1.50571 0.19671 0.41246 0.65164 74261.06 2020-12-09T19:10:02 2020-12-09T19:06:07 None None
ICSI-2 1.13002 0.12413 0.37721 0.49568 41944.78 2020-12-10T16:10:01 2020-12-10T16:09:25 None None
ICSI-2 1.09001 0.12262 0.36268 0.51751 41954.6 2020-12-11T08:00:01 2020-12-11T07:55:49 None None
ICSI-2 1.07481 0.12135 0.35856 0.53111 42274.78 2020-12-11T20:50:01 2020-12-11T20:42:47 None None
ICSI-2 1.43142 0.16489 0.43191 0.58688 39183.44 2020-12-12T17:00:02 2020-12-12T16:59:44 None None
ICSI-2 1.41322 0.16552 0.4248 0.59447 39138.33 2020-12-13T14:00:02 2020-12-13T13:52:10 None None
ICSI-2 2.62744 0.56445 0.37503 0.56079 42759.33 2020-12-14T16:30:02 2020-12-14T16:15:35 None None
Perspecta 0.72157 0.05355 0.25693 0.56074 133200.01 2020-10-30T23:10:01 2020-10-30T23:08:39 None :Timeout:
Perspecta 0.69449 0.02602 0.25049 0.55756 133200.01 2020-11-01T22:50:02 2020-11-01T22:41:37 None :Timeout:
Perspecta 0.6886 0.03362 0.24761 0.60282 30882.73 2020-11-03T18:50:01 2020-11-03T18:41:43 None None
Perspecta 0.65146 0.03456 0.22985 0.66303 26074.27 2020-11-12T20:30:01 2020-11-12T20:24:22 None None
Perspecta 0.68485 0.05133 0.24428 0.6455 25904.39 2020-11-23T20:40:02 2020-11-23T20:37:58 None None
Perspecta 0.62075 0.04045 0.21495 0.71383 26615.78 2020-11-24T04:10:02 2020-11-23T21:06:47 None None
Perspecta 0.62564 0.03845 0.21716 0.71099 25567.71 2020-11-24T22:40:02 2020-11-24T22:33:33 None None

Placeholder text for image-classification-feb2021

Placeholder image

Placeholder image description

Accepting submissions: False
Number of models in image-classification-feb2021, test: 288

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
IceTorch 0.69315 0.0 0.25 0.5 3923.03 2020-12-31T19:30:02 2020-07-27T04:34:41 :No Results: None
trojaicy 0.69315 0.0 0.25 0.5 92.75 2020-12-31T19:30:02 2020-07-27T01:44:11 :No Results: None
Hector 0.69315 0.0 0.25 0.5 3276.88 2020-12-31T19:30:02 2020-07-14T00:09:58 :No Results: None
Cassandra-XF 0.69315 0.0 0.25 0.5 638.42 2020-12-31T19:30:02 2020-07-27T19:44:57 :No Results: None
Perspecta 0.60408 0.03643 0.20829 0.74424 133200.02 2021-01-24T11:10:02 2021-01-24T09:03:07 None :Timeout:
TrinitySRITrojAI 0.33693 0.05587 0.10424 0.93509 128227.02 2021-02-07T19:40:02 2021-02-05T17:17:45 None None
NSEC 4.93282 1.15815 0.31597 0.48949 133200.01 2020-12-31T19:30:02 2020-10-17T13:16:52 None :Timeout:
DRAFFT 0.75865 0.05893 0.27504 0.56149 43160.29 2020-12-31T19:30:02 2020-09-19T21:01:10 None None
PL-GIFT 0.64216 0.03683 0.22631 0.68304 19941.19 2021-02-05T02:30:02 2021-02-04T20:53:59 None None
Perspecta-PurdueRutgers 0.32227 0.08439 0.08784 0.90169 61761.54 2021-01-14T05:00:02 2021-01-14T04:51:04 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 133200.0 2021-01-18T02:40:02 2021-01-17T08:41:27 :Result Parse::No Results: :Timeout:
ARM-UCSD 0.59378 0.07086 0.20042 0.73264 44574.86 2021-01-08T21:50:02 2021-01-08T17:58:03 None None
trojai-example 1.16036 0.32863 0.31203 0.56407 9143.8 2021-02-10T23:20:02 2021-02-10T23:16:39 None None
ICSI-2 0.5196 0.0545 0.17636 0.8077 47442.19 2021-01-26T13:20:01 2021-01-26T13:17:08 None None
ICSI-1 0.69315 0.0 0.25 0.5 133200.02 2021-01-20T01:20:02 2021-01-20T01:11:57 :No Results: :Timeout:
ARM 0.69315 0.0 0.25 0.5 4151.75 2021-01-11T10:00:02 2021-01-11T09:58:12 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
IceTorch 0.69315 0.0 0.25 0.5 3923.03 2020-12-31T19:30:02 2020-07-27T04:34:41 :No Results: None
trojaicy 0.69315 0.0 0.25 0.5 92.75 2020-12-31T19:30:02 2020-07-27T01:44:11 :No Results: None
Hector 0.69315 0.0 0.25 0.5 3276.88 2020-12-31T19:30:02 2020-07-14T00:09:58 :No Results: None
Cassandra-XF 0.69315 0.0 0.25 0.5 638.42 2020-12-31T19:30:02 2020-07-27T19:44:57 :No Results: None
Perspecta 0.67708 0.01859 0.24227 0.61745 33832.79 2020-12-31T19:30:02 2020-12-21T21:51:21 None None
Perspecta 0.87802 0.08411 0.31855 0.61651 53969.76 2021-01-05T01:00:02 2021-01-05T00:54:14 None None
Perspecta 0.66692 0.03275 0.23697 0.64914 54479.81 2021-01-05T18:00:02 2021-01-05T17:51:38 None None
Perspecta 0.70935 0.06281 0.25241 0.6746 133200.02 2021-01-12T22:30:02 2021-01-12T22:25:41 None :Timeout:
Perspecta 0.6663 0.02141 0.23727 0.6203 133200.01 2021-01-18T02:40:02 2021-01-17T22:14:49 None :Timeout:
Perspecta 0.69315 0.0 0.25 0.5 2021-01-20T21:30:02 2021-01-20T21:24:35 :No Results::Info File Missing: None
Perspecta 0.71104 0.03335 0.25789 0.54461 48643.14 2021-01-21T04:40:02 2021-01-21T04:33:24 None None
Perspecta 0.72917 0.05311 0.26554 0.69035 133200.01 2021-01-22T16:00:02 2021-01-22T04:25:35 None :Timeout:
Perspecta 0.69315 0.0 0.25 0.5 2021-01-24T05:10:02 2021-01-23T04:48:40 :No Results::Info File Missing: None
Perspecta 0.60408 0.03643 0.20829 0.74424 133200.02 2021-01-24T11:10:02 2021-01-24T09:03:07 None :Timeout:
Perspecta 0.61598 0.03681 0.21387 0.72541 133200.02 2021-01-26T04:50:02 2021-01-26T04:43:40 None :Timeout:
Perspecta 0.61615 0.03619 0.21379 0.7316 63034.89 2021-01-30T19:20:01 2021-01-30T19:15:50 None None
Perspecta 0.62544 0.03329 0.21712 0.72147 82121.07 2021-02-01T21:30:01 2021-02-01T21:22:26 None None
Perspecta 0.62707 0.05461 0.21723 0.72374 100501.08 2021-02-07T19:40:02 2021-02-05T16:59:01 None None
Perspecta 0.63658 0.05155 0.22232 0.69666 78894.35 2021-02-09T21:30:01 2021-02-09T21:22:04 None None
TrinitySRITrojAI 0.66723 0.03937 0.23804 0.63057 5460.25 2020-12-31T19:30:02 2020-12-21T12:51:50 None None
TrinitySRITrojAI 0.35718 0.0593 0.11105 0.90683 133200.0 2021-01-03T03:10:01 2021-01-03T03:03:33 None :Timeout:
TrinitySRITrojAI 0.34614 0.0586 0.10832 0.93347 133200.01 2021-01-18T18:40:02 2021-01-18T18:31:09 None :Timeout:
TrinitySRITrojAI 0.33693 0.05587 0.10424 0.93509 128227.02 2021-02-07T19:40:02 2021-02-05T17:17:45 None None
NSEC 4.93282 1.15815 0.31597 0.48949 133200.01 2020-12-31T19:30:02 2020-10-17T13:16:52 None :Timeout:
DRAFFT 0.75865 0.05893 0.27504 0.56149 43160.29 2020-12-31T19:30:02 2020-09-19T21:01:10 None None
PL-GIFT 1.01464 0.10331 0.36336 0.61432 17383.79 2020-12-31T19:30:02 2020-12-18T13:55:09 None None
PL-GIFT 1.16171 0.13025 0.38891 0.612 19693.72 2021-01-06T22:30:02 2021-01-06T22:29:58 None None
PL-GIFT 0.69954 0.04301 0.25172 0.6135 17433.25 2021-01-08T14:40:01 2021-01-08T14:39:39 None None
PL-GIFT 0.69315 0.0 0.25 0.5 1445.67 2021-01-12T04:40:02 2021-01-12T04:31:36 :No Results: None
PL-GIFT 0.68832 0.0336 0.2477 0.60209 8261.6 2021-01-12T19:50:01 2021-01-12T19:47:46 None None
PL-GIFT 0.66341 0.04025 0.23541 0.67564 20206.57 2021-01-19T18:20:01 2021-01-19T18:19:33 None None
PL-GIFT 0.69315 0.0 0.25 0.5 1540.18 2021-01-21T04:40:02 2021-01-21T04:34:12 :No Results: None
PL-GIFT 0.65052 0.04275 0.23018 0.68933 20099.79 2021-01-21T14:30:01 2021-01-21T14:20:22 None None
PL-GIFT 0.66009 0.04232 0.23412 0.67853 20316.93 2021-01-21T20:30:01 2021-01-21T15:27:33 None None
PL-GIFT 0.65166 0.03732 0.23052 0.66667 17759.28 2021-01-22T17:30:02 2021-01-22T17:21:04 None None
PL-GIFT 0.65389 0.04042 0.23139 0.67848 17731.78 2021-01-22T23:40:01 2021-01-22T17:41:29 None None
PL-GIFT 0.69997 0.04881 0.25122 0.64041 17433.45 2021-01-25T18:00:01 2021-01-25T17:50:48 None None
PL-GIFT 0.83709 0.07398 0.30736 0.62671 17558.1 2021-01-26T00:00:01 2021-01-25T22:02:44 None None
PL-GIFT 0.66108 0.04834 0.23383 0.69772 19868.22 2021-01-26T14:40:02 2021-01-26T14:36:32 None None
PL-GIFT 0.90406 0.08332 0.33171 0.5882 20167.14 2021-02-02T17:30:02 2021-02-02T17:22:03 None None
PL-GIFT 0.72662 0.05492 0.26272 0.63366 20093.27 2021-02-03T21:20:01 2021-02-03T21:11:02 None None
PL-GIFT 0.76647 0.06155 0.27933 0.62011 20077.39 2021-02-04T14:30:02 2021-02-04T14:28:46 None None
PL-GIFT 0.73604 0.0552 0.26682 0.63797 19892.22 2021-02-04T20:30:02 2021-02-04T15:37:37 None None
PL-GIFT 0.64216 0.03683 0.22631 0.68304 19941.19 2021-02-05T02:30:02 2021-02-04T20:53:59 None None
Perspecta-PurdueRutgers 0.72828 0.12345 0.22303 0.77124 21993.61 2020-12-31T19:30:02 2020-12-17T16:22:59 None None
Perspecta-PurdueRutgers 0.4863 0.09991 0.14771 0.83259 61330.09 2021-01-01T16:30:02 2021-01-01T16:21:00 None None
Perspecta-PurdueRutgers 0.46967 0.09881 0.14142 0.8459 60222.82 2021-01-03T02:10:02 2021-01-03T02:02:09 None None
Perspecta-PurdueRutgers 0.40101 0.08817 0.11779 0.86458 88669.66 2021-01-10T06:40:01 2021-01-10T06:31:02 None None
Perspecta-PurdueRutgers 0.37521 0.09134 0.10719 0.87584 60474.54 2021-01-11T21:40:01 2021-01-11T21:35:57 None None
Perspecta-PurdueRutgers 0.35533 0.08944 0.09921 0.88684 60539.95 2021-01-13T06:30:02 2021-01-13T06:21:21 None None
Perspecta-PurdueRutgers 0.32227 0.08439 0.08784 0.90169 61761.54 2021-01-14T05:00:02 2021-01-14T04:51:04 None None
Perspecta-PurdueRutgers 0.32544 0.08625 0.08817 0.89921 60863.04 2021-01-18T04:40:02 2021-01-18T04:31:26 None None
Perspecta-PurdueRutgers 0.38568 0.0838 0.11368 0.88718 29522.79 2021-01-20T07:10:02 2021-01-20T07:06:58 None None
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 2021-01-20T20:20:01 2021-01-20T19:11:43 :No Results::Info File Missing: None
Perspecta-PurdueRutgers 0.32536 0.08625 0.08815 0.90061 57997.7 2021-01-20T22:20:01 2021-01-20T19:11:43 None None
Perspecta-PurdueRutgers 0.74627 0.11646 0.2492 0.83546 58868.55 2021-01-27T14:40:02 2021-01-27T14:37:20 None None
Perspecta-PurdueRutgers 0.89348 0.09619 0.31256 0.54859 58998.79 2021-02-03T08:00:01 2021-02-03T07:51:52 None None
Perspecta-PurdueRutgers 0.87323 0.07091 0.32292 0.48611 50594.78 2021-02-04T01:50:01 2021-02-04T01:32:02 None None
Perspecta-IUB 0.86668 0.18961 0.28544 0.52057 123593.89 2020-12-31T19:30:02 2020-12-18T11:55:55 None None
Perspecta-IUB 0.76961 0.04875 0.28285 0.51384 124300.98 2021-01-12T17:00:02 2021-01-12T16:50:47 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 133200.0 2021-01-18T02:40:02 2021-01-17T08:41:27 :Result Parse::No Results: :Timeout:
Perspecta-IUB 1.3533 0.48279 0.26717 0.60484 129329.76 2021-01-26T03:10:01 2021-01-26T03:06:21 None None
ARM-UCSD 0.74569 0.10632 0.24576 0.70194 133200.01 2020-12-31T19:30:02 2020-12-07T18:42:57 None :Timeout:
ARM-UCSD 0.60823 0.07171 0.20667 0.72222 44228.23 2021-01-07T20:20:02 2021-01-07T20:18:34 None None
ARM-UCSD 0.59378 0.07086 0.20042 0.73264 44574.86 2021-01-08T21:50:02 2021-01-08T17:58:03 None None
ARM-UCSD 0.9145 0.09265 0.31778 0.52093 39781.08 2021-01-26T20:00:02 2021-01-26T19:57:15 None None
ARM-UCSD 2.63192 0.7914 0.3245 0.49523 52677.36 2021-02-01T20:30:02 2021-02-01T20:21:22 None None
trojai-example 3.42664 0.92187 0.34886 0.48008 4655.98 2020-12-31T19:30:02 2020-10-06T13:34:38 None None
trojai-example 13.81551 1.59561 0.5 0.5 4600.04 2021-01-27T13:40:02 2021-01-27T13:39:19 None None
trojai-example 1.16036 0.32863 0.31203 0.56407 9143.8 2021-02-10T23:20:02 2021-02-10T23:16:39 None None
ICSI-2 0.78322 0.07509 0.27562 0.58652 39585.01 2021-01-05T15:00:02 2021-01-05T14:57:35 None None
ICSI-2 1.16756 0.12907 0.38551 0.4796 23383.43 2021-01-06T19:00:02 2021-01-06T18:55:11 None None
ICSI-2 0.53848 0.06137 0.18285 0.80141 46138.49 2021-01-18T10:10:01 2021-01-18T10:08:25 None None
ICSI-2 0.53969 0.05742 0.18424 0.78942 45665.39 2021-01-19T11:00:02 2021-01-19T10:52:06 None None
ICSI-2 0.56966 0.06379 0.19428 0.76562 45529.72 2021-01-20T07:20:01 2021-01-20T07:16:33 None None
ICSI-2 0.52191 0.05252 0.17727 0.80732 45824.92 2021-01-20T21:20:02 2021-01-20T21:18:47 None None
ICSI-2 0.53064 0.05157 0.17939 0.799 48689.92 2021-01-22T22:40:01 2021-01-22T22:28:38 None None
ICSI-2 0.58294 0.07029 0.19621 0.77141 47767.63 2021-01-23T20:00:02 2021-01-23T19:53:57 None None
ICSI-2 0.52932 0.05363 0.17947 0.79229 45496.04 2021-01-24T13:30:02 2021-01-24T13:26:11 None None
ICSI-2 0.56671 0.08493 0.18766 0.80968 46237.81 2021-01-25T08:20:01 2021-01-25T08:13:58 None None
ICSI-2 0.52636 0.05405 0.17816 0.80577 46825.7 2021-01-25T21:50:01 2021-01-25T21:47:09 None None
ICSI-2 0.5196 0.0545 0.17636 0.8077 47442.19 2021-01-26T13:20:01 2021-01-26T13:17:08 None None
ICSI-2 0.69315 0.0 0.25 0.5 148.89 2021-01-31T22:00:02 2021-01-31T21:56:13 :No Results: None
ICSI-2 1.21081 0.115 0.4265 0.47213 46910.33 2021-02-01T04:00:02 2021-01-31T22:33:52 None None
ICSI-2 7.71025 1.11187 0.49896 0.45923 46795.78 2021-02-01T22:10:02 2021-02-01T22:10:00 None None
ICSI-2 4.43458 0.66481 0.48866 0.53098 46449.56 2021-02-02T14:00:02 2021-02-02T13:50:36 None None
ICSI-2 1.5183 0.16715 0.45474 0.47818 46656.67 2021-02-03T10:00:01 2021-02-03T09:55:40 None None
ICSI-1 0.74035 0.03846 0.27244 0.62071 15878.19 2021-01-11T03:50:02 2021-01-11T03:40:49 None None
ICSI-1 0.69315 0.0 0.25 0.5 133200.02 2021-01-20T01:20:02 2021-01-20T01:11:57 :No Results: :Timeout:
ICSI-1 0.69315 0.0 0.25 0.5 51661.44 2021-01-28T02:50:02 2021-01-28T02:46:25 :No Results: None
ICSI-1 0.69432 0.04693 0.24505 0.62647 99940.74 2021-02-10T02:50:01 2021-02-10T02:42:24 None None
ARM 0.69315 0.0 0.25 0.5 4151.75 2021-01-11T10:00:02 2021-01-11T09:58:12 None None
ARM 0.69315 0.0 0.25 0.5 4131.43 2021-01-19T10:30:01 2021-01-19T10:26:46 None None
ARM 0.8995 0.09038 0.31705 0.53872 72870.05 2021-01-19T20:10:02 2021-01-19T20:04:00 None None

Placeholder text for nlp-sentiment-classification-mar2021

Placeholder image

Placeholder image description

Accepting submissions: False
Number of models in nlp-sentiment-classification-mar2021, test: 504

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 13.81551 1.20617 0.5 0.5 8107.49 2021-02-25T00:50:02 2021-02-24T19:18:51 None None
PL-GIFT 0.25212 0.0448 0.07327 0.95868 10402.44 2021-02-25T06:50:02 2021-02-25T03:48:04 None None
ARM 0.52739 0.04084 0.1785 0.79886 11932.72 2021-03-01T21:40:01 2021-03-01T21:35:22 None None
Perspecta-IUB 0.3118 0.07119 0.09009 0.94916 10401.75 2021-03-16T15:10:01 2021-03-16T15:02:05 None None
Perspecta 0.28266 0.04642 0.08683 0.95351 9823.79 2021-03-17T21:40:02 2021-03-17T21:30:20 None None
ICSI-1 0.33144 0.04582 0.10315 0.92966 37292.64 2021-03-12T19:20:02 2021-03-12T19:14:01 None None
ICSI-2 0.51169 0.06167 0.1651 0.84795 10392.06 2021-03-11T16:00:02 2021-03-11T15:58:38 None None
ARM-UCSD 0.38437 0.06387 0.11159 0.87302 8037.76 2021-03-13T04:00:02 2021-03-13T02:08:35 None None
Perspecta-PurdueRutgers 0.32546 0.05665 0.09817 0.93557 13679.11 2021-03-16T16:10:02 2021-03-16T16:02:46 None None
TrinitySRITrojAI 0.47857 0.04852 0.15616 0.84832 54350.26 2021-03-17T15:30:02 2021-03-17T15:20:20 None None
ARM-UMBC 0.29049 0.06996 0.0847 0.95423 16091.96 2021-03-19T16:50:02 2021-03-19T16:49:50 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 13.81551 1.20617 0.5 0.5 8107.49 2021-02-25T00:50:02 2021-02-24T19:18:51 None None
PL-GIFT 0.69315 0.0 0.25 0.5 2404.08 2021-02-25T00:50:02 2021-02-24T13:36:17 :No Results: None
PL-GIFT 0.25212 0.0448 0.07327 0.95868 10402.44 2021-02-25T06:50:02 2021-02-25T03:48:04 None None
ARM 0.69315 0.0 0.25 0.5 3100.94 2021-03-01T11:40:02 2021-03-01T11:38:33 :No Results: None
ARM 0.52739 0.04084 0.1785 0.79886 11932.72 2021-03-01T21:40:01 2021-03-01T21:35:22 None None
Perspecta-IUB 0.65214 0.03092 0.23018 0.69662 7861.53 2021-03-02T14:50:02 2021-03-02T14:40:08 None None
Perspecta-IUB 0.58793 0.02652 0.20325 0.73468 8011.94 2021-03-06T04:20:02 2021-03-06T04:16:38 None None
Perspecta-IUB 0.60506 0.05596 0.20303 0.76839 8685.28 2021-03-10T02:50:02 2021-03-10T02:43:32 None None
Perspecta-IUB 0.50003 0.07195 0.15545 0.84984 8767.15 2021-03-10T08:50:02 2021-03-10T06:09:10 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 7553.44 2021-03-12T14:30:02 2021-03-12T09:37:51 :No Results: None
Perspecta-IUB 0.47526 0.07628 0.14641 0.89098 10018.15 2021-03-14T05:00:02 2021-03-14T04:58:24 None None
Perspecta-IUB 0.47097 0.07675 0.14398 0.8922 10051.63 2021-03-14T11:10:02 2021-03-14T05:15:29 None None
Perspecta-IUB 0.41643 0.06136 0.13099 0.89795 9987.34 2021-03-14T17:10:02 2021-03-14T15:02:38 None None
Perspecta-IUB 0.39286 0.06754 0.12038 0.91001 9869.28 2021-03-15T03:10:01 2021-03-15T03:07:47 None None
Perspecta-IUB 0.5204 0.06884 0.16968 0.8509 9935.16 2021-03-15T09:10:02 2021-03-15T06:19:38 None None
Perspecta-IUB 0.40195 0.07327 0.12146 0.90964 9715.22 2021-03-16T01:30:02 2021-03-16T01:22:48 None None
Perspecta-IUB 0.3118 0.07119 0.09009 0.94916 10401.75 2021-03-16T15:10:01 2021-03-16T15:02:05 None None
Perspecta 0.69514 0.01641 0.25089 0.54738 13625.22 2021-03-02T18:40:02 2021-03-02T18:34:26 None None
Perspecta 0.50421 0.05877 0.16437 0.84614 8607.85 2021-03-12T18:20:02 2021-03-12T18:18:21 None None
Perspecta 0.36315 0.04148 0.1126 0.92418 9051.7 2021-03-13T00:30:01 2021-03-12T22:31:57 None None
Perspecta 0.28266 0.04642 0.08683 0.95351 9823.79 2021-03-17T21:40:02 2021-03-17T21:30:20 None None
ICSI-1 0.58324 0.03985 0.19957 0.7153 7723.01 2021-03-04T03:30:02 2021-03-04T03:20:19 None None
ICSI-1 0.37058 0.04237 0.11832 0.91482 15075.26 2021-03-11T18:50:02 2021-03-11T18:42:50 None None
ICSI-1 0.33144 0.04582 0.10315 0.92966 37292.64 2021-03-12T19:20:02 2021-03-12T19:14:01 None None
ICSI-2 2.49091 0.43261 0.30162 0.65417 8090.16 2021-03-04T08:10:01 2021-03-04T08:06:30 None None
ICSI-2 0.74547 0.08421 0.24522 0.73361 8737.27 2021-03-09T19:20:02 2021-03-09T19:18:33 None None
ICSI-2 0.56879 0.04209 0.19336 0.77523 8654.23 2021-03-10T16:10:02 2021-03-10T16:07:00 None None
ICSI-2 0.51169 0.06167 0.1651 0.84795 10392.06 2021-03-11T16:00:02 2021-03-11T15:58:38 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 1786.97 2021-03-04T21:10:01 2021-03-04T21:06:31 :No Results: None
ARM-UCSD 0.69315 0.0 0.25 0.5 7260.53 2021-03-05T17:30:02 2021-03-05T17:22:35 :No Results: None
ARM-UCSD 0.69315 0.0 0.25 0.5 7231.46 2021-03-08T22:20:02 2021-03-08T22:16:40 :No Results: None
ARM-UCSD 0.69315 0.0 0.25 0.5 7471.0 2021-03-09T21:20:01 2021-03-09T21:14:00 :No Results: None
ARM-UCSD 0.53035 0.09488 0.14893 0.8373 7118.04 2021-03-10T21:40:02 2021-03-10T21:32:14 None None
ARM-UCSD 0.46609 0.08943 0.12929 0.85913 7868.42 2021-03-11T04:30:02 2021-03-11T04:28:48 None None
ARM-UCSD 0.41925 0.06713 0.12429 0.85714 7659.91 2021-03-11T19:40:02 2021-03-11T19:34:11 None None
ARM-UCSD 3.23458 0.77556 0.11706 0.84755 7855.58 2021-03-12T15:10:01 2021-03-12T15:09:16 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 7194.54 2021-03-12T21:50:01 2021-03-12T21:42:29 :No Results: None
ARM-UCSD 0.38437 0.06387 0.11159 0.87302 8037.76 2021-03-13T04:00:02 2021-03-13T02:08:35 None None
ARM-UCSD 0.58235 0.20017 0.10149 0.92454 7041.35 2021-03-18T20:20:02 2021-03-18T20:13:03 None None
Perspecta-PurdueRutgers 0.48715 0.06704 0.14925 0.81878 20574.91 2021-03-08T06:10:02 2021-03-08T06:01:35 None None
Perspecta-PurdueRutgers 0.43069 0.06168 0.12937 0.85847 20178.69 2021-03-08T19:20:01 2021-03-08T19:12:04 None None
Perspecta-PurdueRutgers 0.40248 0.05956 0.11782 0.86839 20013.31 2021-03-09T05:20:02 2021-03-09T05:11:49 None None
Perspecta-PurdueRutgers 0.34049 0.0531 0.10701 0.9298 9986.22 2021-03-10T18:00:02 2021-03-10T17:52:03 None None
Perspecta-PurdueRutgers 0.34238 0.05325 0.10775 0.92906 11817.86 2021-03-11T02:20:01 2021-03-11T02:19:42 None None
Perspecta-PurdueRutgers 0.33187 0.05215 0.10388 0.93321 13400.03 2021-03-12T18:20:02 2021-03-12T18:18:06 None None
Perspecta-PurdueRutgers 0.32546 0.05665 0.09817 0.93557 13679.11 2021-03-16T16:10:02 2021-03-16T16:02:46 None None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 2199.43 2021-03-08T06:30:02 2021-03-08T06:22:58 :No Results: None
TrinitySRITrojAI 0.69065 0.00308 0.24875 0.63433 7784.53 2021-03-08T12:40:01 2021-03-08T12:25:26 None None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 2039.1 2021-03-16T13:20:01 2021-03-16T13:11:49 :No Results: None
TrinitySRITrojAI 0.47857 0.04852 0.15616 0.84832 54350.26 2021-03-17T15:30:02 2021-03-17T15:20:20 None None
ARM-UMBC 0.69315 0.0 0.25 0.5 291.37 2021-03-14T16:30:02 2021-03-14T16:26:14 :No Results: None
ARM-UMBC 0.35511 0.10202 0.09013 0.95347 8558.2 2021-03-15T02:10:02 2021-03-15T02:09:01 None None
ARM-UMBC 0.31732 0.08479 0.08764 0.95216 11519.4 2021-03-16T00:20:01 2021-03-16T00:11:39 None None
ARM-UMBC 0.29049 0.06996 0.0847 0.95423 16091.96 2021-03-19T16:50:02 2021-03-19T16:49:50 None None

Placeholder text for nlp-sentiment-classification-apr2021

Placeholder image

Placeholder image description

Accepting submissions: False
Number of models in nlp-sentiment-classification-apr2021, test: 480

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 9.44139 1.15093 0.41667 0.5 8134.67 2021-03-24T16:20:02 2021-02-24T19:18:51 None None
IceTorch 0.69315 0.0 0.25 0.5 2353.0 2021-03-24T16:20:02 2020-07-27T04:34:41 :No Results: None
trojaicy 0.69315 0.0 0.25 0.5 129.74 2021-03-24T16:20:02 2020-07-27T01:44:11 :No Results: None
Hector 0.69315 0.0 0.25 0.5 1607.27 2021-03-24T16:20:02 2020-07-14T00:09:58 :No Results: None
Cassandra-XF 0.69315 0.0 0.25 0.5 1385.81 2021-03-24T16:20:02 2020-07-27T19:44:57 :No Results: None
Perspecta 0.57869 0.05927 0.19279 0.7978 9032.31 2021-04-07T20:20:02 2021-04-07T20:18:52 None None
TrinitySRITrojAI 0.36272 0.06091 0.11193 0.91916 49597.55 2021-04-13T08:10:02 2021-04-13T08:01:26 None None
ICSI-1 0.69058 0.05522 0.23872 0.66515 26508.91 2021-03-24T16:20:02 2021-03-12T19:14:01 None None
NSEC 0.69315 0.0 0.25 0.5 2028.32 2021-03-24T16:20:02 2020-10-17T13:16:52 :No Results: None
DRAFFT 0.69315 0.0 0.25 0.5 2437.21 2021-03-24T16:20:02 2020-09-19T21:01:10 :No Results: None
PL-GIFT 0.51837 0.04316 0.17625 0.82359 10750.07 2021-04-16T00:30:01 2021-04-16T00:11:02 None None
Perspecta-IUB 0.59105 0.02518 0.20214 0.78527 8989.4 2021-04-27T11:50:02 2021-04-27T11:49:04 None None
ARM-UCSD 0.48346 0.04259 0.16033 0.81915 44297.56 2021-04-17T19:10:02 2021-04-17T19:02:46 None None
ICSI-2 0.72869 0.04204 0.25737 0.56952 14723.42 2021-04-13T18:50:02 2021-04-13T18:42:36 None None
ARM 0.69315 0.0 0.25 0.5 5054.03 2021-04-19T18:50:01 2021-04-19T18:42:07 :No Results: None
TrinitySRITrojAI-SBU 0.45132 0.10316 0.11794 0.90692 58448.61 2021-04-29T05:20:02 2021-04-29T05:19:50 None None
ARM-UMBC 0.69315 0.0 0.25 0.5 273.82 2021-04-19T14:50:02 2021-04-19T14:48:12 :No Results: None
Perspecta-PurdueRutgers 0.25549 0.05699 0.06789 0.94345 51607.09 2021-04-16T23:10:02 2021-04-16T23:01:11 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 9.44139 1.15093 0.41667 0.5 8134.67 2021-03-24T16:20:02 2021-02-24T19:18:51 None None
trojai-example 9.44139 1.15093 0.41667 0.5 8105.81 2021-03-26T19:40:02 2021-03-26T18:26:14 None None
trojai-example 9.44139 1.15093 0.41667 0.5 7956.63 2021-03-29T20:20:02 2021-03-29T20:11:20 None None
IceTorch 0.69315 0.0 0.25 0.5 2353.0 2021-03-24T16:20:02 2020-07-27T04:34:41 :No Results: None
trojaicy 0.69315 0.0 0.25 0.5 129.74 2021-03-24T16:20:02 2020-07-27T01:44:11 :No Results: None
Hector 0.69315 0.0 0.25 0.5 1607.27 2021-03-24T16:20:02 2020-07-14T00:09:58 :No Results: None
Cassandra-XF 0.69315 0.0 0.25 0.5 1385.81 2021-03-24T16:20:02 2020-07-27T19:44:57 :No Results: None
Perspecta 0.64084 0.05241 0.22166 0.70384 7318.34 2021-03-24T16:20:02 2021-03-17T21:30:20 None None
Perspecta 0.68705 0.00727 0.24709 0.54671 8421.52 2021-03-28T05:30:01 2021-03-28T05:26:53 None None
Perspecta 0.8527 0.07229 0.29798 0.47205 8498.03 2021-03-28T22:30:02 2021-03-28T22:29:48 None None
Perspecta 0.70254 0.00872 0.25468 0.46747 8647.02 2021-03-29T16:10:02 2021-03-29T16:04:58 None None
Perspecta 0.62178 0.07093 0.20706 0.78299 9039.6 2021-04-02T22:30:02 2021-04-02T22:28:07 None None
Perspecta 0.60071 0.06192 0.20275 0.78088 8914.6 2021-04-06T17:50:02 2021-04-06T17:41:45 None None
Perspecta 0.57869 0.05927 0.19279 0.7978 9032.31 2021-04-07T20:20:02 2021-04-07T20:18:52 None None
Perspecta 1.31301 0.18684 0.32023 0.63138 8735.53 2021-04-11T02:00:01 2021-04-11T01:50:59 None None
Perspecta 1.29388 0.17736 0.32701 0.62937 8652.52 2021-04-11T08:00:02 2021-04-11T02:00:40 None None
Perspecta 1.21191 0.17192 0.29772 0.66479 8999.04 2021-04-12T16:10:02 2021-04-12T16:08:20 None None
Perspecta 1.0085 0.13232 0.27942 0.67946 9898.3 2021-04-13T15:00:02 2021-04-13T14:57:22 None None
Perspecta 1.10421 0.1578 0.28701 0.68987 9371.77 2021-04-14T21:20:02 2021-04-14T21:12:29 None None
Perspecta 0.94224 0.11616 0.28005 0.68071 13632.95 2021-04-15T03:20:02 2021-04-15T00:17:58 None None
Perspecta 1.31768 0.15986 0.33172 0.64568 13531.97 2021-04-19T19:40:02 2021-04-19T19:33:40 None None
Perspecta 0.6425 0.05045 0.22136 0.72062 23047.26 2021-04-29T19:10:02 2021-04-29T19:02:09 None None
TrinitySRITrojAI 0.85659 0.10181 0.26043 0.66549 3064.04 2021-03-24T16:20:02 2021-03-22T04:59:49 None None
TrinitySRITrojAI 0.91006 0.11444 0.26852 0.69213 3210.94 2021-03-25T08:50:02 2021-03-25T08:49:53 None None
TrinitySRITrojAI 0.70783 0.02281 0.25569 0.54056 3276.34 2021-03-25T15:00:02 2021-03-25T11:15:22 None None
TrinitySRITrojAI 1.06025 0.14428 0.28757 0.67201 4967.19 2021-03-26T06:40:02 2021-03-26T06:35:11 None None
TrinitySRITrojAI 0.93882 0.09792 0.304 0.62345 48400.35 2021-03-27T16:40:01 2021-03-27T16:32:58 None None
TrinitySRITrojAI 0.85074 0.05868 0.30113 0.49313 9707.71 2021-03-31T10:20:02 2021-03-31T10:14:35 None None
TrinitySRITrojAI 0.9879 0.16691 0.31243 0.50655 9760.78 2021-03-31T16:20:02 2021-03-31T15:05:07 None None
TrinitySRITrojAI 0.91601 0.09537 0.29841 0.62939 47445.58 2021-04-02T16:10:02 2021-04-02T16:09:57 None None
TrinitySRITrojAI 0.68579 0.02608 0.2458 0.58743 10399.63 2021-04-04T00:00:02 2021-04-03T23:51:53 None None
TrinitySRITrojAI 0.68222 0.02849 0.24401 0.6149 8691.88 2021-04-04T19:20:02 2021-04-04T19:18:16 None None
TrinitySRITrojAI 0.67625 0.00617 0.24161 0.6577 8699.07 2021-04-05T05:30:02 2021-04-05T05:25:28 None None
TrinitySRITrojAI 0.7236 0.02736 0.26117 0.53245 10552.1 2021-04-05T18:40:02 2021-04-05T18:35:31 None None
TrinitySRITrojAI 0.69268 0.00322 0.24976 0.53783 8884.15 2021-04-06T01:00:02 2021-04-06T00:50:54 None None
TrinitySRITrojAI 0.70005 0.03085 0.25234 0.59242 8014.53 2021-04-07T21:10:02 2021-04-07T21:05:20 None None
TrinitySRITrojAI 0.66371 0.02087 0.23564 0.66097 7369.48 2021-04-12T04:20:01 2021-04-12T04:15:53 None None
TrinitySRITrojAI 0.36272 0.06091 0.11193 0.91916 49597.55 2021-04-13T08:10:02 2021-04-13T08:01:26 None None
TrinitySRITrojAI 1.33699 0.31572 0.26655 0.65726 11111.17 2021-04-14T07:30:02 2021-04-14T07:26:53 None None
TrinitySRITrojAI 0.67334 0.17389 0.11812 0.91062 34752.1 2021-04-16T19:50:02 2021-04-16T19:43:03 None None
TrinitySRITrojAI 0.5109 0.11544 0.12798 0.91269 34541.5 2021-04-17T17:40:02 2021-04-17T17:34:58 None None
TrinitySRITrojAI 0.44876 0.07442 0.1398 0.88973 21038.54 2021-04-18T06:40:02 2021-04-18T06:33:03 None None
TrinitySRITrojAI 0.41479 0.09237 0.11417 0.92006 34791.28 2021-04-19T08:20:02 2021-04-19T08:15:02 None None
TrinitySRITrojAI 0.43727 0.0493 0.14581 0.86886 133200.01 2021-04-25T21:50:02 2021-04-25T21:47:38 None :Timeout:
ICSI-1 0.69058 0.05522 0.23872 0.66515 26508.91 2021-03-24T16:20:02 2021-03-12T19:14:01 None None
ICSI-1 0.697 0.05755 0.24096 0.68064 27292.92 2021-04-06T02:30:01 2021-04-06T02:23:32 None None
NSEC 0.69315 0.0 0.25 0.5 2028.32 2021-03-24T16:20:02 2020-10-17T13:16:52 :No Results: None
DRAFFT 0.69315 0.0 0.25 0.5 2437.21 2021-03-24T16:20:02 2020-09-19T21:01:10 :No Results: None
PL-GIFT 0.61875 0.03488 0.21706 0.69615 7851.12 2021-03-24T16:20:02 2021-02-25T03:48:04 None None
PL-GIFT 0.67492 0.01708 0.24101 0.60127 9666.16 2021-04-12T00:40:02 2021-04-12T00:18:11 None None
PL-GIFT 0.67491 0.01705 0.24101 0.60128 9547.49 2021-04-12T13:00:01 2021-04-12T12:50:28 None None
PL-GIFT 1.01852 0.077 0.36872 0.55582 10268.44 2021-04-12T23:10:02 2021-04-12T23:03:09 None None
PL-GIFT 1.21555 0.10011 0.41157 0.5722 11025.95 2021-04-13T13:20:02 2021-04-13T13:13:03 None None
PL-GIFT 0.52263 0.04451 0.17933 0.81893 10978.95 2021-04-13T19:20:02 2021-04-13T16:35:53 None None
PL-GIFT 0.52561 0.03981 0.17914 0.80839 11520.23 2021-04-14T01:30:01 2021-04-13T22:43:48 None None
PL-GIFT 0.87538 0.11271 0.25548 0.68883 10213.93 2021-04-15T02:20:02 2021-04-15T02:18:33 None None
PL-GIFT 0.98006 0.13147 0.26237 0.64777 11124.34 2021-04-15T08:20:02 2021-04-15T06:30:38 None None
PL-GIFT 0.53742 0.03804 0.18228 0.80807 9747.65 2021-04-15T18:20:02 2021-04-15T18:15:43 None None
PL-GIFT 0.51837 0.04316 0.17625 0.82359 10750.07 2021-04-16T00:30:01 2021-04-16T00:11:02 None None
PL-GIFT 0.52344 0.04449 0.17969 0.81845 10497.33 2021-04-16T06:30:01 2021-04-16T04:04:22 None None
PL-GIFT 0.69315 0.0 0.25 0.5 2745.7 2021-04-16T14:00:02 2021-04-16T13:57:59 :No Results: None
PL-GIFT 0.54564 0.05035 0.18795 0.81955 36007.3 2021-04-16T16:10:02 2021-04-16T15:44:07 None None
PL-GIFT 0.57724 0.06341 0.19196 0.79295 8060.2 2021-04-17T02:20:02 2021-04-16T18:41:09 None None
PL-GIFT 0.88177 0.10248 0.27638 0.68981 7782.05 2021-04-18T18:40:02 2021-04-18T18:38:47 None None
PL-GIFT 0.69315 0.0 0.25 0.5 8628.14 2021-04-19T00:50:01 2021-04-18T23:58:31 :No Results: None
PL-GIFT 0.53173 0.04891 0.18178 0.8165 9370.79 2021-04-19T13:40:01 2021-04-19T13:37:25 None None
PL-GIFT 0.70734 0.04665 0.25009 0.62916 3115.5 2021-04-19T21:00:01 2021-04-19T21:00:02 None None
Perspecta-IUB 0.80542 0.06942 0.27076 0.55141 8894.23 2021-03-24T16:20:02 2021-03-16T15:02:05 None None
Perspecta-IUB 1.09389 0.13351 0.31056 0.67323 9731.14 2021-04-06T17:10:02 2021-04-06T17:01:49 None None
Perspecta-IUB 0.79254 0.09999 0.24751 0.79 9157.92 2021-04-19T01:20:02 2021-04-19T01:11:08 None None
Perspecta-IUB 0.59105 0.02518 0.20214 0.78527 8989.4 2021-04-27T11:50:02 2021-04-27T11:49:04 None None
ARM-UCSD 0.72242 0.07266 0.24333 0.68889 5841.26 2021-03-24T16:20:02 2021-03-22T22:07:23 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 6994.71 2021-03-24T23:20:02 2021-03-24T23:11:36 :No Results: None
ARM-UCSD 1.10327 0.09787 0.37333 0.54583 6591.29 2021-03-25T05:20:02 2021-03-25T00:32:30 None None
ARM-UCSD 1.0758 0.09761 0.36333 0.55833 6683.07 2021-03-26T07:40:01 2021-03-26T07:37:48 None None
ARM-UCSD 7.19677 1.04988 0.37292 0.4792 6796.96 2021-03-26T20:20:02 2021-03-26T20:17:13 None None
ARM-UCSD 7.19677 1.04988 0.37292 0.47828 6737.53 2021-03-28T16:10:02 2021-03-28T16:05:54 None None
ARM-UCSD 1.08496 0.0977 0.36667 0.55417 6554.84 2021-03-28T22:20:02 2021-03-28T18:42:22 None None
ARM-UCSD 1.20397 0.09828 0.41 0.5 6647.63 2021-03-29T04:20:02 2021-03-28T23:43:17 None None
ARM-UCSD 1.12158 0.09801 0.38 0.5375 9455.42 2021-04-05T19:30:02 2021-04-05T19:26:37 None None
ARM-UCSD 12.3764 1.22923 0.44792 0.39873 9919.47 2021-04-06T01:30:02 2021-04-06T00:01:40 None None
ARM-UCSD 0.98849 0.06167 0.37125 0.44792 10225.31 2021-04-06T07:40:01 2021-04-06T05:23:21 None None
ARM-UCSD 1.19024 0.09828 0.405 0.50625 9650.86 2021-04-08T01:00:02 2021-04-08T00:55:05 None None
ARM-UCSD 0.66031 0.03997 0.2325 0.65625 12450.41 2021-04-08T23:20:02 2021-04-08T23:10:08 None None
ARM-UCSD 0.75156 0.07349 0.26021 0.66042 12366.28 2021-04-09T18:40:01 2021-04-09T18:32:35 None None
ARM-UCSD 0.78047 0.07431 0.27187 0.64375 12277.56 2021-04-12T09:10:02 2021-04-12T09:09:15 None None
ARM-UCSD 0.71875 0.08818 0.23333 0.72083 12493.17 2021-04-12T23:10:02 2021-04-12T23:05:17 None None
ARM-UCSD 0.78742 0.09094 0.25833 0.68958 12845.45 2021-04-13T05:10:02 2021-04-13T04:50:30 None None
ARM-UCSD 0.76911 0.09026 0.25167 0.69792 13213.54 2021-04-13T11:10:02 2021-04-13T07:45:11 None None
ARM-UCSD 0.64644 0.05819 0.22 0.70625 12977.25 2021-04-13T17:10:02 2021-04-13T16:33:52 None None
ARM-UCSD 0.81581 0.06832 0.28167 0.59062 12370.64 2021-04-13T23:10:02 2021-04-13T19:14:37 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 6767.08 2021-04-16T07:50:01 2021-04-16T07:48:03 :No Results: None
ARM-UCSD 0.49719 0.04439 0.16533 0.81003 53552.75 2021-04-17T00:10:01 2021-04-17T00:09:20 None None
ARM-UCSD 0.48346 0.04259 0.16033 0.81915 44297.56 2021-04-17T19:10:02 2021-04-17T19:02:46 None None
ARM-UCSD 0.51877 0.03791 0.17278 0.79734 38875.16 2021-04-18T20:30:01 2021-04-18T20:27:43 None None
ARM-UCSD 0.57659 0.0434 0.19611 0.7476 32774.42 2021-04-19T07:20:02 2021-04-19T02:19:18 None None
ARM-UCSD 0.57659 0.0434 0.19611 0.7476 39723.96 2021-04-19T18:10:01 2021-04-19T18:08:06 None None
ARM-UCSD 0.90159 0.09356 0.3005 0.63121 102254.96 2021-04-24T07:50:01 2021-04-24T07:45:15 None None
ARM-UCSD 0.90159 0.09356 0.3005 0.63127 106056.13 2021-04-25T12:20:02 2021-04-24T08:04:46 None None
ARM-UCSD 0.67697 0.06964 0.227 0.71432 133200.01 2021-04-27T01:00:01 2021-04-27T00:56:01 None :Timeout:
ARM-UCSD 0.56312 0.07983 0.17667 0.79167 97986.25 2021-04-28T18:20:01 2021-04-28T18:04:16 None None
ARM-UCSD 0.58143 0.08098 0.18333 0.78333 92009.84 2021-04-29T23:50:01 2021-04-29T23:43:47 None None
ICSI-2 0.89795 0.06827 0.31883 0.5446 7505.13 2021-03-24T16:20:02 2021-03-11T15:58:38 None None
ICSI-2 0.72869 0.04204 0.25737 0.56952 14723.42 2021-04-13T18:50:02 2021-04-13T18:42:36 None None
ICSI-2 0.73065 0.03586 0.26389 0.55934 18534.35 2021-04-15T18:40:02 2021-04-15T18:39:20 None None
ARM 1.10187 0.22095 0.28809 0.50208 10112.92 2021-03-24T16:20:02 2021-03-01T21:35:22 None None
ARM 0.69315 0.0 0.25 0.5 5054.03 2021-04-19T18:50:01 2021-04-19T18:42:07 :No Results: None
TrinitySRITrojAI-SBU 2.00608 0.23419 0.39375 0.5428 13712.19 2021-03-24T16:20:02 2021-03-19T16:23:09 None None
TrinitySRITrojAI-SBU 0.64742 0.02999 0.22883 0.67667 7489.19 2021-04-13T03:30:02 2021-04-13T03:25:21 None None
TrinitySRITrojAI-SBU 9.54139 1.11201 0.42287 0.50813 7477.56 2021-04-13T14:20:02 2021-04-13T07:22:08 None None
TrinitySRITrojAI-SBU 0.6778 0.02189 0.24298 0.58721 7927.25 2021-04-13T20:20:02 2021-04-13T19:26:00 None None
TrinitySRITrojAI-SBU 9.85254 1.13036 0.41565 0.48482 7325.51 2021-04-14T02:20:02 2021-04-14T01:17:57 None None
TrinitySRITrojAI-SBU 0.71257 0.02125 0.25926 0.52378 7270.41 2021-04-19T21:10:01 2021-04-19T21:01:16 None None
TrinitySRITrojAI-SBU 4.02407 0.85129 0.19008 0.77181 96231.28 2021-04-27T14:10:02 2021-04-27T14:09:26 None None
TrinitySRITrojAI-SBU 0.45132 0.10316 0.11794 0.90692 58448.61 2021-04-29T05:20:02 2021-04-29T05:19:50 None None
ARM-UMBC 0.86848 0.11635 0.25289 0.67792 13701.48 2021-03-24T16:20:02 2021-03-19T16:49:50 None None
ARM-UMBC 0.69315 0.0 0.25 0.5 273.82 2021-04-19T14:50:02 2021-04-19T14:48:12 :No Results: None
ARM-UMBC 12.95205 1.23354 0.46875 0.08247 23592.54 2021-04-19T19:10:02 2021-04-19T18:22:42 None None
Perspecta-PurdueRutgers 0.58139 0.07374 0.18767 0.77292 82238.56 2021-03-27T23:40:02 2021-03-27T23:39:18 None None
Perspecta-PurdueRutgers 0.63088 0.07615 0.20725 0.75799 124626.68 2021-04-02T05:40:02 2021-04-02T05:33:17 None None
Perspecta-PurdueRutgers 0.61606 0.07549 0.20133 0.76076 113219.19 2021-04-07T04:20:01 2021-04-07T04:13:17 None None
Perspecta-PurdueRutgers 0.46043 0.06517 0.14179 0.84271 48977.93 2021-04-10T04:30:02 2021-04-10T04:22:24 None None
Perspecta-PurdueRutgers 0.43246 0.06326 0.13029 0.84271 48965.84 2021-04-11T03:00:01 2021-04-11T02:59:14 None None
Perspecta-PurdueRutgers 0.36725 0.05662 0.10596 0.88889 48974.76 2021-04-12T02:40:02 2021-04-12T02:36:20 None None
Perspecta-PurdueRutgers 0.28282 0.04688 0.07271 0.91597 52449.08 2021-04-13T05:10:02 2021-04-13T05:05:49 None None
Perspecta-PurdueRutgers 0.48782 0.06616 0.15165 0.81465 79938.75 2021-04-14T01:00:02 2021-04-14T00:52:18 None None
Perspecta-PurdueRutgers 0.25549 0.05699 0.06789 0.94345 51607.09 2021-04-16T23:10:02 2021-04-16T23:01:11 None None
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 6698.75 2021-04-19T21:00:01 2021-04-19T20:59:09 :No Results: None
Perspecta-PurdueRutgers 0.47744 0.06193 0.14825 0.81098 97246.18 2021-04-22T02:40:02 2021-04-22T02:35:13 None None
Perspecta-PurdueRutgers 0.5185 0.06926 0.16454 0.82292 114399.38 2021-04-23T19:40:02 2021-04-23T19:38:23 None None

Placeholder text for nlp-named-entity-recognition-may2021

Placeholder image

Placeholder image description

Accepting submissions: False
Number of models in nlp-named-entity-recognition-may2021, test: 384

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
PL-GIFT 0.33097 0.0585 0.10263 0.90831 101515.55 2021-07-06T03:40:02 2021-07-06T03:34:45 None None
Perspecta-PurdueRutgers 0.29714 0.05368 0.08609 0.92019 123880.8 2021-07-02T18:30:01 2021-07-02T18:21:00 None None
TrinitySRITrojAI 0.52495 0.03754 0.17422 0.8005 89033.6 2021-06-23T15:10:01 2021-06-23T08:04:24 None None
Perspecta 0.6429 0.0275 0.22815 0.64944 43763.97 2021-06-03T13:30:02 2021-06-03T13:24:57 None None
TrinitySRITrojAI-SBU 1.57021 0.2154 0.37603 0.52587 6917.32 2021-07-06T19:10:02 2021-07-06T19:05:17 None None
ARM-UMBC 0.69315 0.0 0.25 0.5 355.13 2021-06-02T14:20:01 2021-06-02T14:19:17 :No Results: None
ARM-UCSD 0.57092 0.04807 0.19141 0.74219 21242.56 2021-07-31T06:30:02 2021-07-31T06:23:52 None None
Perspecta-IUB 0.50552 0.05042 0.16806 0.82882 133200.01 2021-07-03T05:50:01 2021-07-03T05:45:55 None :Timeout:
ICSI-2 0.69219 0.01453 0.2494 0.548 112282.45 2021-07-11T07:20:02 2021-07-11T07:16:40 None None
trojai-example 0.69315 0.0 0.25 0.5 2685.55 2021-07-15T23:10:01 2021-07-15T23:05:52 :No Results: None
ICSI-1 0.59968 0.06774 0.2075 0.82981 117195.96 2021-07-29T01:20:01 2021-07-27T04:54:36 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
PL-GIFT 0.70806 0.0244 0.25698 0.52949 6572.62 2021-05-07T20:20:01 2021-05-07T20:13:40 None None
PL-GIFT 0.69812 0.02116 0.2523 0.5325 6604.76 2021-05-08T15:30:01 2021-05-08T15:24:06 None None
PL-GIFT 0.73784 0.02825 0.2704 0.48451 13503.3 2021-05-09T14:10:02 2021-05-09T14:02:36 None None
PL-GIFT 1.79703 0.23035 0.44362 0.60486 36459.46 2021-05-11T22:50:01 2021-05-11T22:48:50 None None
PL-GIFT 0.71553 0.14074 0.23481 0.59991 36382.48 2021-05-12T10:10:02 2021-05-12T10:01:43 None None
PL-GIFT 0.71553 0.14074 0.23481 0.59991 36215.64 2021-05-13T05:40:02 2021-05-13T05:30:46 None None
PL-GIFT 2.07212 0.59251 0.27373 0.6904 118426.92 2021-05-14T00:10:02 2021-05-14T00:04:48 None None
PL-GIFT 1.14674 0.3716 0.22844 0.72894 133200.01 2021-05-15T09:10:02 2021-05-14T19:18:45 None :Timeout:
PL-GIFT 0.65782 0.07743 0.22365 0.71168 133200.01 2021-05-17T01:40:02 2021-05-17T01:33:19 None :Timeout:
PL-GIFT 0.61126 0.06462 0.21133 0.72812 133200.01 2021-05-18T20:50:02 2021-05-18T20:41:20 None :Timeout:
PL-GIFT 0.64193 0.05467 0.22753 0.70535 13894.34 2021-05-29T07:40:02 2021-05-29T07:34:59 None None
PL-GIFT 0.62947 0.05612 0.22174 0.72738 15779.11 2021-05-29T22:40:02 2021-05-29T22:39:14 None None
PL-GIFT 0.54501 0.05109 0.18607 0.79384 37337.69 2021-05-30T19:00:02 2021-05-30T18:54:52 None None
PL-GIFT 0.53383 0.04936 0.18041 0.80364 100961.58 2021-05-31T18:30:01 2021-05-31T18:27:40 None None
PL-GIFT 0.55533 0.04606 0.19014 0.77227 103073.68 2021-06-08T19:20:01 2021-06-08T19:15:38 None None
PL-GIFT 0.46625 0.06051 0.15848 0.84123 109456.16 2021-06-15T06:20:02 2021-06-15T06:18:49 None None
PL-GIFT 0.44945 0.0683 0.14737 0.85691 103728.14 2021-06-17T01:10:01 2021-06-17T01:01:41 None None
PL-GIFT 0.42693 0.0704 0.13558 0.86274 105719.78 2021-06-18T06:10:01 2021-06-17T18:03:36 None None
PL-GIFT 1.15229 0.29601 0.15193 0.8669 105561.95 2021-06-19T20:20:02 2021-06-19T20:11:16 None None
PL-GIFT 0.61067 0.08918 0.19789 0.78815 12790.75 2021-06-21T18:50:02 2021-06-21T18:44:39 None None
PL-GIFT 0.46991 0.07792 0.14899 0.85192 39731.13 2021-06-22T09:30:02 2021-06-22T09:23:30 None None
PL-GIFT 0.42501 0.08539 0.12891 0.89693 39184.72 2021-06-25T03:50:01 2021-06-25T03:46:50 None None
PL-GIFT 0.38743 0.07656 0.11914 0.90443 104555.79 2021-06-25T17:30:01 2021-06-25T17:26:07 None None
PL-GIFT 0.38806 0.07608 0.11875 0.90523 102818.94 2021-06-26T22:40:02 2021-06-26T06:45:54 None None
PL-GIFT 0.39961 0.05947 0.12715 0.89365 103386.52 2021-06-28T19:10:02 2021-06-28T19:00:32 None None
PL-GIFT 0.35737 0.06875 0.11141 0.90068 103014.79 2021-06-30T00:00:01 2021-06-28T23:47:26 None None
PL-GIFT 0.34687 0.06673 0.10778 0.90321 103164.44 2021-07-01T07:10:01 2021-07-01T07:02:52 None None
PL-GIFT 0.37615 0.06326 0.11187 0.90158 103637.7 2021-07-02T17:50:02 2021-07-02T17:46:14 None None
PL-GIFT 0.38088 0.06388 0.1143 0.90217 127027.0 2021-07-03T22:40:02 2021-07-03T12:43:42 None None
PL-GIFT 0.33097 0.0585 0.10263 0.90831 101515.55 2021-07-06T03:40:02 2021-07-06T03:34:45 None None
PL-GIFT 0.37323 0.08036 0.11244 0.91078 102852.44 2021-07-07T18:40:02 2021-07-07T18:37:45 None None
PL-GIFT 0.35465 0.06164 0.1107 0.89883 127404.99 2021-07-09T19:00:02 2021-07-09T18:59:22 None None
Perspecta-PurdueRutgers 0.57664 0.07342 0.18845 0.77127 133200.01 2021-05-15T04:50:02 2021-05-15T04:45:28 None :Timeout:
Perspecta-PurdueRutgers 0.44811 0.07136 0.13771 0.84123 123909.65 2021-05-23T19:10:02 2021-05-23T19:01:29 None None
Perspecta-PurdueRutgers 0.44882 0.07107 0.13786 0.8423 124942.91 2021-05-25T19:30:02 2021-05-25T19:21:02 None None
Perspecta-PurdueRutgers 0.44882 0.07107 0.13786 0.8423 123999.17 2021-06-04T15:50:02 2021-06-04T15:43:21 None None
Perspecta-PurdueRutgers 0.32205 0.05735 0.09036 0.91222 124547.03 2021-06-06T04:10:02 2021-06-06T04:03:13 None None
Perspecta-PurdueRutgers 0.29714 0.05368 0.08609 0.92019 123880.8 2021-07-02T18:30:01 2021-07-02T18:21:00 None None
TrinitySRITrojAI 0.7064 0.02073 0.2562 0.52334 2461.63 2021-05-25T06:40:02 2021-05-25T06:34:02 None None
TrinitySRITrojAI 0.69234 0.00171 0.2496 0.52658 16078.88 2021-05-27T20:00:02 2021-05-27T19:52:51 None None
TrinitySRITrojAI 0.63891 0.02679 0.22755 0.60993 52084.33 2021-06-08T17:00:02 2021-06-08T16:55:24 None None
TrinitySRITrojAI 0.66646 0.03255 0.23785 0.62094 133200.01 2021-06-11T01:30:02 2021-06-11T01:20:17 None :Timeout:
TrinitySRITrojAI 0.77302 0.06106 0.28524 0.65595 125725.95 2021-06-14T06:00:02 2021-06-14T06:00:00 None None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 2021-06-15T17:00:01 2021-06-14T06:41:11 :No Results::Info File Missing: None
TrinitySRITrojAI 0.86889 0.20426 0.26747 0.7149 75162.26 2021-06-17T03:40:02 2021-06-17T03:36:41 None None
TrinitySRITrojAI 0.6598 0.01249 0.23346 0.71501 75917.22 2021-06-18T07:10:01 2021-06-18T07:06:43 None None
TrinitySRITrojAI 0.597 0.04898 0.20704 0.72142 54694.95 2021-06-22T23:10:01 2021-06-22T23:03:02 None None
TrinitySRITrojAI 0.52495 0.03754 0.17422 0.8005 89033.6 2021-06-23T15:10:01 2021-06-23T08:04:24 None None
Perspecta 0.79987 0.05593 0.28375 0.5139 133200.03 2021-05-29T00:50:01 2021-05-29T00:41:47 None :Timeout:
Perspecta 0.69315 0.0 0.25 0.5 2021-06-01T01:10:01 2021-06-01T01:02:02 :No Results::Info File Missing: None
Perspecta 0.67142 0.03029 0.24056 0.58986 25876.23 2021-06-02T02:30:02 2021-06-02T02:21:43 None None
Perspecta 0.6462 0.02832 0.22987 0.63683 25443.02 2021-06-02T21:40:01 2021-06-02T21:32:41 None None
Perspecta 0.6429 0.0275 0.22815 0.64944 43763.97 2021-06-03T13:30:02 2021-06-03T13:24:57 None None
Perspecta 0.66074 0.04103 0.23676 0.63429 43937.63 2021-06-04T21:40:01 2021-06-04T21:39:01 None None
Perspecta 0.66533 0.03484 0.23786 0.62134 50620.72 2021-06-07T04:30:02 2021-06-07T04:29:15 None None
Perspecta 0.69287 0.00394 0.24988 0.5075 133200.01 2021-06-15T05:30:02 2021-06-15T05:26:10 None :Timeout:
Perspecta 1.71414 0.32971 0.42416 0.50715 40354.69 2021-06-21T05:10:02 2021-06-21T05:03:58 None None
Perspecta 0.6937 0.04287 0.24716 0.60208 41705.73 2021-06-26T01:50:02 2021-06-26T01:42:51 None None
Perspecta 0.65628 0.0501 0.22722 0.68369 103119.39 2021-07-16T05:00:02 2021-07-16T04:51:35 None None
TrinitySRITrojAI-SBU 12.66504 1.37696 0.45838 0.54167 47597.09 2021-06-01T05:20:02 2021-06-01T05:14:22 None None
TrinitySRITrojAI-SBU 10.53698 1.32533 0.4363 0.47976 17744.82 2021-06-02T05:30:01 2021-06-02T05:23:42 None None
TrinitySRITrojAI-SBU 10.59776 1.32836 0.43165 0.55115 21158.55 2021-06-02T16:50:01 2021-06-02T16:50:05 None None
TrinitySRITrojAI-SBU 5.3296 1.03547 0.33488 0.53575 133200.02 2021-06-08T05:20:02 2021-06-08T05:19:22 None :Timeout:
TrinitySRITrojAI-SBU 1.57021 0.2154 0.37603 0.52587 6917.32 2021-07-06T19:10:02 2021-07-06T19:05:17 None None
ARM-UMBC 0.69315 0.0 0.25 0.5 355.13 2021-06-02T14:20:01 2021-06-02T14:19:17 :No Results: None
ARM-UCSD 1.12959 0.10963 0.38292 0.53385 44073.68 2021-06-02T19:30:01 2021-06-02T19:18:22 None None
ARM-UCSD 1.16964 0.10983 0.3975 0.51562 42494.83 2021-06-08T18:30:03 2021-06-08T18:22:53 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 2021-07-07T19:30:02 2021-07-07T19:25:17 :No Results::Info File Missing: None
ARM-UCSD 0.72516 0.04201 0.26396 0.5651 109134.77 2021-07-15T00:30:01 2021-07-15T00:29:54 None None
ARM-UCSD 1.1582 0.10979 0.39333 0.52083 110417.47 2021-07-16T07:00:02 2021-07-16T00:50:30 None None
ARM-UCSD 0.85131 0.06902 0.31188 0.54688 112912.88 2021-07-22T00:00:02 2021-07-21T23:56:47 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 2021-07-23T16:40:02 2021-07-23T16:26:34 :No Results::Info File Missing: None
ARM-UCSD 0.68561 0.0389 0.24542 0.61015 10519.79 2021-07-24T06:20:01 2021-07-24T06:18:35 None None
ARM-UCSD 0.97294 0.09436 0.33312 0.55275 110807.04 2021-07-24T12:20:02 2021-07-24T06:30:43 None None
ARM-UCSD 0.69607 0.06574 0.24469 0.65885 11662.72 2021-07-26T06:10:01 2021-07-26T06:01:38 None None
ARM-UCSD 0.64197 0.02935 0.22484 0.65885 11602.76 2021-07-26T17:10:02 2021-07-26T17:03:49 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 359.95 2021-07-27T18:50:01 2021-07-27T18:40:27 :No Results: None
ARM-UCSD 0.69956 0.04154 0.25187 0.59603 12419.93 2021-07-28T00:50:01 2021-07-27T21:28:34 None None
ARM-UCSD 0.86407 0.08505 0.30564 0.59603 12312.11 2021-07-28T16:30:02 2021-07-28T16:20:38 None None
ARM-UCSD 0.67183 0.09613 0.21625 0.74219 23397.29 2021-07-29T18:10:02 2021-07-29T17:56:46 None None
ARM-UCSD 0.67183 0.09613 0.21625 0.74219 21883.46 2021-07-30T18:50:01 2021-07-30T18:47:04 None None
ARM-UCSD 0.57092 0.04807 0.19141 0.74219 21242.56 2021-07-31T06:30:02 2021-07-31T06:23:52 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 2121.13 2021-06-08T15:00:02 2021-06-08T09:01:10 :No Results: None
Perspecta-IUB 7.8332 1.18765 0.43574 0.40114 67248.88 2021-06-15T03:50:02 2021-06-15T03:49:06 None None
Perspecta-IUB 0.64952 0.14528 0.21129 0.78278 58601.36 2021-06-19T03:30:01 2021-06-19T03:28:15 None None
Perspecta-IUB 0.68463 0.10564 0.22642 0.63703 133200.01 2021-06-25T01:40:02 2021-06-25T01:37:23 None :Timeout:
Perspecta-IUB 0.5723 0.0411 0.19763 0.74759 133200.01 2021-06-30T00:30:02 2021-06-30T00:29:48 None :Timeout:
Perspecta-IUB 0.50552 0.05042 0.16806 0.82882 133200.01 2021-07-03T05:50:01 2021-07-03T05:45:55 None :Timeout:
Perspecta-IUB 0.50597 0.04617 0.16946 0.82359 133200.01 2021-07-13T14:20:02 2021-07-13T14:10:22 None :Timeout:
Perspecta-IUB 1.30885 0.29451 0.25251 0.69051 96216.79 2021-07-26T20:40:01 2021-07-26T20:36:54 None None
Perspecta-IUB 1.20488 0.28171 0.24278 0.7299 113093.29 2021-07-29T12:40:01 2021-07-29T12:32:07 None None
Perspecta-IUB 1.19482 0.26423 0.24754 0.72914 126226.86 2021-08-01T02:00:02 2021-08-01T01:53:19 None None
Perspecta-IUB 0.52424 0.04387 0.17684 0.81057 110049.15 2021-08-02T19:30:02 2021-08-02T19:23:49 None None
ICSI-2 0.69315 0.0 0.25 0.5 465.74 2021-07-10T20:20:02 2021-07-10T20:16:54 :No Results: None
ICSI-2 0.69219 0.01453 0.2494 0.548 112282.45 2021-07-11T07:20:02 2021-07-11T07:16:40 None None
trojai-example 0.69315 0.0 0.25 0.5 2685.55 2021-07-15T23:10:01 2021-07-15T23:05:52 :No Results: None
trojai-example 0.69315 0.0 0.25 0.5 2824.34 2021-07-16T15:40:01 2021-07-16T15:33:10 :No Results: None
trojai-example 2.54417 0.67804 0.53029 0.39913 133200.01 2021-07-26T00:20:01 2021-07-26T00:13:16 None :Timeout:
ICSI-1 0.61345 0.06762 0.21477 0.81616 127539.16 2021-07-27T01:20:02 2021-07-27T01:14:29 None None
ICSI-1 0.59968 0.06774 0.2075 0.82981 117195.96 2021-07-29T01:20:01 2021-07-27T04:54:36 None None

Placeholder text for nlp-question-answering-sep2021

Placeholder image

Placeholder image description

Accepting submissions: False
Number of models in nlp-question-answering-sep2021, test: 360

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 0.94248 0.10086 0.31579 0.53727 8574.86 2021-08-10T17:50:02 2021-08-10T16:00:44 None None
Perspecta 0.58932 0.05285 0.2055 0.72611 158979.82 2021-11-20T09:40:02 2021-11-20T09:38:30 None None
TrinitySRITrojAI 0.55413 0.04975 0.188 0.78497 71500.42 2021-11-08T09:20:02 2021-11-08T09:13:04 None None
Perspecta-IUB 0.49644 0.06285 0.16161 0.83991 184118.97 2021-11-05T00:40:01 2021-11-05T00:32:01 None None
TrinitySRITrojAI-SBU 0.59006 0.0571 0.20394 0.76198 7382.34 2021-10-09T21:10:01 2021-10-09T21:08:12 None None
ARM-UCSD 0.65389 0.0535 0.22917 0.66667 95788.92 2021-09-14T00:10:02 2021-09-14T00:03:48 None None
Perspecta-PurdueRutgers 0.30745 0.06534 0.08603 0.92082 168799.14 2021-11-29T20:50:02 2021-11-29T20:46:32 None None
ICSI-2 1.01762 0.13941 0.30814 0.56177 24964.62 2021-10-08T01:20:02 2021-10-08T01:12:15 None None
PL-GIFT 0.57193 0.04204 0.1922 0.78554 212589.38 2021-12-02T16:30:02 2021-12-02T16:27:54 None None
ICSI-1 0.32804 0.04978 0.0895 0.94914 188000.83 2021-11-30T06:20:01 2021-11-30T06:16:20 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 0.94248 0.10086 0.31579 0.53727 8574.86 2021-08-10T17:50:02 2021-08-10T16:00:44 None None
Perspecta 0.68773 0.02821 0.24743 0.54441 216900.01 2021-08-24T16:20:01 2021-08-24T16:16:01 None :Timeout:
Perspecta 0.70172 0.02284 0.25138 0.50647 15482.77 2021-08-27T21:40:01 2021-08-27T21:35:38 None None
Perspecta 0.67398 0.03025 0.2403 0.60952 143751.28 2021-10-12T01:10:01 2021-10-12T01:06:33 None None
Perspecta 0.6727 0.02909 0.24 0.60927 216900.01 2021-10-20T04:50:02 2021-10-20T04:46:01 None :Execute:
Perspecta 0.64121 0.02446 0.22886 0.59605 216900.03 2021-11-09T22:00:01 2021-11-09T21:57:04 None :Execute:
Perspecta 0.6334 0.05221 0.22202 0.70546 170792.94 2021-11-13T21:20:02 2021-11-12T16:18:24 None None
Perspecta 0.64015 0.05539 0.22376 0.67957 216900.01 2021-11-16T00:00:02 2021-11-15T23:51:32 None :Execute:
Perspecta 0.59019 0.05308 0.20547 0.73052 107572.77 2021-11-18T23:20:01 2021-11-18T23:18:28 None None
Perspecta 0.58932 0.05285 0.2055 0.72611 158979.82 2021-11-20T09:40:02 2021-11-20T09:38:30 None None
Perspecta 0.64571 0.07035 0.22362 0.73991 216900.01 2021-12-02T21:40:02 2021-12-02T21:34:36 None :Timeout:
TrinitySRITrojAI 0.70346 0.01537 0.25507 0.45708 10563.66 2021-09-01T02:00:02 2021-09-01T01:51:14 None None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 5254.59 2021-09-01T08:30:02 2021-09-01T07:14:59 :No Results: None
TrinitySRITrojAI 0.67981 0.01029 0.2434 0.58054 17364.15 2021-09-01T14:40:01 2021-09-01T09:32:00 None None
TrinitySRITrojAI 0.57802 0.05535 0.19627 0.78182 126584.9 2021-09-03T22:40:01 2021-09-03T22:36:34 None None
TrinitySRITrojAI 0.68174 0.02912 0.24555 0.56319 142623.99 2021-09-13T17:20:02 2021-09-13T17:13:35 None None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 2021-09-20T23:10:01 2021-09-20T23:09:20 :No Results::Info File Missing: None
TrinitySRITrojAI 0.59827 0.04032 0.20488 0.74752 136817.92 2021-09-21T14:00:02 2021-09-21T11:00:52 None None
TrinitySRITrojAI 0.57492 0.04288 0.1975 0.75931 147327.41 2021-09-26T22:00:02 2021-09-26T21:53:05 None None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 8454.34 2021-10-30T09:00:02 2021-10-30T08:54:25 :No Results: None
TrinitySRITrojAI 0.60368 0.03601 0.2071 0.74671 128941.2 2021-10-30T22:30:02 2021-10-30T22:22:32 None None
TrinitySRITrojAI 0.55413 0.04975 0.188 0.78497 71500.42 2021-11-08T09:20:02 2021-11-08T09:13:04 None None
TrinitySRITrojAI 0.56802 0.04325 0.19341 0.77349 66978.78 2021-11-10T20:10:02 2021-11-10T20:08:56 None None
Perspecta-IUB 6.81026 1.15459 0.44077 0.44853 19551.01 2021-09-06T20:30:02 2021-09-06T20:27:52 None None
Perspecta-IUB 2.00252 0.55902 0.32838 0.52329 68378.21 2021-09-11T21:10:02 2021-09-11T21:09:38 None None
Perspecta-IUB 1.04667 0.10444 0.3601 0.62816 216900.02 2021-09-14T18:00:01 2021-09-14T17:53:20 None :Timeout:
Perspecta-IUB 1.1596 0.27214 0.32196 0.70745 207912.25 2021-09-20T13:50:01 2021-09-20T13:45:30 None None
Perspecta-IUB 1.27477 0.27706 0.35129 0.71477 95205.76 2021-09-27T05:40:02 2021-09-27T05:31:12 None None
Perspecta-IUB 1.07896 0.18155 0.34629 0.75488 178453.78 2021-10-05T04:20:02 2021-10-05T04:15:15 None None
Perspecta-IUB 1.16073 0.26791 0.32926 0.5813 216900.01 2021-10-17T03:40:02 2021-10-17T03:35:58 None :Execute:
Perspecta-IUB 0.69315 0.0 0.25 0.5 216900.01 2021-10-20T20:10:02 2021-10-20T20:03:24 :No Results: :Execute:
Perspecta-IUB 0.62706 0.08326 0.20677 0.79008 216683.17 2021-10-25T14:20:01 2021-10-25T00:31:48 None None
Perspecta-IUB 0.60235 0.10548 0.18508 0.83446 184640.83 2021-10-31T04:40:02 2021-10-31T04:37:30 None None
Perspecta-IUB 3.77639 0.92149 0.26081 0.67344 122655.49 2021-11-03T03:50:01 2021-11-03T03:42:13 None None
Perspecta-IUB 0.49644 0.06285 0.16161 0.83991 184118.97 2021-11-05T00:40:01 2021-11-05T00:32:01 None None
TrinitySRITrojAI-SBU 0.62381 0.04582 0.21672 0.7117 3194.79 2021-09-07T18:00:01 2021-09-07T17:52:29 None None
TrinitySRITrojAI-SBU 0.62483 0.04273 0.2179 0.70326 5633.12 2021-09-20T21:10:01 2021-09-20T20:02:39 None None
TrinitySRITrojAI-SBU 0.62161 0.04306 0.21608 0.71164 6035.63 2021-10-02T19:30:01 2021-10-02T19:23:28 None None
TrinitySRITrojAI-SBU 0.61823 0.04118 0.21454 0.71571 5795.67 2021-10-04T18:50:02 2021-10-04T18:46:30 None None
TrinitySRITrojAI-SBU 0.59006 0.0571 0.20394 0.76198 7382.34 2021-10-09T21:10:01 2021-10-09T21:08:12 None None
TrinitySRITrojAI-SBU 0.61187 0.03865 0.21323 0.71636 7392.59 2021-10-10T03:10:02 2021-10-10T01:05:57 None None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 7283.53 2021-10-11T04:50:02 2021-10-11T04:46:55 :No Results: None
TrinitySRITrojAI-SBU 1.2415 0.24913 0.26045 0.74327 10801.93 2021-10-11T11:00:02 2021-10-11T05:30:39 None None
TrinitySRITrojAI-SBU 1.13724 0.23831 0.24042 0.7248 10820.06 2021-10-11T17:10:02 2021-10-11T16:33:29 None None
TrinitySRITrojAI-SBU 0.59033 0.05711 0.20407 0.7617 10607.92 2021-10-11T23:10:02 2021-10-11T21:22:15 None None
TrinitySRITrojAI-SBU 0.65611 0.15715 0.20414 0.7614 10740.27 2021-10-12T13:40:02 2021-10-12T13:32:49 None None
TrinitySRITrojAI-SBU 0.65611 0.15715 0.20414 0.7614 10740.29 2021-10-12T20:00:01 2021-10-12T19:53:50 None None
TrinitySRITrojAI-SBU 0.60998 0.0599 0.21019 0.75377 11029.68 2021-10-20T02:30:02 2021-10-20T02:28:12 None None
TrinitySRITrojAI-SBU 0.60615 0.05948 0.20756 0.75873 9826.24 2021-10-20T17:50:01 2021-10-20T17:43:38 None None
ARM-UCSD 0.84998 0.10743 0.28111 0.66111 98721.46 2021-09-07T21:50:02 2021-09-07T21:46:04 None None
ARM-UCSD 0.83777 0.107 0.27667 0.66667 98566.32 2021-09-09T01:20:01 2021-09-07T21:51:50 None None
ARM-UCSD 0.81232 0.07079 0.295 0.575 91227.56 2021-09-11T01:10:02 2021-09-11T01:04:52 None None
ARM-UCSD 0.65389 0.0535 0.22917 0.66667 95788.92 2021-09-14T00:10:02 2021-09-14T00:03:48 None None
ARM-UCSD 0.69796 0.0309 0.25075 0.55259 216900.01 2021-10-04T23:20:02 2021-10-04T22:44:47 None :Execute:
ARM-UCSD 0.90859 0.0716 0.33667 0.50556 160052.13 2021-10-08T00:40:02 2021-10-08T00:36:24 None None
ARM-UCSD 0.91042 0.06934 0.33792 0.49384 77691.11 2021-10-12T08:00:02 2021-10-12T07:52:56 None None
ARM-UCSD 0.92713 0.06924 0.34517 0.48019 34020.25 2021-10-19T06:20:02 2021-10-19T06:18:54 None None
ARM-UCSD 0.88934 0.07155 0.32833 0.51944 89481.69 2021-10-25T14:20:01 2021-10-24T01:16:24 None None
Perspecta-PurdueRutgers 0.73775 0.07218 0.26135 0.65412 59653.46 2021-09-13T04:40:01 2021-09-13T04:32:05 None None
Perspecta-PurdueRutgers 0.7006 0.0201 0.25256 0.5131 216900.02 2021-09-15T20:10:02 2021-09-15T20:04:28 None :Timeout:
Perspecta-PurdueRutgers 0.652 0.0363 0.23 0.63889 42393.41 2021-09-20T08:40:01 2021-09-20T08:34:04 None None
Perspecta-PurdueRutgers 0.56716 0.0624 0.18827 0.73333 151680.56 2021-09-21T02:50:01 2021-09-21T02:45:30 None None
Perspecta-PurdueRutgers 0.4619 0.0542 0.14338 0.83148 205280.61 2021-09-24T04:50:01 2021-09-24T04:42:13 None None
Perspecta-PurdueRutgers 0.57749 0.08522 0.18666 0.79594 181991.13 2021-09-29T05:10:02 2021-09-29T05:03:07 None None
Perspecta-PurdueRutgers 0.55324 0.06123 0.18348 0.74611 185884.11 2021-10-03T01:00:02 2021-10-03T01:00:14 None None
Perspecta-PurdueRutgers 0.49423 0.08033 0.15358 0.83745 210165.52 2021-10-05T13:40:02 2021-10-05T13:31:35 None None
Perspecta-PurdueRutgers 0.47487 0.07144 0.14941 0.84968 205601.79 2021-10-08T05:20:01 2021-10-08T05:13:43 None None
Perspecta-PurdueRutgers 0.62987 0.04118 0.22043 0.66861 40399.12 2021-10-10T18:40:02 2021-10-10T18:39:42 None None
Perspecta-PurdueRutgers 0.62554 0.04061 0.21862 0.67296 40386.38 2021-10-11T07:20:02 2021-10-11T07:16:12 None None
Perspecta-PurdueRutgers 0.60242 0.05001 0.20721 0.70395 33381.29 2021-10-13T05:20:01 2021-10-13T05:11:13 None None
Perspecta-PurdueRutgers 0.41178 0.07833 0.12024 0.87415 169552.1 2021-10-14T18:20:01 2021-10-14T18:12:45 None None
Perspecta-PurdueRutgers 0.65602 0.03665 0.23173 0.63423 120176.78 2021-10-17T00:10:02 2021-10-17T00:02:35 None None
Perspecta-PurdueRutgers 0.39113 0.07281 0.11593 0.88622 181532.31 2021-10-18T14:00:02 2021-10-18T13:50:35 None None
Perspecta-PurdueRutgers 0.6767 0.04076 0.23997 0.61083 73794.58 2021-10-22T01:10:02 2021-10-22T01:09:53 None None
Perspecta-PurdueRutgers 0.34162 0.06726 0.09637 0.90756 178807.38 2021-10-25T15:20:02 2021-10-25T15:16:59 None None
Perspecta-PurdueRutgers 1.00645 0.08631 0.36589 0.5 3038.8 2021-10-27T21:20:02 2021-10-27T21:14:22 None None
Perspecta-PurdueRutgers 0.61782 0.04169 0.21509 0.68056 107085.96 2021-11-24T13:40:01 2021-11-24T13:38:58 None None
Perspecta-PurdueRutgers 0.3205 0.07153 0.08818 0.9092 167593.69 2021-11-25T21:40:01 2021-11-25T21:33:12 None None
Perspecta-PurdueRutgers 0.64289 0.04478 0.22505 0.65622 100327.56 2021-11-28T09:20:02 2021-11-28T09:15:04 None None
Perspecta-PurdueRutgers 0.30745 0.06534 0.08603 0.92082 168799.14 2021-11-29T20:50:02 2021-11-29T20:46:32 None None
Perspecta-PurdueRutgers 0.62787 0.04325 0.21908 0.67019 102600.59 2021-12-02T09:00:02 2021-12-02T08:50:40 None None
ICSI-2 1.01762 0.13941 0.30814 0.56177 24964.62 2021-10-08T01:20:02 2021-10-08T01:12:15 None None
ICSI-2 1.31486 0.16853 0.36533 0.53346 24698.02 2021-11-08T23:20:02 2021-11-08T23:19:31 None None
PL-GIFT 0.95822 0.08846 0.34413 0.54475 55743.2 2021-11-19T07:30:02 2021-11-19T07:27:31 None None
PL-GIFT 0.70472 0.02837 0.25488 0.53048 50955.96 2021-11-20T02:50:02 2021-11-20T02:44:16 None None
PL-GIFT 0.73384 0.04854 0.26287 0.57511 96243.79 2021-11-20T19:20:01 2021-11-20T19:20:00 None None
PL-GIFT 0.87097 0.07666 0.30824 0.5434 117536.41 2021-11-21T23:40:02 2021-11-21T23:34:23 None None
PL-GIFT 0.57193 0.04204 0.1922 0.78554 212589.38 2021-12-02T16:30:02 2021-12-02T16:27:54 None None
ICSI-1 0.39745 0.03953 0.1231 0.9179 216900.01 2021-11-26T01:20:02 2021-11-26T01:10:40 None :Execute:
ICSI-1 0.32804 0.04978 0.0895 0.94914 188000.83 2021-11-30T06:20:01 2021-11-30T06:16:20 None None

Placeholder text for nlp-summary-jan2022

Placeholder image

Placeholder image description

Accepting submissions: False
Number of models in nlp-summary-jan2022, test: 420

Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 0.69315 0.0 0.25 0.5 2022-03-04T16:00:02 2022-03-04T15:55:09 :Executed File Update::No Results::Info File Missing: None
TrinitySRITrojAI 0.47508 0.04796 0.15586 0.85257 98697.76 2022-07-08T20:10:02 2022-07-08T20:08:09 None None
ICSI-2 0.69315 0.0 0.25 0.5 2022-02-01T20:50:02 2022-02-01T20:12:05 :No Results::Info File Missing: None
ARM-UCSD 0.67376 0.01746 0.24038 0.60439 3179.67 2022-03-21T07:00:03 2022-03-21T06:54:17 None None
Perspecta-PurdueRutgers 0.3265 0.05829 0.09501 0.91801 97573.01 2022-04-17T15:00:01 2022-04-17T14:55:20 None None
PL-GIFT 0.43045 0.07292 0.12699 0.88906 104873.93 2022-06-20T17:40:02 2022-06-20T17:31:04 None :Container Parameters (jsonschema checker):
TrinitySRITrojAI-SBU 0.5702 0.05871 0.19234 0.77946 15899.61 2022-04-06T18:30:01 2022-04-06T18:23:27 None None
TrinitySRITrojAI-BostonU 0.61365 0.02799 0.21165 0.72985 4602.66 2022-02-25T17:00:01 2022-02-25T16:54:55 None None
Perspecta 0.66285 0.03988 0.23447 0.68909 66745.45 2022-05-23T13:40:01 2022-05-23T13:32:48 None None
Perspecta-IUB 0.59307 0.03302 0.20409 0.74725 105583.58 2022-04-14T20:10:01 2022-04-14T05:35:29 None :Container Parameters (jsonschema checker):
ICSI-1 0.47653 0.04074 0.15546 0.85909 66681.33 2022-04-08T22:40:01 2022-04-08T22:37:06 None :Container Parameters (learned parameters):
ARM 0.54481 0.04059 0.18989 0.75777 3896.28 2022-05-17T16:00:01 2022-05-17T15:51:53 None None

All Results

Team Cross Entropy CE 95% CI Brier Score ROC-AUC Runtime (s) Submission Timestamp File Timestamp Parsing Errors Launch Errors
trojai-example 0.81121 0.05865 0.28367 0.50331 2535.33 2022-01-27T04:10:02 2022-01-27T04:07:20 None None
trojai-example 0.69315 0.0 0.25 0.5 2022-03-04T16:00:02 2022-03-04T15:55:09 :Executed File Update::No Results::Info File Missing: None
trojai-example 0.69315 0.0 0.25 0.5 2022-04-15T02:00:03 2022-03-04T15:55:09 :Executed File Update::No Results::Info File Missing: None
trojai-example 0.69315 0.0 0.25 0.5 2022-04-14T20:00:01 2022-04-14T19:59:00 None None
TrinitySRITrojAI 0.59178 0.02464 0.20187 0.75345 5221.01 2022-01-28T18:30:01 2022-01-28T18:26:24 None None
TrinitySRITrojAI 0.69315 0.0 0.25 0.5 127200.02 2022-02-13T10:10:01 2022-02-13T10:08:22 :No Results: :Timeout:
TrinitySRITrojAI 0.64973 0.02776 0.2286 0.66661 43104.35 2022-02-14T21:40:01 2022-02-14T09:04:31 None None
TrinitySRITrojAI 0.65631 0.06938 0.22594 0.7315 10361.71 2022-02-15T19:10:02 2022-02-15T19:08:12 None None
TrinitySRITrojAI 0.64667 0.02749 0.22724 0.66995 43753.43 2022-02-16T03:10:02 2022-02-16T03:02:56 None None
TrinitySRITrojAI 0.92712 0.09139 0.30947 0.58266 10425.87 2022-03-03T15:20:01 2022-03-03T15:12:47 None None
TrinitySRITrojAI 0.53125 0.03887 0.17866 0.81774 23681.89 2022-03-05T03:30:01 2022-03-05T03:23:49 None None
TrinitySRITrojAI 0.56205 0.06076 0.18773 0.78908 41298.78 2022-03-19T07:40:02 2022-03-19T07:39:00 None None
TrinitySRITrojAI 0.66548 0.06345 0.22688 0.70832 41070.49 2022-03-21T11:20:02 2022-03-21T11:13:53 None None
TrinitySRITrojAI 0.65578 0.0241 0.23143 0.66957 69968.17 2022-04-04T13:50:03 2022-04-04T13:46:34 None None
TrinitySRITrojAI 0.51471 0.05017 0.17474 0.80837 55469.57 2022-04-07T07:40:02 2022-04-07T07:38:04 None None
TrinitySRITrojAI 0.51554 0.04732 0.17143 0.82255 36878.25 2022-05-12T06:40:01 2022-05-12T06:34:12 None None
TrinitySRITrojAI 0.57132 0.03643 0.19491 0.77441 21396.71 2022-06-03T10:40:02 2022-06-03T10:33:34 None None
TrinitySRITrojAI 0.51285 0.05858 0.16867 0.83286 21618.95 2022-06-05T07:20:02 2022-06-05T07:19:26 None None
TrinitySRITrojAI 0.54747 0.05916 0.1833 0.7963 30376.74 2022-06-13T07:30:01 2022-06-13T07:21:13 None None
TrinitySRITrojAI 0.47508 0.04796 0.15586 0.85257 98697.76 2022-07-08T20:10:02 2022-07-08T20:08:09 None None
ICSI-2 0.69315 0.0 0.25 0.5 2022-02-01T20:50:02 2022-02-01T20:12:05 :No Results::Info File Missing: None
ICSI-2 0.69315 0.0 0.25 0.5 1266.38 2022-02-01T23:00:01 2022-02-01T21:07:18 :No Results: :Container Parameters::Container Parameters::Container Parameters:
ICSI-2 0.69315 0.0 0.25 0.5 1274.03 2022-02-07T18:30:02 2022-02-07T18:20:28 :No Results: None
ICSI-2 1.99613 0.36052 0.3086 0.62933 17112.5 2022-02-08T07:00:01 2022-02-08T06:54:12 None None
ICSI-2 0.75425 0.05426 0.26871 0.58462 17188.5 2022-03-08T00:50:01 2022-03-08T00:41:33 :Result Parse: None
ICSI-2 0.7664 0.06349 0.27282 0.63207 17130.38 2022-03-22T17:20:02 2022-03-22T17:18:22 None None
ICSI-2 0.83969 0.06859 0.28949 0.48361 2540.98 2022-02-10T06:50:02 2022-02-10T06:49:31 None None
ARM-UCSD 1.26312 0.1003 0.43438 0.44905 5992.37 2022-02-08T19:40:02 2022-02-08T19:39:52 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 2405.37 2022-02-25T06:20:02 2022-02-25T06:12:53 :No Results: None
ARM-UCSD 0.69315 0.0 0.25 0.5 422.87 2022-02-25T19:00:02 2022-02-25T18:56:07 :No Results: :Container Parameters (metaparameters)::Container Parameters (metaparameters schema)::Container Parameters (learned parameters):
ARM-UCSD 0.69315 0.0 0.25 0.5 674.21 2022-03-04T05:40:01 2022-03-04T05:35:07 :No Results: None
ARM-UCSD 0.90869 0.06987 0.32067 0.50817 2682.4 2022-03-04T11:40:02 2022-03-04T08:01:32 None None
ARM-UCSD 1.23536 0.10503 0.42143 0.5 3367.48 2022-03-18T01:50:02 2022-03-18T01:45:41 None None
ARM-UCSD 1.17258 0.10503 0.39857 0.5 3323.53 2022-03-18T07:50:02 2022-03-18T04:33:11 None None
ARM-UCSD 0.92734 0.09498 0.31324 0.60439 3186.25 2022-03-18T17:20:01 2022-03-18T17:13:45 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 3165.32 2022-03-18T23:20:03 2022-03-18T23:15:28 :No Results: None
ARM-UCSD 0.67376 0.01746 0.24038 0.60439 3179.67 2022-03-21T07:00:03 2022-03-21T06:54:17 None None
ARM-UCSD 0.69315 0.0 0.25 0.5 3300.07 2022-04-05T17:20:02 2022-04-05T17:19:50 :No Results: None
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-04-15T02:20:03 2022-04-15T02:16:00 :Executed File Update::No Results::Info File Missing: None
ARM-UCSD 0.76113 0.05203 0.27798 0.57298 3201.59 2022-04-19T08:20:03 2022-04-19T08:13:35 None :Container Parameters (jsonschema checker):
ARM-UCSD 0.69315 0.0 0.25 0.5 2022-04-28T17:00:01 2022-04-28T16:54:09 :Executed File Update::No Results::Info File Missing: None
ARM-UCSD 11.44714 1.30173 0.41429 0.40556 34300.09 2022-04-28T22:50:02 2022-04-28T22:47:52 None :Container Parameters (jsonschema checker):
ARM-UCSD 11.24978 1.29831 0.40714 0.44608 36378.02 2022-05-05T01:00:01 2022-05-05T00:50:39 None :Container Parameters (jsonschema checker):
Perspecta-PurdueRutgers 0.40009 0.0648 0.12123 0.90239 62228.45 2022-02-21T01:50:01 2022-02-21T01:40:52 None None
Perspecta-PurdueRutgers 0.44151 0.08356 0.126 0.88619 68957.33 2022-02-27T04:00:02 2022-02-27T03:55:11 None None
Perspecta-PurdueRutgers 0.70099 0.01913 0.25283 0.50987 29014.14 2022-03-03T21:40:02 2022-03-03T21:36:00 None None
Perspecta-PurdueRutgers 0.41194 0.05271 0.12569 0.88198 67507.75 2022-03-04T15:50:02 2022-03-04T15:47:28 None None
Perspecta-PurdueRutgers 0.38216 0.05691 0.11242 0.87895 69056.38 2022-03-07T12:50:01 2022-03-07T12:43:50 None None
Perspecta-PurdueRutgers 0.39346 0.05861 0.11638 0.87132 71968.47 2022-03-13T00:20:01 2022-03-13T00:16:37 None None
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 2022-03-18T20:40:03 2022-03-18T20:31:57 :Executed File Update::No Results::Info File Missing: None
Perspecta-PurdueRutgers 0.401 0.05909 0.11965 0.86571 84600.3 2022-03-19T02:50:01 2022-03-19T02:33:30 None None
Perspecta-PurdueRutgers 0.39161 0.05801 0.11604 0.87222 99259.85 2022-03-20T14:30:01 2022-03-20T14:24:53 None None
Perspecta-PurdueRutgers 0.39568 0.06949 0.12288 0.90803 99788.29 2022-03-26T04:30:02 2022-03-26T04:29:33 None None
Perspecta-PurdueRutgers 0.36323 0.0714 0.10538 0.91002 118502.87 2022-03-29T18:30:02 2022-03-29T18:21:40 None None
Perspecta-PurdueRutgers 0.426 0.0813 0.12 0.8904 67146.61 2022-04-03T04:20:01 2022-04-03T04:15:45 None None
Perspecta-PurdueRutgers 0.37583 0.07635 0.10467 0.90572 92898.75 2022-04-08T13:50:01 2022-04-08T13:41:32 None None
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 2022-04-14T04:20:02 2022-04-14T04:19:12 :Executed File Update::No Results::Info File Missing: None
Perspecta-PurdueRutgers 0.69315 0.0 0.25 0.5 2022-04-14T18:30:03 2022-04-14T18:20:41 :Executed File Update::No Results::Info File Missing: None
Perspecta-PurdueRutgers 0.39199 0.07036 0.1155 0.90105 95584.89 2022-04-14T20:10:01 2022-04-14T18:20:41 None None
Perspecta-PurdueRutgers 0.3265 0.05829 0.09501 0.91801 97573.01 2022-04-17T15:00:01 2022-04-17T14:55:20 None None
PL-GIFT 0.58288 0.05991 0.19473 0.77804 73340.4 2022-02-21T23:50:02 2022-02-21T23:46:07 None None
PL-GIFT 0.67086 0.09348 0.21354 0.74638 84607.74 2022-03-08T21:00:02 2022-03-08T20:52:56 None None
PL-GIFT 0.63657 0.09904 0.19192 0.79243 84950.95 2022-03-12T00:30:03 2022-03-12T00:26:23 None None
PL-GIFT 0.56277 0.0492 0.19017 0.78486 85501.15 2022-03-24T12:30:01 2022-03-24T12:25:03 None None
PL-GIFT 0.51936 0.0561 0.17302 0.83003 85809.32 2022-03-25T12:20:03 2022-03-24T21:28:35 None None
PL-GIFT 0.50892 0.04875 0.16716 0.83482 69465.72 2022-03-27T21:40:01 2022-03-27T21:31:19 None None
PL-GIFT 0.53319 0.07306 0.16801 0.8324 70861.18 2022-04-23T01:00:01 2022-04-23T00:56:05 None :Container Parameters (jsonschema checker):
PL-GIFT 0.50619 0.07194 0.1594 0.85125 69477.01 2022-04-26T06:30:02 2022-04-26T06:24:17 None :Container Parameters (jsonschema checker):
PL-GIFT 0.47612 0.07206 0.14749 0.86666 70111.51 2022-04-29T20:10:01 2022-04-29T20:08:30 None :Container Parameters (jsonschema checker):
PL-GIFT 0.48371 0.06758 0.15109 0.86185 69004.05 2022-05-07T07:30:02 2022-05-07T07:22:21 None :Container Parameters (jsonschema checker):
PL-GIFT 0.49454 0.07286 0.15415 0.85815 69424.45 2022-05-08T13:00:01 2022-05-08T12:58:56 None :Container Parameters (jsonschema checker):
PL-GIFT 0.45659 0.06981 0.13966 0.87231 85411.81 2022-05-14T01:40:02 2022-05-14T01:40:02 None :Container Parameters (jsonschema checker):
PL-GIFT 0.45611 0.06843 0.13749 0.86766 86599.22 2022-05-15T04:30:02 2022-05-15T04:26:18 None :Container Parameters (jsonschema checker):
PL-GIFT 0.47084 0.0714 0.14246 0.86629 86494.3 2022-05-17T08:00:01 2022-05-17T07:59:53 None None
PL-GIFT 0.46459 0.08225 0.13309 0.87996 86517.84 2022-06-15T11:00:02 2022-06-15T10:50:58 None :Container Parameters (jsonschema checker):
PL-GIFT 0.50163 0.07998 0.14882 0.85052 86274.17 2022-06-16T15:40:01 2022-06-16T15:35:56 None :Container Parameters (jsonschema checker):
PL-GIFT 0.46849 0.08115 0.13735 0.87368 104801.5 2022-06-18T06:40:01 2022-06-18T06:35:05 None :Container Parameters (jsonschema checker):
PL-GIFT 0.43045 0.07292 0.12699 0.88906 104873.93 2022-06-20T17:40:02 2022-06-20T17:31:04 None :Container Parameters (jsonschema checker):
TrinitySRITrojAI-SBU 0.61822 0.02764 0.21464 0.71296 5660.08 2022-02-24T01:30:01 2022-02-24T01:27:18 None None
TrinitySRITrojAI-SBU 0.69315 0.0 0.25 0.5 2022-04-05T03:30:02 2022-04-05T03:22:19 :Executed File Update::No Results::Info File Missing: None
TrinitySRITrojAI-SBU 0.5702 0.05871 0.19234 0.77946 15899.61 2022-04-06T18:30:01 2022-04-06T18:23:27 None None
TrinitySRITrojAI-SBU 0.68436 0.03318 0.24343 0.58186 127203.36 2022-05-03T03:10:01 2022-05-03T03:00:42 None :Timeout:
TrinitySRITrojAI-SBU 0.6232 0.06735 0.21149 0.73611 92190.2 2022-05-31T17:30:02 2022-05-31T17:21:21 None None
TrinitySRITrojAI-BostonU 0.61365 0.02799 0.21165 0.72985 4602.66 2022-02-25T17:00:01 2022-02-25T16:54:55 None None
Perspecta 0.70636 0.01992 0.25353 0.49303 31313.09 2022-02-26T08:40:02 2022-02-26T08:36:02 None None
Perspecta 0.73158 0.04203 0.26644 0.62743 87355.06 2022-03-26T23:50:01 2022-03-26T23:46:48 None None
Perspecta 0.77727 0.0424 0.28648 0.50126 97796.64 2022-04-04T06:20:01 2022-04-04T06:11:32 None None
Perspecta 0.75772 0.04223 0.27559 0.54565 127203.39 2022-04-18T00:40:02 2022-04-18T00:35:55 None :Container Parameters (jsonschema checker)::Timeout:
Perspecta 0.74528 0.04861 0.26804 0.58052 127203.26 2022-04-20T13:10:03 2022-04-20T13:02:41 None :Container Parameters (jsonschema checker)::Timeout:
Perspecta 0.71878 0.04024 0.25924 0.57792 124236.25 2022-04-27T23:40:02 2022-04-27T23:33:04 None None
Perspecta 0.69606 0.03004 0.25135 0.54419 127203.22 2022-05-07T22:10:02 2022-05-07T22:06:51 None :Timeout:
Perspecta 0.66285 0.03988 0.23447 0.68909 66745.45 2022-05-23T13:40:01 2022-05-23T13:32:48 None None
Perspecta-IUB 0.69074 0.00348 0.24901 0.5049 1816.72 2022-02-26T17:00:01 2022-02-26T16:53:18 None :Container Parameters (learned parameters):
Perspecta-IUB 0.69315 0.0 0.25 0.5 28404.19 2022-03-07T20:40:01 2022-03-08T05:25:39 :No Results: None
Perspecta-IUB 0.69315 0.0 0.25 0.5 61110.49 2022-03-11T19:50:02 2022-03-11T19:48:00 :No Results: None
Perspecta-IUB 0.70668 0.02528 0.25616 0.52743 98956.63 2022-03-13T18:30:01 2022-03-13T18:26:07 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 2454.63 2022-03-22T05:10:01 2022-03-22T05:09:18 :No Results: None
Perspecta-IUB 0.72281 0.03359 0.26283 0.54878 127203.4 2022-03-22T15:10:02 2022-03-22T15:08:50 None :Timeout:
Perspecta-IUB 1.5103 0.34201 0.36372 0.54424 127203.04 2022-03-30T15:30:02 2022-03-30T15:23:21 None :Timeout:
Perspecta-IUB 0.6455 0.0431 0.22697 0.70275 105470.86 2022-04-03T05:30:03 2022-04-03T05:22:06 None None
Perspecta-IUB 0.62241 0.04621 0.21658 0.74073 104013.01 2022-04-05T04:30:01 2022-04-06T01:13:23 None None
Perspecta-IUB 0.63507 0.0328 0.22239 0.68614 104430.23 2022-04-12T00:40:02 2022-04-12T00:33:17 None None
Perspecta-IUB 0.69315 0.0 0.25 0.5 2022-04-14T17:40:01 2022-04-14T05:35:29 :Executed File Update::No Results::Info File Missing: None
Perspecta-IUB 0.59307 0.03302 0.20409 0.74725 105583.58 2022-04-14T20:10:01 2022-04-14T05:35:29 None :Container Parameters (jsonschema checker):
Perspecta-IUB 0.60999 0.04172 0.20978 0.73051 104915.06 2022-04-18T04:40:01 2022-04-18T04:37:52 None :Container Parameters (jsonschema checker):
Perspecta-IUB 0.69315 0.0 0.25 0.5 104021.13 2022-05-17T16:10:02 2022-05-17T16:08:48 :No Results: :Container Parameters (jsonschema checker):
Perspecta-IUB 0.78431 0.074 0.27305 0.64937 103087.41 2022-05-31T15:10:01 2022-05-31T15:01:13 None :Container Parameters (jsonschema checker):
ICSI-1 0.69315 0.0 0.25 0.5 2022-03-10T15:00:01 2022-01-31T17:03:07 :No Results::Info File Missing: None
ICSI-1 0.47653 0.04074 0.15546 0.85909 66681.33 2022-04-08T22:40:01 2022-04-08T22:37:06 None :Container Parameters (learned parameters):
ICSI-1 0.48666 0.04194 0.16015 0.85613 120070.65 2022-04-27T15:10:01 2022-04-27T15:05:16 None :Container Parameters (learned parameters)::Container Parameters (jsonschema checker):
ARM 0.69315 0.0 0.25 0.5 2022-04-04T18:40:02 2022-04-04T18:39:57 :Executed File Update::No Results::Info File Missing: None
ARM 1.99396 0.60497 0.20741 0.74653 3039.62 2022-04-06T17:30:01 2022-04-06T17:29:06 None None
ARM 1.99396 0.60497 0.20741 0.74653 3053.42 2022-05-03T03:00:02 2022-05-03T02:51:49 None None
ARM 0.78387 0.25561 0.19227 0.7587 3038.62 2022-05-03T12:10:02 2022-05-03T12:05:23 None None
ARM 0.54481 0.04059 0.18989 0.75777 3896.28 2022-05-17T16:00:01 2022-05-17T15:51:53 None None

Status and Error Codes

Messages about the status of submissions that may be awaiting execution.

Code Description
None No jobs are submitted.
Queued Job has been queued for processing.
Awaiting Timeout Job is submitted, but must wait for time until next execution.
Pending Job is in processing queue and is pending availability of system resources.
Running Job is running
Disabled No longer accepting jobs from the team.

Messages about the status of files that are shared with the TrojAI google drive.

Code Description
None No shared files found in TrojAI google drive.
Multiple files submitted Team has more than one file shared with the TrojAI google drive account per evaluation server. You can share one file which starts with 'test' (for STS) and one which does not (for ES). Unshare/Delete shared files until only one is shared per server.
Ok Found one file shared with TrojAI google drive per server.

Messages about the general status of submissions (global across all leaderboards).

Code Description
Shared File Error Team has an issue with one or more of the shared files. "Format" indicates incorrect file name, should be "leaderboard_name-data_split_name". "Leaderboard name" indicates invalid leaderboard name. "Data split name" indicates invalid data split name."
Ok All files shared have no issues.

Error codes during processing result and metadata after a submission has completed.

Code Description
None No errors found while parsing the results of the submission.
:Result Parse: Unable to parse one ore more result files generated by the submission container.
:No Results: Unable to find and parse any results generated by the container.
:Executed File Update: Unable to update the submission metadata to reflect the Drive File which was actually executed, if different from the one initially submitted.
:Log File Missing: Unable to find the log file from the container execution.
:Info File Missing: Unable to find the job information file from the container execution.
:Confusion File Missing: Unable to find the confusion matrix file from the container execution.
:File Upload: Unable to upload a file from the test server to Drive.

Error codes during execution of a submission.

Code Description
None No errors found while launching the submission.
:Container Parameters: There was an error with the container, one or more required parameters may not be valid or not exist.
:Slurm Script Error: There was an error when submitting to the processing queue (Slurm may be offline)
Hypervisor offline Unable to establish connection to the virtual machine hypervisor.
:GPU: Failed to communicate with the GPU within the VM
:Copy in: Failed to copy in the actor-shared file into the VM.
:Timeout: The actor's execution timed out. Failed to finish processing all input models within allocated time.
:Execute: Other errors during execution of actor execution. (check slurm log file)
:Copy out: Failed to copy out results (they may not exist).
:Shutdown: Failed to shutdown the VM.
:VM: Issue with the VM resource (major error).

Disclaimer

NIST-developed software is provided by NIST as a public service. You may use, copy and distribute copies of the software in any medium, provided that you keep intact this entire notice. You may improve, modify and create derivative works of the software or any portion of the software, and you may copy and distribute such modifications or works. Modified works should carry a notice stating that you changed the software and should note the date and nature of any such change. Please explicitly acknowledge the National Institute of Standards and Technology as the source of the software.

NIST-developed software is expressly provided "AS IS." NIST MAKES NO WARRANTY OF ANY KIND, EXPRESS, IMPLIED, IN FACT OR ARISING BY OPERATION OF LAW, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT AND DATA ACCURACY. NIST NEITHER REPRESENTS NOR WARRANTS THAT THE OPERATION OF THE SOFTWARE WILL BE UNINTERRUPTED OR ERROR-FREE, OR THAT ANY DEFECTS WILL BE CORRECTED. NIST DOES NOT WARRANT OR MAKE ANY REPRESENTATIONS REGARDING THE USE OF THE SOFTWARE OR THE RESULTS THEREOF, INCLUDING BUT NOT LIMITED TO THE CORRECTNESS, ACCURACY, RELIABILITY, OR USEFULNESS OF THE SOFTWARE.

You are solely responsible for determining the appropriateness of using and distributing the software and you assume all risks associated with its use, including but not limited to the risks and costs of program errors, compliance with applicable laws, damage to or loss of data, programs or equipment, and the unavailability or interruption of operation. This software is not intended to be used in any situation where a failure could cause risk of injury or damage to property. The software developed by NIST employees is not subject to copyright protection within the United States.

The Cross Entropy loss values (and confidence intervals) reported by the TrojAI leaderboard are only indicative of trojan detector performance on the specific dataset the detector was evaluated on. The TrojAI leaderboard results do not indicate general purpose trojan detection algorithm quality.